Software test code updating method based on neural machine translation

Abstract

The invention belongs to the technical field of machine learning, and discloses a neural machine translation-based software test code updating method, which comprises the following steps of: acquiring large-scale method level change data in a Java project from GitHub, and constructing a data set; the data set is processed, and moderate code samples are screened by controlling the nesting depth and the variable number of codes; performing context abstraction, editing serialization and word segmentation coding on the screened data samples; by training the outdated test recognition model and the test case updating model, recognition of whether the test case needs to be updated or not and generation of a new test case are achieved respectively; a Transform updater model is combined with beam search to generate a plurality of test case candidate results, and a legal updated test case is generated through code abstraction recovery and grammar check. According to the method, the automation level of co-evolution of production codes and test codes is improved, and the method is suitable for code maintenance and test optimization tasks in software development.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning, and in particular relates to a software test code updating method based on neural machine translation. Background Art

[0002] As the complexity of software development continues to increase, the co-evolution of production code and test code has become a critical issue in software engineering. Updates to production code often require synchronous updates to corresponding test code to ensure the stability and reliability of the software system. However, in actual development, the maintenance of test code often lags behind changes to production code. This phenomenon may lead to test code failure and insufficient test coverage, thus causing system defects and potential production risks.

[0003] Traditional software test code maintenance mainly relies on manual operations, including manually checking the impact of production code changes and updating the corresponding test code. This method is not only time-consuming and labor-intensive, but also prone to errors. Although some rule-based test code generation methods have been proposed, these methods rely on manually defined rules and templates, have poor versatility, and cannot adapt to complex code update scenarios. In recent years, with the rapid development of machine learning technology, especially the application of deep learning in the fields of natural language processing and program understanding, automatic code generation and update methods based on deep learning have gradually become a research hotspot. For example, neural machine translation (NMT) technology has been widely used in natural language translation, and its end-to-end learning ability and excellent performance provide a new solution for software code updates. The NMT model can regard changes to production code as the "source language" and updates to test code as the "target language", and achieve the co-evolution of the two through the encoder-decoder architecture.

[0004] Existing machine learning-based code update methods still have the following challenges:

[0005] 1) Insufficient sample data quality: In the dataset used to train the model, there are usually fewer samples of co-evolution of production code and test code, and the quality of the samples varies, which affects the generalization ability of the model.

[0006] 2) Long sequence processing problem: The context information of the code is complex and of varying lengths. Traditional models are prone to performance degradation when processing long code sequences.

[0007] 3) Semantic consistency problem: Code updates need to ensure the semantic consistency between production code and test code, but existing methods still have limitations in capturing semantic relationships. Summary of the invention

[0008] The purpose of the present invention is to provide a software test code updating method based on neural machine translation to solve the above-mentioned technical problems.

[0009] In order to solve the above technical problems, the specific technical solution of a software test code update method based on neural machine translation of the present invention is as follows:

[0010] A method for updating software test code based on neural machine translation comprises the following steps:

[0011] S1: Obtain 5163K method-level changes in Java projects from Github to build a dataset;

[0012] S2: Filter and split the data set obtained in S1;

[0013] S3: Apply complexity constraints to the dataset and filter out overly complex or overly simple codes based on the code complexity index to ensure that the model can learn representative and general features more efficiently.

[0014] S4: Label the samples in the dataset to provide clear positive and negative sample labels for the model;

[0015] S5: Use context abstraction, editing serialization, word segmentation and encoding, and sequence alignment methods to transform the code into a form suitable for model processing;

[0016] S6: Use the processed part of the samples to train models, including an outdated test identification model and a test case update model, to train a neural network classifier to identify test cases that need to be updated and a Transformer updater model to generate new test cases.

[0017] S7: predict the updated test code through the neural network model and output the generated modified test code sequence.

[0018] Furthermore, the 5163K method-level changes in S1 include non-test methods and unit tests. The collected co-evolved Java methods and their corresponding test cases annotated with @Test are the following four-tuple:

[0019] <original method,updated method,original test,updated test> , and build a data set;

[0020] In the modification of production code and test code, only EE type modification is selected as the research object. Further, S2 includes the following specific steps:

[0021] We extract method-level production-test code change pair samples from the historical data set obtained from S1. We use regular expressions to match useless lines in the code for these change pair samples and classify production code changes into 12 types, which belong to two categories: programming language structure modifications and natural language structure modifications. The former includes modifications to package-id, import-stmt, class-dec, method-sig, method-body, field, and annotation; the latter includes modifications to copyright, javadoc, and comment. Modifications to package-id, import-stmt, and natural language structures are actually modifications to useless lines for the ultimate goal, because these modifications are actually just corresponding replacement operations in the test code. So we use the following regular expression:

[0022] comment = re.compile(r' [] *(()|()|()|import|package)′)

[0023] comment_mod=re.compile(r′^[+-][]\*)|(VV*)|(VV)|import|package)′)

[0024] empty_row=re.compile(r′^[]*$′)

[0025] empty_row_mod=re.compile(r′^[+-][]*$)

[0026] The processed code is then formatted, with the production and test source codes represented as individual characters separated by a space; the spaces at the beginning and end of the source code are removed, and the extra spaces in the middle of the code are deleted, leaving only one space.

[0027] Furthermore, the S3 includes the following specific steps:

[0028] For each code sample, calculate the sample's nesting depth and the number of scalars, and select samples with moderate indicators based on the obtained indicators;

[0029] (1) Nesting depth: measures the depth of nested structures in a code block:

[0030] Nesting Depth=max(Depth(B i ))

[0031] (2) Number of variables: Count the number of variables declared in the code:

[0032]

[0033] Furthermore, the S4 includes the following specific steps:

[0034] Construct a dataset containing positive and negative samples. If a test case should be updated as its corresponding production code changes, mark it as a positive sample; otherwise, mark it as a negative sample. For the test update task, only the positive samples whose test cases need to be updated in the recognition task are used to construct the dataset. The details of the positive / negative sample construction and test update dataset are as follows:

[0035] (1) Construction of positive samples: First, extract the changed test cases with @test annotations, i.e., t and t′, from the collected method-level code changes. Second, use Tree-Sitter to extract the names of the test cases. Third, perform name matching to find the production code. If the production code is in the same project and has also been changed in the same commit, it is considered a positive sample. Finally, 4,397 positive samples are obtained, each of which consists of four parts, i.e., the original method, the updated method, the original test, and the updated test.

[0036] (2) Negative sample construction: To construct negative samples, we extract the changed methods with test cases that do not need to be updated. First, we exclude the methods in the positive samples and extract the test cases of the remaining changed methods. For each candidate changed method, we search for its test cases as follows:

[0037] Path matching: According to the best practices of Junit testing, the production code and its corresponding test cases are located in the mirror folder. Through path matching, the test class is heuristically found, the file name of the method is obtained, and the file path of the test case is determined;

[0038] Name matching: After path matching, we extract corresponding tests for candidate methods. Similar to the process of building positive samples, we match production code and test code through name matching. Then, we exclude samples whose test cases have changed. We get 13,269 negative samples, each of which consists of three parts: the original method, the updated method, and the original test.

[0039] Furthermore, the S5 comprises the following specific steps:

[0040] First, the code is abstracted and artificial restrictions are imposed when processing the model to determine the number of the most common words, that is, the vocabulary size. In order to reduce the vocabulary, the context must be abstracted while retaining the semantic information of the source code. NPTCC will maintain a dictionary to store the mapping relationship between the original tokens and the replacement variables so that the original tokens can be refilled after the updated test code is generated. Through the above code abstraction process and controlling the size of the vocabulary, the Transformer model focuses on learning common patterns from different code changes;

[0041] Fine-grained production code changes are then better represented by edit serialization, with code edits represented as: Medium i and x i ′ are the tokens of the original version and the new version production method at position i, a i Indicates that x i Convert to x i There are four types of edit operations, namely insertion, deletion, equality, and replacement. To obtain the code change representation, the original method and the modified method are first split into code token sequences and used to construct the edit sequence; then the token-level alignment is calculated to obtain the edit triple at position i. <x i ,x i ′,a i >Finally, the edit triplets at all positions form the edit sequence e of the production code method change. For a single edit e i = <x i ,x i ′,a′ i > Connect the three parts together to indicate editing i , by comparing the token x at position i i and x i 'Get a i , then, input x,x i ,x i ′, t are connected into an input I to predict the probability distribution in the target. For the recognition task, the target l = {0, 1} indicates that the test code does not need to be updated or needs to be updated. For the update task, the target is the updated test case in the positive sample. The model aims to generate new test cases t′ as accurately as possible.

[0042] Furthermore, the S6 comprises the following specific steps:

[0043] The model training phase includes two phases, namely, the identification of outdated tests and the updating of outdated tests. In the identification phase of outdated tests, it is necessary to build a dataset containing positive and negative samples, and train a neural network classifier to identify outdated tests that need to be updated. In the updating phase of outdated tests, only positive samples whose test cases need to be updated in the identification task are used to build a dataset, and a Transformer updater model is trained to generate new test cases.

[0044] (2) Obsolete test identification

[0045] In the outdated test identification phase, NPTCC proposed an outdated test identifier to identify whether the test case needs to be updated. It includes an encoder and a classifier:

[0046] 1) Encoder

[0047] At this stage, the encoder is responsible for obtaining the context representation embedding vector of the input sequence. NPTCC uses the pre-trained CodeT5 encoder to initialize the input I. CodeT5 is a pre-trained model based on Transformer. The Transformer encoder converts the input I into a context vector representation. For each input token I i , Transformer generates three embedding vectors for it, namely the query vector q i , key vector k j Sum value vector v i , the Transformer encoder uses dot products by using the query vector q for each token in the input i and the key vector k j To calculate I i The attention score is as follows:

[0048]

[0049] where d is q i and k j The dimension of attention score α i,j It indicates the degree of attention paid to the jth input when encoding the i-th input, and then the normalized score is obtained through the softmax function:

[0050]

[0051] To learn related / unrelated tokens, each value vector is multiplied using softmax and then these vectors are summed up:

[0052]

[0053] Use the last hidden state z |I| The context vector representation of input I

[0054] 2) Classifier

[0055] Based on the learned context vector representation, binary classification is performed to better capture the four input information (I i ={x,x′,e,t}), use a dense layer with a nonlinear function to learn the potential interactions between the four input information, and then use the output of the dense layer to predict the possibility of the final label I = {0,1}. The classifier is defined as follows:

[0056]

[0057] P(l|<x,x′,e,t> )=Softmax(f)

[0058] W and b represent the weight matrix and bias respectively, tanh is the activation function of the dense layer perceptron, and the softmax function will output the final probability of label l, whose value is between 0 and 1. For the probability score, the score is higher when the original test needs to be updated, and lower when the test does not need to be updated; (2) Outdated test update At this stage, NPTCC proposed an updater for outdated test cases to update outdated test cases that need to be updated. It also consists of two parts, namely the encoder and the decoder:

[0059] 1) Encoder

[0060] The encoder uses the same encoder as the recognizer for outdated tests. The input to the updater still consists of four parts, namely I = {x, x′, e, t}. Since different parts of the input I affect different parts of the generated test, the correlation between each token of the input I and the target updated test t′ should be learned. Therefore, unlike the recognition task, the attention score z = {z1,…,z |I|}, instead of the last hidden state as the input to the decoder;

[0061] 2) Decoder

[0062] Different from the recognition task, where outdated test updating should generate new test cases based on the input, the decoder learns to generate the corresponding new test t′ token by token based on the input and all preceding tokens generated so far. Mathematically, the test updating task is defined as finding Make Where P θ (t′|I u )for:

[0063]

[0064] P θ (t′|I u ) is regarded as the conditional log-likelihood of a new test case t′ predicted given an input I. The model is trained by minimizing the negative log-likelihood between the predicted test case and the ground truth. The decoder architecture consists of two parts, namely the self-attention layer and the encoder-decoder attention layer. The calculation of the self-attention layer is similar to that of the encoder, except that it only processes the words generated so far. The encoder-decoder attention layer learns the association between the output sequence and the input sequence. When calculating the attention score between the encoder and the decoder, the key vector K comes from the encoder output z=(z1,…,z |I| ), target token y j and source code tokens w1,…,w m The attention distribution between j yes:

[0065]

[0066] Specifically, both the recognizer and updater are initialized by the pre-trained Transformer model and fine-tuned on the two tasks respectively.

[0067] Furthermore, the S7 includes the following specific steps:

[0068] The two models trained by S6 are combined together to keep the production code and test code identified and updated. When developers make changes to the production code, this method can use these changes and determine whether the test case should be updated. If the test is identified as outdated, a new test case will be generated to test the new production code. Update test case generation includes two steps to solve these problems: code abstraction recovery and syntax checking. Code abstraction recovery refills the original tokens based on the relevant mapping information such as related variables, constants, and strings stored in the dictionary before; syntax checking performs legality checks on the generated test cases through static detection and dynamic testing. The corresponding code abstraction recovery and syntax checking are performed through the original outputs of these models. Finally, updated test cases without syntax errors are generated and provided to developers.

[0069] The software test code update method based on neural machine translation of the present invention has the following advantages: the production code and test code co-evolution method based on neural machine translation proposed by the present invention realizes automatic test code update through high-quality data processing, context abstraction and semantic modeling. It significantly improves the test efficiency in the software development process and reduces maintenance costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 It is a schematic diagram of the framework of the software test code updating method based on neural machine translation of the present invention;

[0071] Figure 2 A simple example schematic diagram of the editing sequence of the present invention. DETAILED DESCRIPTION

[0072] In order to better understand the purpose, structure and function of the present invention, the following is a further detailed description of a software test code updating method based on neural machine translation of the present invention in conjunction with the accompanying drawings.

[0073] like Figure 1 As shown, a method for updating software test code based on neural machine translation of the present invention comprises the following steps:

[0074] S1: Obtain 5163K method-level changes in Java projects from Github to build a dataset;

[0075] 5163K method-level changes include non-test methods and unit tests. We collect the co-evolved Java methods and their corresponding test cases annotated with @Test, i.e., a quaternion:

[0076] <original method,updated method,original test,updated test> , and build a data set;

[0077] In the modification of production code and test code, the most common type combination is EE (Edit-Edit), which means that the developer updates the existing production class and its corresponding test class at the same time. Since EE type modifications account for the majority of all modification type combinations, only EE type modifications are selected as the research object in this method sample.

[0078] S2: Filter and split the data set obtained in S1;

[0079] We extract the production-test code change pair samples at the method level from the historical data set obtained from S1. For the change pair samples, we use regular expressions to match the useless lines in the code. Through relevant research, we divide the changes in the production code into 12 types, which belong to two categories, namely, the modification of programming language structure and the modification of natural language structure. The former includes the modification of package-id, import-stmt, class-dec, method-sig, method-body, field, and annotation, and the latter includes the modification of copyright, javadoc, and comment. Among them, the modification of package-id, import-stmt, and natural language structure (such as comments, copyright, etc.) is actually the modification of useless lines for the final goal, because these modifications are actually just corresponding replacement operations in the test code. So we use the following regular expression:

[0080] comment = re.compile(r' [] *(()|()|()|import|package)′)

[0081] comment_mod=re.compile(r′^[+-][]\*)|(VV\*)|(VV)|import|package)′)

[0082] empty_row=re.compile(r′^[]*$′)

[0083] empty_row_mod=re.compile(r′^[+-][]*$′)

[0084] The processed code is then formatted, with the source code for production and testing represented as individual characters separated by a space. In addition, the spaces at the beginning and end of the source code need to be removed, and the extra spaces in the middle of the code are deleted, leaving only one space. This can improve the model's ability to learn sample features.

[0085] S3: Apply complexity constraints to the dataset and filter out overly complex or overly simple codes based on the code complexity index to ensure that the model can learn representative and general features more efficiently.

[0086] In order to further improve the representativeness and generalization of the model learning samples, the complexity constraints are imposed on the data here. Because overly complex code may increase the difficulty of model learning, and overly simple code may not provide enough diversity features. Complexity constraints help select moderate code samples and balance the learning ability of the model. This method calculates the nesting depth and number of scalars of each code sample, and selects samples with moderate indicators based on the obtained indicators.

[0087] (1) Nesting depth: measures the depth of nested structures in code blocks, such as if, for, while, etc.

[0088] Nesting Depth=max(Depth(B i ))

[0089] (2) Number of variables: Count the number of variables declared in the code.

[0090]

[0091] S4: Label the samples in the dataset to provide clear positive and negative sample labels for the model;

[0092] Since the quality of the dataset is critical to the deep learning model, the data needs to be classified. For the recognition task, this experiment needs to construct a dataset containing positive and negative samples. If a test case should be updated with the change of its corresponding production code, this article marks it as a positive sample. Otherwise, this article marks it as a negative sample. For the test update task, only the positive samples whose test cases need to be updated in the recognition task are used to construct the dataset. The details of the positive / negative sample construction and test update dataset are as follows:

[0093] (1) Positive sample construction. First, extract the positive samples with

[0094] @test annotation (i.e., t and t′). Secondly, Tree-Sitter is used to extract the names of the test cases. Third, name matching is performed to find the production code. If the production code is in the same project and has also been changed in the same commit, it is considered a positive sample. In the end, this experiment obtained 4,397 positive samples, each of which consists of four parts, namely (original method, updated method, original test, updated test).

[0095] (2) Negative sample construction. In order to construct negative samples, the present invention should extract the modified methods with test cases that do not need to be updated. First, exclude the methods in the positive sample and extract the test cases of the remaining modified methods. For each candidate modified method, search for its test cases according to the following steps:

[0096] Path matching: According to the best practices of Junit testing, the production code and its corresponding test cases are located in the mirror folder. Therefore, the present invention uses path matching to heuristically find the test class, obtain the file name where the method is located, and determine the file path of the test case. For example,

[0097] The test file for / src / main / java / Connect.java is usually located in / src / test / java / ConnectTest.java or / src / test / java / TestConnect.java.

[0098] Name matching: After path matching, the corresponding tests are extracted for the candidate methods. Similar to the process of constructing positive samples, the present invention matches the production code and the test code through name matching. Then, the present invention excludes samples with changes in test cases, and obtains 13,269 negative samples, each of which consists of three parts, namely (original method, updated method, original test).

[0099] S5: Use context abstraction, editing serialization, word segmentation and encoding, and sequence alignment methods to transform the code into a form suitable for model processing;

[0100] First, the code is abstracted. There are some limitations in the training process of the neural machine translation model. Since the Transformer model outputs the probability distribution of tokens, the model calculation becomes quite slow when there are multiple possible tokens. It is necessary to impose artificial restrictions when processing the model to determine the number of the most common words, the so-called vocabulary size. In order to reduce the vocabulary, the context must be abstracted while retaining the semantic information of the source code. For example, the variable name is replaced with "Var_1",..., "Var_n", and each constant and string is also replaced with "Num_1",..., "Num_n" and "Str". The reasons for doing this are as follows: 1) Reduce the vocabulary and the frequency of occurrence of specific tokens; 2) Reduce data redundancy and improve data consistency. NPTCC will maintain a dictionary to store the mapping relationship between the original tokens and the replacement variables so that the original tokens can be refilled after the updated test code is generated. Through the above code abstraction process and controlling the size of the vocabulary, the Transformer model can focus on learning common patterns from different code changes.

[0101] Fine-grained production code changes are then better represented by edit serialization. Code edits are represented as: Medium i and x i′ are the tokens at position i of the original version and the new version production method respectively. i Indicates that x i Convert to x i ′. There are four types of edit operations, namely insertion, deletion, equality, and substitution. To obtain the code change representation, this method first splits the original method and the modified method into code token sequences and uses them to construct the edit sequence. Then the token-level alignment is calculated to obtain the edit triplet at position i. <x i ,x i ′,a i >Finally, the edit triples at all positions form the edit sequence e of the production code method change. For a single edit e i = <x i ,x i ′,a′ i > Connect the three parts together to indicate editing i By comparing the token x at position i i and x i 'Get a i Then, input x,x i ,x i ′,t are connected into an input I to predict the probability distribution in the target. For the recognition task, the target l={0,1} indicates that the test code does not need to be updated or needs to be updated, respectively. For the update task, the target is the updated test case in the positive sample. The model aims to generate new test cases t′ and be as accurate as possible.

[0102] Figure 2 Here is a simple example of editing a sequence:

[0103] The inputs to the model include the original production method x, the updated production method x′, and an original test t.

[0104] For each code segment, first split it into a series of tokens, x = {x1,…,x m},x′={x′1,…,x′ n},t={t1,…,t l}where m, n, and l represent the lengths of the original method, updated method, and original test, respectively. Roberta's tokenizer is then used to obtain these tokens. Instead of constructing tokens by separating source code by spaces, it uses Byte Pair Encoding (BPE) to mitigate the Out-of Vocabulary (OoV) problem.

[0105] S6: Use the processed part of the samples to train models, including an outdated test identification model and a test case update model, to train a neural network classifier to identify test cases that need to be updated and a Transformer updater model to generate new test cases.

[0106] The model training phase consists of two phases, namely, the identification of outdated tests and the updating of outdated tests. In the identification phase of outdated tests, a dataset containing positive and negative samples needs to be constructed, and a neural network classifier is trained to identify outdated tests that need to be updated. In the updating phase of outdated tests, only positive samples whose test cases need to be updated in the identification task are used to construct the dataset, and a Transformer updater model is trained to generate new test cases.

[0107] (1) Obsolete test identification

[0108] In the outdated test identification phase, NPTCC proposed an outdated test identifier to identify whether the test case needs to be updated. It includes an encoder and a classifier.

[0109] 1) Encoder In this stage, the encoder is responsible for obtaining the contextual representation embedding vector of the input sequence. In this paper, NPTCC uses the pre-trained CodeT5 encoder to initialize the input I. Since CodeT5 has a strong ability to capture code semantics and convey code semantics from the identifiers assigned by developers, it has achieved encouraging results in code understanding and generation. CodeT5 is a pre-trained model based on Transformer. The encoder of Transformer converts the input I into a contextual vector representation For each input token I i , Transformer generates three embedding vectors for it, namely the query vector q i , key vector k j Sum value vector v i The Transformer encoder uses a dot product to generate a query vector q for each token in the input. i and the key vector k j To calculate I i The attention score is as follows:

[0110]

[0111] where d is q i and k j The dimension of the attention score α i,j It indicates the degree of attention paid to the jth input when encoding the i-th input. Then, the normalized score is obtained through the softmax function:

[0112]

[0113] To learn related / unrelated tokens, each value vector is multiplied using softmax and then these vectors are summed up:

[0114]

[0115] Use the last hidden state z |I| The context vector representation of input I

[0116] 2) Classifier

[0117] This step aims to perform binary classification based on the learned context vector representation. In order to better capture these four input information (I i ={x,x′,e,t}), this method uses a dense layer with a nonlinear function to learn the potential interactions between them. Then, the output of the dense layer is used to predict the possibility of the final label I = {0,1}. More specifically, the classifier is defined as follows:

[0118]

[0119] P(l|<x,x′,e,t> )=Softmax(f)

[0120] W and b represent the weight matrix and bias respectively. Tanh is the activation function of the dense layer perceptron. The softmax function will output the final probability of label l, whose value is between 0 and 1. For the probability score, this paper hopes that the score is high when the original test needs to be updated, and low when the test does not need to be updated.

[0121] (2) Update of outdated tests

[0122] At this stage, NPTCC proposed an updater for outdated test cases, which is used to update outdated test cases that need to be updated. It also consists of two parts, namely the encoder and the decoder.

[0123] 1) Encoder

[0124] The encoder uses the same encoder as the recognizer for outdated tests. The input to the updater still consists of four parts, namely I = {x, x′, e, t}. Since different parts of the input I affect different parts of the generated test, the correlation between each token of the input I and the target updated test t′ should be learned. Therefore, unlike the recognition task, the attention score z = {z1,…,z |I|}, instead of the last hidden state as the input to the decoder.

[0125] 2) Decoder

[0126] Different from the recognition task, outdated test updating should generate new test cases based on the input. The decoder learns to generate the corresponding new test t′ token by token based on the input and all the preceding tokens generated so far. Mathematically speaking, the test updating task is defined as finding Make Where P θ (t′|I u )for:

[0127]

[0128] P θ (t′|I u ) can be viewed as the conditional log-likelihood of a new test case t′ predicted given an input I. The model can be trained by minimizing the negative log-likelihood between the predicted test case and the ground truth. Specifically, the decoder architecture consists of two parts, the self-attention layer and the encoder-decoder attention layer. The self-attention layer is calculated similarly to the encoder, except that it only processes the words generated so far. The encoder-decoder attention layer learns the association between the output sequence and the input sequence. When calculating the attention score between the encoder and decoder, the key vector K comes from the encoder output z=(z1,…,z |I| ). Target token y j and source code tokens w1,…,w m The attention distribution between j yes:

[0129]

[0130] Specifically, both the recognizer and updater are initialized by the pre-trained Transformer model and fine-tuned on the two tasks respectively.

[0131] S7: predict the updated test code through the neural network model and output the generated modified test code sequence.

[0132] The two models trained by S6 are combined to keep production code and test code identified and updated. When developers make changes to production code, this method can use these changes and determine whether the test case should be updated. If the test is identified as outdated, a new test case will be generated to test the new production code. However, before generating the final updated test case, the Transformer updater model generates multiple candidate updated test outputs through beam search. The original output of the Transformer updater model is far from the test case version that can be successfully compiled because these outputs are processed by S5 and S6, that is, they contain code abstraction and tokenization, and may even contain syntax errors. Therefore, the update test case generation of this method includes two steps to solve these problems: code abstraction recovery and syntax checking. Code abstraction recovery refills the original tokens based on the relevant mapping information such as related variables, constants, and strings stored in the previous dictionary; syntax checking performs legality detection on the generated test cases through static detection (FindBugs) and dynamic testing (compiler and related plug-ins). The original outputs of these models are used to perform corresponding code abstraction recovery and syntax checking, and finally updated test cases without syntax errors are generated and provided to developers.

[0133] It is to be understood that the present invention is described by some embodiments, and it is known to those skilled in the art that various changes or equivalent substitutions may be made to these features and embodiments without departing from the spirit and scope of the present invention. In addition, under the teachings of the present invention, these features and embodiments may be modified to adapt to specific circumstances and materials without departing from the spirit and scope of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the scope of protection of the present invention.

Claims

1. A method for updating software test code based on neural machine translation, characterized in that: The steps include: S1: Obtain 5163K method-level changes in Java projects from Github to build a dataset; S2: Filter and split the data set obtained in S1; S3: Apply complexity constraints to the dataset and filter out overly complex or overly simple codes based on the code complexity index to ensure that the model can learn representative and general features more efficiently. S4: Label the samples in the dataset to provide clear positive and negative sample labels for the model; S5: Use context abstraction, editing serialization, word segmentation and encoding, and sequence alignment methods to transform the code into a form suitable for model processing; S6: Use the processed part of the samples to train models, including an outdated test identification model and a test case update model, to train a neural network classifier to identify test cases that need to be updated and a Transformer updater model to generate new test cases. S7: predict the updated test code through the neural network model and output the generated modified test code sequence.

2. The method for updating software test code based on neural machine translation according to claim 1, characterized in that: The 5163K method-level changes in S1 include non-test methods and unit tests, and the collected co-evolved Java methods and their corresponding test cases annotated with @Test are a quadruple: <original method,updated method,original test,updated test> , and build a data set; Among the modifications of production code and test code, only EE type modifications are selected as research objects.

3. The method for updating software test code based on neural machine translation according to claim 1, characterized in that: The S2 comprises the following specific steps: We extract method-level production-test code change pair samples from the historical data set obtained from S1. We use regular expressions to match useless lines in the code and classify production code changes into 12 types, which belong to two categories: programming language structure modifications and natural language structure modifications. The former includes package-id, import-stmt, class-dec, method-sig, method-body, field, and annotation modifications, and the latter includes The modifications of copyright, javadoc, and comment, including the modifications of package-id, import-stmt, and natural language structure, are actually useless line modifications for the final goal, because these modifications are actually just corresponding replacement operations in the test code, so the following regular expression is used: comment=re.compile(r' [] *(0|0|0|import|package)') comment_mod=re.compile(r'^[+-][]\*)|(V\*)|(VV)|import|package)′) empty_row=re.compile(r′^[]*$') empty_row_mod=re.compile(r'[+-][]*$′) The processed code is then formatted, with the production and test source codes represented as individual characters separated by a space; the spaces at the beginning and end of the source code are removed, and the extra spaces in the middle of the code are deleted, leaving only one space.

4. The method for updating software test code based on neural machine translation according to claim 1, characterized in that: The S3 comprises the following specific steps: For each code sample, calculate the sample's nesting depth and the number of scalars, and select samples with moderate indicators based on the obtained indicators; (1) Nesting depth: measures the depth of nested structures in a code block: Nesting Depth=max(Depth(B i )) (2) Number of variables: Count the number of variables declared in the code:

5. The method for updating software test code based on neural machine translation according to claim 1, characterized in that: The S4 comprises the following specific steps: Construct a dataset containing positive and negative samples. If a test case should be updated as its corresponding production code changes, mark it as a positive sample; otherwise, mark it as a negative sample. For the test update task, only the positive samples whose test cases need to be updated in the recognition task are used to construct the dataset. The details of the positive / negative sample construction and test update dataset are as follows: (1) Positive sample construction: First, the changed test cases with @test annotations, i.e., t and t′, are extracted from the collected method-level code changes. Second, the names of the test cases are extracted using Tree-Sitter; Third, name matching is performed to find the production code. If the production code is in the same project and has also been changed in the same commit, it is considered a positive sample. Finally, 4,397 positive samples are obtained, each of which consists of four parts, namely the original method, the updated method, the original test, and the updated test. (2) Negative sample construction: To construct negative samples, we extract the changed methods with test cases that do not need to be updated. First, we exclude the methods in the positive samples and extract the test cases of the remaining changed methods. For each candidate changed method, we search for its test cases as follows: Path matching: According to the best practices of Junit testing, the production code and its corresponding test cases are located in the mirror folder. Through path matching, the test class is heuristically found, the file name of the method is obtained, and the file path of the test case is determined; Name matching: After path matching, we extract corresponding tests for candidate methods. Similar to the process of building positive samples, we match production code and test code through name matching. Then, we exclude samples whose test cases have changed. We get 13,269 negative samples, each of which consists of three parts: the original method, the updated method, and the original test.

6. The method for updating software test code based on neural machine translation according to claim 1, characterized in that: The S5 comprises the following specific steps: First, the code is abstracted and artificial restrictions are imposed when processing the model to determine the number of the most common words, that is, the vocabulary size. In order to reduce the vocabulary, the context must be abstracted while retaining the semantic information of the source code. NPTCC will maintain a dictionary to store the mapping relationship between the original tokens and the replacement variables so that the original tokens can be backfilled after the updated test code is generated. Through the above code abstraction process and controlling the size of vocabulary, the Transformer model focuses on learning common patterns from different code changes; Fine-grained production code changes are then better represented by edit serialization, with code edits represented as: Medium i and x′ i are the tokens at position i of the original version and the new version production method, respectively. i Indicates that x i Convert to x′ i There are four types of edit operations, namely insertion, deletion, equality, and replacement. To obtain the code change representation, the original method and the modified method are first split into code token sequences and used to construct the edit sequence; then the token-level alignment is calculated to obtain the edit triplet at position i. <x i ,x′ i ,a i >Finally, the edit triplets at all positions form the edit sequence e of the production code method change. For a single edit e i = <x i ,x′ i ,a′ i > Connect the three parts together to indicate editing i , by comparing the token x at position i i and x′ i Get a i , then, input x,x i ,x′ i ,t is connected into an input I to predict the probability distribution in the target. For the recognition task, the target l = {0,1} indicates that the test code does not need to be updated or needs to be updated, respectively. For the update task, the target is the updated test case in the positive sample. The model aims to generate new test cases t′ as accurately as possible.

7. The method for updating software test code based on neural machine translation according to claim 1, characterized in that: The S6 comprises the following specific steps: The model training phase includes two phases, namely, the identification of outdated tests and the updating of outdated tests. In the identification phase of outdated tests, it is necessary to build a dataset containing positive and negative samples, and train a neural network classifier to identify outdated tests that need to be updated. In the updating phase of outdated tests, only positive samples whose test cases need to be updated in the identification task are used to build a dataset, and a Transformer updater model is trained to generate new test cases. (1) Obsolete test identification In the outdated test identification phase, NPTCC proposed an outdated test identifier to identify whether the test case needs to be updated. It includes an encoder and a classifier: 1) Encoder At this stage, the encoder is responsible for obtaining the context representation embedding vector of the input sequence. NPTCC uses the pre-trained CodeT5 encoder to initialize the input I. CodeT5 is a pre-trained model based on Transformer. The Transformer encoder converts the input I into a context vector representation. For each input token I i , Transformer generates three embedding vectors for it, namely the query vector q i , key vector k j Sum value vector v i , the Transformer encoder uses dot products by using the query vector q for each token in the input i and the key vector k j To calculate I i The attention score is as follows: where d is q i and k j The dimension of attention score α i,j It indicates the degree of attention paid to the jth input when encoding the i-th input, and then the normalized score is obtained through the softmax function: To learn related / unrelated tokens, each value vector is multiplied using softmax and then these vectors are summed up: Use the last hidden state z |I| The context vector representation of input I 2) Classifier Based on the learned context vector representation, binary classification is performed to better capture the four input information (I i ={x,x′,e,t}), use a dense layer with a nonlinear function to learn the potential interactions between the four input information, and then use the output of the dense layer to predict the possibility of the final label I = {0,1}. The classifier is defined as follows: P(l|<x,x′,e,t> )=Softmax(f) W and b represent the weight matrix and bias respectively, tanh is the activation function of the dense layer perceptron, and the softmax function will output the final probability of label l, whose value is between 0 and 1. For the probability score, the score is higher when the original test needs to be updated, and lower when the test does not need to be updated; (2) Outdated test update At this stage, NPTCC proposed an updater for outdated test cases, which is used to update outdated test cases that need to be updated. It also consists of two parts, namely the encoder and the decoder: 1) Encoder The encoder uses the same encoder as the recognizer for outdated tests. The input to the updater still consists of four parts, namely I = {x, x′, e, t}. Since different parts of the input I affect different parts of the generated test, the correlation between each token of the input I and the target updated test t′ should be learned. Therefore, unlike the recognition task, the attention score z = {z1,…,z |I| }, instead of the last hidden state as the input to the decoder; 2) Decoder Different from the recognition task, where outdated test updating should generate new test cases based on the input, the decoder learns to generate the corresponding new test t′ token by token based on the input and all preceding tokens generated so far. Mathematically, the test updating task is defined as finding Make Where P θ (t′|I u )for: P θ (t′|I u ) is regarded as the conditional log-likelihood of a new test case t′ predicted given an input I. The model is trained by minimizing the negative log-likelihood between the predicted test case and the ground truth. The decoder architecture consists of two parts, namely the self-attention layer and the encoder-decoder attention layer. The calculation of the self-attention layer is similar to that of the encoder, except that it only processes the words generated so far. The encoder-decoder attention layer learns the association between the output sequence and the input sequence. When calculating the attention score between the encoder and the decoder, the key vector K comes from the encoder output z=(z1,…,z |I| ), target token y j and source code tokens w1,…,w m The attention distribution between j yes: Specifically, both the recognizer and updater are initialized by the pre-trained Transformer model and fine-tuned on the two tasks respectively.

8. The method for updating software test code based on neural machine translation according to claim 1, characterized in that: The S7 comprises the following specific steps: Combine the two models trained by S6 to keep the production code and test code identified and updated. When developers make changes to the production code, this method can use these changes and determine whether the test case should be updated. If the test is identified as outdated, a new test case will be generated to test the new production code. The update test case generation includes two steps to solve these problems: code abstraction recovery and syntax checking. Code abstraction recovery re-fills the original tokens based on the relevant mapping information such as related variables, constants, and strings stored in the dictionary before; Syntax checking checks the legality of generated test cases through static detection and dynamic testing, and performs corresponding code abstraction recovery and syntax checking through the original output of these models. Finally, updated test cases without syntax errors are generated and provided to developers.

Images