A Software Test Code Update Method Based on Neural Machine Translation

Abstract

This invention belongs to the field of machine learning technology and discloses a method for updating software test code based on neural machine translation. It involves acquiring large-scale method-level change data from Java projects on GitHub to construct a dataset; processing the dataset by controlling the nesting depth and number of variables to select suitable code samples; performing context abstraction, editing serialization, and word segmentation encoding on the selected data samples; training an obsolete test identification model and a test case update model to identify whether test cases need updating and generate new test cases, respectively; using a Transformer updater model combined with beam search to generate multiple candidate test cases; and generating valid update test cases through code abstraction and restoration and syntax checking. This invention improves the automation level of the co-evolution of production code and test code and is applicable to code maintenance and test optimization tasks in software development.

Description

Technical Field

[0001] This invention belongs to the field of machine learning technology, and in particular relates to a method for updating software test code based on neural machine translation. Background Technology

[0002] As software development becomes increasingly complex, the co-evolution of production code and test code has become a crucial issue in software engineering. Updates to production code often require corresponding updates to test code to ensure the stability and reliability of the software system. However, in practice, test code maintenance often lags behind changes to production code. This can lead to test code failures, insufficient test coverage, and consequently, system defects and potential production risks.

[0003] Traditional software test code maintenance relies heavily on manual operations, including manually checking the impact of production code changes and updating the corresponding test code. This approach is not only time-consuming and labor-intensive but also prone to errors. Although some rule-based test code generation methods have been proposed, these methods depend on manually defined rules and templates, resulting in poor versatility and an inability to adapt to complex code update scenarios. In recent years, with the rapid development of machine learning technology, especially the application of deep learning in natural language processing and program understanding, deep learning-based automatic code generation and update methods have gradually become a research hotspot. For example, neural machine translation (NMT) technology has been widely used in natural language translation, and its end-to-end learning capabilities and excellent performance provide a new solution for software code updates. NMT models can treat changes to production code as the "source language" and updates to test code as the "target language," achieving co-evolution of both through an encoder-decoder architecture.

[0004] Existing machine learning-based code update methods still face the following challenges:

[0005] 1) Insufficient sample data quality: In the dataset used to train the model, there are usually few co-evolutionary samples of production code and test code, and the sample quality is uneven, which affects the generalization ability of the model.

[0006] 2) Long sequence processing problem: The context information of the code is complex and the length varies. Traditional models are prone to performance degradation when processing long code sequences.

[0007] 3) Semantic consistency problem: Code updates need to ensure semantic consistency between production code and test code, but existing methods still have limitations in capturing semantic relationships. Summary of the Invention

[0008] The purpose of this invention is to provide a software test code update method based on neural machine translation to solve the above-mentioned technical problems.

[0009] To address the aforementioned technical problems, the specific technical solution of the software test code update method based on neural machine translation of the present invention is as follows:

[0010] A method for updating software test code based on neural machine translation includes the following steps:

[0011] S1: Obtain the 5163K method-level change components from the Java project on GitHub and build the dataset;

[0012] S2: Perform data filtering and splitting on the dataset obtained in S1;

[0013] S3: Constrain the dataset on complexity, and filter out overly complex or overly simple code based on the code complexity index to ensure that the model can learn representative and generalized features more efficiently.

[0014] S4: Label the samples in the dataset to provide the model with clear positive and negative sample labels;

[0015] S5: Uses context abstraction, editing serialization, word segmentation and encoding, and sequence alignment methods to transform the code into a form suitable for model processing;

[0016] S6: Use the processed partial samples to train the model, including the outdated test identification model and the test case update model, to train a neural network classifier to identify the test cases that need to be updated and a Transformer updater model to generate new test cases.

[0017] S7: Predict the updated test code using a neural network model and output the generated sequence of modified test code.

[0018] Furthermore, the 5163K method-level changes in S1 include non-test methods and unit tests, the collected co-evolved Java methods and their corresponding test cases annotated with @Test, i.e., the quadruple:

[0019] <original method,updated method,original test,updated test> This is how the dataset is constructed;

[0020] In modifying production and test code, only modifications of the EE type are selected as the research object. Further, S2 includes the following specific steps:

[0021] Extract method-level production-test code change pairs from the historical dataset obtained from S1. For these change pairs, use regular expressions to match useless lines in the code. Production code changes are categorized into 12 types, falling into two main categories: modifications to programming language structures and modifications to natural language structures. The former includes modifications to package-id, import-stmt, class-dec, method-sig, method-body, field, and annotation; the latter includes modifications to copyright, javadoc, and comments. Modifications to package-id, import-stmt, and natural language structures are ultimately considered useless lines because these modifications are simply replaced in the test code. Therefore, the following regular expressions are used:

[0022] comment = re.compile(r') [] *(()|()|()|import|package)′)

[0023] comment_mod=re.compile(r′^[+-][]\*)|(VV*)|(VV)|import|package)′)

[0024] empty_row=re.compile(r′^[]*$′)

[0025] empty_row_mod=re.compile(r′^[+-][]*$)

[0026] The processed code is then formatted, representing the production and test source code as individual characters separated by a space; leading and trailing spaces are removed from the source code, and extra spaces in the code are deleted, leaving only one space.

[0027] Furthermore, step S3 includes the following specific steps:

[0028] For each code sample, calculate the nesting depth and scalar number of samples, and select samples with appropriate metrics based on the obtained metrics.

[0029] (1) Nesting depth: measures the depth of nesting structures within a code block.

[0030] Nesting Depth = max(Depth(B i ))

[0031] (2) Number of variables: The number of variables declared in the statistics code:

[0032]

[0033] Furthermore, step S4 includes the following specific steps:

[0034] A dataset containing positive and negative samples is constructed. If a test case should be updated with changes to its corresponding production code, it is marked as a positive sample; otherwise, it is marked as a negative sample. For test update tasks, only positive samples whose test cases need to be updated in the identification task are used to construct the dataset. Details on positive / negative sample construction and test update dataset are as follows:

[0035] (1) Positive Sample Construction: First, extract change test cases with the @test annotation from the collected method-level code changes, i.e., t and t′. Second, use Tree-Sitter to extract the names of the test cases; third, perform name matching to find production code, and if the production code is in the same project and was also changed in the same commit, it is considered a positive sample; finally, 4397 positive samples are obtained, each sample includes four parts, i.e., original method, updated method, original test, and updated test;

[0036] (2) Negative Sample Construction: To construct negative samples, we extract the changed methods with test cases that do not need to be updated. First, we exclude the methods in the positive samples and extract the test cases for the remaining changed methods. For each candidate changed method, we search for its test cases according to the following steps:

[0037] • Path matching: Based on JUnit testing best practices, production code and its corresponding test cases are located in the image folder. The test class is found heuristically by matching the path, the file name of the method is obtained, and the file path of the test case is determined.

[0038] • Name matching: After path matching, corresponding tests are extracted for candidate methods. Similar to the process of building positive samples, production code and test code are matched by name matching. Then, samples where test cases have changed are excluded, resulting in 13,269 negative samples. Each sample includes three parts: the original method, the updated method, and the original test.

[0039] Furthermore, step S5 includes the following specific steps:

[0040] First, the code is abstracted. Artificial constraints are imposed when processing the model to determine the number of most common words, i.e., the vocabulary size. To reduce the vocabulary size, contextual abstraction must be performed while preserving the semantic information of the source code. NPTCC maintains a dictionary to store the mapping between original symbols and replacement variables, so that the original symbols can be refilled after generating updated test code. Through this code abstraction process and controlling the vocabulary size, the Transformer model focuses on learning common patterns from different code variations.

[0041] Then, fine-grained production code changes are better represented by editing the serialization, with the code edits represented as follows: Chinese x i and x i ′ represents the token at position i in the original and new version production methods, respectively, and a i Indicates x i Convert to x i There are four types of editing operations: insertion, deletion, equality, and replacement. To obtain the code change representation, the original method and the modified method are first segmented into code token sequences, and these sequences are used to construct the edit sequence. Then, token-level alignment is calculated to obtain the edit triple at position i. <x i ,x i ′,a i Finally, the edit triples at all positions form the edit sequence e of the production code method change, for a single edit e. i = <x i ,x i ′,a′ i Connecting the three parts together indicates editing e i By comparing the symbol x at position i i and x i ′Get a i Then, input x,x i ,x i The input I is formed by concatenating ' and t to predict the probability distribution in the target. For the recognition task, the target l = {0, 1} represents whether the test code does not need to be updated or needs to be updated, respectively. For the update task, the target is the updated test cases in the positive samples. The model aims to generate new test cases t' as accurately as possible.

[0042] Furthermore, step S6 includes the following specific steps:

[0043] The model training phase consists of two stages: the identification of outdated tests and the updating of outdated tests. In the stage of identifying outdated tests, a dataset containing positive and negative samples needs to be built, and a neural network classifier is trained to identify outdated tests that need to be updated. In the stage of updating outdated tests, only positive samples that need to be updated in the identification task are used to build the dataset, and a Transformer updater model is trained to generate new test cases.

[0044] (2) Outdated test identification

[0045] In the obsolete test identification phase, NPTCC proposes an obsolete test identifier to determine whether test cases need to be updated. It consists of an encoder and a classifier:

[0046] 1) Encoder

[0047] At this stage, the encoder is responsible for obtaining the context representation embedding vector of the input sequence. NPTCC uses the pre-trained CodeT5 encoder to initialize the input I. CodeT5 is a pre-trained model based on Transformer. The Transformer encoder transforms the input I into a context vector representation. For each input token I i The Transformer generates three embedding vectors for it, namely the query vector q. i Key vector k j Sum vector v i The Transformer encoder utilizes dot products by using a query vector q for each token in the input. i and key vector k j To calculate I i The attention score is calculated using the formula shown below:

[0048]

[0049] Where d is q i and k j The dimension of attention score α i,j This represents the level of attention given to the j-th input when encoding the i-th input, and then the normalized score is obtained through the softmax function:

[0050]

[0051] To learn relevant / irrelevant tokens, softmax is used to multiply each value vector, and then these vectors are summed:

[0052]

[0053] Use the last hidden state z |I| The context vector representation of input I

[0054] 2) Classifier

[0055] Binary classification is performed based on the learned context vector representation in order to better capture the four input information (I... i The relationship between the four inputs (x, x′, e, t) is learned using a dense layer with a non-linear function. Then, the output of the dense layer is used to predict the probability of the final label I = {0, 1}. The classifier is defined as follows:

[0056]

[0057] P(l|<x,x′,e,t> = Softmax(f)

[0058] W and b represent the weight matrix and bias respectively. tanh is the activation function of the dense layer perceptron. The softmax function will output the final probability of label l, whose value is between 0 and 1. For the probability score, the score is higher when the original test needs to be updated, and lower when the test does not need to be updated; (2) Update of obsolete test cases. At this stage, NPTCC proposes an updater for obsolete test cases to update obsolete test cases that need to be updated. It also includes two parts, namely encoder and decoder:

[0059] 1) Encoder

[0060] The encoder uses the same encoder as the recognizer used in the outdated test. The updater's input still consists of four parts, i.e., I = {x, x′, e, t}. Since different parts of the input I affect different parts of the generated test, the correlation between each word in the input I and the target updated test t′ should be learned. Therefore, unlike the recognition task, the attention score z = {z1, ..., z} of I is used here. |I|} instead of using the last hidden state as input to the decoder;

[0061] 2) Decoder

[0062] Unlike recognition tasks, stale test updates should generate new test cases based on the input. The decoder learns to generate corresponding new tests t′ word by word based on the input and all the preceding words generated so far. Mathematically, the test update task is defined as finding... Make Where P θ (t′|I u )for:

[0063]

[0064] P θ (t′|I u The model is viewed as the conditional log-likelihood of a new test case t′ predicted given an input I. It is trained by minimizing the negative log-likelihood between the predicted test case and the ground truth. The decoder architecture consists of two parts: a self-attention layer and an encoder-decoder attention layer. The self-attention layer is computed similarly to the encoder, except it only processes the words generated so far. The encoder-decoder attention layer learns the association between the output sequence and the input sequence. When calculating the attention score between the encoder and decoder, the key vector K comes from the encoder's output z = (z1, ..., z...). |I| ), target symbol y j and source code symbols w1,…,w m Attention distribution between a j yes:

[0065]

[0066] Specifically, both the recognizer and the updater are initialized by a pre-trained Transformer model and fine-tuned on the two tasks respectively.

[0067] Furthermore, step S7 includes the following specific steps:

[0068] This method combines two models trained with S6 to maintain the identification and updating of production and test code. When developers make changes to the production code, this method can use these changes to determine whether test cases should be updated. If a test is identified as outdated, a new test case is generated to test the new production code. The generation of updated test cases involves two steps to address these issues: code abstraction and restoration, and syntax checking. Code abstraction and restoration refills the original tokens based on the relevant mapping information such as variables, constants, and strings stored in the dictionary. Syntax checking performs validity checks on the generated test cases through static and dynamic testing. Based on the original output of these models, corresponding code abstraction and restoration and syntax checking are performed to ultimately generate updated test cases without syntax errors for developers.

[0069] The software test code update method based on neural machine translation proposed in this invention has the following advantages: This invention proposes a method for the co-evolution of production code and test code based on neural machine translation. Through high-quality data processing, contextual abstraction, and semantic modeling, it achieves automated test code updates. This significantly improves testing efficiency and reduces maintenance costs in the software development process. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the software test code update method framework based on neural machine translation of the present invention;

[0071] Figure 2 This is a simplified example diagram of the editing sequence of the present invention. Detailed Implementation

[0072] To better understand the purpose, structure, and function of this invention, the following detailed description of a software test code update method based on neural machine translation is provided in conjunction with the accompanying drawings.

[0073] like Figure 1 As shown, a software test code update method based on neural machine translation according to the present invention includes the following steps:

[0074] S1: Obtain the 5163K method-level change components from the Java project on GitHub and build the dataset;

[0075] The 5163K method-level changes include non-test methods and unit tests. We have collected the co-evolved Java methods and their corresponding test cases annotated with `@Test`, i.e., quadruplets:

[0076] <original method,updated method,original test,updated test> This is how the dataset is constructed;

[0077] In modifications to production and test code, the most common type combination is EE (Edit-Edit), indicating that developers simultaneously updated both the existing production class and its corresponding test class. Since EE type modifications account for the majority of all modification type combinations, this method sample only selects EE type modifications as the research object.

[0078] S2: Perform data filtering and splitting on the dataset obtained in S1;

[0079] Extracting method-level production-test code change pairs from the historical dataset obtained from S1, regular expressions are used to match useless lines in the code for these change pairs. Based on relevant research, production code changes are categorized into 12 types, falling into two main categories: modifications to programming language structures and modifications to natural language structures. The former includes modifications to package-id, import-stmt, class-dec, method-sig, method-body, field, and annotation; the latter includes modifications to copyright, javadoc, and comments. Modifications to package-id, import-stmt, and natural language structures (e.g., comments, copyright) are essentially useless lines in the final target, as these modifications are simply replaced in the actual test code. Therefore, the following regular expression is used:

[0080] comment = re.compile(r') [] *(()|()|()|import|package)′)

[0081] comment_mod=re.compile(r′^[+-][]\*)|(VV\*)|(VV)|import|package)′)

[0082] empty_row=re.compile(r′^[]*$′)

[0083] empty_row_mod=re.compile(r′^[+-][]*$′)

[0084] The processed code is then formatted, representing production and test source code as individual characters separated by spaces. Leading and trailing spaces are removed, and extra spaces within the code are deleted, leaving only one space. This improves the model's ability to learn sample features.

[0085] S3: Constrain the dataset on complexity, and filter out overly complex or overly simple code based on the code complexity index to ensure that the model can learn representative and generalized features more efficiently.

[0086] To further improve the representativeness and generalization of the features learned by the model, a complexity constraint is imposed on the data. Overly complex code may increase the difficulty of model learning, while overly simple code may not provide sufficient diversity features. The complexity constraint helps select appropriate code samples to balance the model's learning ability. This method calculates the nesting depth and scalar number of samples for each code sample, and selects samples with appropriate metrics based on the obtained indicators.

[0087] (1) Nesting depth: measures the depth of nesting structures in code blocks, such as if, for, while, etc.

[0088] Nesting Depth = max(Depth(B i ))

[0089] (2) Number of variables: The number of variables declared in the statistics code.

[0090]

[0091] S4: Label the samples in the dataset to provide the model with clear positive and negative sample labels;

[0092] Since the quality of the dataset is crucial for deep learning models, data classification is necessary. For the recognition task, this experiment requires constructing a dataset containing both positive and negative samples. A test case is labeled as a positive sample if it should be updated with changes to its corresponding production code. Otherwise, it is labeled as a negative sample. For the test update task, only positive samples that require updating their test cases in the recognition task are used to construct the dataset. Details regarding the construction of positive / negative samples and the test update dataset are as follows:

[0093] (1) Positive Sample Construction. First, positive samples are extracted from the method-level code changes collected in this invention.

[0094] The test cases (i.e., t and t′) annotated with @test were modified. Next, Tree-Sitter was used to extract the names of the test cases. Third, name matching was performed to find production code. If the production code was in the same project and also changed in the same commit, it was considered a positive sample. Ultimately, this experiment yielded 4397 positive samples, each consisting of four parts: (original method, updated method, original test, updated test).

[0095] (2) Negative Sample Construction. To construct negative samples, this invention should extract modified methods with test cases that do not require updating. First, methods in the positive samples are excluded, and test cases for the remaining modified methods are extracted. For each candidate modified method, its test cases are searched according to the following steps:

[0096] • Path matching: Following JUnit best practices, production code and its corresponding test cases are located in the mirror folder. Therefore, this invention uses path matching to heuristically locate the test class, obtain the filename of the method, and determine the file path of the test case. For example,

[0097] The test file for / src / main / java / Connect.java is usually located in / src / test / java / ConnectTest.java or / src / test / java / TestConnect.java.

[0098] • Name Matching: After path matching, corresponding tests are extracted for candidate methods. Similar to the process of building positive samples, this invention uses name matching to match production code and test code. Then, this invention excludes samples where test cases have changed, resulting in 13,269 negative samples. Each sample consists of three parts: (original method, updated method, original test).

[0099] S5: Uses context abstraction, editing serialization, word segmentation and encoding, and sequence alignment methods to transform the code into a form suitable for model processing;

[0100] First, code abstraction is crucial in the training of neural machine translation models. Since the Transformer model outputs the probability distribution of words, computation becomes quite slow when multiple possible words exist. Therefore, artificial constraints need to be imposed during model processing to determine the number of most common words, known as the vocabulary size. To reduce the vocabulary size, contextual abstraction must be performed while preserving the semantic information of the source code. For example, variable names are replaced with "Var_1", ..., "Var_n", and each constant and string is replaced with "Num_1", ​​..., "Num_n" and "Str". This is done for several reasons: 1) reducing the vocabulary size and the frequency of specific words; 2) reducing data redundancy and improving data consistency. NPTCC maintains a dictionary to store the mapping between original words and replacement variables, allowing the original words to be refilled after generating updated test code. Through this code abstraction process and by controlling the vocabulary size, the Transformer model can focus on learning common patterns from different code variations.

[0101] Then, fine-grained production code changes are better represented by editing the serialization. Code edits are represented as: Chinese x i and x i′ represents the token at position i in the original and new version production methods, respectively. a i Indicates x i Convert to x i The method performs four types of editing operations: insertion, deletion, equality, and replacement. To obtain a representation of the code changes, this method first segments the original and modified methods into sequences of code tokens and uses them to construct the edit sequence. Then, token-level alignment is calculated to obtain the edit triplet at position i. <x i ,x i ′,a i Finally, the edit triples at all positions form the edit sequence e for this production code method change. For a single edit e... i = <x i ,x i ′,a′ i Connecting the three parts together indicates editing e i By comparing the symbol x at position i. i and x i ′Get a i Then, input x,x i ,x i The values ​​t' and t' are concatenated into an input I to predict the probability distribution in the target. For the recognition task, the target l = {0, 1} represents whether the test code does not need to be updated or needs to be updated, respectively. For the update task, the target is the updated test cases in the positive samples. The model aims to generate new test cases t' as accurately as possible.

[0102] Figure 2 Here is a simple example of editing a sequence:

[0103] The model's inputs include the original production method x, the updated production method x′, and an original test t.

[0104] For each code segment, it is first divided into a series of tokens, x = {x1, ..., x...} m},x′={x′1,…,x′ n},t={t1,…,t l} where m, n, and l represent the lengths of the original method, the update method, and the original test, respectively. These tokens are then obtained using Roberta's tokenizer. Unlike constructing tokens by separating source code based on spaces, it utilizes Byte Pair Encoding (BPE) to mitigate the Out-of-Vocabulary (OoV) problem.

[0105] S6: Use the processed partial samples to train the model, including the outdated test identification model and the test case update model, to train a neural network classifier to identify the test cases that need to be updated and a Transformer updater model to generate new test cases.

[0106] The model training phase consists of two stages: outdated test identification and outdated test updating. In the outdated test identification stage, a dataset containing both positive and negative samples is constructed, and a neural network classifier is trained to identify outdated tests that need updating. In the outdated test updating stage, only positive samples that require updating their test cases in the identification task are used to construct the dataset, and a Transformer updater model is trained to generate new test cases.

[0107] (1) Outdated test identification

[0108] In the obsolete test identification phase, NPTCC proposes an obsolete test identifier to determine whether test cases need to be updated. It consists of an encoder and a classifier.

[0109] 1) The encoder, at this stage, is responsible for acquiring the context representation embedding vector of the input sequence. In this paper, NPTCC utilizes a pre-trained CodeT5 encoder to initialize the input I. CodeT5 has achieved encouraging results in code understanding and generation due to its powerful ability to capture code semantics and convey code meaning from identifiers assigned by developers. CodeT5 is a pre-trained Transformer-based model. The Transformer encoder transforms the input I into a context vector representation. For each input token I i The Transformer generates three embedding vectors for it, namely the query vector q. i Key vector k j Sum vector v i The Transformer encoder utilizes the dot product, by using the query vector q for each token in the input. i and key vector k j To calculate I i The attention score is calculated using the formula shown below:

[0110]

[0111] Where d is q i and k j The dimension of attention score α. i,j This represents the level of attention given to the j-th input when encoding the i-th input. Then, the normalized score is obtained using the softmax function.

[0112]

[0113] To learn relevant / irrelevant tokens, softmax is used to multiply each value vector, and then these vectors are summed:

[0114]

[0115] Use the last hidden state z |I| The context vector representation of input I

[0116] 2) Classifier

[0117] This step aims to perform binary classification based on the learned context vector representation. This is to better capture the four input information (I... i Given the relationships between x, x′, e, and t, this method uses a dense layer with a non-linear function to learn the potential interactions between them. Then, the output of the dense layer is used to predict the probability of the final label I = {0, 1}. More specifically, the classifier is defined as follows:

[0118]

[0119] P(l|<x,x′,e,t> = Softmax(f)

[0120] W and b represent the weight matrix and bias, respectively. tanh is the activation function of the dense perceptron. The softmax function outputs the final probability of label l, with a value between 0 and 1. For the probability score, this paper aims for a higher score when the original test requires an update, and a lower score when the test does not require an update.

[0121] (2) Deprecated test updates

[0122] At this stage, NPTCC proposed an updater for obsolete test cases, used to update obsolete test cases that need updating. It also consists of two parts: an encoder and a decoder.

[0123] 1) Encoder

[0124] The encoder uses the same encoder as the recognizer used in the outdated test. The updater's input still consists of four parts, namely I = {x, x′, e, t}. Since different parts of the input I affect different parts of the generated test, the correlation between each word in the input I and the target updated test t′ should be learned. Therefore, unlike the recognition task, the attention score z = {z1, ..., z} of I is used here. |I| Instead of using the last hidden state as input to the decoder, we use}.

[0125] 2) Decoder

[0126] Unlike recognition tasks, stale test updates should generate new test cases based on the input. The decoder learns to generate new test cases t′ token by token, based on the input and all leading words generated so far. Mathematically, the test update task is defined as finding... Make Where P θ (t′|I u )for:

[0127]

[0128] P θ (t′|I u The conditional log-likelihood of a new test case t′ predicted given an input I can be viewed as the conditional log-likelihood of that predicted test case t′. The model can be trained by minimizing the negative log-likelihood between the predicted test case and the ground truth. Specifically, the decoder architecture consists of two parts: a self-attention layer and an encoder-decoder attention layer. The computation of the self-attention layer is similar to that of the encoder, except that it only processes the words generated so far. The encoder-decoder attention layer learns the association between the output sequence and the input sequence. When calculating the attention score between the encoder and decoder, the key vector K comes from the encoder's output z = (z1,…,z...). |I| ). Target symbol y j and source code symbols w1,…,w m Attention distribution between a j yes:

[0129]

[0130] Specifically, both the recognizer and the updater are initialized by a pre-trained Transformer model and fine-tuned on the two tasks respectively.

[0131] S7: Predict the updated test code using a neural network model and output the generated sequence of modified test code.

[0132] This method combines two models trained in S6 to maintain the identification and updating of production and test code. When developers make changes to the production code, this method uses these changes to determine whether test cases should be updated. If a test is identified as obsolete, a new test case is generated to test the new production code. However, before generating the final updated test cases, the Transformer updater model generates multiple candidate updated test outputs through beam search. The original outputs of the Transformer updater model differ significantly from the version of test cases that can be successfully compiled because these outputs have undergone S5 and S6 processing, i.e., code abstraction and lexicalization, and may even contain syntax errors. Therefore, the updated test case generation of this method includes two steps to address these issues: code abstraction restoration and syntax checking. Code abstraction restoration refills the original tokens based on the relevant mapping information such as variables, constants, and strings stored in the dictionary; syntax checking performs validity checks on the generated test cases through static detection (FindBugs) and dynamic testing (compiler and related plugins). By performing code abstraction and restoration and syntax checking on the raw output of these models, updated test cases without syntax errors can be generated and provided to developers.

[0133] It is understood that the present invention has been described through some embodiments, and those skilled in the art will recognize that various changes or equivalent substitutions can be made to these features and embodiments without departing from the spirit and scope of the invention. Furthermore, under the teachings of the present invention, these features and embodiments can be modified to adapt to specific situations and materials without departing from the spirit and scope of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed herein, and all embodiments falling within the scope of the claims of this application are within the protection scope of the present invention.

Claims

1. A method for updating software test code based on neural machine translation, characterized in that, Includes the following steps: S1: Obtain the 5163K method-level change components from the Java project on GitHub and build the dataset; S2: Perform data filtering and splitting on the dataset obtained in S1; S3: Constrain the dataset on complexity and filter out overly complex or overly simple code based on code complexity metrics to ensure that the model learns representative and generalized features more efficiently. S4: Label the samples in the dataset to provide the model with clear positive and negative sample labels; S5: Uses context abstraction, editing serialization, word segmentation and encoding, and sequence alignment methods to transform the code into a form suitable for model processing; First, the code is abstracted, and artificial constraints are imposed when processing the model to determine the size of the vocabulary of the most common words. In order to reduce the vocabulary size, the context must be abstracted while preserving the semantic information of the source code. NPTCC will maintain a dictionary to store the mapping relationship between the original tokens and the replacement variables so that the original tokens can be refilled after generating updated test code. Through the above code abstraction process and controlling the size of the vocabulary, the Transformer model can focus on learning common patterns from different code variations. Then, fine-grained production code changes are better represented by edit serialization, with the code edits represented as follows: ,in and These are the original version and the new version production methods. i Positional symbols, Indicates will Convert to The editing operations include four types: insertion, deletion, equality, and replacement. To obtain a representation of the code changes, the original and modified methods are first segmented into sequences of code tokens, and these sequences are used to construct the edit sequence. Then, token-level alignment is calculated to obtain the positions. i Editing Triples Finally, the edit triples at all positions form the edit sequence for this production code method change. e For a single editor Connecting the three parts together indicates editing. By comparing the token at position i and get Then, input The concatenation of these components forms an input I used to predict the probability distribution within the target. For a recognition task, the target... l={0,1} These represent test code that does not need updating or that needs updating, respectively. For update tasks, the target is the updated test cases in the positive samples. This model aims to generate new test cases. And as accurately as possible; S6: Use the processed partial samples to train the model, including the outdated test identification model and the test case update model, to train a neural network classifier to identify the test cases that need to be updated and a Transformer updater model to generate new test cases. S7: Predict the updated test code using a neural network model and output the generated sequence of modified test code.

2. The software test code update method based on neural machine translation according to claim 1, characterized in that, The 5163K method-level changes in S1 include non-test methods and unit tests. The collected co-evolved Java methods and their corresponding test cases annotated with @Test are represented as a quadruple: < original method, updated method, original test, updated test >, thereby constructing the dataset; In modifying production and test code, only modifications of the EE type are selected as the research object.

3. The software test code update method based on neural machine translation according to claim 1, characterized in that, S2 includes the following specific steps: Extract method-level production-test code change pairs from the historical dataset obtained from S1. For these change pairs, use regular expressions to match useless lines in the code. Production code changes are categorized into 12 types, falling into two main categories: modifications to programming language structures and modifications to natural language structures. The former includes modifications to package-id, import-stmt, class-dec, method-sig, method-body, field, and annotation; the latter includes modifications to copyright, javadoc, and comments. Modifications to package-id, import-stmt, and natural language structures are ultimately considered useless lines because these modifications are simply replaced in the test code. Therefore, the following regular expressions are used: ， The processed code is then formatted, representing the production and test source code as individual characters separated by a space; leading and trailing spaces are removed from the source code, and extra spaces in the code are deleted, leaving only one space.

4. The software test code update method based on neural machine translation according to claim 1, characterized in that, S3 includes the following specific steps: For each code sample, calculate the nesting depth and scalar number of samples, and select samples with appropriate metrics based on the obtained metrics. (1) Nesting depth: measures the depth of nesting structures within a code block. ， (2) Number of variables: The number of variables declared in the statistical code: 。 5. The software test code update method based on neural machine translation according to claim 1, characterized in that, S4 includes the following specific steps: A dataset containing positive and negative samples is constructed. If a test case should be updated with changes to its corresponding production code, it is marked as a positive sample; otherwise, it is marked as a negative sample. For test update tasks, only positive samples whose test cases need to be updated in the identification task are used to construct the dataset. Details on positive / negative sample construction and test update dataset are as follows: (1) Positive Sample Construction: First, extract change test cases with the @test annotation from the collected method-level code changes, for... t and Secondly, Tree-Sitter was used to extract the names of the test cases; thirdly, name matching was performed to find production code, and if the production code was in the same project and had also been changed in the same commit, it was considered a positive sample; finally, 4397 positive samples were obtained, each sample including the original method, the updated method, the original test, and the updated test. (2) Negative Sample Construction: To construct negative samples, extract the changed methods with test cases that do not need to be updated. First, exclude the methods in the positive samples and extract the test cases for the remaining changed methods. For each candidate changed method, search for its test cases according to the following steps: • Path matching: Based on JUnit testing best practices, production code and its corresponding test cases are located in the image folder. The test class is found heuristically by matching the path, the file name of the method is obtained, and the file path of the test case is determined. • Name matching: After path matching, name matching is used to match production code and test code. Then, samples where test cases have changed are excluded, resulting in 13,269 negative samples. Each sample includes the original method, the updated method, and the original test.

6. The software test code update method based on neural machine translation according to claim 1, characterized in that, S6 includes the following specific steps: The model training phase includes two stages: outdated test identification and outdated test updating. In the outdated test identification stage, a dataset containing positive and negative samples needs to be built, and a neural network classifier is trained to identify outdated tests that need to be updated. In the outdated test updating stage, only positive samples that need to be updated in the identification task are used to build the dataset, and a Transformer updater model is trained to generate new test cases. (1) Outdated test identification In the obsolete test identification phase, NPTCC proposes an obsolete test identifier to determine whether test cases need to be updated. It consists of an encoder and a classifier: 1) Encoder At this stage, the encoder is responsible for obtaining the context representation embedding vector of the input sequence. NPTCC uses a pre-trained CodeT5 encoder to initialize the input. I CodeT5 is a pre-trained model based on Transformer. The Transformer encoder takes the input... I Convert to context vector representation , For each input word The Transformer generates query vectors for it. Key vectors Sum value vector The Transformer encoder utilizes dot products by using a query vector for each word in the input. and key vector To calculate The attention score is calculated using the formula shown below: ， in d yes and Dimensions, attention score Indicates when the first i When encoding the input, for the first input... j The level of attention given to each input, and then, through softmax The function obtains the normalized score: ， To learn related / irrelevant symbols, use softmax Multiply each value vector, then add the vectors together: ， Use the last hidden state As input I Context vector representation ; 2) Classifier Binary classification is performed based on the learned context vector representation in order to better capture the four input information ( The algorithm uses a dense layer with a non-linear function to learn the potential interactions between four input information items, and then uses the output of the dense layer to predict the final label. I={0,1} The probability of a classifier is determined as follows: ， ， W and b These represent the weight matrix and the bias, respectively. tanh It is the activation function of the dense layer perceptron. softmax The function will output the label. l The final probability, which has a value between 0 and 1, is higher when the original test needs to be updated and lower when the test does not need to be updated. (2) Deprecated test updates At this stage, NPTCC proposed an updater for obsolete test cases, used to update obsolete test cases that need to be updated. It also includes an encoder and a decoder: 1) Encoder The encoder uses the same encoder as the one used in the obsolete test; the updater's input still consists of four parts, represented as follows: Due to input I Different parts of the input affect different parts of the generated test, and the input should be learned. I Each word and target update test The correlation between them, therefore, unlike the recognition task, is used here. I Attention score Instead of using the last hidden state as the input to the decoder; 2) Decoder Unlike recognition tasks, outdated test updates should generate new test cases based on the input. The decoder learns to generate corresponding new test cases word by word based on the input and all the preceding words generated so far. Mathematically speaking, the test update task is defined as finding , making ,in for: ， Treat as given input I Predicted new test cases The conditional log-likelihood model is trained by minimizing the negative log-likelihood between predicted test cases and ground facts. The decoder architecture consists of two parts: a self-attention layer and an encoder-decoder attention layer. The self-attention layer processes only the words generated so far, while the encoder-decoder attention layer learns the association between the output sequence and the input sequence. Key vectors are used to calculate the attention score between the encoder and decoder. K Output from encoder target token and source code symbols Attention distribution between yes: ， Specifically, both the recognizer and the updater are initialized by a pre-trained Transformer model and fine-tuned on the two tasks respectively.

7. The software test code update method based on neural machine translation according to claim 1, characterized in that, S7 includes the following specific steps: The two models trained in S6 are combined to maintain the identification and updating of production and test code. When developers make changes to the production code, this method uses these changes to determine whether test cases should be updated. If a test is identified as obsolete, a new test case is generated to test the new production code. The test case generation update involves two steps to address these issues: code abstraction restoration and syntax checking. Code abstraction restoration refills the original tokens based on the relevant variables, constants, and string-related mapping information stored in the dictionary. Syntax checking verifies the validity of generated test cases through static and dynamic testing. Based on the raw output of these models, corresponding code abstraction and restoration and syntax checking are performed, ultimately generating updated test cases without syntax errors for developers.

Images