Unit test method for generating discrimination model based on retrieval

Abstract

The invention belongs to the technical field of software engineering, and particularly relates to a unit test method for generating a discrimination model based on retrieval. Searching a to-be-tested function, namely a test case pair, and establishing a code library; constructing an efficient code retriever based on code embedding and mixed feature extraction; constructing a multi-input matching discriminator network, preparing an own data set, and training the network; integrating the code library, the retriever and the discriminator into a local knowledge base, and performing fine adjustment on the large model based on the local knowledge base to obtain a retrieval generation discrimination model; and generating a test case based on the retrieval generation discrimination model, and testing. According to the method, the discrimination model is generated by using retrieval, corresponding change is automatically carried out according to the previous test case, and the test with higher coverage rate and quicker speed can be realized.

Description

Technical Field

[0001] This invention belongs to the field of software engineering technology, and particularly relates to software testing, unit testing, deep learning, artificial intelligence, and other technical fields. Specifically, it relates to a unit testing method based on a retrieval-generated discriminant model. Background Technology

[0002] Unit testing primarily refers to the inspection and verification of the smallest testable unit in software. Through unit testing, developers can identify defects and problems in the code early in the software development process, thereby effectively ensuring the correctness of the program. Unit testing is often performed by testers; however, the process of writing unit test cases requires testers to first understand the function's functionality and branches, as well as boundary conditions and possible abnormal situations, which is time-consuming and labor-intensive, increasing the workload of testers.

[0003] While automated testing frameworks have brought convenience to software testing, current software testing processes still face several challenges. The results of automated unit tests are heavily influenced by the unit under test and the testing environment. Changes in either require adjustments to test cases; otherwise, test failures or false positives may occur. Existing methods lack the ability to adapt to subtle changes. Furthermore, for systems reliant on external environments or data, automated unit testing may struggle to fully simulate real-world scenarios, thus failing to accurately assess software performance in actual operation.

[0004] In summary, existing automated unit testing processes have several problems in test case design and test script writing. These problems not only affect the quality of software testing but also increase the workload of testers. Therefore, how to optimize the unit testing process, reduce the workload of testers, and improve the efficiency and quality of software testing has become an urgent problem to be solved in the current software development field. To address these problems, this invention, building upon previous research, proposes a method for generating and testing test cases using a retrieval-based discriminative model. Summary of the Invention

[0005] The purpose of this invention is to address the problems of time-consuming and labor-intensive unit test case writing, low test case coverage, lack of systematic and standardized automated testing methods, and generation illusion of large models in code tasks. This invention proposes a unit testing method based on generating large models and matching networks, which can generate and correct unit tests through a retrieval generation discrimination mechanism.

[0006] The technical solution of this invention is: A unit testing method based on a retrieval-generated discriminant model is proposed. This method modifies and generates unit tests through a discriminant generation mechanism. It consists of three phases: retrieval, training, and testing. In the retrieval phase, a codebase and retrieval mechanism are established. The codebase contains a large number of completed test functions—test case pairs. The retrieval mechanism finds the most similar test functions—test case pairs—based on the input test function. In the training phase, the retrieval engine and multi-input matching discriminator are trained using a proprietary dataset. Retrieval keys are generated, and the discriminator is given matching and difference analysis capabilities. The trained retrieval keys and discriminator are then added to a larger model for fine-tuning, constructing the retrieval-generated discriminant model. In the testing phase, the test function is input into the retrieval-generated discriminant model, and test cases are output for unit testing.

[0007] Furthermore, the specific steps are as follows: Step 1: Establish a code repository and retrieval mechanism; Step 1.1: Collect open-source code datasets and combine them with private datasets to form a codebase.

[0008] Step 1.2: Embed the functions to be tested from the codebase into vectors using vector embedding methods. This involves embedding the functions to be tested from the dataset... Perform vector embedding, excluding features obtained by the embedding method. In addition, various features of the code are extracted, including syntactic features, semantic features, and textual features. Specifically, syntactic features are determined by the abstract syntax tree, assuming a code snippet... The abstract syntax tree is Its node set is The edge set is Each node There is a type tag type( v ), forming the node type feature vector:

[0009] semantic features Extracted using the BERT method. Text features include variable and function set features, as well as code comment features. For variable and function set features, the Word2Vec method is used to map variable names to a vector space.

[0010] in It is an embedding vector of variable and function names.

[0011] Code comment features The annotations are vectorized using the BERT model.

[0012] Constructing a hybrid feature vector:

[0013] in , , , , Here are the weight parameters, and their sum is 1.

[0014] Step 1.3: Construct an efficient code retrieval tool based on code embedding and hybrid feature extraction.

[0015] By comprehensively considering various characteristics of the code and combining inverted index retrieval and vector retrieval, functions that are similar to the input function can be quickly found in a large codebase.

[0016] For the input function to be tested, after performing steps including vector embedding and multidimensional feature extraction, feature fusion, index construction, coarse search, and fine search, the function that approximates the code library and its corresponding test cases are output.

[0017] Vector embedding, multidimensional feature extraction, and feature fusion are shown in step 1.2.

[0018] Index construction consists of two parts: inverted index construction and vector index construction. Inverted index construction is divided into three steps: tokenization, inverted index initialization, and inverted index population. For each code snippet... Tokenize and extract its token set. :

[0019] in It is a specific token in the code snippet, which is then used to initialize an empty inverted index table:

[0020] Tokens for each code snippet Add to the inverted index:

[0021] Assume there is Each code snippet The set of tokens is The construction of an inverted index is represented as follows:

[0022] Vector index construction consists of three steps: feature vector extraction, vector index initialization, and vector index filling. For each code snippet... Extract its comprehensive feature vector Then initialize an empty vector index:

[0023] Add the feature vector of each code snippet to the vector index:

[0024] Assume there is Each code snippet The feature vector is The construction of vector indices can be represented as:

[0025] Coarse searching is implemented based on an inverted index, which is used to quickly filter out a set of potentially related code snippets. Let the inverted index be: ={"for":{1,2,3},"int":{1,3},"i":{1,2}}. Query code The set of tokens is If the query result is {"for","i"}, then the coarse search result will be: {1,2,3}.

[0026] Fine-grained search is implemented based on vector indexes. Assuming the query code... The mixed feature vector is Vector index The system has been built. The goal of fine-grained search is to refine the candidate set from the coarse search. Filter the set of code snippets that are most similar to the query code. .

[0027]

[0028] in, Represents the largest value The similarity is calculated using cosine similarity.

[0029] This efficient code retrieval tool uses code embedding and hybrid feature extraction to efficiently retrieve code snippets similar to the query code from a large-scale codebase.

[0030] Step 2: Build and train a multi-input matching discriminator; Step 2.1: Construct a multi-input matching discriminator network. A multi-input matching discriminator network is a neural network model that distinguishes and matches multiple pairs of test cases (functions to be tested). The network mainly consists of an encoder module, a matching module, a difference detection module, and a synthesis processing module.

[0031] The encoder module takes code segments and test cases as input and consists of a preprocessing layer, a Transformer encoder, and a pooling layer. The preprocessing layer performs tokenization and removes extra spaces and newlines. The Transformer encoder, composed of six Transformer encoder blocks, performs semantic processing and embedding on the input sequence. The pooling layer uses average pooling, averaging the encoder output sequence over time to obtain a fixed-dimensional vector. The encoder module outputs a semantic vector of the code segment. and the semantic vectors of test cases .

[0032] The input to the matching module is the semantic vector of the code segment output by the encoder module. and the semantic vectors of test cases It consists of a similarity calculation layer and a matching classifier. The similarity calculation layer calculates the code segment vector. and vectors The similarity between code segments is calculated using cosine similarity. The matching classifier determines whether the code segment and test case match based on the similarity value, outputting the matching probability of each test case and the similarity value between the code segments. The matching classifier consists of multiple fully connected layers, and the final output is an activation function. The matching results are evaluated using a combined loss function based on contrastive loss and weighted sum loss.

[0033] The combined loss function, based on contrastive loss and weighted sum loss, is a loss function used to evaluate matching performance in multi-input scenarios. Contrastive loss is applicable to paired inputs and measures whether the similarity between multiple pairs of inputs meets expectations.

[0034] in, and They are the first The code segment and the first The semantic vector of each test case. It is a label, indicating and Whether it matches, 1 means match, 0 means no match. yes and The Euclidean distance between them. It is a marginal value used to control the loss of mismatched pairs.

[0035] Taking into account the similarity factors among different functions under test, and combining weighted sum loss, a loss function suitable for multi-input matching modules is constructed:

[0036]

[0037] in, and These are the number of code segments and the number of test cases, respectively. yes The code segment and the first The weights of each test case pair. By minimizing... The model can learn which code segments match which test cases, and at the same time, it can appropriately penalize those that do not match.

[0038] The input to the difference detection module is the semantic vector of the code segment output by the encoder module. It consists of a cross-attention layer, a difference feature extraction layer, and a difference description generator. The cross-attention layer calculates the cross-attention weights between two code segments to determine their similarities and differences. For each code segment pair... , Calculate the cross-attention weights:

[0039]

[0040] in, It is the first Semantic vectors of code segments It is the dimension of the vector.

[0041] The differential feature extraction layer uses cross-attention weights to extract differential features:

[0042]

[0043] The difference description generator uses an encoder-decoder structure to generate difference description text:

[0044]

[0045]

[0046]

[0047] The integration module combines the outputs from the matching and difference detection modules to provide input for the subsequent generator. The input test function is tagged as follows: For each retrieved code segment pair Construct a discriminator to synthesize the output. :

[0048] in It was obtained from the matching module. and The similarity value, The differences between the two are shown in the following example: =(0.98,"Change variable name from x to y"), =(0.85,"A loop structure has been added"). The discriminator's synthesized output, along with the retrieved test function—the test cases—are used together to generate the generator.

[0049] Step 2.2: Train the network using the dataset to obtain the retrieval key and the trained discriminator.

[0050] The proprietary dataset, derived from local work scenarios, contains 3618 pairs of test cases for C++, 2415 pairs for Java, and 2148 pairs for Python. This dataset is then incorporated into the network for training, enabling automatic learning of model parameters and saving the corresponding search keys.

[0051] Step 3: Fine-tune the retrieval and generation discriminative model; Step 3.1: Integrate the code repository, search keys, and discriminator into a local knowledge base.

[0052] Step 3.2: Fine-tune the large model based on the local knowledge base to obtain the retrieval generation discriminant model.

[0053] Download the pre-trained model and load its weights and word segmenter. Fine-tune the model using the LoRA method, train it using a local knowledge base, and merge the LoRA weights into the original model to obtain a retrieval generation and discriminative model. This model can perform retrieval based on the input function and provide similar test function pairs, automatically generating test cases.

[0054] Step 4: Generate test cases based on the retrieval-generated discriminant model and conduct tests; Step 4.1: Input the function to be tested and perform preprocessing.

[0055] Adjust the format of the function under test according to programming standards to ensure neat code. Rewrite macro definitions in the code to restore their original meaning and eliminate the output illusion caused by large models due to input macro definitions. Tag and categorize the functions under test to speed up retrieval.

[0056] Step 4.2: Input the processed function to be tested into the retrieval and generate the discrimination model, generate test cases, and perform testing.

[0057] The processed test function is input into a retrieval and discriminant model. The model searches for the most similar complex code segment and its corresponding test cases based on the input function. Combining the similarity and difference information provided by the discriminator, test cases are automatically generated. After the test cases are generated, a written script is used to automatically perform the tests.

[0058] Furthermore, it also includes step 5: feeding the test results back to the model for iterative upgrades; After completing the test, the coverage metric is obtained. It is then determined whether the coverage meets the standard. If it does, the new function to be tested—the test case pair—is added to the code library.

[0059] Furthermore, in step 5, if the requirements are not met, the valid unit test parts are retained and the uncovered information is sent back to the unit test thought chain. The test cases are generated again and tested, and the test report information of each iteration is retained until the coverage meets the requirements.

[0060] Furthermore, step 1.1 specifically includes: Methods2Test: A large-scale supervised dataset containing 780,944 pairs of Java functions to be tested – test case pairs.

[0061] UniTSyn contains 2.7 million test case pairs for functions under test, covering five major programming languages: Python, Java, Go, C++, and JavaScript.

[0062] Private: There are a total of 16,541 sets of proprietary test cases.

[0063] Furthermore, in step 2.2, the proprietary dataset is integrated into the codebase after training and given a higher retrieval priority. By leveraging the similarity in code style within the same working environment, the retrieval process can be accelerated.

[0064] Furthermore, in step 3.1, the data in the code repository is organized into a format suitable for retrieval, such as text files or JSON. Based on different work scenarios, code segments in the code repository are tagged to clarify the functional category of each code segment. A code segment may have multiple tags to ensure quick location during retrieval.

[0065] Furthermore, step 3.1 includes the following types of tags: I. Data Processing It mainly includes three subcategories: data transformation, data calculation, and data filtering.

[0066] II. Logic Control Class It mainly includes two subcategories: conditional judgment and loop control.

[0067] Conditional judgment functions mainly judge based on input conditions and perform different operations.

[0068] III. Input / Output Classes It mainly includes two subcategories: input processing and output generation.

[0069] Input processing functions are primarily responsible for receiving user input or external data input and performing preliminary processing.

[0070] IV. Interactive Classes It mainly includes two subcategories: internal interaction and external interaction.

[0071] Internal interaction functions refer to functions that call other functions within the system; one function is responsible for calling a function in another module to obtain data.

[0072] The trained discriminator network is also added to the local knowledge base. The knowledge base is then deployed to a local server using Docker containerization to ensure its stability and scalability.

[0073] The advantages of this invention are: This method achieves a breakthrough in intelligent industrial-grade unit testing by constructing a collaborative framework of retrieval enhancement generation, discriminative verification, and closed-loop iteration. Its core value lies in three innovations: First, it proposes a multimodal hybrid retrieval mechanism, integrating syntactic structure (AST node distribution), semantic logic (BERT embedding), textual features (variable / function name vectorization), and code comments. Through cascaded retrieval using inverted and vector indices, it accurately matches similar functions in a million-level codebase, improving recall by over 40% and controlling retrieval latency to the millisecond level, completely solving the misjudgment problem of "superficial similarity but fundamental difference" in traditional code clone detection. Second, it innovatively designs a multi-input matching discriminator network, employing a four-level architecture of encoder-matching module-difference detection module-comprehensive processing module. Combined with a cross-attention mechanism, it captures subtle differences between codes (such as ±1 changes in boundary values), and supplements this with a weighted loss function to enhance key sample learning. This reduces the misuse rate of test cases for similar code by 60%, and achieves a rejection rate of up to 92% for code with 95% logical similarity but functional differences. Finally, a closed loop for the co-evolution of data and models is established: when the coverage target is met, the <function and test case> pairs are automatically added to the knowledge base; when the target is not met, valid test cases are retained and supplemented accordingly. Each iteration promotes a 15%-30% increase in coverage. At the same time, incremental data is used to dynamically fine-tune the model, which improves the adaptation efficiency of private domain code by 35%. Attached Figure Description

[0074] Figure 1 The flowchart shows the unit testing method based on the retrieval generation discriminant model provided by this invention. Figure 2A schematic diagram of the efficient code retrieval tool based on code embedding and hybrid feature extraction provided by the present invention; Figure 3 A schematic diagram of a multi-input matching discriminator network is provided for this invention; Figure 4 This is a schematic diagram of the retrieval generation and discrimination model provided by the present invention.

[0075] Figure 5 This is a schematic diagram of the unit testing method based on the retrieval generation discriminant model provided by the present invention. Detailed Implementation

[0076] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] The features and illustrative embodiments of various aspects of the present invention will now be described in detail. Numerous specific details are set forth in the following detailed description to provide a thorough understanding of the invention. However, it will be apparent to those skilled in the art that the invention may be practiced without requiring some of these specific details. The following description of embodiments is merely intended to provide a better understanding of the invention by illustrating examples of the invention. The invention is by no means limited to any specific setups and methods set forth below, but covers any improvements, substitutions, and modifications to structures, methods, and devices without departing from the spirit of the invention. Well-known structures and techniques are not shown in the drawings and the following description to avoid unnecessarily obscuring the invention.

[0078] Example 1 A unit testing method based on a retrieval-generated discriminant model can analyze code features and retrieve similar code and corresponding unit tests from a codebase. Modifications are then made and unit tests are generated through a discriminant generation mechanism. The method consists of three phases: retrieval, training, and testing. In the retrieval phase, a codebase and retrieval mechanism are established. The codebase contains a large number of completed test function-test case pairs. The retrieval mechanism can find the most similar test function-test case pair based on the input test function. In the training phase, the retrieval engine and multi-input matching discriminator are trained using a proprietary dataset. Retrieval keys are generated, and the discriminator is given matching and difference analysis capabilities. The trained retrieval keys and discriminator are then added to a larger model for fine-tuning, constructing the retrieval-generated discriminant model. In the testing phase, the test function is input into the retrieval-generated discriminant model, which outputs test cases and performs unit testing. The technical solution of this invention is: a unit testing method based on a retrieval-generated discriminant model, comprising: Step 1: Establish a code repository and retrieval mechanism; Step 1.1: Collect open-source code datasets and combine them with private datasets to form a codebase.

[0079] Methods2Test: A large-scale supervised dataset containing 780,944 pairs of Java functions to be tested – test case pairs.

[0080] UniTSyn: A large-scale dataset containing 2.7 million test case pairs of functions under test, covering five major programming languages: Python, Java, Go, C++, and JavaScript.

[0081] Private: There are a total of 16,541 sets of proprietary test cases.

[0082] Step 1.2: Embed the functions to be tested from the codebase into vectors using vector embedding methods. This involves embedding the functions to be tested from the dataset... Based on vector embedding, features obtained by general embedding methods, in addition to... In addition, various features of the code are extracted, including syntactic features, semantic features, and textual features. Specifically, syntactic features are mainly determined by the abstract syntax tree, assuming a code snippet... The abstract syntax tree is Its node set is The edge set is Each node There is a type tag type( v ), forming the node type feature vector:

[0083] semantic features Extracted using the BERT method. Text features include variable and function set features, as well as code comment features. For variable and function set features, the Word2Vec method is used to map variable names to a vector space.

[0084] in It is an embedding vector of variable and function names.

[0085] Code comment features The annotations are vectorized using the BERT model.

[0086] Constructing a hybrid feature vector:

[0087] in , , , , Here are the weight parameters, and their sum is 1.

[0088] Step 1.3: Construct an efficient code retrieval tool based on code embedding and hybrid feature extraction.

[0089] This invention proposes an efficient code retrieval method based on code embedding and hybrid feature extraction. It comprehensively considers multiple features of the code, combining inverted index retrieval and vector retrieval to quickly find functions approximating the input function in a large codebase. Its structure diagram is shown below. Figure 2 As shown.

[0090] For the input function to be tested, after performing steps including vector embedding and multidimensional feature extraction, feature fusion, index construction, coarse search, and fine search, the function that approximates the code library and its corresponding test cases are output.

[0091] Vector embedding, multidimensional feature extraction, and feature fusion are shown in step 1.2.

[0092] Index construction consists of two parts: inverted index construction and vector index construction. Inverted index construction is divided into three steps: tokenization, inverted index initialization, and inverted index population. For each code snippet... Tokenize and extract its token set. :

[0093] in These are specific tokens (such as words, variable names, function names, etc.) within the code snippet. Then, an empty inverted index table is initialized:

[0094] Tokens for each code snippet Add to the inverted index:

[0095] Assume there is Each code snippet The set of tokens is The construction of an inverted index can be represented as:

[0096] Vector index construction consists of three steps: feature vector extraction, vector index initialization, and vector index filling. For each code snippet... Extract its comprehensive feature vector Then initialize an empty vector index:

[0097] Add the feature vector of each code snippet to the vector index:

[0098] Assume there is Each code snippet The feature vector is The construction of vector indices can be represented as:

[0099] Coarse searching is implemented based on an inverted index, which is used to quickly filter out a set of potentially related code snippets. Let the inverted index be: ={"for":{1,2,3},"int":{1,3},"i":{1,2}}. Query code The set of tokens is If the query result is {"for","i"}, then the coarse search result will be: {1,2,3}.

[0100] Fine-grained search is implemented based on vector indexes. Assuming the query code... The mixed feature vector is Vector index The system has been built. The goal of fine-grained search is to refine the candidate set from the coarse search. Filter the set of code snippets that are most similar to the query code. .

[0101]

[0102] in, Represents the largest value The similarity is calculated using cosine similarity.

[0103] An efficient code retrieval tool that combines code embedding and hybrid feature extraction can efficiently retrieve code snippets similar to the query code from a large-scale codebase.

[0104] Step 2: Build and train a multi-input matching discriminator; Step 2.1: Construct a multi-input matching discriminator network. The multi-input matching discriminator network is a neural network model proposed in this invention for distinguishing and matching multiple test case pairs (functions to be tested). Its network structure diagram is shown below. Figure 3 As shown, the network mainly consists of an encoder module, a matching module, a difference detection module, and a comprehensive processing module.

[0105] The encoder module takes code segments and test cases as input and consists of a preprocessing layer, a Transformer encoder, and a pooling layer. The preprocessing layer performs tokenization and removes extra spaces and newlines. The Transformer encoder, composed of six Transformer encoder blocks, performs semantic processing and embedding on the input sequence. The pooling layer uses average pooling, averaging the encoder output sequence over time to obtain a fixed-dimensional vector. The encoder module outputs a semantic vector of the code segment. and the semantic vectors of test cases .

[0106] The input to the matching module is the semantic vector of the code segment output by the encoder module. and the semantic vectors of test cases It consists of a similarity calculation layer and a matching classifier. The similarity calculation layer calculates the code segment vector. and vectors The similarity between code segments is calculated using cosine similarity. The matching classifier determines whether the code segment and test case match based on the similarity value, outputting the matching probability of each test case and the similarity value between the code segments. The matching classifier consists of multiple fully connected layers, and the final output is an activation function. The matching results are evaluated using a combined loss function based on contrastive loss and weighted sum loss.

[0107] The combined loss function based on contrastive loss and weighted sum loss is a loss function proposed in this invention for evaluating matching performance in multi-input scenarios. The contrastive loss function is applicable to paired inputs and measures whether the similarity between multiple pairs of inputs meets expectations.

[0108] in, and They are the first The code segment and the first The semantic vector of each test case. It is a label, indicating and Whether it matches (1 indicates a match, 0 indicates no match). yes and The Euclidean distance between them. It is a marginal value used to control the loss of mismatched pairs.

[0109] Taking into account the similarity factors among different functions under test, and combining weighted sum loss, a loss function suitable for multi-input matching modules is constructed:

[0110]

[0111] in, and These are the number of code segments and the number of test cases, respectively. yes The code segment and the first The weights of each test case pair. By minimizing... The model can learn which code segments match which test cases, and at the same time, it can appropriately penalize those that do not match.

[0112] The input to the difference detection module is the semantic vector of the code segment output by the encoder module. It consists of a cross-attention layer, a difference feature extraction layer, and a difference description generator. The cross-attention layer calculates the cross-attention weights between two code segments to determine their similarities and differences. For each code segment pair... , Calculate the cross-attention weights:

[0113]

[0114] in, It is the first Semantic vectors of code segments It is the dimension of the vector.

[0115] The differential feature extraction layer uses cross-attention weights to extract differential features:

[0116]

[0117] The difference description generator uses an encoder-decoder structure to generate difference description text:

[0118]

[0119]

[0120]

[0121] The integration module combines the outputs from the matching and difference detection modules to provide input for the subsequent generator. The input test function is tagged as follows: For each retrieved code segment pair Construct a discriminator to synthesize the output. :

[0122] in It was obtained from the matching module. and The similarity value, The differences between the two are shown in the following example: =(0.98,"Change variable name from x to y"), =(0.85,"A loop structure has been added"). The discriminator's synthesized output, along with the retrieved test function—the test cases—are used together to generate the generator.

[0123] Step 2.2: Train the network using the dataset to obtain the retrieval key and the trained discriminator.

[0124] The proprietary dataset, derived from local work scenarios, contains 3618 pairs of test cases for C++, 2415 pairs for Java, and 2148 pairs for Python. This dataset is then incorporated into the network for training, enabling automatic learning of model parameters and saving the corresponding search keys.

[0125] After training, our own dataset is integrated into the codebase and given a higher retrieval priority. By leveraging the similarity in coding style within the same working environment, the retrieval process can be accelerated.

[0126] Step 3: Fine-tune the retrieval and generation discriminative model; Step 3.1: Integrate the code repository, search keys, and discriminator into a local knowledge base.

[0127] Organize the data in the code repository into a searchable format, such as text files or JSON. Tag code segments in the code repository according to different work scenarios, clearly defining the functional category of each segment. A code segment may have multiple tags to ensure quick location during retrieval. Specifically, the tags include the following: I. Data Processing It mainly includes three subcategories: data transformation, data calculation, and data filtering.

[0128] Data conversion functions transform data from one format to another, such as converting RGB image data to grayscale image data in image processing software. Data calculation functions perform mathematical operations, such as calculating the product of two matrices, or calculating the mean, median, and standard deviation of a set of data. Data filtering functions are used to filter data from a dataset to select those that meet specific criteria, such as selecting students with scores greater than 90 from a list containing information on multiple students.

[0129] II. Logic Control Class It mainly includes two subcategories: conditional judgment and loop control.

[0130] Conditional functions primarily determine whether a user is an adult based on input conditions. For example, they might determine if a user is an adult based on their age. Loop control functions are used to control loops, such as calculating the sum of numbers from 1 to a given positive integer.

[0131] III. Input / Output Classes It mainly includes two subcategories: input processing and output generation.

[0132] Input processing functions are primarily responsible for receiving user input or external data input and performing preliminary processing. For example, a function might receive a username and password from the user and encrypt the password. Output generation functions are primarily responsible for outputting data to external files. For example, generating a report file or writing data to a file according to a specific format.

[0133] IV. Interactive Classes It mainly includes two subcategories: internal interaction and external interaction.

[0134] Internal interaction functions are functions that call other functions within the system; one function is responsible for calling a function in another module to retrieve data. External interaction functions are functions that interact with external databases, such as functions that insert data into the database.

[0135] The trained discriminator network is also added to the local knowledge base. The knowledge base is then deployed to a local server using Docker containerization to ensure its stability and scalability.

[0136] Step 3.2: Fine-tune the large model based on the local knowledge base to obtain the retrieval generation discriminant model.

[0137] Download the pre-trained model and load its weights and word segmenter. Fine-tune the model using the LoRA method, train it using a local knowledge base, and merge the LoRA weights into the original model to obtain a retrieval generation and discriminative model. This model can perform retrieval based on the input function and provide similar test function pairs, automatically generating test cases.

[0138] Step 4: Generate test cases based on the retrieval-generated discriminant model and conduct tests; Step 4.1: Input the function to be tested and perform preprocessing.

[0139] Adjust the format of the function under test according to programming standards to ensure neat code. Rewrite macro definitions in the code to restore their original meaning and eliminate the output illusion caused by large models due to input macro definitions. Tag and categorize the functions under test to speed up retrieval.

[0140] Step 4.2: Input the processed function to be tested into the retrieval and generate the discrimination model, generate test cases, and perform testing.

[0141] The processed test function is input into a retrieval and discriminant model. The model searches for the most similar complex code segment and its corresponding test cases based on the input function. Combining the similarity and difference information provided by the discriminator, test cases are automatically generated. After the test cases are generated, a written script is used to automatically perform the tests.

[0142] Step 5: Feed the test results back to the model for iterative upgrades; After completing the test, the coverage metric is obtained. It is then determined whether the coverage meets the standard. If it does, the new function to be tested—the test case pair—is added to the code library.

[0143] If the requirements are not met, retain the valid unit test parts and re-transfer the uncovered information back to the unit test thought chain, re-execute to generate test cases for iteration and testing, retain the test report information for each iteration until the coverage meets the requirements.

[0144] Example 2 Step 1: Establish a code repository and retrieval mechanism; Step 1.1: Collect open-source code datasets and combine them with private datasets to form a codebase.

[0145] Methods2Test: A large-scale supervised dataset containing 780,944 pairs of Java functions for testing.

[0146] UniTSyn: A large-scale dataset containing 2.7 million test case pairs of functions under test, covering five major programming languages: Python, Java, Go, C++, and JavaScript.

[0147] Private: There are a total of 16,541 sets of proprietary test cases.

[0148] Step 1.2: Use vector embedding to embed the function to be tested in the code library into a vector.

[0149] Syntax: AST, 512 dimensions; Semantics: BERT, 768 dimensions; Text: variable names + comments, 256 dimensions Step 2: Train the multi-input matching discriminator Data set: 8181 pairs.

[0150] Loss: Comparative loss.

[0151] Output: discriminator.pt + key_py_discount. Step 3: Fine-tune the retrieval and generation discriminative model; Integrate the codebase, search keys, and discriminators into a local knowledge base.

[0152] CodeLlama-7B + LoRA fine-tuning for 3 epochs yields RGD-Model-py.

[0153] Step 4: Generate test cases based on the retrieval-generated discriminant model and conduct tests. Step 4.1: Input the function to be tested and perform preprocessing.

[0154] Input the function to be tested: def discount_price(prices, rate=0.1): If not 0 <= rate <= 1: raise ValueError("rate must be between 0 and 1") if not all(p>= 0 for p in prices): raise ValueError("all prices must be non-negative") return [round(p * (1 - rate), 2) for p in prices] Preprocessing Tag: ["price","discount","float_round"].

[0155] Step 4.2: Input the retrieval to generate the discriminant model, generate test cases, and perform testing.

[0156] ① Search top-3

[0157] Retrieve the corresponding function: func_id = disc_0073 def apply_discount(prices, rate): If not 0 <= rate <= 1: raise ValueError("rate must be in [0,1]") return [round(p * (1 - rate), 2) for p in prices] func_id = disc_0142 def discount_single_or_list(items, discount): if isinstance(items, (int, float)): return round(items * (1 - discount), 2) return [round(i * (1 - discount), 2) for i in items] func_id = disc_0205 def safe_discount(values, pct): if not 0 <= pct <= 100: raise ValueError("pct must be 0~100") return [round(v * (100 - pct) / 100, 2) for v in values] ② Discriminator output Added list comprehension + round(p,2) Add non-negative checks ③ Generate test cases RGD-Model-py automatically outputs test_discount_price.py: import pytest from discount import discount_price def test_normal(): assert discount_price([100, 50], 0.2) == [80.0, 40.0] def test_zero_rate(): assert discount_price(

[100] , 0) ==

[100] def test_invalid_rate(): with pytest.raises(ValueError): discount_price(

[100] , 1.5) def test_negative_price(): with pytest.raises(ValueError): discount_price([-10], 0.1) ④ Perform the test pytest test_discount_price.py -q Initial results: Row coverage 88%, Branch coverage 75%.

[0158] ⑤ Iterative Improvement Uncovered branch: round behavior boundary (e.g., 9.995 → 10.00).

[0159] Model supplementary use cases: def test_round_edge(): assert discount_price([9.995], 0.01) == [9.9] Step 5: Feed the test results back to the model for iterative upgrades; After completing the tests, the coverage metric is obtained. It is then determined whether the coverage meets the target. If it does, the new function to be tested—the test case pair—is added to the codebase. If the requirements are not met, the valid unit tests are retained, and the uncovered information is returned to the unit test thought process. The test cases are then regenerated and tested, and the test report information for each iteration is retained until the coverage requirement is met.

[0160] Run again: row coverage 100%, branch coverage 100%.

[0161] Will<discount_price, test_discount_price.py> Added codebase, version number py-disc-v1.1.

[0162] It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other, and the various embodiments can be referenced and cited in each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should be covered within the protection scope of the present invention.

Claims

1. A unit testing method based on a retrieval-generation discriminant model, characterized in that, Modifications are made and unit tests are generated through a discrimination mechanism. There are three stages: retrieval, training, and testing. In the retrieval stage, a code library and retrieval mechanism are established. The code library contains a large number of completed functions to be tested - test case pairs. The retrieval mechanism finds the most similar function to be tested - test case pair based on the input function to be tested. During the training phase, the retrieval engine and multi-input matching discriminator are trained using a proprietary dataset to generate retrieval key values ​​and enable the discriminator to perform matching and difference analysis. The trained retrieval key values ​​and the discriminator are then added to a larger model for fine-tuning to construct a retrieval generation and discriminator model. During the testing phase, the function to be tested is input into the retrieval generation and discriminator model, test cases are output, and unit tests are performed.

2. The method as described in claim 1, characterized in that, The specific steps are as follows: Step 1: Establish a code repository and retrieval mechanism; Step 1.1: Collect open-source code datasets and combine them with the private datasets to form a codebase; Step 1.2: Use vector embedding methods to embed the function to be tested from the code library into a vector; embed the function to be tested from the dataset... Perform vector embedding, excluding features obtained by the embedding method. In addition, various features of the code are extracted, including syntactic features, semantic features, and textual features; specifically, syntactic features are determined by the abstract syntax tree, assuming a code snippet... The abstract syntax tree is Its node set is The edge set is Each node There is a type tag type( v ), forming the node type feature vector: semantic features Extracted using the BERT method; text features include variable and function set features as well as code comment features. For variable and function set features, the Word2Vec method is used to map variable names to a vector space. in It is an embedding vector of variable and function names; Code comment features The annotations are vectorized using the BERT model. Constructing a hybrid feature vector: in , , , , These are the weight parameters, and their sum is 1. Step 1.3: Construct an efficient code retrieval tool based on code embedding and hybrid feature extraction; By comprehensively considering various characteristics of the code and combining inverted index retrieval and vector retrieval, functions that are similar to the input function can be quickly found in a large codebase. For the input function to be tested, after performing steps including vector embedding and multidimensional feature extraction, feature fusion, index construction, coarse search, and fine search, the function that approximates the code library and its corresponding test cases are output. Vector embedding, multidimensional feature extraction, and feature fusion are shown in step 1.2; Index construction consists of two parts: inverted index construction and vector index construction. Inverted index construction is divided into three steps: tokenization, inverted index initialization, and inverted index filling. For each code snippet... Tokenize and extract its token set. : in It is a specific token in the code snippet, which is then used to initialize an empty inverted index table: Tokens for each code snippet Add to the inverted index: Assume there is Each code snippet The set of tokens is The construction of an inverted index is represented as follows: Vector index construction consists of three steps: feature vector extraction, vector index initialization, and vector index filling; for each code snippet Extract its comprehensive feature vector Then initialize an empty vector index: Add the feature vector of each code snippet to the vector index: Assume there is Each code snippet The feature vector is The construction of vector indices is represented as follows: Coarse-grained search is implemented based on an inverted index, which is used to quickly filter out a set of potentially related code snippets; let the inverted index be: ={"for":{1,2,3},"int":{1,3},"i":{1,2}};Query code The set of tokens is If the query result is {"for","i"}, then the coarse search result will be: {1,2,3}; Fine-grained search is implemented based on vector indexes. Assuming the query code... The mixed feature vector is Vector index It has been built successfully; The goal of fine-grained search is to refine the candidate set from the coarse search. Filter the set of code snippets that are most similar to the query code. ; in, Represents the largest value The similarity is calculated using cosine similarity. An efficient code retrieval tool that combines code embedding and hybrid feature extraction can efficiently retrieve code snippets similar to the query code from a large-scale codebase. Step 2: Build and train a multi-input matching discriminator; Step 2.1: Construct a multi-input matching discriminator network; a multi-input matching discriminator network is a neural network model that distinguishes and matches multiple pairs of test functions and test cases; the network mainly consists of an encoder module, a matching module, a difference detection module, and a comprehensive processing module; The encoder module takes code segments and test cases as input and consists of a preprocessing layer, a Transformer encoder, and a pooling layer. The preprocessing layer performs tokenization and removes extra spaces and newlines. The Transformer encoder, composed of six Transformer encoder blocks, performs semantic processing and embedding on the input sequence. The pooling layer uses average pooling, averaging the encoder output sequence over time to obtain a fixed-dimensional vector. The encoder module outputs a semantic vector of the code segment. and the semantic vectors of test cases ; The input to the matching module is the semantic vector of the code segment output by the encoder module. and the semantic vectors of test cases It consists of a similarity calculation layer and a matching classifier; the similarity calculation layer calculates the code segment vector. and vectors The similarity between the code segments is calculated using cosine similarity. The matching classifier determines whether the code segments and test cases match based on the similarity values, and outputs the matching probability of each test function and test case, as well as the similarity value between the code segments. The matching classifier consists of multiple fully connected layers and is finally output by an activation function. The matching results are evaluated using a comprehensive loss function based on contrastive loss and weighted sum loss. The combined loss function, based on contrastive loss and weighted sum loss, is a loss function used to evaluate matching performance in multi-input scenarios. Contrastive loss is applicable to paired inputs and measures whether the similarity between multiple pairs of inputs meets expectations. in, and They are the first The code segment and the first Semantic vectors of each test case; It is a label, indicating and Whether it matches, 1 indicates a match, 0 indicates no match; yes and The Euclidean distance between them; It is a marginal value used to control the loss of mismatched pairs; Taking into account the similarity factors among different functions under test, and combining weighted sum loss, a loss function suitable for multi-input matching modules is constructed: in, and These are the number of code segments and the number of test cases, respectively. yes The code segment and the first The weights of each test case pair; by minimizing The model can learn which code segments match which test cases, and at the same time, it can appropriately penalize pairs that do not match. The input to the difference detection module is the semantic vector of the code segment output by the encoder module. It consists of a cross-attention layer, a differential feature extraction layer, and a differential description generator. The cross-attention layer calculates the cross-attention weights between two code segments to determine their similarities and differences. For each code segment... , Calculate the cross-attention weights: in, It is the first Semantic vectors of code segments It is the dimension of the vector; The differential feature extraction layer uses cross-attention weights to extract differential features: The difference description generator uses an encoder-decoder structure to generate difference description text: After receiving the outputs from the matching and difference detection modules, the integration module combines this information to provide input for the subsequent generator; the input test function is tagged as... For each retrieved code segment pair Construct a discriminator to synthesize the output. : in It was obtained from the matching module. and The similarity value, The differences between the two are shown in the following example: =(0.98,"Change variable name from x to y"), =(0.85,"A loop structure has been added"); The discriminator's output and the retrieved test function—test cases—are used together in the generator to generate the test cases; Step 2.2: Train the network using the dataset to obtain the retrieval key and the trained discriminator; The proprietary dataset comes from a local work environment and contains 3,618 pairs of C++ language, 2,415 pairs of Java language, and 2,148 pairs of Python language functions to be tested—test case pairs. The dataset is added to the network for training, the model parameters are automatically learned, and the corresponding retrieval key values ​​are saved. Step 3: Fine-tune the retrieval and generation discriminative model; Step 3.1: Integrate the code repository, search keys, and discriminator into a local knowledge base; Step 3.2: Fine-tune the large model based on the local knowledge base to obtain the retrieval generation discriminant model; Download the pre-trained model and load its weights and word segmenter; fine-tune the model using the LoRA method, load the local knowledge base for training, and merge the LoRA weights into the original model to obtain the retrieval generation discriminant model; it can perform retrieval based on the input function and provide similar test function-test case pairs, and automatically generate test cases; Step 4: Generate test cases based on the retrieval-generated discriminant model and conduct tests; Step 4.1: Input the function to be tested and perform preprocessing; Adjust the format of the function under test according to the programming specifications to ensure neat code; rewrite the macro definitions in the code to restore the macro definitions and eliminate the output illusion caused by the large model due to the input macro definitions; categorize the functions under test by tags to speed up the retrieval speed. Step 4.2: Input the processed function to be tested into the retrieval and generate the discrimination model, generate test cases, and perform testing; The processed function to be tested is input into the retrieval and a discriminant model is generated. The model searches based on the input function to find the most similar complex code segment and the corresponding test case. The test cases are automatically generated by combining the similarity and difference information given by the discriminator. After the test cases are generated, the written script is used to automatically perform the test.

3. The method as described in claim 2, characterized in that, It also includes step 5: feeding the test results back to the model for iterative upgrades; obtaining the coverage metric after completing the test, determining whether the coverage meets the standard, and if it does, adding the new function to be tested—the test case pair—to the code library.

4. The method as described in claim 3, characterized in that, In step 5, if the requirements are not met, the valid unit test parts are retained and the uncovered information is sent back to the unit test thought chain. The test cases are generated again and tested, and the test report information of each iteration is retained until the coverage meets the requirements.

5. The method as described in claim 2, characterized in that, Step 1.1 specifically includes: Methods2Test: A large-scale supervised dataset containing 780,944 pairs of Java function-test case pairs; UniTSyn contains 2.7 million test case pairs for functions under test, covering five mainstream programming languages: Python, Java, Go, C++, and JavaScript. Private: There are a total of 16,541 sets of proprietary test cases.

6. The method as described in claim 2, characterized in that, In step 2.2, the proprietary dataset is integrated into the codebase after training and given a higher retrieval priority. By leveraging the similarity in code style within the same working environment, the retrieval process can be accelerated.

7. The method as described in claim 2, characterized in that, In step 3.1, the data in the code library is organized into a format suitable for retrieval; according to different work scenarios, the code segments in the code library are tagged to clarify the functional category to which each code segment belongs. A code segment may have multiple tags to ensure that it can be quickly located during retrieval.

8. The method as described in claim 7, characterized in that, Step 3.1 includes the following types of tags: I. Data Processing It includes three subcategories: data transformation, data calculation, and data filtering. II. Logic Control Class It includes two subcategories: conditional judgment and loop control. Conditional functions primarily evaluate input conditions and perform different operations accordingly. III. Input / Output Classes It includes two subcategories: input processing and output generation. Input processing functions are primarily responsible for receiving user input or external data input and performing preliminary processing; IV. Interactive Classes It includes two subcategories: internal interaction and external interaction. Internal interaction functions refer to functions that call other functions within the system, where one function is responsible for calling a function in another module to obtain data.

Images