A Software Defect Localization Method Based on Enhanced Embedded Vector Semantic Representation

Abstract

This invention provides a software defect localization method based on enhanced embedding vector semantic representation, belonging to the field of computer technology, and solves the technical problem of insufficient semantic information representation of multimodal embedding vectors. The technical solution includes the following steps: S1: Data augmentation of the source code; S2: Constructing positive and negative sample pairs between and within modalities; S3: Text preprocessing of the defect report to obtain a text sequence; S4: Inputting the text sequence into a CodeBert pre-trained model to obtain embedding vector representations; S5: Learning the similarity within and between modalities; S6: Fine-tuning the pre-trained model by jointly performing retrieval and binary classification tasks; S7: Sorting the source code files to obtain prediction results. The beneficial effects of this invention are: through comparative learning, better embedding vector representations are obtained; and by jointly sorting the source code files using the retrieval and classification models, the effectiveness of defect localization is further improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method for locating software defects based on enhanced embedded vector semantic representation. Background Technology

[0002] Defect localization refers to the task of locating potentially faulty source code files within a software project given defect reports. Developers must meticulously analyze error reports and examine numerous source code files; manually identifying all incoming error reports requires significant human and material resources. Therefore, automated defect localization can significantly improve the efficiency of software defect fixing and software maintenance.

[0003] Many techniques for generating text embedding vectors have been proposed, but these methods always treat both defect reports written in natural language and source code files written in programming languages ​​as belonging to the same modality (natural language text). In reality, programming languages ​​have a different writing style than natural language and contain a large amount of structural information; therefore, they should not be considered to belong to the same modality as natural language. Some current defect localization methods obtain high-quality embedding vector representations by using large pre-trained models, but how to further fine-tune these pre-trained models to obtain more reliable embedding representations for specific tasks has not been extensively studied.

[0004] How to solve the above-mentioned technical problems is the challenge facing this invention. Summary of the Invention

[0005] The purpose of this invention is to provide a software defect localization method based on enhanced embedded vector semantic representation. This method can mark the corresponding defect file according to the content described in the defect report, thus solving the problem of not being able to fully represent the semantic information unique to programming languages.

[0006] The core idea of ​​this invention is as follows: This invention proposes a software defect localization method based on enhanced embedding vector semantic representation. Specifically, by contrastive learning of the objective function, it learns the similarity between natural language and programming language, as well as within natural language and programming language. It further fine-tunes the model parameters of the pre-trained model CodeBERT to obtain embedding vector representations with richer semantic information. Then, it combines a retrieval model and a binary classification model, weights the retrieval score and classification score to obtain a similarity score, and ranks the source code files by relevance.

[0007] This invention is achieved through the following measures: a software defect localization method based on enhanced embedded vector semantic representation, comprising the following steps:

[0008] (1) Perform data augmentation on the source code file, specifically including the following steps:

[0009] (1-1) First, convert the functions in the source code file s into an abstract syntax tree (AST) one by one;

[0010] (1-2) Then traverse the nodes in the abstract syntax tree to find the nodes where function is defined and variable is assigned;

[0011] (1-3) After finding the target node, replace the function name with the func marker and use var. i The i-th variable name appearing in the mark-replacement function is obtained from the enhanced source code file s′.

[0012] (2) Pair the defect report, source code file, and data-enhanced source code file to construct positive and negative sample pairs between and within modalities, specifically including the following steps:

[0013] (2-1) Report the defect b i and its corresponding source code file s i Defect Report b i and its corresponding enhanced source code file s i 'Consider it as a positive sample pair between modes (b) i ,s i ), (b i ,s i ′);

[0014] (2-2) Report the defect b i and its non-corresponding source code file s j Considered as negative sample pairs between modes (b i ,s j ), where the subscript i ≠ j;

[0015] (2-3) Transfer the source code file s i And its enhanced source code file s i 'Consider positive sample pairs (s) within the modality i ,s i ′);

[0016] (2-4) Transfer the source code file s i and other source code files appearing in the batch j Defect Report b i Defect reports in batches j Considered as negative sample pairs within the modality (s i ,s j ), (b i ,b j The batch size is set to 16.

[0017] (3) Perform text preprocessing on the constructed positive and negative sample training data pairs to generate text sequences, specifically including the following steps:

[0018] (3-1) Use the RobertaTokenizer tokenizer to segment the defect report, source code file, and enhanced source code file into token sequences, each token sequence being a list of words separated by spaces;

[0019] (3-2) Insert a [CLS] token at the beginning of the token sequence and concatenate a [SEP] token at the end of the token sequence to obtain a new {b} i}, {s i}, {s i '}sequence;

[0020] (4) Input the preprocessed text sequence into the CodeBERT pre-trained model to obtain word embedding vector representations containing contextual semantic information. Specifically, this includes the following steps: using the CodeBERT pre-trained model to input {b i}, {s i}, {s i The sequence is encoded into the embedding space, and the embedding vector marked with [CLS] is taken as the embedding vector of the entire sequence, where the dimension of the embedding vector is 1×768.

[0021] (5) Based on the objective function of contrastive learning, learn the similarity within and between modalities, and fine-tune the parameters of the CodeBERT pre-trained model. The specific steps include: First, define a single defect report b. i The cross-modal contrastive loss function is:

[0022]

[0023] Define a single source code file s i The cross-modal contrastive loss function is:

[0024]

[0025] Among them, the molecule is related to the current b. i Or s i The relevant positive sample data pairs, the denominator is all positive and negative sample data pairs, and λ and μ are weight parameters. t is a scalar temperature parameter. The final cross-modal contrastive learning loss function is the arithmetic mean of the loss values ​​from all defect reports and source code files, defined as:

[0026]

[0027] Where N is the batch size.

[0028] (6) Fine-tuning the pre-trained model for joint retrieval and binary classification tasks, specifically including the following steps:

[0029] (6-1) In the retrieval task, the similarity score between the defect report and the source code file is calculated using vector dot product. The ultimate goal of training the retrieval model is to maximize the dot product between positive sample pairs while minimizing the dot product between negative sample pairs.

[0030] The loss function for training an information retrieval model is defined as:

[0031]

[0032] Where N is the batch size, and exp(·) represents the exponential function.

[0033] (6-2) In the classification task, first classify the defect report b i and source code files i Perform vector concatenation, then calculate the vector difference b. i -s i sum of vector products b i ⊙s i This is also concatenated to the original vector, and then input into a feedforward neural network, as shown in the following formula:

[0034] r i =tanh(W1·concat(b) i ,s i ,b i -s i ,b i ⊙s i ))

[0035] Where W1 represents the parameters of the feedforward neural network, and tanh(·) is the activation function. Then, the output r... i Input a single-layer perceptron classifier:

[0036] s i =sigmoid(W2·r) i )

[0037] Where W2 represents the parameters of a single-layer perceptron classifier. Finally, the training objective function used for classification is defined as the binary cross-entropy loss function:

[0038]

[0039] Where N is the batch size.

[0040] (6-3) In the optimization phase, the training objectives for the retrieval model obtained in 7.1, the classification model obtained in 7.2, and the cross-modal contrastive learning training objective obtained in 6.1 are combined to obtain the final fine-tuning objective function, defined as follows: L = αL 检索模型 +βL 对比学习 +γL 分类模型

[0041] Where α, β, and γ are the weighting coefficients for different losses.

[0042] (7) Based on the similarity score obtained by weighting the retrieval score and the classification score, the source code files are ranked according to their relevance. This includes the following steps: In the prediction stage, candidate source code files are ranked from largest to smallest using the similarity score weighted by the retrieval score and the classification score to obtain the final prediction result. The final similarity score is calculated as follows: sim(b i ,s i )=ω1·sim1(b i ,s i )+ω2·sim2(b i ,s i )

[0043] Where sim1 is the retrieval similarity, sim2 is the classification similarity, and ω1 and ω2 are weight parameters.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0045] (1) Use the pre-trained model CodeBERT containing contextual semantic information to provide embedding vector representations of defect reports and source code files. The rich contextual information can help learn the correlation between defect reports and source code files.

[0046] (2) To reduce the sensitivity of function and variable name editing, code enhancement is used to rename functions and variables to meaningless tags; more code snippets are generated while keeping the functionality of the code snippets unchanged as much as possible, and positive and negative training sample pairs across modalities are constructed to learn the similarity within and between modalities.

[0047] (3) Construct a contrastive objective function to fine-tune the embedding vector of the pre-trained model by narrowing the distance between positive samples and widening the distance between negative samples in order to obtain a better embedding representation.

[0048] (4) In the fine-tuning stage, the retrieval training objective, classification objective and cross-modal contrastive learning objective are combined, and the source code files are ranked using a similarity score weighted by retrieval score and classification score, which further improves the effectiveness of defect localization. Attached Figure Description

[0049] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0050] Figure 1 This is a framework diagram of a software defect localization method based on enhanced embedded vector semantic representation provided by the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. Of course, the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0052] Example 1

[0053] See Figure 1 As shown, this invention provides a software defect localization method based on enhanced embedded vector semantic representation, specifically including the following:

[0054] (1) Extract a batch of source code files from the original dataset and perform data augmentation on the code:

[0055] (1-1) First, convert the functions in the source code file s into an abstract syntax tree (AST);

[0056] (1-2) Then traverse the nodes in the AST to find the nodes where functions are defined and variables are assigned;

[0057] (1-3) After finding the target node, replace the function name with the func marker and use var. i The i-th variable name appearing in the mark-replacement function is obtained from the enhanced source code file s′.

[0058] (2) Report defects in a batch i , where i∈(1, batch), and source code file s i and the data-enhanced source code file s i Pairing: Constructing positive and negative sample pairs between and within modes.

[0059] (2-1) Report the defect b i and its corresponding source code file s i Defect Report b i and its corresponding enhanced source code file s i 'Consider it as a positive sample pair between modes (b) i ,s i ), (b i ,s i ′);

[0060] (2-2) Report the defect b i and its non-corresponding source code file s j Considered as negative sample pairs between modes (b i ,s j ), where the subscript i ≠ j;

[0061] (2-3) Transfer the source code file s i And its enhanced source code file s i 'Consider positive sample pairs (s) within the modality i ,s i ′);

[0062] (2-4) Transfer the source code file s i and other source code files appearing in the batch j Defect Report b i Defect reports in batches j Considered as negative sample pairs within the modality (s i ,s j ), (b i ,b j );

[0063] (3) Preprocess the text data into a format that can be input into the pre-trained model.

[0064] (3-1) b i ,s i ,s i Use the tokenizer module to segment the tokens into a sequence of words.

[0065] (3-2) Concatenate a [CLS] token at the beginning of the token sequence and a [SEP] token at the end of the token sequence to obtain {b i}, {s i}, {s i '}sequence.

[0066] (4) Input the newly generated token sequence into the pre-trained model and extract the word embedding vector of the [CLS] tag as the embedding vector representation of the entire sequence.

[0067] (5) Input the embedded vector into the contrastive learning module and perform contrastive learning by minimizing the contrastive learning loss function.

[0068] (6) Further fine-tuning of the pre-trained model for joint retrieval and binary classification tasks.

[0069] (6-1) In the retrieval task, the vector dot product between the defect report and the source code file is calculated as the similarity score. The ultimate goal of training the retrieval model is to maximize the dot product between positive sample pairs while minimizing the dot product between negative sample pairs.

[0070] The loss function for training an information retrieval model is defined as:

[0071]

[0072] Where N is the batch size.

[0073] (6-2) In the classification task, first classify the defect report b i and source code files i Perform vector concatenation, then calculate the vector difference b. i -s i sum of vector products b i ⊙s i This is also concatenated to the original vector, and then input into a feedforward neural network, as shown in the following formula:

[0074] r i =tanh(W1·concat(b) i ,s i ,b i -s i ,b i ⊙s i ))

[0075] Where W1 represents the parameters of the feedforward neural network, and tanh(·) is the activation function. Then, the output r... i Input a single-layer perceptron classifier:

[0076] s i =sigmoid(W2·r) i )

[0077] Where W2 represents the parameters of a single-layer perceptron classifier. Finally, the training objective function used for classification is defined as the binary cross-entropy loss function:

[0078]

[0079] Where N is the batch size.

[0080] (6-3) In the optimization phase, the training objectives for the retrieval model obtained in 7.1, the classification model obtained in 7.2, and the cross-modal contrastive learning training objective obtained in 6.1 are combined. The final fine-tuning objective function is then defined as follows:

[0081] L=αL 检索模型 +βL对比学习 +γL 分类模型

[0082] Where α, β, and γ are the weighting coefficients for different losses.

[0083] (7) In the prediction phase, candidate source code files are sorted from largest to smallest using a similarity score weighted by retrieval score and classification score to obtain the final prediction result. The final similarity score is calculated as follows:

[0084] sim(b i ,s i )=ω1·sim1(b i ,s i )+ω2·sim2(b i ,s i )

[0085] Where sim1 is the retrieval similarity, sim2 is the classification similarity, and ω1 and ω2 are weight parameters.

[0086] (9) The method of this invention and existing flawed methods are evaluated on the same dataset. The quality of this method is automatically evaluated using three performance metrics commonly used in information retrieval tasks (i.e., Accuracy, MAP, and MRR):

[0087] Table 1. Comparison of results between the method of this invention and other methods.

[0088]

[0089] Experiments show that the software defect localization method based on enhanced embedded vector semantic representation proposed in this invention achieves higher accuracy in defect localization compared to baseline methods. Specifically, this method, through comparative learning and fine-tuning of the parameters of the pre-trained model, and by fusing a retrieval model and a binary classification model, outperforms these baseline methods in performance. Specifically, for Accuracy@1, the proposed method improves performance by 17%–20% across various software projects; for MAP, it improves performance by 8%–17%; and for MRR, it improves performance by at least 3%–11%. These experimental results demonstrate the effectiveness of the proposed method. These results indicate that the proposed method has high competitiveness and advantages.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A software defect localization method based on enhanced embedded vector semantic representation, characterized in that, Includes the following steps: 1.1: Perform data augmentation on the source code files; Step 1.1, which involves data augmentation of the source code file, specifically includes the following steps: 2.1: First, save the source code file. The functions in the code are converted into an Abstract Syntax Tree (AST); 2.2: Then traverse the nodes in the AST to find the nodes where functions are defined and variables are assigned values; 2.3: After finding the target node, use Mark the replacement function name, use The enhanced source code file is obtained by marking the i-th variable name appearing in the substitution function. ; 1.2: Pair the defect report with the source code file and the data-enhanced source code file to construct positive and negative sample pairs between and within modalities; 1.3: Perform text preprocessing on the constructed positive and negative sample training data pairs to generate text sequences; 1.4: Input the preprocessed text sequence into the CodeBert pre-trained model to obtain word embedding vector representations containing contextual semantic information; 1.5: Design the objective function for contrastive learning so that similarity within and between modes can be considered simultaneously during the fine-tuning of the pre-trained model; Step 1.5, which fine-tunes the pre-trained model across modalities using a contrastive learning objective function, specifically includes the following steps: 6.1: To better represent defect reports and source code files, it is necessary to consider the similarity between defect reports and source code files, as well as the internal similarity between defect reports and source code files; Define a single defect report The cross-modal contrastive loss function is: ； Where λ and μ are weight parameters, t is a scalar temperature parameter; Define a single source code file The cross-modal contrastive loss function is: ； Among them, the molecule is related to the current or The relevant positive sample data pairs, the denominator is all positive and negative sample data pairs, and λ and μ are weight parameters. Where t is a scalar temperature parameter, the final cross-modal contrastive learning loss function is the arithmetic mean of the loss values ​​from all defect reports and source code files, defined as: ; Where N is the batch size; 1.6: Fine-tuning the pre-trained model for joint retrieval and binary classification tasks; 1.7: Rank the source code files based on their relevance according to the similarity score obtained by weighting the retrieval score and the classification score.

2. The software defect localization method based on enhanced embedded vector semantic representation according to claim 1, characterized in that, Step 1.2, which constructs positive and negative sample pairs between and within modes, specifically includes the following steps: 3.1: Report the defect and its corresponding source code files Defect Report and its corresponding enhanced source code files 'Consider them as positive sample pairs between modes' ; 3.2: Report the defect and its incompatible source code files Consider as negative sample pairs between modes subscript ; 3.3: Transfer the source code file and its enhanced source code files 'Consider positive sample pairs within the modality' ; 3.4: Transfer the source code file and other source code files appearing in the batch Defect Report Defect reports in batches Negative sample pairs within a modality , .

3. The software defect localization method based on enhanced embedded vector semantic representation according to claim 1, characterized in that, Step 1.3, which involves text preprocessing of the training data, specifically includes the following steps: 4.1: Use the RobertaTokenizer tokenizer to segment the defect report, source code file, and enhanced source code file into token sequences; 4.2: Add a [CLS] token to the beginning of the token sequence and a [SEP] token to the end of the token sequence to obtain... , , sequence.

4. The software defect localization method based on enhanced embedded vector semantic representation according to claim 1, characterized in that, Step 1.4 involves feeding the data into CodeBERT to obtain word embedding vector representations, specifically including the following steps: using a CodeBERT pre-trained model to... , , The sequence is encoded into the embedding space, and the embedding vector marked with [CLS] is taken as the embedding vector of the entire sequence.

5. The software defect localization method based on enhanced embedded vector semantic representation according to claim 1, characterized in that, Step 1.7 ranks the source code files based on the relevance score obtained by weighting the retrieval score and the classification score. Specifically, this includes the following steps: In the prediction phase, candidate source code files are sorted from largest to smallest using the similarity score weighted by the retrieval score and the classification score to obtain the final prediction result. The final similarity score is calculated as follows: ； in, To retrieve similarity, For classification similarity, and These are the weight parameters.

Images