A software defect prediction method based on a position-sensitive hierarchical attention network

Abstract

The application relates to the field of artificial intelligence and source code semantic feature extraction, and relates to a software defect prediction method based on a position-sensitive hierarchical attention network, which mainly comprises the following steps: source code preprocessing into an embedding vector; sample data set construction; position-sensitive hierarchical attention network model construction; source code semantic feature extraction through an additive attention mechanism and a self-attention mechanism; calculation of the gradient of a loss function on model parameters by using a back propagation algorithm; adaptive adjustment of parameters by using an Adam optimization algorithm to realize model training and optimization; and finally, input of a to-be-detected source code file into the model to output whether the source code has defects. The application can more accurately predict software defects and improve software development and testing efficiency by capturing local and global features of source code semantic information from different levels through hierarchical segmentation and reconstruction, and by introducing relative position coding to solve the position information problem and the long-distance path dependence problem.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and source code semantic feature extraction, specifically to a software defect prediction method based on a position-sensitive hierarchical attention network. Background Technology

[0002] Software defect prediction refers to using historical project data to identify potentially defective software modules in the source code of a new version of the project before defects occur, thereby preventing and correcting defects in advance to ensure software quality and reliability. The earlier software defects are discovered, the lower the cost of fixing them. For example, addressing defects during the software development phase can reduce the software costs incurred by discovering defects during the testing phase. By allocating more testing resources to high-risk modules, software quality and testing efficiency can be guaranteed.

[0003] Early research on software defect prediction focused on traditional manual features. However, due to the various syntactic structures and semantic information inherent in programming languages, traditional manual metrics could not capture this complex information in the code. With the rise of deep learning, the acquisition of deep semantic features from source code through deep learning has been widely applied. In the early days, source code was treated as a general text sequence, and natural language processing methods were used to encode each word. However, because source code is special text data with certain syntactic rules, some semantics are represented by symbols, such as assignment operators and other operators. Direct encoding can easily produce useless ambiguous information.

[0004] Abstract Syntax Trees (ASTs), as a representation of the syntactic structure of source code, contain semantic information such as symbols within their nodes, and the order of the nodes reflects the syntactic structure of the source code. In the field of software defects, ASTs are frequently used as intermediate representations of code to extract semantic information. Common processing methods include simply traversing the tree to generate a one-dimensional sequence, treating the AST as a graph, and inputting it into a tree-like network. Whether treating it as a text sequence, directly flattening the AST into a sequence of nodes, or using a tree structure, these methods still serialize the AST as a whole at a coarse-grained level, ignoring the fine-grained expression of the project's source code. Fine-grained defect prediction enables further defect localization, helping developers focus their limited energy on high-risk areas of the source code, resulting in higher efficiency.

[0005] In addition, source code naturally possesses a hierarchical structure, and the same code appearing in different locations is likely to have different semantics, closely related to its context. Therefore, predicting the defective tendencies of code should vary depending on its location, and most current deep learning-based methods have not fully utilized this information. Summary of the Invention

[0006] To address the technical problems existing in the prior art, this invention provides a software defect prediction method based on a location-sensitive hierarchical attention network. It employs an improved hierarchical attention network as the backbone network and designs an AST representation learning method based on path segmentation and reconstruction. Attention mechanisms are used to dynamically assign different weights to different inputs at both the node and path context levels. Relative position encoding is introduced at the path layer to distinguish the contributions of paths at different locations, enabling the model to capture defective regions. This solves the long-distance path dependency problem and improves the accuracy and robustness of software defect prediction.

[0007] The objective of this invention can be achieved by adopting the following technical solutions:

[0008] A software defect prediction method based on a location-sensitive hierarchical attention network, the method comprising:

[0009] S1. Source code preprocessing: The source code is parsed into an abstract syntax tree using a toolkit. The parser generates a word stream according to the syntax structure and then obtains the paths from all root nodes to leaf nodes in the abstract syntax tree by preorder traversal.

[0010] S2. Construct a sample dataset. Use the preprocessed path set of source code files of historical version projects as the training set and the preprocessed path set of source code files of new version projects as the test set. Set a label for each source code file. The label is used to indicate whether the source code file has defects.

[0011] S3. Construct a position-sensitive hierarchical attention network model. The embedding layer converts each node into a node embedding vector. The hierarchical attention layer improves upon the hierarchical attention network by changing one of the two additive attention mechanisms to a self-attention mechanism and introducing relative position encoding into this self-attention mechanism. This captures the position information of different paths in the tree structure at the path level, resulting in a position-sensitive hierarchical attention network model.

[0012] S4. Use the training sample set to train and optimize the position-sensitive hierarchical attention network model, and input the source code semantic features extracted from the position-sensitive hierarchical attention network model into a logistic regression classifier to obtain convergence.

[0013] S5. Input the code of the item to be detected into the trained position-sensitive hierarchical attention network model for defect prediction, and output whether the code of the item to be detected has defects.

[0014] Specifically, step S1 includes:

[0015] Before coding, the source code is parsed into an abstract syntax tree using a code processing toolkit, and a node stream is generated based on the syntax structure. All paths from the root node to the leaf node are extracted by traversing the abstract syntax tree, and the maximum number of paths and the number of nodes in each path are limited to L and T, respectively.

[0016] L = coe1 * max_path_len,

[0017] T = coe2 * max_node_len,

[0018] Where max_path_len is the number of paths in the maximum tree in the AST, max_node_len is the number of nodes in the longest path in the AST, and coe1 and coe2 are adjustable coefficients that limit these two lengths, respectively.

[0019] Specifically, step S2 includes:

[0020] The dataset from the historical version of the project code is used as the training set to train the software prediction model; the dataset from the new version of the project code is used as the test set to evaluate the performance of the software prediction model.

[0021] Perform random oversampling on the training set to increase the number of minority class samples.

[0022] Specifically, step S3 includes:

[0023] The abstract syntax tree is divided into three different granularities: node granularity, path granularity, and abstract syntax tree granularity.

[0024] A vocabulary is constructed based on the node flow. After mapping each node to a unique index, the nodes are initialized as vectors through an embedding layer to obtain node granularity vectors.

[0025] Additive attention is used at the node level to obtain the path granularity vector. Self-attention is used at the path level, and relative position embedding is introduced into the self-attention mechanism to obtain the abstract syntax tree granularity vector. The abstract syntax tree granularity vector is used as the semantic feature of the source code.

[0026] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0027] 1. This invention provides a software defect prediction method based on a position-sensitive hierarchical attention network. By constructing a position-sensitive hierarchical attention network model, the hierarchical attention layer is improved on the basis of the hierarchical attention network. One of the two additive attention mechanisms is changed to a self-attention mechanism, and relative position encoding is introduced into this self-attention mechanism. Compared with the features selected manually and the features extracted by convolutional neural networks, this method uses hierarchical segmentation and reconstruction to capture local and global features of source code semantic information from different levels. At the same time, the introduction of relative position encoding solves the problem of position information and long-distance path dependence, which can more accurately predict software defects and improve the efficiency of software development and testing.

[0028] 2. This invention takes into account the lack of regularity in ASTs representing different source codes. By imposing several restrictions on the AST, including the node type, path length, and number of paths in the syntax tree, resources are saved while retaining effective information. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0030] Figure 1 This is a flowchart of a software defect prediction method based on a location-sensitive hierarchical attention network according to an embodiment of the present invention.

[0031] Figure 2 This is the overall network architecture diagram of the model method in this embodiment of the invention;

[0032] Figure 3 This is a detailed network diagram of the node coding layer and path coding layer in an embodiment of the present invention. Detailed Implementation

[0033] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments, and the implementation of the present invention is not limited thereto. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0034] Example 1

[0035] like Figure 1The flowchart shown illustrates a software defect prediction method based on a location-sensitive hierarchical attention network. The software defect prediction method based on a location-sensitive hierarchical attention network described in this invention includes the following steps:

[0036] S1. Source code preprocessing: The source code is parsed into an abstract syntax tree using a toolkit. The parser generates a word stream according to the syntax structure, and then obtains the paths from all root nodes to leaf nodes in the abstract syntax tree by preorder traversal, so as to facilitate subsequent path vector representation.

[0037] Preprocessing the source code is a preliminary step in software defect prediction tasks based on deep learning models. Its purpose is to transform the source code into vectors to match the model's input, while filtering out unnecessary redundant information. In this method, the representation of the software source code is unrestricted and can be achieved in various ways, such as traditional manual code metrics, text sequences, abstract syntax trees (ASTs), control flow graphs, system dependency graphs, etc. Since programming languages ​​are a special type of text with specific grammatical rules, some of their semantics are represented as assignment operations and arithmetic operators, potentially involving useless ambiguous information. Preferably, this embodiment uses an Abstract Syntax Tree (AST) to represent the source code. As a tree-like representation of the code's grammatical structure, the AST inherently contains the grammatical structure and semantic information of the programming language, independent of specific grammar and language details. It transforms the source code into a more abstract form, applicable to various programming languages, and possesses higher versatility.

[0038] Specifically, step S1 includes:

[0039] Before coding, the source code is parsed into an abstract syntax tree using a code processing toolkit, and a node stream is generated based on the syntax structure. Medium-granularity AST nodes are filtered and retained. All paths from the root node to the leaf node are extracted by traversing the abstract syntax tree, and the maximum number of paths and the number of nodes in each path are limited respectively.

[0040] Before encoding, a code processing toolkit is used to parse the source code into an AST and generate a node stream based on the syntactic structure. Considering that the granularity of node types affects the sparsity or redundancy of extracted information, this method only retains medium-granularity AST nodes. In the node stream after granularity filtering, all paths from root nodes to leaf nodes are extracted by traversing the AST. To limit the complexity of the AST, the method limits the maximum number of paths and the number of nodes in each path to L and T, respectively.

[0041] L = coe1 * max_path_len

[0042] T = coe2 * max_node_len

[0043] Where `max_path_len` is the number of paths in the maximum tree of the AST, `max_node_len` is the number of nodes in the longest path of the AST, and `coe1` and `coe2` are adjustable coefficients that limit these two lengths. Node sequences and path sequences with lengths less than the limits are padded with 0s, while node sequences exceeding the length limits are truncated at the end. Preferably, the adjustment coefficients `coe1` and `coe2` are both set to 0.9 to avoid excessive redundancy when converting the code into feature vectors, while retaining most of the effective information.

[0044] S2. Construct a sample dataset. Use the preprocessed path set of source code files of historical version projects as the training set and the preprocessed path set of source code files of new version projects as the test set. Set a label for each source code file. The label is used to indicate whether the source code file has defects.

[0045] The sample dataset includes a training set and a test set. In order to ensure that the training set and the test set have similar data distributions and that the training set can accurately represent the test set, this embodiment selects historical versions and newly released versions of multiple projects.

[0046] Specifically, step S2 includes:

[0047] The dataset from the historical version of the project code is used as the training set. The dataset includes source code, package name, software metrics, number of defects, and whether defects exist, etc., to train the software prediction model. The dataset from the new version of the project code is used as the test set. It includes source code, package name, software metrics, number of defects, and whether defects exist, etc., to evaluate the performance of the software prediction model.

[0048] Preferably, to ensure that the training set can effectively train the model, the distribution of defect categories in the training set should be reasonable. This embodiment performs random oversampling on the training dataset to increase minority class samples and address the class imbalance problem, thus ensuring the quality and reliability of the dataset.

[0049] S3. Construct a position-sensitive hierarchical attention network model. The embedding layer converts each node into a node embedding vector. The hierarchical attention layer improves upon the hierarchical attention network by changing one of the two additive attention mechanisms to a self-attention mechanism and introducing relative position encoding into this self-attention mechanism. This captures the position information of different paths in the tree structure at the path level, resulting in a position-sensitive hierarchical attention network model.

[0050] Specifically, step S3 includes:

[0051] The abstract syntax tree is divided into three different granularities: node granularity, path granularity, and abstract syntax tree granularity.

[0052] A vocabulary is constructed based on the node flow. After mapping each node to a unique index, the nodes are initialized as vectors through an embedding layer to obtain node granularity vectors.

[0053] Additive attention is used at the node level to obtain the path granularity vector. Self-attention is used at the path level, and relative position embedding is introduced into the self-attention mechanism to obtain the abstract syntax tree granularity vector. The abstract syntax tree granularity vector is used as the semantic feature of the source code.

[0054] like Figure 2 As shown in the diagram, the overall network architecture of the model method, the position-sensitive hierarchical attention network model, includes an embedding layer and a hierarchical attention layer. The hierarchical attention layer mainly comprises a node encoding layer, a path encoding layer, and a relative position encoding embedding. The input historical and new version source code is parsed to obtain corresponding abstract syntax trees. The node flow of the abstract syntax tree is then fed into the embedding layer to obtain a vector representation of each node. These representation vectors are then fed into the hierarchical attention layer, where, combined with the relative position encoding embedding, semantic features of the source code are extracted. Finally, these features are input into a binary classifier to determine if the code contains defects.

[0055] The embedding layer is used to embed nodes into node vector representations. It constructs a vocabulary based on the node flow, maps each node to a unique index, and then initializes the nodes as vectors through the embedding layer to obtain node-granular vectors.

[0056] nv it =Embedding(n it ),t∈[1,T],i∈[1,L]

[0057] Among them, nv it Let be the node granularity vector of the t-th node in the i-th path. Assuming the node vector length is vector_size, the entire AST tree is converted into an L*T*vector_size matrix, which serves as the input to subsequent node-level coding layers.

[0058] This method does not restrict any deep learning framework for any network model. In this embodiment, PyTorch open-source machine learning library is preferred because it is easy to use, flexible and efficient. It is used to quickly build and train deep learning models.

[0059] Hierarchical attention networks (HNNs), proposed in 2016, are deep learning models for natural language processing. The hierarchical attention mechanism uses the same additive attention mechanism at two levels to integrate contextual information of different granularities. This model models and classifies input text by introducing attention mechanisms at different granularities of text. A natural language-based HNN consists of three levels: word level, sentence level, and document level. At the word level, a bidirectional recurrent neural network is used to encode each word. At the sentence level, an attention mechanism is used to focus on the most important words to capture sentence-level information. At the document level, an attention mechanism is again used to focus on the most important sentences to capture document-level information. Finally, the information from all levels is integrated and fed into a fully connected layer for classification.

[0060] Preferably, this method applies an improved hierarchical attention network to programming languages, similarly dividing the source code into three different granularities: node level, path level, and abstract syntax tree (source file) level. A two-level encoding method is employed during the encoding phase, extracting semantic features from the source code at both the node and path levels. At the node level, additive attention is used to focus attention on the most important node information in the entire task, a method more aligned with human cognitive abilities.

[0061] At the node-level encoding, an additive attention mechanism is used to obtain the path-granular vector. Attention weights are calculated using a trainable feedforward neural network, and these weights are then applied to each position in the input sequence, specifically including:

[0062] A set of node granularity vectors is linearly transformed to obtain a set of context vector representations. These context vector representations are summed and activated by a nonlinear function tanh to obtain a nonlinear representation. The output of this nonlinear function is normalized and used as the node-level attention weight. The node-level attention weights are weighted and averaged with the context representations of key nodes to obtain the path granularity vector.

[0063] like Figure 3 As shown, the left side is a detailed network diagram of the node encoding layer in this invention. Specifically, for a given set of node granularity vectors generated by the embedding layer, N = [nv i1 ,nv i2 ,...,nv iTAs the input sequence, the intermediate vector representation of each node is first obtained through a Bi-GRU neural network. The Bi-GRU neural network consists of gated recurrent units in two directions, reading the input sequence in both directions and progressively passing sequence information to both time steps. Through bidirectional processing, Bi-GRU can simultaneously consider historical and future information in the sequence, thus capturing the semantic information more accurately. This is achieved by connecting positive information... and reverse information The output of yields the node context vector representation h. i,t :

[0064]

[0065]

[0066]

[0067] Bi-GRU, as a variant of RNN, inherently possesses the ability to learn positional information without requiring additional positional encoding. Subsequently, the node context vector h... i,t Using the tanh function as the activation function, directly inputting it into a single-layer perceptron yields its nonlinear representation u. i,t :

[0068] u i,t =tanh(W n h i,t +b n )

[0069] Then calculate the nonlinear representation u. i,t With hidden representation u n The similarity is used as an importance measure, and this similarity is normalized using a softmax function to obtain the node-level attention weights.

[0070]

[0071] Among them, u n They are initialized as trainable parameters and are continuously updated during the model's training process.

[0072] Finally, the context representations of all key nodes and the node-level attention weights are weighted and summed to obtain the encoded vector of the path composed of these nodes, i.e., the path granularity vector. For example, the vector representation of the i-th path in the AST is pv. i :

[0073]

[0074] Therefore, the path granularity vector pv iIt gathers information about all nodes on the i-th path.

[0075] like Figure 3 As shown, the right side is a detailed network diagram of the path encoding layer in this invention. In the path encoding layer, a similar method to the node encoding layer can be used to obtain the abstract syntax tree granular vector, thereby aggregating the information of all paths in the AST. However, in the software defect task, the number of paths L in the AST parsed from the project code is significantly greater than the number of nodes T in the path. This results in all paths forming a long sequence. Although Bi-GRU can merge contextual information from two directions, the node encoding layer method struggles to capture long-term dependencies in the long sequence because the information gradually decays during transmission.

[0076] Specifically, a self-attention mechanism is used in path-level encoding. Relative position embedding encoding is introduced into the self-attention mechanism to obtain an abstract syntax tree granular vector, which includes:

[0077] For each input path granularity vector, it is transformed into a query vector Q, a key vector K, and a value vector V. The similarity between each query vector Q and all key vectors K is calculated using the dot product method. These similarities are then converted into path-level self-attention weights using the softmax function. All value vectors V are then weighted and summed using the path-level self-attention weights to obtain a path vector that captures the relationships between other paths in the sequence.

[0078] Position-sensitive path vectors are obtained by using relative position embedding encoding instead of absolute position encoding in the path vectors. All position-sensitive path vectors are then weighted and summed to obtain the abstract syntax tree granular vector.

[0079] This embodiment proposes a position-sensitive self-attention mechanism at the path encoding level. The self-attention mechanism introduces relative position embedding encoding to capture the positional information of the path within the tree structure. The self-attention mechanism mainly includes: calculating the similarity between each input sequence element and other elements, and then using these similarities to perform a weighted summation on each element to capture the relationships between different elements in the sequence.

[0080] For a given set of path granularity vectors P = [pv1,...,pv] generated by the coding layer of a node, L The self-attention mechanism is first defined as:

[0081]

[0082] K = W K P,

[0083] Q = W Q P,

[0084] V = WV P

[0085] Where d represents the dimension of each path vector pv, P is a set of path granularity vectors, and W K W Q W V All are trainable parameter matrices, where Q is the query vector, K is the key vector, and V is the value vector.

[0086] Self-attention mechanisms allow for parallel processing of all path vectors and can more effectively capture long-term dependencies, but they do not inherently consider the position of each path. However, the position of a path in an AST often has significant semantic meaning and contributes to defect prediction. Preferably, this invention introduces relative position embedding in the self-attention layer.

[0087] In the first step of the self-attention mechanism, path-level self-attention weights The importance of path i and path j is represented by the query vector q. i ∈Q and key vector k j The dot product value of ∈K is used for measurement. In self-attention mechanisms, positional information is typically introduced using absolute positional encoding. Absolute positional encoding assigns an absolute representation to each position, embedding this absolute representation into a vector pos. This absolute position embedding vector pos is then directly added to the path vector to generate new key vector k and value vector v, thus obtaining the path-level self-attention weights.

[0088]

[0089] Where: d represents the dimension of each path vector pv, pv i and PV j Let pos represent the path vector of the i-th path and the path vector of the j-th path, respectively. i and pos j Let represent the absolute position embedding vectors of the i-th path and the j-th path, respectively. T This represents the transpose operation.

[0090] The query vector is removed from the path-level self-attention weight calculation formula, and the key vector is replaced with relative position embedding encoding to calculate the path-level self-attention weight based on relative position. Unlike absolute position encoding, the position of each path in relative position encoding is not absolutely represented; instead, the relative distance between the current position and the attention position is considered when calculating the attention score. To achieve this, this embodiment calculates the path-level self-attention weight... query vector W Q pos i Delete, and simultaneously remove the key vector W Kpos j Replace with relative position embedding encoding It is a trainable binary position vector The transformed path-level self-attention weights are used to characterize the positional relationship between the i-th and j-th paths. Therefore, the path-level self-attention weights contain relative positional information between the paths. The calculation formula is:

[0091]

[0092] In the path vector, relative position embedding encoding is used instead of absolute position encoding to compute a position-sensitive path vector. Similarly, in the second step of the self-attention mechanism, absolute position encoding obtains the value vector v by adding the absolute position embedding vector pos to the path vector. j ∈V, and the path-level self-attention weights relative to the position Perform multiplication and weighting to obtain the path vector o that captures other path relationships in the sequence. i In this embodiment, relative position embedding encoding is also used in this step. Substitute value vector W V pos j , It is also a trainable binary position vector used to represent the positional relationship between the i-th path and the j-th path, and a position-sensitive path vector o. i The calculation formula is:

[0093]

[0094] In the path encoding layer, the position-sensitive self-attention mechanism enhances the model's ability to model the path order in the AST, thereby better capturing the subtle semantics conveyed by the path order.

[0095] Finally, all critical path information is processed through the location-sensitive path vector set O = [o1,...,o...]. L We perform a weighted summation to obtain the encoding of the entire AST, called the abstract syntax tree granular vector astv.

[0096]

[0097] Therefore, the abstract syntax tree granularity vector (ASTV) aggregates information from all paths in the AST. This represents a trainable parameter matrix. The vector ASTV serves as the extracted semantic features of the source code in the software defect prediction task, and is subsequently input into a binary classifier for prediction. Preferably, this paper uses Logistic Regression as the software defect classifier and trains the model by minimizing the cross-entropy loss.

[0098] S4. Use the training sample set to train and fine-tune the position-sensitive hierarchical attention network model, and input the source code semantic features extracted from the position-sensitive hierarchical attention network model into a logistic regression classifier to obtain convergence.

[0099] Preferably, the location-sensitive hierarchical attention network model is trained using the historical version project sample dataset from step S2; wherein the training set samples include defective samples and non-defective samples, with labels 1 and 0 respectively, and the loss function is a binary classification cross-entropy function.

[0100] The goal of model optimization is to minimize the loss function, that is, to improve the model's predictive ability by minimizing the difference between the model's predicted values ​​and the true values. The basic idea is to adjust the model's internal parameters using training data so that the model can better fit the training data. Specifically, this embodiment uses the backpropagation algorithm to calculate the gradient of the loss function with respect to the model parameters for a defined loss function, and employs the Adam optimization algorithm to adaptively adjust the learning rate of each parameter. Combining the advantages of momentum gradient descent and adaptive learning rate, it can converge to the optimal solution more quickly. Simultaneously, to effectively reduce the risk of model overfitting, a Dropout layer is added after the word embedding layer, node encoding layer, and path encoding layer. In each training iteration, the output of 0.3% of neurons is randomly set to zero, thereby increasing the model's robustness.

[0101] In the binary classification stage, this embodiment selects the StepLR scheduler as the learning rate decay scheme. At every predefined 3 epochs, the learning rate is multiplied by 0.5 (gamma factor), which enables more precise adjustment of parameters in the later stage of training.

[0102] S5. Input the code of the item to be detected into the trained position-sensitive hierarchical attention network model for defect prediction, and output whether the code of the item to be detected has defects.

[0103] In this embodiment, once the position-sensitive hierarchical attention network model is trained, the project code to be detected is input into the trained position-sensitive hierarchical attention network model for defect prediction. The project source code file is automatically parsed into an abstract syntax tree, and vector embedding is performed on each node. After passing through the hierarchical attention network model, a binary classification result value is output. When the binary classification result value is 1, it indicates that the project code file is predicted to have a defect; when the binary classification result value is 0, it indicates that the project code file is predicted to have no defect.

[0104] In summary, this invention first preprocesses the project code into an AST and initializes vector embeddings. Then, it constructs a sample dataset to train and optimize the proposed new model. Finally, it uses the trained model to predict whether the new project code has defects. Because this invention extracts source code semantic features hierarchically from a finer-grained perspective, the attention mechanism and multi-level structure enable the model to better understand the structure and important information of the programming language, thus possessing high feature extraction capabilities and improving classification performance. Simultaneously, the relative position encoding introduced at the path level enhances the model's ability to process positional information in sequences, solving the long-distance path dependency problem and enabling it to better capture path regions with defects influenced by code position.

[0105] The above embodiments are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above embodiments. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A software defect prediction method based on a location-sensitive hierarchical attention network, characterized in that, Includes the following steps: S1. Source code preprocessing: The source code is parsed into an abstract syntax tree using a toolkit. The parser generates a word stream according to the syntax structure and then obtains the paths from all root nodes to leaf nodes in the abstract syntax tree by preorder traversal. S2. Construct a sample dataset. Use the preprocessed path set of source code files of historical version projects as the training set and the preprocessed path set of source code files of new version projects as the test set. Set a label for each source code file. The label is used to indicate whether the source code file has defects. S3. Construct a position-sensitive hierarchical attention network model. The embedding layer converts each node into a node embedding vector. The hierarchical attention layer improves upon the hierarchical attention network by changing one of the two additive attention mechanisms to a self-attention mechanism and introducing relative position encoding into this self-attention mechanism. This captures the position information of different paths in the tree structure at the path level, resulting in a position-sensitive hierarchical attention network model. The abstract syntax tree is divided into three different granularities: node granularity, path granularity, and abstract syntax tree granularity. A vocabulary is constructed based on the node flow. After mapping each node to a unique index, the nodes are initialized as vectors through an embedding layer to obtain node granularity vectors. Additive attention mechanism is used to obtain path granularity vector in node-level encoding, self-attention mechanism is used in path-level encoding, relative position embedding is introduced in self-attention mechanism to obtain abstract syntax tree granularity vector, and abstract syntax tree granularity vector is used as source code semantic feature. At the node level, an additive attention mechanism is used to obtain the path-granular vector, specifically including: A set of node granularity vectors is linearly transformed to obtain a set of context vector representations. These context vector representations are summed and activated by a nonlinear function tanh to obtain a nonlinear representation. The output of this nonlinear function is normalized and used as the node-level attention weight. The node-level attention weights are weighted and averaged with the context representations of key nodes to obtain the path granularity vector. A self-attention mechanism is used in path-level encoding. Relative position embedding encoding is introduced into the self-attention mechanism to obtain an abstract syntax tree granular vector, specifically including: For each input path granularity vector, transform it into a query vector. Key vector Sum value vector Each query vector is calculated using the dot product method. With all key vectors The similarity between them is transformed into path-level self-attention weights using the softmax function. Use path-level self-attention weights on all value vectors. By performing a weighted summation, we obtain a path vector that captures the relationships between other paths in the sequence; Path-level self-attention weights The query vector in the calculation formula is deleted, and the key vector is replaced with a relative position embedding code. The path-level self-attention weight of the relative position is then calculated. Position-sensitive path vectors are obtained by using relative position embedding encoding instead of absolute position encoding in the path vector. The abstract syntax tree granular vector is obtained by weighted summation of all position-sensitive path vectors. ; The path-level self-attention weights The calculation formula is: ； in, Represents each path vector Dimensions , All of these are trainable parameter matrices. and Let represent the path vector of the i-th path and the path vector of the j-th path, respectively. and Let represent the absolute position embedding vectors of the i-th path and the j-th path, respectively. This represents the transpose operation; The path-level self-attention weights of the relative positions The calculation formula is: ， in, Encode trainable relative position embeddings; The location-sensitive path vector The calculation formula is: ； in, For a trainable parameter matrix, Encode trainable relative position embeddings; The abstract syntax tree granularity vector The calculation formula is: ； in, is a trainable parameter matrix; S4. Use the training sample set to train and optimize the position-sensitive hierarchical attention network model, and input the source code semantic features extracted from the position-sensitive hierarchical attention network model into a logistic regression classifier to obtain convergence. S5. Input the code of the item to be detected into the trained position-sensitive hierarchical attention network model for defect prediction, and output whether the code of the item to be detected has defects.

2. The software defect prediction method based on a location-sensitive hierarchical attention network as described in claim 1, characterized in that, Step S1 specifically includes: Before coding, the source code is parsed into an abstract syntax tree using a code processing toolkit, and a node stream is generated based on the syntax structure. All paths from the root node to the leaf node are extracted by traversing the abstract syntax tree, and the maximum number of paths and the number of nodes in each path are limited to L and T, respectively. ， ， in, The maximum number of paths in the AST (Extreme Tree Syntax). This represents the number of nodes in the longest path in the AST. and These are the adjustable coefficients that limit these two lengths, respectively.

3. The software defect prediction method based on a location-sensitive hierarchical attention network as described in claim 2, characterized in that, The Adjustable coefficient and All adjustable coefficients are set to 0.

9.

4. The software defect prediction method based on a location-sensitive hierarchical attention network as described in claim 1, characterized in that, Step S2 specifically includes: The dataset from the historical version of the project code is used as the training set to train the software prediction model; the dataset from the new version of the project code is used as the test set to evaluate the performance of the software prediction model. Perform random oversampling on the training set to increase the number of minority class samples.

5. The software defect prediction method based on a position-sensitive hierarchical attention network as described in claim 1, characterized in that, Step S4 specifically includes: The location-sensitive hierarchical attention network model was trained using a historical version of the project sample dataset. The training set samples included defective and non-defective samples, labeled 1 and 0, respectively. The loss function used was the binary cross-entropy function. The gradient of the loss function with respect to the model parameters was calculated using the backpropagation algorithm. The learning rate of each parameter was adaptively adjusted using the Adam optimization algorithm. A Dropout layer is added after the word embedding layer, node encoding layer, and path encoding layer. In each training iteration, the output of neurons with a ratio of 0.3 is randomly set to zero. In the binary classification stage, the StepLR scheduler is selected as the learning rate decay scheme, and the learning rate is multiplied by 0.5 every 3 predefined epochs.

6. The software defect prediction method based on a location-sensitive hierarchical attention network as described in claim 1, characterized in that, Step S5 specifically includes: The project code to be detected is input into the trained position-sensitive hierarchical attention network model for defect prediction. The project source code file is parsed into an abstract syntax tree, and vector embedding is performed on each node. The hierarchical attention network model outputs a binary classification result value. When the binary classification result value is 1, it means that the project code file is predicted to have a defect. When the binary classification result value is 0, it means that the project code file is predicted to have no defect.

Images