Software row-level defect prediction method based on hierarchical attention mechanism

Abstract

The invention discloses a software row-level defect prediction method based on a hierarchical attention mechanism, and the method comprises the steps: converting each row of statements in a source code file into vector representation through employing a CodeBERT pre-training model, so as to obtain the semantic information of a code row; extracting a program dependency graph of the source code, recording a mapping relation between each node in the program dependency graph and a source code line, and calculating a network measurement index corresponding to each node; fusing the semantic vectors of the code lines and the network measurement indexes to generate fused node features so as to construct a program dependency graph after the node features are expanded; constructing a HAGLineDP model, wherein the HAGLineDP model comprises a graph feature extraction network and a dual-task classification network; the graph feature extraction network designs three-level feature evolution paths from local to global and from structure to semantic, and sequentially comprises a local structure aggregation layer, a structure generalization enhancement layer and a semantic importance weighting layer; the dual-task classification network comprises a node classification path and a graph classification path; and training the HAGLineDP model by using a joint loss function and an AdamW optimizer, outputting the defect probability of each code line for a to-be-predicted source code file through a node classification path, and sorting according to the defect probability to locate a high-risk code line. According to the method, fine positioning of code defects can be effectively completed, and efficient distribution and utilization of software testing resources are promoted.

Description

Technical Field

[0001] This invention relates to software engineering technology, specifically to a hierarchical attention mechanism (…). A software line-level defect prediction method. Background Technology

[0002] Software defects can lead to various forms of loss, including economic losses, reputational damage, security risks, and threats to personal safety, causing serious negative impacts on individuals, businesses, governments, and society. In recent years, security incidents caused by software defects have been numerous. For example, the 2017 Equifax data breach was caused by the software's failure to apply security patches to fix the Apache Struts vulnerability (CVE-2017-5638) in a timely manner, allowing attackers to exploit this vulnerability to compromise the system and resulting in the leakage of personal information of 143 million American consumers. In 2020, Zoom's user growth surge led to a large-scale leak of user videos on the platform, raising public concerns about Zoom's privacy protection capabilities.

[0003] With the rapid development of machine learning and deep learning technologies, the field of software defect prediction has received unprecedented impetus. These modern artificial intelligence technologies have changed our perspective and methods for software quality management. In the past, the identification of software defects mainly relied on manual review and testing, methods that were both time-consuming and error-prone. However, the emergence of deep learning algorithms has made it possible to automatically identify defect patterns from large amounts of historical data, greatly improving the efficiency and reliability of software development. However, previous research has mainly suffered from two limitations.

[0004] (1) The prediction granularity of most current methods is still coarse-grained. In recent years, the prediction granularity of most methods has remained at the file level, which is clearly not fine enough. After all, not all lines of code in a defective file are defective. Statistics show that only 1-3% of the lines of code in a file are defective, which means that when a source code file is predicted to be defective, developers still need to spend a lot of time and effort to locate the defective lines of code. However, line-level defect prediction can quickly help developers locate high-risk lines of code, thereby reducing the cost of software defect repair.

[0005] (2) Insufficient extraction of contextual information from defective lines. Traditional defect prediction models are mostly based on statistical features, and then use typical machine learning algorithms (e.g., support vector machines, logistic regression, or decision trees) to train the prediction model. These methods are good at handling structured data, such as the number of lines of code, module complexity, Halstead features based on operator and operand counts, CK features based on function and inheritance counts, etc. This surface information can only roughly reflect the statistical measures of the code itself and does not involve semantic information. This leads to the simplification or neglect of contextual information, even though it exists. Semantic information helps the prediction model understand the specific logic and intent of code execution, rather than just analyzing the external form of the code (e.g., the number of lines of code, complexity, etc.). By analyzing the semantic information of the code, the model can more accurately analyze the actual execution path and identify the risk points of the code, because these factors may directly affect the occurrence of defects. For example, problems such as incorrect control flow and improper resource handling can usually only be captured through semantic information. Summary of the Invention

[0006] The purpose of this invention is to propose a software line-level defect prediction method based on a hierarchical attention mechanism to solve the problems of existing defect prediction methods having coarse granularity, difficulty in accurately locating high-risk code statements, and insufficient extraction of defect line context information, resulting in superficial code semantic analysis.

[0007] The technical solution for achieving the present invention is: a software row-level defect prediction method based on a hierarchical attention mechanism, characterized by the following steps:

[0008] Step 1: Use the CodeBERT pre-trained model to convert each line of statements in the source code file into a vector representation in order to obtain the semantic information of the code line;

[0009] Step 2: Extract the program dependency graph from the source code, record the mapping relationship between each node in the program dependency graph and the source code line, and calculate the network metric corresponding to each node.

[0010] Step 3: Merge the semantic vectors of lines of code with network metrics to generate fused node features, in order to construct a program dependency graph with expanded node features;

[0011] Step 4: Construct the HAGLineDP model. The HAGLineDP model includes a graph feature extraction network and a dual-task classification network. The graph feature extraction network is designed with a three-level feature evolution path from local to global and from structure to semantics, including a local structure aggregation layer, a structure generalization enhancement layer, and a semantic importance weighting layer, which are used to perform hierarchical feature extraction and aggregation on the expanded program dependency graph. The dual-task classification network includes a node classification path and a graph classification path, which are used to perform defect probability prediction at the line-of-code level and defect classification prediction at the source code file level based on the output of the graph feature extraction network.

[0012] Step 5: Train the HAGLineDP model using the joint loss function and the AdamW optimizer. The joint loss function consists of node classification loss and graph classification loss. After the model is trained, for the source code file to be predicted, output the defect probability of each line of code through the node classification path, and sort them according to the defect probability to locate high-risk lines of code.

[0013] Further, in step 1, the CodeBERT pre-trained model is used to convert each line of statements in the source code file into a vector representation to obtain the semantic information of the code line. The specific method is as follows:

[0014] The code file organization is described using a sequential structure, where each code file is represented as a series of lines of code, denoted as < >

[0015] The CodeBERT pre-trained model is used to convert each line of code into a 768-dimensional vector representation.

[0016] Further, in step 2, the program dependency graph of the source code is extracted, the mapping relationship between each node in the program dependency graph and the source code line is recorded, and the network metric corresponding to each node is calculated. The specific method is as follows:

[0017] use The tool analyzes source code files and generates an initial program dependency graph containing data flow and control flow information;

[0018] Extract the names of each node in the initial program dependency graph and their corresponding source code line numbers, and establish a mapping relationship between nodes and code lines;

[0019] Based on social network analysis methods, the computational program relies on network metrics for each node in the graph. There are 16 network metrics in total, including Size, Ties, Pairs, Density, Weak Comp, Normalized Weak Comp, 2Step Reach, Reach Efficacy, Brokerage, Normalized Brokerage, Ego Between, Normalized Ego Between, Eigenvector Centrality (EC), Close Centrality (CC), Betweenness Centrality (BC), and Degree.

[0020] Further, in step 3, the semantic vectors of code lines are fused with network metrics to generate fused node features, which are then used to construct a program dependency graph with expanded node features. The specific method is as follows:

[0021] The semantic vectors of code lines and network metrics are mapped to a unified 32-dimensional space through fully connected layers, and then concatenated and fused to form the fused node features;

[0022] Based on the fused node features and the topology of the program dependency graph, an extended program dependency graph is constructed.

[0023] Further, in step 4, the HAGLineDP model is constructed, specifically as follows:

[0024] (1) Graph Feature Extraction Network

[0025] Local structure aggregation layer: This layer aggregates the direct neighbor information of nodes for nodes within tightly packed control and data flow information between code statements. Its output characteristics Represented as:

[0026]

[0027] in It is a learnable weight matrix. Neighboring nodes Features Let N(i) be the degree of a node, and N(i) be the set of neighbors of node i. For activation functions;

[0028] Structural generalization enhancement layer: For sparse connection regions in the program dependency graph, the output features of the local structure aggregation layer are sampled and aggregated to expand the receptive field of the nodes and enhance the local structural representation of the nodes.

[0029] Semantic Importance Weighted Layer: Addressing the characteristic that code defects are often triggered by a few statements on the critical path, an attention mechanism is introduced to dynamically learn the dependency weights between nodes. Attention coefficients are used to identify the influence weights of key code lines on defect formation, solving the problem of insufficient long-distance semantic dependency capture. and his neighbors , No. A person's attention The attention coefficient is calculated as follows:

[0030]

[0031] in , Representing the target node respectively and neighboring nodes The feature vector output by the structural generalization enhancement layer Indicates the first A learnable weight matrix shared by each attention head. This represents the transpose of the learnable attention weight vector under the k-th attention head. For normalization function, It is a non-linear activation function. This represents the dot product operation. This indicates a concatenation operation, where the outputs of multiple attention heads are concatenated and then integrated through a single-head attention layer to obtain the final node embedding vector.

[0032] (2) Dual-task classification network

[0033] Node classification path: Through a linear layer, the final node embedding vector is independently mapped to the defect prediction logical value corresponding to each node;

[0034] Graph classification path: First, a global attention pooling layer is used to weight and aggregate the final node embedding vector to obtain a graph-level representation vector, and then another linear layer is used to map it to a file-level defect prediction logical value.

[0035] The logical values ​​output by the node classification path and the graph classification path are converted into defect probabilities using the softmax function.

[0036] Further, in step 5, the HAGLineDP model is trained using the joint loss function and the AdamW optimizer. Both the node classification loss and the graph classification loss are calculated using the cross-entropy loss function with class weights. The joint loss function is expressed as follows: During model training, the ReduceLROnPlateau strategy is used to dynamically adjust the learning rate, and gradient clipping is used to prevent gradient explosion.

[0037] Furthermore, the method also includes: after the model training is completed, for source code files classified as defective, sorting them in descending order based on the defect probabilities of each node output by the node classification path to generate a sorted list of high-risk code lines in the file.

[0038] A software row-level defect prediction system based on a hierarchical attention mechanism includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the software row-level defect prediction method based on the hierarchical attention mechanism.

[0039] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the software row-level defect prediction method based on a hierarchical attention mechanism.

[0040] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the software row-level defect prediction method based on a hierarchical attention mechanism.

[0041] Compared with the prior art, the significant advantages of this invention are: 1) it utilizes a pre-trained model Learning the semantic information of code lines solves the problem that most existing methods predict at the file level and cannot accurately locate defective code lines. 2) By analyzing the program dependency graph ( Social network analysis ), using graph network metrics to extend the semantics of a single line of code, which solves the problem of insufficient extraction of contextual information of defective lines; 3) considering the characteristics of line-level and file-level code semantics, it can not only enhance the prediction accuracy, but also achieve simultaneous output of multi-task prediction results.

[0042] The present invention will now be described in further detail with reference to the accompanying drawings. Attached Figure Description

[0043] Figure 1 This is a flowchart illustrating a software row-level defect prediction method based on a hierarchical attention mechanism.

[0044] Figure 2 Build a flowchart for the extended program dependency graph.

[0045] Figure 3 for Model structure diagram.

[0046] Figure 4 This is a schematic diagram of the model training process. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0048] like Figure 1 As shown, a software row-level defect prediction method based on a hierarchical attention mechanism includes the following steps:

[0049] Step 1: Use The pre-trained model converts each line of statements in the source code file into a vector representation to obtain the semantic information of the code line.

[0050] Step 1-1: Using the... A defect prediction dataset was constructed, covering 7 open-source projects and 24 software versions. This dataset includes projects of varying sizes and defect rates, with file counts ranging from 757 to 8,846 and code lines ranging from 74K to 567K. File defect rates range from 2% to 19%, and code line defect rates range from 0.05% to 2.90%.

[0051] Steps 1-2: Use a sequential structure to describe the organization of code files. Each code file is represented as a series of lines of code, denoted as < >

[0052] Steps 1-3: Using pre-trained models This converts line statements in the source code into a 768-dimensional vector representation to obtain the semantic information of the code.

[0053] Step 2: Extract the program dependency graph of the source code, record the mapping relationship between each node in the program dependency graph and the source code line, and calculate the network metric corresponding to each node, as shown in Table 1.

[0054] Step 2-1: Use The tool analyzes the source code files to obtain an initial program dependency graph containing data flow and control flow information;

[0055] Step 2-2: For each program dependency graph, extract the node name and corresponding source code line number by finding each edge, and finally associate and map this structured information with the original code line data;

[0056] Steps 2-3: Analyze each node of the initial program dependency graph using social network analysis methods to obtain 16 network metrics for each node. These metrics are used to quickly understand which node in the network is more important and has a wider impact. Social network analysis uses network and graph theory methods to study social structures, characterizing network structure features through nodes and the edges connecting them. In the field of software engineering, SNA (Social Network Analysis) is widely used to model communication and collaboration among developers, and also to study the architecture of technical information (such as program dependency graphs), thus providing key information for software development projects. Specific metrics are shown in the table below:

[0057] Table 1. Description of Network Metrics

[0058]

[0059] Step 3: Merge the semantic vectors of code lines with network metrics to generate fused node features, thereby constructing a program dependency graph with expanded node features, such as... Figure 2 As shown:

[0060] Step 3-1: Use a fully connected layer to transform the dimensions of the network metrics from Step 2 and the line-of-speech semantic vector from Step 1, ultimately forming two 32-dimensional vectors, denoted as... .

[0061]

[0062]

[0063] Step 3-2: Fuse the two 32-dimensional features by concatenation to generate a larger tensor. This increases the number of channels, or feature dimension, in deep learning models, allowing for better capture of relationships between different features in subsequent network layers and obtaining multi-dimensional feature representations. The calculation formula is as follows, where || represents concatenation:

[0064]

[0065] Step 3-3: Based on the row numbers in Step 2-2, convert the above... At each node of the corresponding program dependency graph, an expanded new program dependency graph is formed.

[0066] Steps 3-4: Divide the training set and validation set into an 8:2 ratio. Our experimental method is to test on the last version of each project and use the remaining versions as the training set.

[0067] Step 4: Build Model; such as Figure 3 As shown, the The model includes a graph feature extraction network and a dual-task classification network. The graph feature extraction network is designed with a three-level feature evolution path from local to global and from structure to semantics, which includes a local structure aggregation layer, a structure generalization enhancement layer, and a semantic importance weighting layer, used to perform hierarchical feature extraction and aggregation on the expanded program dependency graph. The dual-task classification network includes a node classification path and a graph classification path, used to perform defect probability prediction at the line-of-code level and defect classification prediction at the source code file level based on the output of the graph feature extraction network.

[0068] Step 4-1: First, from the expanded program dependency graph ( The feature vectors and connection relationships of each node are extracted and used as input data for subsequent model training. To fully consider the semantic information of each node and its surrounding code structure, a local structure aggregation layer is designed to efficiently learn features and process information on the expanded program dependency graph. By fusing the neighborhood information of nodes in the model, not only can the representational power of the nodes themselves be improved, but the relationships and contextual logic between codes can also be reflected more comprehensively. For each node... When performing feature updates, we aggregate the information of its surrounding neighboring nodes to ultimately generate the underlying representation of that node. The specific calculation method is as follows:

[0069]

[0070] in It is a node Updated features It is a node The set of neighboring nodes, From node To the node The edge weight (default is 1). It is a learnable weight matrix. It is a learnable bias term. Neighboring nodes Input features, It is a node The normalization degree is calculated as follows:

[0071]

[0072] Step 4-2: Because program dependency graphs often exhibit significant sparsity when handling global calls or cross-functional dependencies, traditional local convolutions are prone to information loss. This step focuses on output features. Building upon this foundation, a structural generalization enhancement layer based on a sampling mechanism is superimposed. This layer, while preserving the core features of the nodes themselves, employs a mean aggregation function to capture contextual information within a broader receptive field, thereby simultaneously obtaining both local details and global structural features. The specific calculation process is as follows:

[0073]

[0074] It is a node Input features, It is a node The output characteristics, It is a node The neighborhood group, , It is a learnable weight matrix, and mean is the mean aggregation function.

[0075] Step 4-3: Multi-head attention first extracts features from different representation subspaces, with each "head" focusing on a different aspect of the input. Increasing the number of heads expands the model's capacity, providing a breadth-based expansion. Subsequent self-attention layers can globally model these multi-head processed features, offering a depth-based extension to enhance the model's non-linear representation capabilities and uncover deeper correlations between them. Combining these two approaches allows for the learning of more complex sequence internal structures and long-range dependencies. Traditional recurrent neural networks suffer from long-range dependency problems, while the self-attention mechanism can directly establish connections between tokens at arbitrary distances, unrestricted by sequence distance, thus making it easier to capture long-range dependencies. The specific calculation process is shown below:

[0076] First, calculate the neighbor nodes from different attention head perspectives. For the central node The original relevance score is calculated in the same way for each attention head. The calculation method for the h-th attention head is as follows:

[0077]

[0078] in Output features from the previous layer, It is a linear transformation matrix that maps input features to the attention space. || represents the concatenation operation. It is a learnable weight vector used to compress the concatenated high-dimensional vector into a scalar value. It is a nonlinear activation function used to introduce nonlinearity and allow negative gradient propagation.

[0079] Next, we use the Softmax function to convert the original scores above. Transform it into a probability distribution (weights) to ensure that the sum of the contributions of all neighbors to node i is 1. The larger the value, the more important neighbor j is to node i in the h-th feature subspace. The calculation formula is as follows:

[0080]

[0081] Then, the above weighted summation process is executed in parallel nheads times, and the results are concatenated, as shown in the following formula:

[0082]

[0083] Finally, to integrate the features of the multi-head subspace and unify the output dimension, a single-head self-attention layer is introduced as the output fusion layer. This layer processes the concatenated multi-head features. Weighted aggregation is performed to obtain the final node embedding vector. The calculation is as follows:

[0084]

[0085] in, This indicates that neighbor node j is related to the current node. The weight value indicates the importance or contribution of the neighboring node. The higher the weight value, the more important the information provided by that neighboring node.

[0086] Step 4-4: After the above steps, we have obtained the features of each node. To obtain the file's features, we perform global summation pooling on all node features and finally predict the file's defects through a fully connected layer. For line-level predictions, the node features are directly input into the fully connected layer for calculation, and then... The layer obtains the probability that each row has a defect.

[0087] Steps 4-5: Save the trained model and use it to predict test files. This will provide us with the binary classification results of the files and the defect probability of each line of code. By sorting the defect probabilities of all lines of code, we have achieved defect prediction for each line of code.

[0088] Step 5: Train the HAGLineDP model using a joint loss function and the AdamW optimizer; the joint loss function consists of node classification loss and graph classification loss; after the model is trained, for the source code file to be predicted, output the defect probability of each line of code through node classification path, and sort them according to the defect probability, such as... Figure 4 As shown, this is used to locate high-risk lines of code.

[0089] Step 5-1: Load the preprocessed graph data in batches, including node features, adjacency relationships, and defect labels at the line-of-code and file-level. For each batch of data, the model... Forward propagation is performed, generating both node-level and graph-level predictions. To address multi-task learning objectives and class imbalance, a cross-entropy loss function with class weights is used to calculate the loss. The total loss consists of node classification losses. And graph classification loss Composition. The weighted cross-entropy loss function is calculated as shown in the formula:

[0090]

[0091] in, It is the sample size. It is an indicator function (if the sample) The true category is The value is 1 if it is 1, otherwise it is 0. The model predicts the sample. Category The probability, This is a category The assigned weights are used to penalize misclassifications of the minority class. The total loss function is the sum of the node loss and the graph loss, calculated using the formula:

[0092]

[0093] Step 5-2: Use The optimizer updates the network parameters based on the gradient of the composite loss. To improve training stability and convergence, a dynamic learning rate adjustment strategy is employed. After each training epoch, if the validation set loss does not decrease within a specified patience period, the learning rate is reduced by a preset percentage. Furthermore, gradient clipping is introduced during training to limit the norm of the gradient to within a threshold of 5, preventing gradient explosion.

[0094] This invention also proposes a software row-level defect prediction system based on a hierarchical attention mechanism, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the software row-level defect prediction method based on the hierarchical attention mechanism.

[0095] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the software row-level defect prediction method based on a hierarchical attention mechanism.

[0096] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the software row-level defect prediction method based on a hierarchical attention mechanism.

[0097] In summary, this invention proposes a framework based on a hierarchical attention mechanism. This enables software defect prediction at the line-of-code level, helping developers pinpoint higher-risk lines of code.

[0098] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0099] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A software line-level defect prediction method based on a hierarchical attention mechanism, characterized in that, The steps are as follows: Step 1: Use the CodeBERT pre-trained model to convert each line of statements in the source code file into a vector representation in order to obtain the semantic information of the code line; Step 2: Extract the program dependency graph from the source code, record the mapping relationship between each node in the program dependency graph and the source code line, and calculate the network metric corresponding to each node. Step 3: Merge the semantic vectors of code lines with network metrics to generate fused node features, in order to construct a program dependency graph with expanded node features; Step 4: Construct the HAGLineDP model. The HAGLineDP model includes a graph feature extraction network and a dual-task classification network. The graph feature extraction network is designed with a three-level feature evolution path from local to global and from structure to semantics, including a local structure aggregation layer, a structure generalization enhancement layer, and a semantic importance weighting layer, which are used to perform hierarchical feature extraction and aggregation on the expanded program dependency graph. The dual-task classification network includes a node classification path and a graph classification path, which are used to perform defect probability prediction at the line-of-code level and defect classification prediction at the source code file level based on the output of the graph feature extraction network. Step 5: Train the HAGLineDP model using the joint loss function and the AdamW optimizer. The joint loss function consists of node classification loss and graph classification loss. After the model is trained, for the source code file to be predicted, output the defect probability of each line of code through the node classification path, and sort them according to the defect probability to locate high-risk lines of code.

2. The software row-level defect prediction method based on hierarchical attention mechanism according to claim 1, characterized in that, Step 1: Using the CodeBERT pre-trained model, each line of code in the source code file is converted into a vector representation to obtain the semantic information of the code line. The specific method is as follows: The code file organization is described using a sequential structure, where each code file is represented as a series of lines of code, denoted as < > The CodeBERT pre-trained model is used to convert each line of code into a 768-dimensional vector representation.

3. The software row-level defect prediction method based on hierarchical attention mechanism according to claim 1, characterized in that, Step 2: Extract the program dependency graph from the source code, record the mapping relationship between each node in the program dependency graph and the source code line, and calculate the network metric corresponding to each node. The specific method is as follows: use The tool analyzes source code files and generates an initial program dependency graph containing data flow and control flow information; Extract the names of each node in the initial program dependency graph and their corresponding source code line numbers, and establish a mapping relationship between nodes and code lines; Based on social network analysis methods, the computational program relies on network metrics for each node in the graph. There are 16 network metrics in total, including Size, Ties, Pairs, Density, Weak Comp, Normalized Weak Comp, 2Step Reach, Reach Efficacy, Brokerage, Normalized Brokerage, Ego Between, Normalized Ego Between, Eigenvector Centrality (EC), Close Centrality (CC), Betweenness Centrality (BC), and Degree.

4. The software row-level defect prediction method based on hierarchical attention mechanism according to claim 1, characterized in that, Step 3: Merge the semantic vectors of code lines with network metrics to generate fused node features, thereby constructing a program dependency graph with expanded node features. The specific method is as follows: The semantic vectors of code lines and network metrics are mapped to a unified 32-dimensional space through fully connected layers, and then concatenated and fused to form the fused node features; Based on the fused node features and the topology of the program dependency graph, an extended program dependency graph is constructed.

5. The software row-level defect prediction method based on segmentation graph attention according to claim 1, characterized in that, Step 4, construct the HAGLineDP model, the specific method is as follows: (1) Graph Feature Extraction Network Local structure aggregation layer: This layer aggregates the direct neighbor information of nodes for nodes within tightly packed control and data flow information between code statements. Its output characteristics Represented as: ； in It is a learnable weight matrix. Neighboring nodes Features Let N(i) be the degree of a node, and N(i) be the set of neighbors of node i. For activation functions; Structural generalization enhancement layer: For sparse connection regions in the program dependency graph, the output features of the local structure aggregation layer are sampled and aggregated to expand the receptive field of the nodes and enhance the local structural representation of the nodes. Semantic Importance Weighted Layer: Addressing the characteristic that code defects are often triggered by a few statements on the critical path, an attention mechanism is introduced to dynamically learn the dependency weights between nodes. Attention coefficients are used to identify the influence weights of key code lines on defect formation, solving the problem of insufficient long-distance semantic dependency capture. and his neighbors , No. A person's attention The attention coefficient is calculated as follows: ； in , Representing the target node respectively and neighboring nodes The feature vector output by the structural generalization enhancement layer Indicates the first A learnable weight matrix shared by each attention head. This represents the transpose of the learnable attention weight vector under the k-th attention head. For normalization function, It is a non-linear activation function. This represents the dot product operation. This indicates a concatenation operation, where the outputs of multiple attention heads are concatenated and then integrated through a single-head attention layer to obtain the final node embedding vector. (2) Dual-task classification network Node classification path: Through a linear layer, the final node embedding vector is independently mapped to the defect prediction logical value corresponding to each node; Graph classification path: First, a global attention pooling layer is used to weight and aggregate the final node embedding vector to obtain a graph-level representation vector, and then another linear layer is used to map it to a file-level defect prediction logical value. The logical values ​​output by the node classification path and the graph classification path are converted into defect probabilities using the softmax function.

6. The software row-level defect prediction method based on hierarchical attention mechanism according to claim 1, characterized in that, Step 5: Train the HAGLineDP model using the joint loss function and the AdamW optimizer. Both the node classification loss and the graph classification loss are calculated using the cross-entropy loss function with class weights. The joint loss function is expressed as follows: During model training, the ReduceLROnPlateau strategy is used to dynamically adjust the learning rate, and gradient clipping is used to prevent gradient explosion.

7. The software row-level defect prediction method based on hierarchical attention mechanism according to claim 1, characterized in that, Also includes: After the model is trained, for source code files classified as defective, the defect probabilities of each node output by the node classification path are sorted in descending order to generate a sorted list of high-risk code lines in the file.

8. A software row-level defect prediction system based on a hierarchical attention mechanism, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the software row-level defect prediction method based on a hierarchical attention mechanism as described in any one of claims 1-7.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the software row-level defect prediction method based on a hierarchical attention mechanism as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, wherein when executed by a processor, the computer program implements the software row-level defect prediction method based on a hierarchical attention mechanism as described in any one of claims 1-7.

Images