Vulnerability detection method and system based on semantic sensitive contrast learning and graph representation

Abstract

The invention discloses a vulnerability detection method and system based on semantic sensitive contrast learning and graph representation, and belongs to the technical field of vulnerability detection.The method comprises the steps that source codes and transformed codes are input into a vulnerability detection model to be processed, and a vulnerability classification result is obtained; the vulnerability detection model comprises a semantic learning module, a graph representation learning module, a feature fusion module and a support vector machine; wherein the construction process of the vulnerability detection model comprises the following steps: taking a source code and a transformed code as a positive sample pair for comparative learning to obtain code semantic feature embedding; the source code is represented as a graph structure, the graph structure is processed, and then code structure feature embedding is obtained; and performing feature fusion on the code semantic feature embedding and the code structure feature embedding to obtain fusion vectors, and classifying the fusion vectors to obtain a vulnerability classification result. The model is guided to learn semantics related to vulnerabilities through comparative learning, and meanwhile, the accuracy of vulnerability detection is improved in combination with grammatical structure features of codes.

Description

Technical Field

[0001] The present invention belongs to the technical field of vulnerability detection, and in particular relates to a vulnerability detection method and system based on semantically sensitive contrastive learning and graph representation. Background Art

[0002] The statements in this section merely provide background information related to the present disclosure and do not necessarily constitute prior art.

[0003] Software vulnerabilities have become a major threat to information security, leading to numerous cyberattacks and data breaches. With the increasing complexity of software development, the number and variety of vulnerabilities are growing exponentially. In particular, the frequent occurrence of vulnerabilities in certain core software systems poses a serious threat to the stability of the entire software and network ecosystem. Therefore, improving the accuracy and applicability of vulnerability detection technology has become a key priority for ensuring software security.

[0004] Vulnerability detection technology has garnered widespread attention in recent years, with researchers continuously exploring more efficient detection methods. Deep learning has become a key area of ​​focus in software vulnerability detection, primarily due to its high degree of automation and ability to identify unknown vulnerabilities. However, many current deep learning-based vulnerability detection methods are limited by the black-box nature of their training process, which often lacks a clearly defined modeling objective for vulnerability characteristics. Consequently, during the learning phase, these models struggle to effectively identify and focus on key semantic information. Furthermore, current vulnerability detection methods often rely on global context modeling, failing to fully capture local structural features within the source code. While global information helps understand the overall semantics of the code, local dependencies (such as control flow and data flow) are equally important for identifying subtle and high-risk vulnerability patterns. Due to the lack of adequate modeling of these fine-grained structures, models can easily miss key features when encountering complex or subtle vulnerabilities, resulting in reduced detection accuracy. Summary of the Invention

[0005] To overcome the shortcomings of the above-mentioned prior art, the present invention provides a vulnerability detection method and system based on semantically sensitive contrastive learning and graph representation. By contrastive learning, the model is guided to learn vulnerability-related semantics, and the grammatical structure characteristics of the code are combined to improve the accuracy of vulnerability detection.

[0006] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: In a first aspect, the present invention provides a vulnerability detection method based on semantically sensitive contrastive learning and graph representation, comprising: Obtaining source code to be tested, normalizing and transforming the source code to obtain transformed code; Inputting the source code and the transformed code into a vulnerability detection model for processing to obtain a vulnerability classification result; the vulnerability detection model includes a semantic learning module, a graph representation learning module, a feature fusion module and a support vector machine; The process of building the vulnerability detection model includes: The source code and the transformed code are used as positive sample pairs for comparative learning to obtain the code semantic feature embedding; Representing the source code as a graph structure, processing the graph structure, outputting node representations, and thereby obtaining code structure feature embedding; The code semantic feature embedding and the code structural feature embedding are fused to obtain a fusion vector, and the fusion vector is classified to obtain a vulnerability classification result.

[0007] A further technical solution is to transform the source code by generating an abstract syntax tree from the source code and performing structural conversion using the abstract syntax tree.

[0008] A further technical solution is that the semantic learning module includes a parallel query encoder and a momentum encoder. The query encoder and the momentum encoder have the same structure and are composed of multiple self-attention layers. Each layer includes a multi-head self-attention mechanism, residual connection and layer normalization, a feedforward network, residual connection and layer normalization in sequence.

[0009] A further technical solution is to encode the query samples, positive samples and negative sample queues through a programming language model to obtain the query sample vector representation, positive sample vector representation and negative sample queue vector representation; the query sample vector representation is input into the query encoder, and the positive sample vector representation and negative sample queue vector representation are input into the momentum encoder, and the code semantic feature embedding is output.

[0010] A further technical solution uses a token-level co-occurrence graph construction method to represent the source code as a graph structure. Specifically, tokens in the source code are extracted and regarded as nodes in the graph; a sliding window mechanism is used to construct token relationships, and edges are established between tokens appearing in the same window to obtain a graph structure.

[0011] In a further technical solution, the graph representation learning module includes a multi-layer graph convolutional neural network that introduces residual connections.

[0012] A further technical solution is to obtain the code structure feature embedding by converting the node representation into a new node representation, performing a mixed operation of sum pooling and maximum pooling on the new node representations of all nodes to obtain the code structure feature embedding.

[0013] In a second aspect, the present invention provides a vulnerability detection system based on semantically sensitive contrastive learning and graph representation, comprising: A data acquisition module is configured to: acquire source code to be detected, normalize and transform the source code to obtain transformed code; a model detection module configured to: input the source code and the transformed code into a vulnerability detection model for processing to obtain a vulnerability classification result; the vulnerability detection model includes a semantic learning module, a graph representation learning module, a feature fusion module, and a support vector machine; The process of building the vulnerability detection model includes: The source code and the transformed code are used as positive sample pairs for comparative learning to obtain the code semantic feature embedding; Representing the source code as a graph structure, processing the graph structure, outputting node representations, and thereby obtaining code structure feature embedding; The code semantic feature embedding and the code structural feature embedding are fused to obtain a fusion vector, and the fusion vector is classified to obtain a vulnerability classification result.

[0014] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the vulnerability detection method based on semantically sensitive contrastive learning and graph representation as described in the first aspect.

[0015] In a fourth aspect, the present invention provides a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the vulnerability detection method based on semantically sensitive contrastive learning and graph representation as described in the first aspect are implemented.

[0016] One or more of the above technical solutions have the following beneficial effects: This paper employs contrastive learning to guide the model's learning of vulnerability semantics and uses token graphs and GCNs to capture the structural features of code. Specifically, we first design seven code transformation methods for code samples based on different vulnerability semantics. The transformed samples retain the same semantics as the original samples. By combining these transformed samples with a contrastive learning model, we guide the neural network to capture the core semantics of the vulnerability, thereby reducing the influence of superficial features in the training set. We then introduce a graph-based representation method that uses token-level co-occurrence graphs and GCNs to learn local structural features of the code. During GCN training, we incorporate residual connections and employ sum and max pooling strategies to aggregate node-level features into a comprehensive global representation of the code. Finally, we fuse these two types of features to learn their complementary information, aiming to provide a more discriminative representation for vulnerability detection. We then use a support vector machine for vulnerability classification. This method outperforms baseline models on three benchmark datasets (FFMPeg+Qemu, Reveal, and Big_vul), demonstrating excellent performance in vulnerability detection tasks.

[0017] The present invention addresses the problems of inaccurate vulnerability feature modeling and difficulty in effectively identifying key semantic information caused by the black-box characteristics of deep learning models in existing vulnerability detection technologies. By combining comparative learning with token-level graph representation, the model is effectively guided to learn semantic features related to vulnerabilities, while capturing local structural features of the code. This breaks through the limitations of black-box characteristics, significantly improves the accuracy and applicability of vulnerability detection, can better cope with complex and hidden vulnerability detection tasks, reduces dependence on large amounts of manually annotated data, and enhances the generalization ability and robustness of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0019] Figure 1 This is a flowchart of a vulnerability detection method based on semantically sensitive contrastive learning and graph representation according to an embodiment of the present invention; Figure 2 This is a flow chart of obtaining code semantic features during the comparative learning process according to an embodiment of the present invention; Figure 3 Schematic diagram of a token-level co-occurrence graph construction method according to an embodiment of the present invention; Figure 4 This is a flow chart of a graph convolutional neural network for obtaining code structure features according to an embodiment of the present invention; Figure 5 1 is a graph showing the comparison results of the method proposed in the embodiment of the present invention with the baseline model on three data sets; Figure 6 This is a diagram showing the results of evaluating different common types of vulnerabilities according to an embodiment of the present invention. DETAILED DESCRIPTION

[0020] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0021] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present invention. As used herein, unless the context clearly indicates otherwise, the singular form is intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0022] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0023] Explanation of terms Contrastive learning: Contrastive learning is a self-supervised learning method whose core idea is to enable the model to learn more discriminative feature representations by constructing positive and negative sample pairs. During training, samples with the same semantics or origin are considered positive pairs, while samples with different semantics or origins are considered negative pairs. By optimizing the loss function, the model is forced to shorten the distance between positive pairs and increase the distance between negative pairs in feature space, thereby improving the model's expressiveness and generalization capabilities in unsupervised or weakly supervised scenarios. This method is widely used in fields such as image recognition, natural language processing, and recommendation systems, effectively improving model performance without requiring large amounts of manually labeled data.

[0024] Transformer: The Transformer is a deep neural network architecture based on the self-attention mechanism, originally developed for natural language processing tasks. Its key feature is that it does not rely on the traditional recurrent neural network (RNN) architecture, but instead uses a multi-head self-attention mechanism to capture dependencies between arbitrary positions in a sequence. The Transformer consists of two parts: an encoder and a decoder. Each layer is composed of a multi-head attention mechanism and a feedforward neural network. Residual connections and layer normalization operations are introduced to improve the stability and expressiveness of model training. Due to its efficient parallel computing capabilities and powerful modeling results, the Transformer has been widely used in various fields such as text, images, and speech, becoming one of the mainstream deep learning model architectures.

[0025] Abstract Syntax Tree: An Abstract Syntax Tree (AST) is a tree-like representation of source code structure, used to depict the grammatical structure of a program while removing irrelevant syntactic details (such as brackets and indentation). In an AST, each node represents a syntactic structure (such as a variable declaration, function call, conditional statement, etc.), reflecting the program's semantic hierarchy. Abstract Syntax Trees are commonly used in compilers, static analysis, code transformation, vulnerability detection, and other fields. They help systems perform structured parsing, semantic understanding, and further processing of source code, and serve as an important intermediate representation for source code analysis and manipulation.

[0026] Graph Convolutional Neural Network: A Graph Convolutional Network (GCN) is a deep learning model for processing graph-structured data. Its core idea is to learn the structural representation of each node in the graph by aggregating the neighbor information of the nodes in the graph. In GCN, each layer of the network performs a weighted combination of the features of the node and its adjacent nodes to achieve the propagation and fusion of information in the graph structure. This method can effectively model complex relationships in non-Euclidean space and is widely used in tasks such as social network analysis, recommendation systems, knowledge graphs, and chemical molecular structure modeling. Compared with traditional neural networks, GCN can capture the implicit topological relationships and contextual information in the graph, thereby improving the model's expression and reasoning capabilities on graph data.

[0027] Residual Connections: Residual connections are a structural optimization technique used in deep neural networks, designed to alleviate the problems of vanishing gradients and performance degradation during deep network training. The basic idea is to bypass the intermediate nonlinear transformations by adding the input signal of a layer directly to the output of that layer, thus forming a "skip connection." This structure allows the model to retain the original input information while maintaining nonlinear modeling capabilities, making deep models easier to optimize and converging faster. Residual connections are a key component in many advanced network architectures, such as ResNet and Transformer, and play an important role in improving the expressive power and training stability of deep models.

[0028] Support Vector Machine (SVM): A Support Vector Machine (SVM) is a supervised learning algorithm based on statistical learning theory and is widely used in classification, regression, and anomaly detection tasks. Its basic principle is to construct an optimal hyperplane that separates samples of different categories with the greatest possible margin, thereby improving the model's generalization ability. For linearly inseparable problems, SVM uses a kernel function to map data into a high-dimensional feature space, achieving linear separability in this space, thereby solving nonlinear classification problems. SVMs offer excellent robustness and theoretical interpretability, making them particularly suitable for small to medium-sized data scenarios with high-dimensional features.

[0029] Example 1 like Figure 1 As shown, this embodiment discloses a vulnerability detection method based on semantically sensitive contrastive learning and graph representation, which includes the following steps: S1: Obtain source code to be tested, normalize and transform the source code to obtain transformed code; In this embodiment, code normalization and transformation techniques are first applied to process the source code.

[0030] S101: Code normalization can eliminate redundant information caused by differences in programming style, variable naming, and formatting, thereby enhancing structural consistency, improving the semantic understanding of the model, and reducing false positives. The process mainly includes: (1) removing redundant spaces, blank lines, and comments to improve readability and analysis efficiency. Comments and trailing spaces, while helpful for human understanding, introduce noise without affecting program behavior; (2) standardizing variable and function names by replacing user-defined identifiers with common identifiers (e.g., var_1, func_0), while retaining standard library functions and language keywords to maintain code semantics and avoid compromising vulnerability detection performance.

[0031] S102: In order to convert the original code data, first use Tree-sitter to generate its abstract syntax tree AST. By using AST for structural transformation, semantically equivalent but syntactically different code representations can be generated. In order to address the limitations of existing models in directly capturing vulnerability semantics, the present invention proposes seven AST-based transformation methods to convert source code into variants with diverse structures but equivalent semantics. These converted code samples serve as comparison pairs. During the training process, contrastive learning is used to guide the model to capture vulnerability semantics while ignoring surface interference. The seven transformation methods are described and given examples as follows: (1) Conditional Swap: Conditional Swap adjusts the form of conditional evaluation, for example, converting a > b to b < a while maintaining logical equivalence. This mitigates the model's tendency to associate fixed patterns with specific labels and encourages attention to semantic consistency. This transformation enhances the model's ability to recognize semantically equivalent conditions, thereby improving the detection of boundary-related vulnerabilities such as access control flaws and insufficient input validation.

[0032] (2) Arithmetic Transformation: Arithmetic Transformation rewrites mathematical expressions into equivalent forms, such as rewriting g + 1 as 1 + g or a - (-b) as a + b without changing the calculation result. This prevents the model from overfitting to specific expression patterns and helps deepen the understanding of arithmetic semantics. It enhances the robustness to expression variations, improves the detection of arithmetic-related vulnerabilities such as integer overflows and precision errors, and reduces false positives and false negatives.

[0033] (3) While-for Swap: While-for Swap converts a while loop into an equivalent for loop and vice versa without changing the loop semantics. This addresses the model's tendency to associate specific loop structures with certain behaviors, for example, labeling a while loop as high-risk due to its higher frequency in the training data. By shifting the focus to loop conditions and boundaries, this transformation improves the detection of loop-related vulnerabilities such as infinite loops and out-of-bounds accesses, enhances the adaptability to code style changes, and reduces structure-based false detections.

[0034] (4) Insertion of Invalid Statements: Insertion of Invalid Statements adds operations that do not change the program semantics, such as g = g + 0 or NOP statements. This transformation prevents error detection based on surface features and mitigates the model's bias towards statement density or code complexity. Inserting irrelevant statements encourages the model to focus on semantic content, enhances the robustness to code obfuscation, and improves the accuracy of vulnerability detection.

[0035] (5) if Condition Transformation: The transformation of if conditions involves reordering conditional statements without changing the program logic to ensure the same execution result. Due to the black-box nature of the model, it may overfit to shallow patterns in conditional expressions, wrongly associating forms such as a > b with specific labels (such as vulnerabilities) based on training data rather than understanding the true semantics. This shallow pattern learning may lead to the model misjudging conditions, resulting in incorrect execution paths and potential vulnerabilities such as unauthorized access due to flawed permission checks. By transforming if conditions, these logical errors can be simulated and help detection tools identify vulnerabilities.

[0036] (6) Statement Sequence Adjustment: Statement sequence adjustment refers to reordering statements whose execution order can be changed without affecting the program semantics. This transformation enhances the robustness of detection tools and mitigates model bias caused by fixed statement patterns (such as "declare first, then check condition" or "initialize first, then assign value"), which can lead to false positives or false negatives. By refactoring the code, it improves the model's ability to capture dependencies between statements and reason about data flow, reduces reliance on a specific execution order, and helps detect issues such as access control violations.

[0037] (7) Switch-to-if transformation: The switch-to-if transformation transforms a switch-case statement into a series of equivalent if-else conditional statements. Learning-based models, due to their black-box nature, often rely on surface features such as token patterns or syntax, mistakenly associating the switch structure itself with vulnerabilities. This transformation reduces this bias by removing structural dependencies and encouraging the model to focus on the actual logic behind the branches. It is particularly useful for multi-branch logic, as different combinations of conditions can lead to different execution paths and reveal hidden vulnerabilities. By replacing switch statements with logically equivalent if-else statements, the model can reason about behavior based on logical completeness rather than syntactic type, thereby promoting a deeper understanding of branch correctness and coverage.

[0038] S2: Input the source code and the transformed code into a vulnerability detection model for processing to obtain vulnerability classification results. The vulnerability detection model includes a semantic learning module, a graph representation learning module, a feature fusion module, and a support vector machine.

[0039] (1) Semantic Learning Module The semantic learning module receives the source code and the transformed code, and performs comparative learning on the source code and the transformed code as positive sample pairs to obtain the code semantic feature embedding.

[0040] The first stage of the vulnerability detection model uses a semantic learning module, which uses the transformed code as a positive sample. This allows the vulnerability detection model to ignore surface changes during the comparative learning process and focus on the intrinsic semantic representation of the real vulnerability. In the previous step, the abstract syntax tree transformation of the code fragment has been obtained, and the semantic function of the code remains unchanged but the grammatical structure is changed. The transformed code As a positive sample pair, that is: . Negative sample pairs is randomly selected from the entire code base, and The semantically different code snippets are: .

[0041] In standard contrastive learning methods, negative samples typically only come from the current batch, which can lead to an insufficient number of negative samples and affect training effectiveness. Therefore, this paper also introduces a negative sample queue, which is used to store past negative samples to enhance training stability and improve the effectiveness of contrastive learning. It uses a fixed-size FIFO (first-in-first-out) queue to store negative samples from historical batches. Assuming the maximum capacity of the queue is K, when the queue is full and a new negative sample enters, the queue automatically removes the oldest sample at the head of the queue, ensuring that the queue size always remains at K. This strategy helps keep the data in the queue up to date while preventing outdated negative samples from affecting the training of the semantic learning module.

[0042] Code data is usually input into the semantic learning module in the form of a token sequence. Each token represents a basic component of the code. Since the code has hierarchical structural information, directly using the token sequence may not be able to effectively learn the semantics of the code. Therefore, the present invention uses Roberta for encoding. During the encoding process, the position encoding automatically generated by Roberta is also added as an input feature, and combined with the attention mask to optimize the Transformer's ability to process different codes. Figure 2 As shown in the figure, the semantic learning module includes a parallel query encoder (Transformer encoder) and a momentum encoder. The query encoder adopts the Transformer structure and consists of multiple self-attention layers. Each layer includes a multi-head self-attention mechanism, residual connection and layer normalization, a feedforward network, residual connection and layer normalization.

[0043] Since the Transformer's parameters are updated quickly during training, this may lead to unstable embeddings for positive samples. To this end, this paper introduces a momentum encoder, which acts as a slowly updating encoder to calculate the embeddings of positive samples. The momentum encoder uses the same Transformer structure as the query encoder, and its parameters are updated using an exponential sliding average, as shown in the following formula:

[0044] in, represents the parameters of the momentum encoder, represents the momentum coefficient (set to 0.999 in this embodiment), Denotes the parameters of the query encoder. The parameters of the query encoder are trained and updated normally via backpropagation. The parameters of the momentum encoder are not trained via backpropagation but are updated using the above update formula.

[0045] During training, the present invention uses the contrastive loss function InfoNCE to optimize the semantic learning module. The module inputs are a query, a positive sample, and a negative sample queue. (The query and positive sample form a pair of code snippets. The semantically transformed code is the positive sample, while the remaining semantically transformed code forms the negative sample queue. The query and each negative sample form a pair of code snippets.) The query is typically the original code snippet, while the positive sample is a code snippet obtained through some transformation (e.g., variable renaming, conditional swapping, etc.). They are semantically similar but syntactically different.

[0046] The query sample, positive sample, and negative sample queues are converted into vector representations using Roberta (a programming language model). The query sample vector representation is then fed into the query encoder to obtain the query sample embedding vector. The vector representations of the positive and negative sample queues are then fed into the momentum encoder to obtain the embedding vectors for the positive and negative samples, respectively. Finally, the code semantic feature embedding is output. The following formula is then used to maximize the similarity between the query sample and the positive sample, bringing their embedding vectors as close together in space as possible. Simultaneously, the similarity between the query sample and the negative sample is minimized, keeping their embedding vectors as far apart as possible in space.

[0047]

[0048] in, represents the loss function of InfoNCE in contrastive learning, represents a query sample, represents a positive sample, represents negative samples, Represents the temperature parameter. It controls the smoothness of the softmax function. The smaller the value, the more sensitive the difference.

[0049] (2) Graph Representation Learning Module The source code is represented as a graph structure, the graph structure is processed, and the node representation is output to obtain the code structure feature embedding.

[0050] In this embodiment, the source code is represented as a graph structure, the node features are initialized using a pre-trained programming language model, and a graph convolutional neural network with residual connections is used to learn the structural features of the code.

[0051] In order to enhance the learning of syntactic structure features in the code, this paper introduces a token-level co-occurrence graph construction method in the second stage of the vulnerability detection model. This method is used to represent the source code as a graph structure. Specifically, the process is as follows: for a given code snippet, the tokens in the code are extracted, possible duplicate tokens are removed, and the unique tokens are retained and regarded as nodes in the graph. The result is as follows: Figure 3 The circled content is shown in the figure. Then, based on the co-occurrence relationships between source code and the window size, corresponding edges are added. A sliding window mechanism is used to construct token relationships, establishing edges between tokens that appear within the same window. These co-occurrence relationships are encoded into an adjacency matrix (graph structure), and the edge weights reflect the frequency of co-occurrence.

[0052] The pre-trained programming language model RoBERTa is used to initialize each node in the graph structure to obtain the embedding vector of each node feature, which is used as the initial node feature input for subsequent graph neural network training.

[0053] The token-level co-occurrence graph construction formula is expressed as:

[0054] in, Indicates the relationship between tokens. 、 Indicates a token node.

[0055] like Figure 4 As shown in the figure, the semantic learning module includes a multi-layer graph convolutional neural network and soft attention. Soft attention is used in the graph-level readout layer. By weighting the representation of each node output by the graph convolutional neural network, the importance of each node in the graph representation is dynamically adjusted. Specifically, soft attention assigns different weights to the features of each node, and then aggregates the weighted representations of all nodes into the final graph-level feature embedding, i.e., the code structure feature embedding, through a hybrid pooling (sum + max pooling) method.

[0056] To improve the training performance of graph convolutional neural networks (GCNs) and alleviate the problem of vanishing or exploding gradients, residual connections are introduced in the GCN layer. Residual connections allow information to be passed directly from lower layers to higher layers, which can better preserve and transmit low-level features while enhancing the stability of the model during training. The update formula for the graph structure node is as follows:

[0057] in, Indicates the The node representation of the layer, Indicates the The node representation of the layer, Represents the adjacency matrix (graph structure).

[0058] After passing through the graph convolutional neural network with residual connections, the node representation is obtained; then the node representation is used to generate the embedded representation of the graph through the graph-level readout pooling layer. For each node , from Node representation of the layer It will be converted into a new node representation through the following formula :

[0059] in, represents the activation function (such as Sigmoid), represents the learnable weights, Representation node In GCN The layer representation, represents the bias term, represents element-wise multiplication, represents a nonlinear transformation function (such as ReLU), Represents a linear transformation of the node representation.

[0060] Node representation for all nodes Perform a hybrid operation of sum pooling and max pooling to obtain graph-level embedding , expressed as:

[0061] in, Represents the MIX function, which indicates one of three mixing methods, including SUM, MUL, and CONCAT operations; Represents a collection of nodes in a graph structure.

[0062] SUM is expressed as: ; MUL is expressed as: ; CONCAT is represented as: .

[0063] (3) Feature Fusion Module In this embodiment, code semantic feature embedding and code structural feature embedding are fused to obtain a fusion vector, which is then classified to obtain a vulnerability classification result.

[0064] In the third stage of the vulnerability detection model, the code semantic feature embedding and the code structure feature embedding are spliced ​​to obtain a fusion vector. After the fusion vector is normalized, all samples are split into feature matrices. and tags ,Support Vector Machine (SVM) is used to classify the fusion vector and obtain the vulnerability classification results.

[0065] The technical effects achieved by the present invention are specifically as follows: Breaking through black-box limitations and improving vulnerability detection accuracy: Traditional deep learning-based vulnerability detection methods are limited by the black-box nature of the model, making it difficult to clearly define and effectively model vulnerability characteristics, resulting in low accuracy in complex vulnerability detection. This paper guides the model to learn vulnerability-related semantic features through comparative learning. Combining token-level graph representations with graph convolutional neural networks to capture the local structural features of the code, this model breaks through the limitations of black-box characteristics and more comprehensively understands and identifies vulnerability patterns, significantly improving vulnerability detection accuracy.

[0066] Enhanced model generalization to adapt to diverse vulnerability detection tasks: Contrastive learning methods eliminate the need for extensive manually labeled data. By constructing positive and negative sample pairs to optimize the model, the model possesses stronger expressiveness and generalization capabilities in unsupervised or weakly supervised scenarios. This allows the model to transcend the limitations of black-box nature and better adapt to diverse vulnerability detection tasks. Its generalization capabilities enable effective detection even of unseen vulnerability types.

[0067] Improving model robustness and resilience to code style changes and obfuscation: We employ multiple code transformation methods to generate semantically equivalent but syntactically distinct code variants, which serve as positive sample pairs for comparative learning. This allows the model to transcend the limitations of black-box nature, ignoring surface variations and focusing on the intrinsic semantic representation of real vulnerabilities. This enhances the model's robustness to code style changes and obfuscation, reducing false positives and negatives caused by code style differences or obfuscation.

[0068] Optimizing model training and performance: Introducing residual connections into graph convolutional neural networks alleviates the problem of vanishing or exploding gradients, improving model training and stability. Furthermore, strategies such as negative sample queues and momentum encoders further optimize the contrastive learning process, enabling the model to break through the limitations of black-box features, more effectively learn vulnerability signatures, and enhance overall model performance.

[0069] The following is a detailed description of the experimental content.

[0070] During the experiment, the proposed method was evaluated based on three vulnerability datasets (FFmpeg+QEM, ReVeal, and Big_Vul). The data details are shown in Table 1: Table 1 Evaluation table of vulnerability detection methods based on vulnerability datasets

[0071] The experiments were run on a server equipped with an Intel 12th Gen Core i5-12600KF CPU and an NVIDIA GeForce GTX 4060TI 16G GPU, 32GB of DDR4 memory, and Windows 11. In the first phase, the data was processed using the RoBERTaTokenizer and then partitioned into 80% training, 10% validation, and 10% test sets. DataLoader training was performed with a batch size of 64. The model was trained on a GPU using the Adam optimizer (lr=1e-4) for 100 epochs and evaluated on the validation and test sets. In the second phase, the data was processed using the RoBERTaTokenizer and then partitioned into 80% training, 10% validation, and 10% test sets. Due to the large memory footprint of the graph, DataLoader training was performed with a batch size of 4. The model was trained on a GPU using the Adam optimizer (5e-5) for 50 epochs and evaluated on the validation and test sets.

[0072] To comprehensively evaluate the performance of the proposed model in vulnerability detection tasks, this paper uses four commonly used evaluation metrics: Accuracy, Precision, Recall, and F1 Score. The definitions and meanings of each metric are as follows: (1) Accuracy: Accuracy = (TP + TN) / (TP + TN + FP + FN) Accuracy indicates the ratio of correctly classified samples (including vulnerability samples and non-vulnerability samples) to the total number of samples, and is used to measure the overall classification performance of the model.

[0073] Among them, TP is the true positive example (the number of samples correctly predicted by the model as vulnerabilities); TN is the true negative example (the number of samples correctly predicted by the model as non-vulnerabilities); FP is the false positive example (the number of non-vulnerability samples incorrectly predicted by the model as vulnerabilities); FN is the false negative example (the number of vulnerability samples incorrectly predicted by the model as non-vulnerabilities).

[0074] (2) Precision: Precision = TP / (TP + FP) Precision measures the proportion of samples predicted by the model as "vulnerable" that actually have vulnerabilities. A higher precision means a lower false positive rate, indicating that the model is more reliable in identifying vulnerabilities.

[0075] (3) Recall: Recall = TP / (TP + FN) Recall (also known as sensitivity or true positive rate) indicates the proportion of samples with real vulnerabilities that are correctly identified by the model. A higher recall indicates a stronger model's ability to detect real vulnerabilities.

[0076] (4) F1 score: F1 Score = 2 × (Precision × Recall) / (Precision +Recall) The F1 score is the harmonic mean of precision and recall, and is a comprehensive measure of a model's accuracy and completeness in positive example detection. It is particularly useful in scenarios with class imbalanced data.

[0077] like Figure 5 As shown in the figure, the proposed model is compared with various baseline models, including VulDeePecker, Russell et al., SySeVR, Devign, Reveal, GRACE, AMPLE, and MAGNET. This experiment aims to compare the vulnerability detection performance of different models on three datasets: FFmpeg+Qemu, Reveal, and big_vul. The experimental results show that the proposed model significantly outperforms other baseline methods on all datasets. In particular, the F1 scores on the FFmpeg+Qemu and big_vul datasets reach 91.50% and 92.14%, respectively, far exceeding those of other models, demonstrating stronger generalization and detection performance. This demonstrates that the proposed method has higher accuracy and stability in vulnerability detection tasks.

[0078] like Figure 6 As shown in the figure, this experiment compared and evaluated the detection performance of multiple models on different CWE types of vulnerabilities, covering 11 common vulnerability types, and used Devign, Reveal, IVDetect, MAGNET and the "model of the present invention" for performance comparison.

[0079] CWE (Common Weakness Enumeration) numbers represent different types of software security vulnerabilities. CWE-20 is Improper Input Validation: The program fails to properly verify whether the input data conforms to the expected format or value range; CWE-22 is Path Traversal: The attacker accesses files or directories that should not be allowed to access on the system by manipulating input; CWE-79 is Cross-Site Scripting Attack: User input is not properly sanitized, resulting in malicious scripts being executed in the browser; CWE-119 is Improper Buffer Boundary Restriction: Such as buffer overflow, resulting in memory corruption or code execution; CWE-125 is Out-of-Bounds Read: The program reads data outside the array or buffer range, which may cause information leakage or crash; CWE-190 is Integer Overflow or Wraparound: The integer value is calculated The error exceeds the representable range and may be exploited to execute arbitrary code or bypass security checks; CWE-200 is the exposure of sensitive information: such as confidential data contained in error messages, logs or responses; CWE-416 is the use after free: references to freed memory may lead to arbitrary code execution or crash; CWE-476 is the dereference of a null pointer: the program attempts to access a NULL pointer, resulting in a crash; CWE-732 is the improper allocation of critical resource permissions: the program grants too broad permissions to files, devices, etc.; CWE-787 is the out-of-bounds write: the program writes data beyond the buffer boundary, which may cause memory corruption, privilege escalation or the execution of malicious code.

[0080] like Figure 6 The results show that the proposed model achieved the highest or near-highest accuracy across all vulnerability types, with an average accuracy of 96.90%, significantly outperforming the second-place MAGNET at 75.70%. In particular, for the high-proportion CWE-20 and CWE-119 types, the proposed model achieved accuracies of 97.49% and 97.72%, respectively, demonstrating its strong adaptability and robustness in both mainstream and complex vulnerability detection tasks. Experiments validate the comprehensive superiority of the proposed model (vulnerability detection model) in fine-grained vulnerability detection scenarios.

[0081] Example 2 This embodiment discloses a vulnerability detection system based on semantically sensitive contrastive learning and graph representation, including: A data acquisition module is configured to: acquire source code to be detected, normalize and transform the source code to obtain transformed code; a model detection module configured to: input the source code and the transformed code into a vulnerability detection model for processing to obtain a vulnerability classification result; the vulnerability detection model includes a semantic learning module, a graph representation learning module, a feature fusion module, and a support vector machine; The process of building the vulnerability detection model includes: The source code and the transformed code are used as positive sample pairs for comparative learning to obtain the code semantic feature embedding; Representing the source code as a graph structure, processing the graph structure, outputting node representations, and thereby obtaining code structure feature embedding; The code semantic feature embedding and the code structural feature embedding are fused to obtain a fusion vector, and the fusion vector is classified to obtain a vulnerability classification result.

[0082] Example 3 The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the method of embodiment 1 when executing the program.

[0083] Example 4 The purpose of this embodiment is to provide a computer-readable storage medium, a computer-readable storage medium having a computer program stored thereon, which performs the steps of the method of embodiment 1 when executed by a processor.

[0084] The steps involved in the apparatuses of Examples 3 and 4 above correspond to those of Method Example 1. For detailed implementation, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any of the methods of the present invention.

[0085] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0086] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

[0087] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A vulnerability detection method based on semantically sensitive contrastive learning and graph representation, characterized by: include: Obtaining source code to be tested, normalizing and transforming the source code to obtain transformed code; Inputting the source code and the transformed code into a vulnerability detection model for processing to obtain a vulnerability classification result; The vulnerability detection model includes a semantic learning module, a graph representation learning module, a feature fusion module and a support vector machine; The process of building the vulnerability detection model includes: The source code and the transformed code are used as positive sample pairs for comparative learning to obtain the code semantic feature embedding; Representing the source code as a graph structure, processing the graph structure, outputting node representations, and thereby obtaining code structure feature embedding; The code semantic feature embedding and the code structural feature embedding are subjected to feature fusion to obtain a fusion vector, and the fusion vector is classified to obtain a vulnerability classification result.

2. The vulnerability detection method based on semantically sensitive contrastive learning and graph representation according to claim 1, characterized in that: The transformation of the source code specifically includes: generating an abstract syntax tree from the source code, and performing structural transformation using the abstract syntax tree.

3. The vulnerability detection method based on semantically sensitive contrastive learning and graph representation according to claim 1, characterized in that: The semantic learning module includes a parallel query encoder and momentum encoder. The query encoder and momentum encoder have the same structure and are composed of multiple self-attention layers. Each layer includes a multi-head self-attention mechanism, residual connection and layer normalization, a feedforward network, residual connection and layer normalization.

4. The vulnerability detection method based on semantically sensitive contrastive learning and graph representation according to claim 3, characterized in that: The query samples, positive samples and negative sample queues are encoded through the programming language model to obtain the query sample vector representation, positive sample vector representation and negative sample queue vector representation; the query sample vector representation is input into the query encoder, and the positive sample vector representation and negative sample queue vector representation are input into the momentum encoder, which outputs the code semantic feature embedding.

5. The vulnerability detection method based on semantically sensitive contrastive learning and graph representation according to claim 1, characterized in that: A token-level co-occurrence graph construction method is used to represent the source code as a graph structure. Specifically, tokens in the source code are extracted and regarded as nodes in the graph. A sliding window mechanism is used to construct token relationships, and edges are established between tokens appearing in the same window to obtain a graph structure.

6. The vulnerability detection method based on semantically sensitive contrastive learning and graph representation according to claim 1, characterized in that: The graph representation learning module includes a multi-layer graph convolutional neural network that introduces residual connections.

7. The vulnerability detection method based on semantically sensitive contrastive learning and graph representation according to claim 6, characterized in that: The code structure feature embedding is obtained specifically as follows: the node representation is converted into a new node representation, and a mixed operation of sum pooling and maximum pooling is performed on the new node representations of all nodes to obtain the code structure feature embedding.

8. A vulnerability detection system based on semantically sensitive contrastive learning and graph representation, characterized by: include: A data acquisition module is configured to: acquire source code to be detected, normalize and transform the source code to obtain transformed code; A model detection module is configured to: input the source code and the transformed code into a vulnerability detection model for processing to obtain a vulnerability classification result; The vulnerability detection model includes a semantic learning module, a graph representation learning module, a feature fusion module and a support vector machine; The process of building the vulnerability detection model includes: The source code and the transformed code are used as positive sample pairs for comparative learning to obtain the code semantic feature embedding; Representing the source code as a graph structure, processing the graph structure, outputting node representations, and thereby obtaining code structure feature embedding; The code semantic feature embedding and the code structural feature embedding are subjected to feature fusion to obtain a fusion vector, and the fusion vector is classified to obtain a vulnerability classification result.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the vulnerability detection method based on semantic-sensitive contrastive learning and graph representation as described in any one of claims 1 to 7 are implemented.

10. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps in the vulnerability detection method based on semantic-sensitive contrastive learning and graph representation are implemented as described in any one of claims 1 to 7.

Images