A source code vulnerability intelligent detection method and system

Abstract

The application relates to the technical field of code intelligent analysis, and provides a source code vulnerability intelligent detection method and system, which comprises the following steps: obtaining function-level source code text to be detected, generating a word unit sequence through a word segmenter, extracting a hidden state after length alignment; extracting three-granularity features through a multi-granularity feature extraction module, obtaining fusion features through an adaptive gate fusioner; the adaptive gate fusioner scores the three-granularity features through a gate network, then performs weighted fusion on the three-granularity features to obtain fusion representation, and obtains a vulnerability probability through a classifier. In the training stage, a double-view enhanced input of semantic equivalence is introduced, a closed-loop training mechanism is constructed by jointly optimizing cross-entropy loss, supervised contrast loss and consistency distillation loss, and a weighted sampling strategy based on the reciprocal of the category frequency is adopted. The application can effectively capture local and cross-scale vulnerability patterns, and improve the recognition ability and detection stability of vulnerabilities triggered by local sentences.

Description

Technical Field

[0001] This invention belongs to the field of intelligent code analysis technology, and in particular relates to an intelligent source code vulnerability detection method and system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development of intelligent code analysis technology, code language models pre-trained based on large-scale code corpora have been widely used in the field of source code vulnerability detection due to their strong semantic representation and transfer capabilities. These models are used for tasks such as function-level vulnerability binary classification and line-level vulnerability localization, which have improved the efficiency of automated vulnerability analysis to a certain extent.

[0004] However, in practical engineering applications and complex software scenarios, existing technologies still have limitations in terms of vulnerability identification accuracy and model stability. In particular, the model performance remains unstable in the following situations: local vulnerabilities triggered by a small number of key statements are difficult to capture stably; cross-scale vulnerability patterns that rely on both local patterns and overall semantic constraints are not adequately modeled; the robustness of the model decreases under perturbation conditions such as changes in variable naming and differences in code style; and the model's generalization ability is limited when the distribution of vulnerability samples is highly unbalanced.

[0005] In the above application scenarios, the existing technology still has the following shortcomings: The problem of reliably capturing locally triggered vulnerability patterns: In reality, vulnerabilities are often triggered by a small number of key statements / key tokens (e.g., dangerous API (Application Programming Interface) calls lacking boundary checks, missing null pointer checks, etc.). Even if the Transformer (encoder model) has global modeling capabilities, in function-level binary classification scenarios, the training objective usually only supervises the final classification result. This can easily cause the model to "mix and average" a large amount of context unrelated to the vulnerability with a small amount of key triggering evidence, resulting in insensitivity to key local evidence and an increase in false negatives. This problem is more pronounced in long functions and samples with a high proportion of noisy context. The problem of insufficient cross-scale semantic vulnerability pattern modeling: local triggers rely more on token / statement level patterns, while global logic / control and data dependency vulnerabilities rely more on function level semantics. Many existing works introduce multi-granularity representations, but common fusion strategies tend to be "static fusion" (such as fixed ratio, average fusion or simple splicing followed by linear mapping) in implementation, which makes it difficult to adaptively decide "which granularity to believe" for different samples, thus bringing problems of fusion noise and unstable generalization. The problem of insufficient robustness to code style and semantic equivalence transformations: The same vulnerability may present different surface forms in different projects, such as different variable names, insertion of redundant statements, and formatting changes. These transformations are semantically approximately equivalent, but they will change the token distribution. If only cross-entropy classification is used for training, the model is prone to learning surface correlations that are unrelated to the vulnerability, resulting in a decrease in cross-project generalization. Therefore, how to use the constraint mechanism in the training phase to keep the semantically equivalent code consistent in the representation space and prediction results, thereby reducing the model's sensitivity to surface form differences, is one of the key technical requirements for improving the robustness of the model. The highly imbalanced distribution of vulnerability samples leads to insufficient generalization ability: In vulnerability datasets, the sample distribution is usually significantly imbalanced, with the proportion of non-vulnerable samples generally being significantly higher than that of vulnerability samples. This often causes the model to be dominated by the high proportion of class samples (usually non-vulnerable samples) during training, resulting in insufficient ability to distinguish the minority class (vulnerable samples), thus affecting the overall generalization performance and detection effect of the model. Summary of the Invention

[0006] To address the technical problems mentioned above, this invention provides an intelligent source code vulnerability detection method and system. By simultaneously extracting Token-level, Statement Block-level, and Function-level semantic features based on a unified encoding representation, and by introducing an adaptive gating fusion mechanism, the importance weights of different granular features are dynamically calculated based on the input sample features. This enables adaptive integration of local and overall semantic information, allowing for the automatic selection of more effective semantic evidence sources in different vulnerability scenarios, thereby improving the adaptability to complex vulnerability patterns and overall detection accuracy.

[0007] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a source code vulnerability intelligent detection method, comprising: The function-level source code text to be detected is obtained, a word segmenter is used to generate a word sequence, the word sequence is length-aligned, and the hidden state is extracted by an encoder. Based on the hidden state, a three-granularity feature extraction module is used to extract three-granularity features, which include word granularity features, sentence block granularity features, and function granularity features. Based on three-granularity features, an adaptive gated fusion fusion derives a fused feature. The adaptive gated fusion derives a fused feature by scoring the three-granularity features separately through a gated network, obtaining the three-granularity attention weights through a normalized exponential function, and then performing weighted fusion on the three-granularity features to obtain a fused representation. The fused representation is then subjected to linear transformation, linear correction unit and regularization to obtain the fused feature. Based on the fusion features, the vulnerability probability is obtained through a classifier.

[0008] Furthermore, the extraction of the word granular features employs linear projection and average pooling.

[0009] Furthermore, the extraction of the statement block granular features employs one-dimensional convolution and max pooling aggregation.

[0010] Furthermore, the extraction of the function granularity features employs linear projection.

[0011] Furthermore, the multi-granularity feature extraction module, the adaptive gating fusion unit, and the classifier constitute a source code vulnerability detection model. Under dual-view augmented input, the source code vulnerability detection model jointly optimizes the cross-entropy loss, the supervised comparison loss, and the consistency distillation loss for closed-loop training.

[0012] Furthermore, the source code vulnerability detection model introduces a weighted sampling strategy based on the reciprocal of category frequency during the training phase.

[0013] Furthermore, the weights of the supervised contrast loss and the consistency distillation loss are dynamically calculated based on the imbalance of the training set samples and the training rounds.

[0014] A second aspect of the present invention provides a source code vulnerability intelligent detection system, comprising: The pre-trained encoding module is configured to: acquire the function-level source code text to be detected, generate a word sequence through a word segmenter, align the word sequence by length, and extract the hidden state through an encoder; The multi-granularity feature encoding module is configured to extract three-granularity features based on the hidden state through the multi-granularity feature extraction module. The three-granularity features include word granularity features, sentence block granularity features, and function granularity features. An adaptive gated fusion module is configured to: obtain fused features based on three-granularity features through an adaptive gated fusion unit; the adaptive gated fusion unit scores the three-granularity features separately through a gated network, obtains the three-granularity attention weights through a normalized exponential function, performs weighted fusion on the three-granularity features to obtain a fused representation, and performs linear transformation, linear correction unit and regularization on the fused representation to obtain the fused features; The vulnerability classification module is configured to obtain the vulnerability probability based on fused features and a classifier.

[0015] A third aspect of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the source code vulnerability intelligent detection method described above.

[0016] A fourth aspect of the present invention provides a computer device including a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, wherein the processor executes the program to implement the steps of the source code vulnerability intelligent detection method described above.

[0017] Compared with the prior art, the beneficial effects of the present invention are: This invention extracts token-level, statement-block-level, and function-level semantic features simultaneously based on a unified encoding representation. This allows for simultaneous attention to local triggering patterns and overall semantic structure, reducing the dilution of critical vulnerability triggering statements during global semantic aggregation. Consequently, it enhances the ability to identify and detect vulnerabilities triggered by local statements.

[0018] This invention introduces an adaptive gating fusion mechanism to dynamically calculate the importance weights of features at different granularities based on the features of the input samples. This enables adaptive integration of local and overall semantic information, allowing for the automatic selection of more effective sources of semantic evidence in different vulnerability scenarios. This improves the adaptability to complex vulnerability patterns and the overall detection accuracy.

[0019] This invention introduces semantically equivalent multi-view enhancement and contrast consistency constraints during the training phase, enabling the model to maintain stable semantic representation and prediction results under conditions such as variable naming changes, redundant statement insertion, and code format differences. This reduces the model's dependence on surface features and improves its generalization ability in cross-project and cross-coding style scenarios.

[0020] This invention introduces a weighted sampling strategy based on category frequency statistics to increase the frequency of vulnerability samples during training, enabling the model to learn vulnerability sample knowledge more fully, thereby improving the ability to identify vulnerability function features and expanding the overall vulnerability detection coverage. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart of a source code vulnerability intelligent detection method according to Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of a computer device according to Embodiment 4 of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] Example 1 This embodiment provides a method for intelligent detection of source code vulnerabilities.

[0026] This embodiment provides a source code vulnerability intelligent detection method. Through multi-granularity feature joint modeling and contrast consistency training mechanism, the model learns a more stable and discriminative vulnerability semantic representation in the feature space, thereby reducing the over-reliance on majority class sample features, improving the ability to identify minority class (vulnerability samples), and improving the overall detection coverage and generalization performance.

[0027] This embodiment provides an intelligent source code vulnerability detection method that achieves stable modeling of complex vulnerability patterns through multi-granularity feature extraction, adaptive gating fusion, and a contrastive learning consistency constraint training mechanism. Specifically, it addresses the following technical issues: (1) The problem of local triggering vulnerability patterns is difficult to highlight: When using only a single function-level representation for binary classification, a small number of key sentences / tokens are easily "diluted" by a large number of non-key contexts, resulting in missed detections. This embodiment constructs token-level features, line / sentence block-level features (using 1D-CNN (one-dimensional convolutional neural network) to simulate local semantic blocks), and function-level features (CLS) on the same encoder hidden layer sequence, so that the model has the input representation foundation of "seeing both local and global at the same time" during training and inference. The line-level branch adopts one-dimensional convolutional neural network layer Conv1d (convolution kernel=10, stride=5, padding=5) and MaxPooling aggregation to enhance the ability to extract local triggering patterns. (2) The problem of cross-scale semantic information is difficult to model collaboratively: When solving the problem of different information focus and different noise sources of multi-granularity features, the fixed ratio / average fusion is difficult to adapt to different samples, resulting in fusion noise and unstable performance. In this embodiment, a three-way gated network is designed to calculate the output scores of three granularities: Token (word), Line (sentence), and Function (function). The three granularity attention weights are obtained by Softmax (normalized exponential function) and the three granularity features are weighted and fused. After fusion, the fusion is transformed by linear transformation + ReLU (linear correction unit) + regularized random deactivation Dropout (0.1) to obtain a stable fusion representation. At the same time, the output granular weights are used for interpretable analysis. (3) Insufficient robustness to code style changes and semantic equivalence transformations: To address the issue that when relying solely on cross-entropy supervision, the model easily learns superficial correlations such as variable names, formats, and redundant statements, leading to instability in "semantic equivalence transformations" and decreased generalization across projects, this embodiment performs two random augmentations on the same code sample during the training phase to generate View1 (view1). Figure 1 / View2 (view) Figure 2 It uses "dual-view input" to enter the model training process, providing a data structure foundation for comparative learning and consistency constraints; (4) The problem of insufficient generalization ability caused by extreme class imbalance: In actual vulnerability detection datasets, the ratio of vulnerability samples to non-vulnerability samples differs greatly. In order to solve the problem that when there are few vulnerability samples and many normal samples, the model optimization process is easily dominated by the majority class and the recall rate is unstable, this embodiment is based on the statistical class frequency of the training set labels, constructs class weights according to the inverse of the frequency, and maps them to sample-level weights. Then, a WeightedRandomSampler is used for sampling training, so that minority class samples (vulnerability samples) are sampled more frequently during training, thereby alleviating the situation of insufficient vulnerability learning due to extreme imbalance of dataset types.

[0028] This embodiment provides an intelligent source code vulnerability detection method that constructs three feature paths—Token (lexical) granularity, Statement-level granularity, and Function-level granularity—on the hidden layer representation output by the same pre-trained encoder. It then uses a gating network to adaptively weight and fuse the three feature paths. Combined with supervised contrastive learning (SupCon) and self-distillation consistency constraints (KL) during the training phase, it achieves collaborative modeling of locally triggered vulnerabilities and cross-scale semantic vulnerabilities, while improving robustness to surface equivalence deformation.

[0029] This embodiment provides an intelligent source code vulnerability detection method, such as... Figure 1 As shown, it includes the following steps: Step 1: Input standardization and pre-trained encoding: Generate length-aligned token sequences and output hidden state sequences.

[0030] Step 101: Construct and align the input sequence with its length.

[0031] Given the function-level source code text S to be detected, tokenization is performed using a pre-trained model and its corresponding token segmenter to obtain a token sequence: X=(x1,x2,…,x n ); Where, x iLet represent the i-th lexical unit, and n represent the number of lexical units.

[0032] To ensure the stability of batch processing and convolution operations, a maximum sequence length L (preferably 512) is set; sequences with a length exceeding L are truncated; sequences with a length less than L are padded with PAD (padding tokens) at the end to ensure that the final input length is consistent with L.

[0033] In this embodiment, the training configuration selects block_size as 512.

[0034] Meanwhile, an attention mask is constructed: the PAD (padding marker) positions are set to 0, and the non-PAD (non-padding marker) positions are set to 1 to avoid the padding from interfering with the encoding. The corresponding implementation is attention_mask=input_ids.ne(1); where attention_mask is a 0 / 1 marker sequence used to indicate which positions participate in the calculation and which positions should be ignored, and input_ids represents the number (ID) corresponding to each token in the input sequence, which is an integer sequence of length L. In the specific implementation code statement, ne=not equal, which is equivalent to input_ids!=1.

[0035] Step 102: Pre-trained encoding output.

[0036] The length-aligned input_ids (the ID corresponding to each token in the input sequence) are input into the pre-trained encoder (Transformer Encoder, such as the RoBERTa / CodeBERT backbone, where RoBERTa and CodeBERT are a pre-trained language representation model based on the Transformer architecture and a pre-trained language model oriented towards source code, respectively), to obtain the hidden state sequence: H=(h1,h2,…,h L ),h i ∈ Q ; Among them, h i Let Q represent the hidden state of the i-th token, Q be the feature dimension (hidden size, typically 768) of the encoder's hidden layers (TransformerLayers), B be the batch size, and the output tensor shape be [B, L, Q].

[0037] Step 2, MultiGrainFeatureExtractor: Extracts three-granularity features (Token, Line, and Function) from the hidden state and unifies them to the same dimension.

[0038] The goal of step 2 is to simultaneously extract fine-grained lexical patterns, medium-grained local sentence block patterns, and global function semantics from the same hidden state sequence H, and project the three features onto a unified dimension to avoid dimensional inconsistencies and scale imbalances during subsequent fusion.

[0039] Let the unified fusion dimension be d = Q / 2 (in the code implementation, hidden_size / / 2, i.e., 768 → 384), where hidden_size represents the dimension of the hidden layer features output by the pre-trained encoder. In the actual code, the symbol " / / " represents integer division.

[0040] (1) Token granularity features, using average pooling + linear projection.

[0041] First, for each token, the hidden vector h i Perform a linear projection to obtain the projected vector: ; Obtain the sequence ,in, The weight matrix represents the linear projection. d represents the bias vector, and d represents the unified feature dimension (fusion dimension).

[0042] The corresponding code implementation is token_proj:Linear(hidden_size,hidden_size / / 2), where token_proj represents the linear projection module, and Linear(a,b) represents a linear transformation.

[0043] Then, for the sequence Perform mean pooling to obtain token granular features: , ; The corresponding code implementation is torch.mean(token_feat, dim=1), which outputs a shape of [B, d]; where torch.mean represents the mean calculation function, and token_feat represents the feature sequence obtained by linearly projecting the hidden state sequence H. dim=1 indicates that an averaging operation is performed on the sequence length dimension L, thereby aggregating all word features in the sequence into a single vector representation.

[0044] (2) Line / sentence block granularity features are achieved using 1D-CNN (one-dimensional convolutional neural network) + MaxPooling.

[0045] To enhance the extraction capability of local continuous patterns (e.g., dangerous API (Application Programming Interface) calls + missing checks), a one-dimensional convolution is used to slide along the token sequence to simulate “statement block / line level” pattern extraction.

[0046] First, transpose the hidden state H into the input required for the convolution: H'∈ B×Q×L .

[0047] Then, perform a one-dimensional convolution Conv1D to obtain the convolution output: C=Conv1D(H';k=10,s=5,p=5), C∈ B ×d×L’ The convolution parameters are: kernel_size (k) = 10, stride (s) = 5, padding (p) = 5, and out_channels (number of output channels) = d. The number of output channels, out_channels, is denoted as d, which represents the feature dimension extracted by the convolution operation, i.e., the dimension of the feature representation generated at each position. L' represents the length of the sequence after convolution.

[0048] Finally, the convolutional output is aggregated by max pooling to obtain sentence block-level features: ,in, This represents the convolutional feature slice at position t, which is the channel feature representation of all samples at that position.

[0049] The corresponding implementation is torch.max(line_feat,dim=2)[0], which outputs a shape of [B,d]. Here, line_feat is the "local semantic block feature" extracted by convolution, dim=2 represents the operation on the second dimension (sequence dimension L′), and torch.max() selects the maximum value among all positions.

[0050] (3) Function-level features are projected using CLS (classification label).

[0051] During the encoding process, the model typically introduces a CLS token (classification token) at the beginning of the input sequence and calculates the hidden vector corresponding to that position through the encoder. This hidden vector is the first vector h of the hidden state sequence mentioned earlier. l This is used to represent the global semantic information of the entire input sequence. The hidden vector h at the CLS (classification label) location is taken. l As a global semantic representation: h cls =h l ; And by linear projection onto a unified dimension, we obtain the function-level features: f func =W f h cls +b f W f ∈ H×d Among them, W f The weight matrix, b, represents the linear mapping. f This represents the bias vector.

[0052] The corresponding implementation is func_proj(hidden_states[:,0,:]), which outputs a shape of [B,d].

[0053] In summary, we obtain a three-granularity vector: f token ,f line ,f func ∈ d ; Where d=384.

[0054] Step 3, Adaptive Hierarchical Attention: The three-granularity weights are calculated and weighted by a gating network and Softmax (normalized exponential function), and the fusion vector and weights are output for visualization.

[0055] Step 3 is used to solve the static fusion failure problem caused by different focus and different noise sources of multi-granularity features. The core idea is to score the three granularity features of each sample through a gating network, and then obtain the weights through Softmax (normalized exponential function) to realize dynamic weighted fusion that is adaptive to the sample; at the same time, the output weights are used for subsequent visualization and interpretation.

[0056] Step 301: Gating scoring and Softmax normalization (normalization exponential function).

[0057] Calculate the gating score for each of the three granularities: , , ; Wherein, weight w * ∈ d×1 .

[0058] The implementation consists of three linear layers: gate_token / gate_line / gate_func:Linear(d,1), where gate_token / gate_line / gate_func are three linear gate networks that map token-level features, sentence block-level features, and function-level features. Each linear gate is a linear transformation structure Linear(d,1), whose input is a feature vector of dimension d and whose output is a scalar gate score, which represents the importance of the corresponding granular feature in the current sample.

[0059] The concatenation of the three gating scores yields a concatenation vector: G=[g t ,g l ,g f ]∈ B×3 Among them, g t g l and g f , respectively, represent the gating scores calculated for the three granular features through the corresponding linear gating layer, and B represents the batch size.

[0060] Then, the attention weights are obtained by processing the gating vector G at the granular dimension using Softmax (normalized exponential function): [α] t ,α l ,α f ]=Softmax(G), α t +α l +α f =1.

[0061] The corresponding code implementation is attention_weights=F.softmax(gate_scores,dim=1), with an output shape of [B,3]; where gate_scores represents a vector G composed of gating scores calculated by the gating network from the three granular features; F.softmax() is the normalized exponential function; and attention_weights are the attention weights.

[0062] Step 302: Weighted fusion and nonlinear stabilization.

[0063] The three granular features are weighted and summed according to their weights to obtain the fused representation: ffused =α t f token +α l f line +α f f func ; The corresponding implementation is to multiply each item by its weight and then add them together.

[0064] To enhance the nonlinear expressive power of the fusion representation and suppress overfitting, a linear transformation (Linear) + ReLU (nonlinear activation function) + Dropout (random deactivation operation) is performed on the fusion representation to obtain the fusion features: ; Where the probability of Dropout (random deactivation) is p=0.1, It is a non-linear activation function. This represents a random deactivation operation, where W and b represent the weight matrix and bias vector of the linear transformation, respectively.

[0065] The corresponding implementation is fusion_transform=Linear(d,d)+ReLU+Dropout(0.1). In the code implementation, fusion_transform represents the fused features, Linear(d,d) is the linear transformation, and d represents the feature dimension, which is used to represent the vector length of each granular feature and the fused feature.

[0066] Final output: fused features ∈ d and attention weights [α] t ,α l ,α f ]∈ 3 Used for visual analysis.

[0067] Step 4, Vulnerability Classification (RobertaClassificationHead): Perform binary classification on the fused representation and output the vulnerability probability.

[0068] Fusion features ∈ d The system performs binary classification output, with the classification header employing a two-layer structure: Dropout: The ratio is config.hidden_dropout_prob (the probability of hidden dropout in the configuration file); Activation is achieved by the linear mapping d→d+tanh (hyperbolic tangent activation function); Dropout (random deactivation); The output layer d→2 yields logits (unnormalized classification scores).

[0069] The corresponding formula is: u=Dropout( ); v = tanh(W1u + b1); z = W2Dropout(v) + b2, z∈ 2 ; Where u represents the fusion feature The intermediate feature representation after random dropout processing is defined as follows: v represents the feature representation of intermediate feature u after linear transformation and activation by hyperbolic tangent function (tanh); W1 represents the weight matrix of the first layer linear transformation; b1 represents the bias vector of the first layer linear transformation; z represents the unnormalized score (logits) of the classification output, which is used to represent the prediction result of the input sample belonging to different categories; W2 represents the weight matrix of the output layer; and b2 represents the bias vector of the output layer.

[0070] Finally, based on z, the vulnerability probability P(y=1|x) can be obtained through Softmax (normalized exponential function).

[0071] Step 5, Training Phase: Comparative Learning and Consistency Constraints (Dual-View Loop Closure): Under dual-view augmented input, the cross-entropy loss, supervised comparative loss, and consistency distillation loss are jointly optimized to achieve robust training loop closure.

[0072] Step 5 is enabled only during the training phase to improve the model’s robustness to equivalent surface deformations and prevent gated fusion from degenerating into a single branch.

[0073] Step 501: Enhanced input with dual views.

[0074] Perform two random enhancements on the same code segment, including variable renaming, semantic equivalence substitution, and insertion of redundant code, to obtain View1. Figure 1 ) and View2 (view Figure 2 The inputs are converted into model inputs and the training dataset is returned as (input_ids_view1, input_ids_view2, label). During the validation / testing phase, only a single view input is returned. The first view input, input_ids_view1, represents the first set of lexical number sequences obtained after random augmentation of the original code. The second view input, input_ids_view2, represents the second set of lexical number sequences obtained after another random augmentation of the same original code. The label represents the true category label corresponding to the code sample.

[0075] Step 502: Supervised contrastive learning (SupCon) projector head and temperature coefficient.

[0076] To perform SupCon (supervised contrastive learning), the fused features ∈ d The mapped fused features are obtained by mapping to 128 dimensions using a projection head: p=W2(ReLU(W1 ))∈ 128 ; The corresponding implementation is projection_head:Linear(d,d)+ReLU+Linear(d,128); where d represents the dimension of the fused features, 128 represents the dimension of the embedding space used in contrastive learning, and the weight parameters in the linear transformation are all learnable parameters; projection_head is a feature transformation module used to map the fused features to the embedding space required for contrastive learning.

[0077] The temperature coefficient of the loss function SupConLoss for supervised contrastive learning is temperature=0.1.

[0078] Under dual-view conditions, the two enhanced views of the same code sample are constructed into a feature tensor [B,2,128], and positive / negative sample sets are formed with label supervision, bringing samples with the same label closer and pushing samples with different labels further apart.

[0079] Step 503, Self-distillation consistency constraint (KL Consistency Loss).

[0080] For the two views, we obtain logits (classification output scores) z1 and z2 respectively. Through bidirectional KL constraints (also known as self-distillation consistency constraints), their prediction distributions are consistent (corresponding to Self-Distillation (KL divergence) in the implementation, where KL divergence represents KL divergence, which is used to measure the difference between two probability distributions; Self-Distillation represents the self-distillation mechanism, which is used to constrain the consistency of model prediction results under different views; it is calculated simultaneously with SupCon).

[0081] Step 504: Joint loss function and training loop closure.

[0082] During the training phase, the following joint loss is jointly optimized: L=L ce +λL supcon +αL kl ; Among them, Lce Let L represent the cross-entropy loss, and λ and α represent the contrastive loss. supcon With consistency loss L kl The weighting coefficients.

[0083] As one implementation method, the formula for calculating the weighting coefficient is as follows: ; ; Where, imbalance_ratio represents the sample imbalance degree, and imbalance_ratio = number of non-vulnerable samples / number of vulnerable samples (the larger the value, the more severe the imbalance); epoch represents the current training epoch; total_epochs represents the total number of training epochs; and the base value, peak value, and decay coefficient of the comparison loss weight are λ. base , λ peak and λ decay The values ​​are 0.01, 0.15, and 0.8, respectively; imb_thresh_λ is the sensitivity threshold of the contrastive loss weights to imbalance, typically set to 8; the base value, maximum value, and smoothing coefficient of the contrastive loss weights are α, β, γ, β ... base α max and α smooth The values ​​are 0.3, 3 and 2 respectively; imb_thresh_α is the sensitivity threshold of the consistency loss weight to the degree of imbalance, which is generally set to 15.

[0084] λ (contrastive loss weight) and α (consistency loss weight) are dynamically calculated based on the imbalance of the training set samples and the training epochs. The core objective is to allow the weights to adaptively adjust with the training process and data distribution, ensuring that the model first learns basic classification capabilities while adapting to the characteristics of sample imbalance and maximizing the effect of auxiliary loss. The contrastive loss weight focuses on the imbalance and dynamically adjusts the constraint strength with the training process, aiming to accurately address the problem of insufficient feature learning for minority class vulnerability samples, while balancing the contribution of contrastive learning and classification tasks throughout the entire training cycle. The consistency loss weight focuses on steady growth, responding only mildly to the imbalance, ensuring the consistency of predictions between the two views throughout the process, and adapting to the requirements of robust training closed loop.

[0085] This embodiment employs a training loop consisting of dual-view enhancement → three-granularity extraction → gating fusion → classification output → SupCon (supervised contrastive learning) representation alignment + KL prediction alignment → reverse update. This allows the model to learn a feature space that is semantically invariant and semantically separable from vulnerabilities, and to stabilize the learning of gating weights.

[0086] Step 6: Weighted sampling training strategy for class imbalance.

[0087] To alleviate the training bias caused by the uneven distribution of vulnerability samples and non-vulnerability samples, a weighted sampling strategy based on the reciprocal of class frequency is introduced during the training phase.

[0088] Specifically, the process involves: statistically analyzing the distribution of labels in the training set, calculating the number of samples in each class (class_sample_count), and taking its reciprocal as the class weight (weight = 1 / class_sample_count). Further, based on the class label of each training sample i, the corresponding class weight is assigned to that sample, thus constructing a sample weight vector (samples_weight) of the same length as the total number of samples in the training set (if the training set has 10,000 samples, then the sample weight is a vector of length 10,000, reflecting the "probability weight" of each specific sample being selected during sampling). Then, a weighted random sampler (WeightedRandomSampler(samples_weight, len(samples_weight)) is constructed for sampling training, ensuring that minority class vulnerability samples are sampled more fully during training. This sampling mechanism artificially increases the frequency of minority class vulnerability samples in each training batch, achieving a logically balanced data distribution at the training input. This ensures that the model can more fully learn the vulnerability pattern features under imbalanced class distribution, effectively solving the technical defects of insufficient model training and recall fluctuations caused by sample imbalance, thereby significantly enhancing the recall stability of the detection results.

[0089] In the model execution process of this embodiment, the input source code sample is first processed through input standardization and pre-training encoding steps to obtain a length-aligned token sequence, which is then converted into a unified hidden layer representation sequence by a pre-trained encoder. Subsequently, this hidden layer representation is input into a multi-granularity feature encoding step to extract semantic features at the token, statement-level, and function-level granularities, and uniformly mapped to the same dimensional space. Then, the multi-granularity features enter an adaptive gating fusion step, where the contribution weights of different granularity features in the current sample are calculated through a gating network, and dynamic weighted fusion is achieved through Softmax (normalized exponential function) normalization to generate a comprehensive semantic representation vector. The fused semantic representation is first input into a vulnerability classification step, and the function-level vulnerability prediction result is output by the classifier to realize vulnerability identification and risk assessment of the input code sample.

[0090] During the training phase, after obtaining the classification prediction results and corresponding feature representations, semantically equivalent multi-view inputs are constructed for the same code sample. Combined with contrastive learning and prediction consistency constraints, the classification loss, representation alignment loss, and prediction consistency loss are jointly optimized to form a stable training loop and improve the model's generalization ability and robustness in complex real-world code environments.

[0091] Compared with existing vulnerability detection methods based on single-granularity semantic representation or fixed feature fusion strategies, the source code vulnerability detection method based on adaptive multi-granularity semantic fusion and contrast consistency constraints proposed in this embodiment has the following advantages in terms of model stability, detection accuracy, and engineering applicability: (1) Improve the detection stability of local triggering vulnerabilities.

[0092] By simultaneously extracting token-level, statement-block-level, and function-level semantic features based on a unified encoding representation, the model can simultaneously focus on local triggering patterns and overall semantic structure, reducing the dilution phenomenon of critical vulnerability triggering statements in the global semantic aggregation process, thereby improving the ability to identify and detect vulnerabilities triggered by local statements.

[0093] (2) Achieve collaborative modeling of cross-scale semantic information.

[0094] By introducing an adaptive gating fusion mechanism, the importance weights of features at different granularities are dynamically calculated based on the features of the input samples, enabling adaptive integration of local and overall semantic information. This allows the model to automatically select more effective sources of semantic evidence in different vulnerability scenarios, thereby improving the model's adaptability to complex vulnerability patterns and overall detection accuracy.

[0095] (3) Enhance the robustness of the model to changes in code style.

[0096] By introducing semantically equivalent multi-view enhancement and contrast consistency constraints during the training phase, the model can maintain stable semantic representation and prediction results under conditions such as variable naming changes, redundant statement insertion, and code format differences. This reduces the model's dependence on surface features and improves its generalization ability in cross-project and cross-coding style scenarios.

[0097] (4) Alleviate the training bias problem caused by the imbalance of dataset classes.

[0098] By introducing a weighted sampling strategy based on category frequency statistics, the frequency of vulnerability samples appearing during the training process is increased, enabling the model to learn low-frequency category samples more fully, thereby improving the ability to identify vulnerability types and expanding the overall vulnerability detection coverage.

[0099] (5) Improve the overall stability and engineering applicability of the model.

[0100] By constructing a training closed loop of "multi-granularity feature extraction - adaptive fusion - classification prediction - consistency constraint optimization", the model can stably output vulnerability prediction results in complex real code environments, reduce dependence on specific data distributions or coding styles, and improve the engineering application value of the model in automatic code auditing and vulnerability detection systems.

[0101] Therefore, this embodiment, while ensuring detection accuracy, improves the stability and generalization ability of the model in complex real-world software engineering scenarios, providing an efficient and reliable technical path for automated code vulnerability detection.

[0102] Example 2 This embodiment provides an intelligent source code vulnerability detection system, including: The pre-trained encoding module is configured to: acquire the function-level source code text to be detected, generate a word sequence through a word segmenter, align the word sequence by length, and extract the hidden state through an encoder; The multi-granularity feature encoding module is configured to extract three-granularity features based on the hidden state through the multi-granularity feature extraction module. The three-granularity features include word granularity features, sentence block granularity features, and function granularity features. An adaptive gated fusion module is configured to: obtain fused features based on three-granularity features through an adaptive gated fusion unit; the adaptive gated fusion unit scores the three-granularity features separately through a gated network, obtains the three-granularity attention weights through a normalized exponential function, performs weighted fusion on the three-granularity features to obtain a fused representation, and performs linear transformation, linear correction unit and regularization on the fused representation to obtain the fused features; The vulnerability classification module is configured to obtain the vulnerability probability based on fused features and a classifier.

[0103] It should be noted that each module in this embodiment corresponds one-to-one with each step in Embodiment 1, and their specific implementation processes are the same, so they will not be repeated here.

[0104] Example 3 This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the source code vulnerability intelligent detection method described in Embodiment 1 above.

[0105] Example 4 This embodiment provides a computer device, such as... Figure 2As shown, the system includes a computer-readable storage medium 1003, a processor 1001, a communication interface 1002, and a computer program stored on the computer-readable storage medium 1003 and executable on the processor 1001. The processor 1001, communication interface 1002, and computer-readable storage medium 1003 can be connected via a bus or other means. The communication interface 1002 is used to receive and send data. When the processor 1001 executes the program, it implements the steps of the source code vulnerability intelligent detection method described in Embodiment 1 above.

[0106] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A source code vulnerability intelligent detection method, characterized in that, include: The function-level source code text to be detected is obtained, a word segmenter is used to generate a word sequence, the word sequence is length-aligned, and the hidden state is extracted by an encoder. Based on the hidden state, a multi-granularity feature extraction module is used to extract three-granularity features from the hidden layer representation output by the same pre-trained encoder. The three-granularity features include word granularity features, sentence block granularity features, and function granularity features. Based on three-granularity features, an adaptive gated fusion fusion derives a fused feature. The adaptive gated fusion derives a fused feature by scoring the three-granularity features separately through a gated network, obtaining the three-granularity attention weights through a normalized exponential function, and then performing weighted fusion on the three-granularity features to obtain a fused representation. The fused representation is then subjected to linear transformation, linear correction unit and regularization to obtain the fused feature. The multi-granularity feature extraction module, adaptive gating fusion unit, and classifier constitute the source code vulnerability detection model. Under dual-view augmented input, the source code vulnerability detection model jointly optimizes the cross-entropy loss, supervised contrast loss, and consistency distillation loss for closed-loop training. The source code vulnerability detection model introduces a weighted sampling strategy based on the reciprocal of the category frequency during the training phase; The weights of the supervised contrast loss and the consistency distillation loss are dynamically calculated based on the imbalance of the training set samples and the training rounds. Based on the fusion features, the vulnerability probability is obtained through a classifier.

2. The intelligent source code vulnerability detection method as described in claim 1, characterized in that, The extraction of word-level features employs linear projection and average pooling.

3. The intelligent source code vulnerability detection method as described in claim 1, characterized in that, The extraction of statement block granular features employs one-dimensional convolution and max pooling aggregation.

4. The intelligent source code vulnerability detection method as described in claim 1, characterized in that, The extraction of the function granularity features uses linear projection.

5. A source code vulnerability intelligent detection system, employing the source code vulnerability intelligent detection method as described in any one of claims 1-3, characterized in that, include: The pre-trained encoding module is configured to: acquire the function-level source code text to be detected, generate a word sequence through a word segmenter, align the word sequence by length, and extract the hidden state through an encoder; The multi-granularity feature encoding module is configured to extract three-granularity features based on the hidden state through the multi-granularity feature extraction module. The three-granularity features include word granularity features, sentence block granularity features, and function granularity features. An adaptive gated fusion module is configured to: obtain fused features based on three-granularity features through an adaptive gated fusion unit; the adaptive gated fusion unit scores the three-granularity features separately through a gated network, obtains the three-granularity attention weights through a normalized exponential function, performs weighted fusion on the three-granularity features to obtain a fused representation, and performs linear transformation, linear correction unit and regularization on the fused representation to obtain the fused features; The vulnerability classification module is configured to obtain the vulnerability probability based on fused features and a classifier.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the source code vulnerability intelligent detection method as described in any one of claims 1-4.

7. A computer device comprising a computer-readable storage medium, a processor, and a computer program stored on the computer-readable storage medium and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the source code vulnerability intelligent detection method as described in any one of claims 1-4.

Images