Vulnerability detection method-mVulD-DO based on multi-mode combined distillation optimization

Abstract

A vulnerability detection method-mVulD-DO based on multi-modal joint distillation optimization comprises the steps that firstly, after a key code structure diagram is generated, a function name, a variable name and auxiliary representation information are extracted from the key code structure diagram, code slices are combined and input into a pre-training semantic encoder for encoding, and feature tensors of semantic modals are generated; in addition, a heterogeneous adjacency matrix is constructed by using nodes of the key code structure diagram and edges with different attributes, GAT is input for coding, and a feature tensor of the diagram structure is generated. And distilling the semantic feature tensor through a multi-head distillation network to obtain a corresponding single-peak feature, and further extracting a long-distance dependency relationship of the code through BLSTM to obtain a final semantic auxiliary feature. The distribution distance between a graph structure feature space and a semantic auxiliary feature space is calculated by using a dynamic Sinkhorn algorithm, and the features of the optimized mode are further fused by using a global attention layer to ensure the coordination of the features in the feature space. According to the invention, the detection efficiency of the model and the accuracy of vulnerability detection are improved.

Description

Technical Field

[0001] The present invention relates to a multi-modal joint distillation optimization vulnerability detection method - mVulD-DO. Background Art

[0002] Traditional rule-based vulnerability detection methods usually rely on rules and vulnerability features predefined by experts, and identify potential vulnerabilities by statically analyzing source code and matching it with a rule library. However, the limitation of this method is that the coverage of the rule library is limited, and it often fails to effectively detect new types of vulnerabilities with a high false positive rate. Machine learning-based vulnerability detection methods classify by automatically learning features from a large number of source code samples without manually defining vulnerability features. Although such methods can handle a wider range of vulnerability types, they require feature extraction from the code, and feature design still depends on domain knowledge, suffering from problems such as coarse granularity and insufficient accuracy. In contrast, deep learning methods can automatically extract complex features in source code, avoiding the limitations of manual feature design. Deep learning models can learn more abstract and efficient representations from a large amount of data, thus showing superior performance in vulnerability detection. Especially when dealing with large-scale, complex, and diverse vulnerabilities, they have better generalization ability and are thus widely applied to vulnerability detection tasks.

[0003] Among them, the Recurrent Neural Network (RNN) has become a very important model because it can effectively capture the dependencies in time series data. To address the problem that RNN is prone to vanishing or exploding gradients when dealing with long sequences, researchers proposed variants of RNN - Bidirectional Long Short-Term Memory Network (BLSTM) and Bidirectional Gated Recurrent Unit (BGRU). These two models can not only handle forward time dependencies but also backward dependencies, making them more efficient in capturing long-term dependencies and context information in source code. Li et al. (Li Z, Zou D, Xu S, et al. VulDeePecker: A Deep Learning-Based System for Vulnerability Detection[C]. 2018.) and Zou et al. (Zou D, Wang S, Xu S, et al. µVulDeePecker: A Deep Learning-Based System for Multiclass Vulnerability Detection[J], 2019) constructed code snippets using data dependencies and control dependencies and used the BLSTM model to learn the semantic features of code snippets. Although these two methods consider the features of syntax and semantics comprehensively, they lack consideration of the structural features of the code. Further research, such as Zhou et al. (Zhou, Yaqin, et al. "Devign: Effective vulnerability identification by learning comprehensive program semantics via graph neural networks." Advances in neural information processing systems 32 (2019).) adopted Graph Neural Network (GNN), which can better capture structural information such as control flow and data flow in source code by modeling the source code as a graph structure. Although data dependencies and control dependencies are considered, the attributes of nodes and edges are not considered, resulting in insufficient capture of semantic information.Subsequently, Wu et al. (Wu, Bolun, et al. "SlicedLocator: Code vulnerability locator based on sliced dependence graph." Computers & Security 134 (2023): 103469.) considered both sequence features and graph structure features, defined attributes for each edge in the graph structure according to the dependence relationship, and achieved good results in the vulnerability detection task. However, it did not consider higher-level semantic information, and did not fully solve the heterogeneity problem of the feature distributions of the two different modalities of sequence and graph. Patent CN201910447971 uses the program slicing technique to identify code statements related to candidate vulnerability elements based on the program dependence graph (PDG). However, this method focuses on detecting vulnerabilities in a single function, while vulnerabilities in source code are often not limited to a single function, especially in scenarios involving cross-function calls, global variables, etc. Patent CN202110770650 uses the Bert model and BLSTM to detect C source code vulnerabilities. However, BERT and BiLSTM are based on sequence models and have weak capabilities for capturing the graph structure information of source code. Patent CN113868650A discloses a vulnerability detection method based on code heterogeneous intermediate graph representation, which converts each syntax unit (such as statements, functions, variables, etc.) in the source code into nodes and edges in the graph, thereby retaining the structure information of the code. However, this method is difficult to capture long-distance and cross-function dependence relationships. Patent CN118296612A proposes a source code vulnerability detection method based on a sequence and graph dual-channel model. Although this method comprehensively considers semantic and syntactic information, the fusion of the outputs of the graph model and the sequence model through the fully connected layer is not detailed enough, and the heterogeneity between different modality features is not considered. Summary of the Invention

[0004] The present invention discloses a multi-modal joint distillation optimization vulnerability detection method mVulD-DO (Multi-modal Joint Distillation Optimization for Vulnerability Detection), which improves the detection efficiency of the model and the accuracy of vulnerability detection.

[0005] The technical solution adopted by the present invention is as follows: A multi-modal joint distillation optimization vulnerability detection method - mVulD-DO mainly includes the following steps: Step A: Multimodal feature extraction; Extract multi-dimensional structural dependency information from the source code, fuse function calls, execution paths, and data transfer relationships to construct a program dependency network; Based on this network, extract function names, variable names, and auxiliary representation Token_type in the key code structure, and then combine with local code snippets to obtain four semantic modal feature tensors; At the same time, use the nodes and their dependency relationships in the subgraph to construct a heterogeneous adjacency matrix to generate a graph modal feature tensor; Step B: Multi-head feature distillation layer; Input the four semantic modal feature tensors generated in Step A into the multi-head attention mechanism respectively to aggregate global context information and filter out key region features; Subsequently, the feed-forward network performs step-by-step non-linear transformation on the features at each time step to enhance the expression ability of the model; The multi-scale convolutional layer extracts features from different receptive fields to further improve the ability to capture local multi-scale context patterns, and at the same time compresses the feature dimension to reduce redundancy; Finally, the four deeply distilled semantic feature tensors are concatenated in the 0th dimension, which not only retains the independent information of each modality, but also models the global dependencies in the time dimension through BLSTM, thereby further enhancing the cross-modal feature fusion and collaborative representation to generate the final semantic auxiliary feature tensor; Step C: Multimodal feature fusion and optimization; In this step, first, use the global attention layer to perform weighted fusion on the graph modal feature tensor and the semantic auxiliary feature tensor; Through the multi-head attention mechanism, the attention weights are dynamically assigned to the features of different modalities to capture the cross-modal global dependency information, and the normalization layer is used to ensure the stability of the fused features; Then, based on the Sinkhorn distance as a regularization term, optimize the features of the two modalities, minimize their distribution differences, and further improve the alignment and complementarity of the features; The optimized two features will be re-fused in the next iteration and detected; Finally, generate the class prediction of each sample through forward propagation and the fully connected layer, and the classification loss and the Sinkhorn loss together constitute the total loss function, and the model parameters are updated through joint learning for backpropagation, thereby improving the accuracy of classification detection.

[0006] Step A further includes: A1: First, extract multi-dimensional structural dependency information from the source code, including function calls, execution paths, and data transfer relationships, to construct a comprehensive program dependency network; Based on the pre-designed vulnerability feature descriptions: arrays, pointers, operators, API calls, perform directional screening on this network, identify the key code structures that reflect potential vulnerability risks from it, and separate the key subgraphs from the overall network to obtain local and focused structural information; A2: After extracting the key subgraphs, use a tree parser to extract function names, variable names, and auxiliary representation Token_type from them, and then combine with local code snippets to convert them into feature tensors of multiple semantic modalities through an advanced pre-trained semantic encoder; at the same time, construct an adjacency matrix of the heterogeneous graph structure based on the nodes and their dependencies in the subgraph, and use the graph attention network to generate the graph modality feature tensor.

[0007] Step B further includes: B1: First, the multi-head attention layer maps the feature tensors of the four semantic modalities to multiple subspaces, and calculates the vector representations of queries, keys, and values in each subspace respectively; calculates the attention scores through the scaled dot-product attention algorithm, that is, measures the correlation between the query vector and the key vector in the feature space; subsequently, uses the Softmax function to normalize the scores and converts them into a probability distribution form to generate attention weights; the high-weight regions correspond to more important features, which play a key role in aggregating context information when weighted summing through the value vectors, thereby dynamically screening important regions in the input semantic features and capturing global context information; B2: The feed-forward network consists of two fully connected layers; the first layer maps the input features from a low dimension to a high dimension, and then introduces non-linearity through the ReLU activation function; the second layer maps the features from a high dimension back to a low dimension to ensure that the output dimension is consistent with the input; the context information obtained in step B1 is input into the feed-forward network to perform independent non-linear transformations on the features of each time step; B3: The multi-scale convolutional layer takes the time-step feature representation enhanced by the feed-forward network as input, and uses convolutional kernels with three different receptive fields [a×a], [b×b], [c×c] to capture different local feature detail patterns respectively; each convolutional kernel slides on the input feature map to extract local features within the corresponding receptive field, thereby generating feature representations of three scales; subsequently, through 1×1 point convolution, the multi-scale features are weighted and summed along the channel dimension to remove redundant information and compress the feature dimension, generating a more compact information representation, the unimodal features of each modality; B4: Finally, the unimodal feature tensors of each modality obtained in step B3 are concatenated along the 0th dimension. This way not only preserves the independence between modalities but also lays a foundation for subsequent cross-modal interactions; the concatenated tensor is processed by the BLSTM to capture the global dependencies in the time dimension, and finally forms a unified semantic auxiliary feature tensor.

[0008] Step C further includes: C1: By performing weighted calculations on the feature matrix X of the graph modality output in step A and the feature matrix Y of the semantic auxiliary features output in step B, the complex relationships between modalities are dynamically captured to ensure the effective flow of information from each modality. Specifically, for each modality, its features are adjusted to the same dimension through a linear transformation, and the adjusted features are subject to multi-head attention calculation. The model can focus on the important parts of different modality features and perform weighting according to their relevance. Finally, the output is normalized through a normalization layer to obtain the feature tensor after the fusion of different modalities. C2: Calculate the square of the Euclidean distance between the two modality feature matrices X and Y, which is used as the cost matrix C. At the same time, initialize the marginal distributions of the two modality feature matrices X and Y using a uniform distribution and initialize the dual variables as all-1 vectors. Subsequently, use the alternating optimization algorithm Sinkhorn-Knopp to update the dual variables until they converge. C3: After the dual variables converge, convert the cost matrix C into a transport plan matrix according to the dual variables, and calculate the final sinkhorn distance using the regularized Wasserstein distance formula based on the transport plan matrix. C4: Input the tensor after the fusion of different modalities in step C1 into the detector, which uses Cross-EntropyLoss as the cross-entropy loss function for binary classification tasks. At the same time, add the calculated Sinkhorn distance as a regularization term to the loss function, and the regularization coefficient is dynamically adjusted according to the loss value of model training and the sinkhorn distance. , represents the overall scaling, controlling the influence degree of the classification loss in the regulation. When the modality distributions are quite different, automatically increases to strengthen the alignment. When the classification error is large, automatically decreases to preferentially improve the discrimination ability. This process continuously adjusts the distributions of the two modality feature spaces through a joint optimization method to make them better aligned and achieve better feature fusion. Finally, the detection result output by the model performs binary classification based on the optimized feature representation.

[0009] The mVulD-DO of the present invention first collects samples from the CVEfixes and SARD datasets to generate a subgraph of the Program Dependency Graph with Calls (PDGC). Then, feature extraction is carried out from two aspects: on the one hand, function names, variable names, and auxiliary characterization information (Token_type) are extracted from the PDGC subgraph, and combined with code slices and input into CodeBERT for encoding to generate four semantic feature tensors; on the other hand, a heterogeneous adjacency matrix is constructed using the nodes and edges with different attributes of the PDGC subgraph and input into GAT for encoding to generate a graph structure feature tensor. Next, the four semantic feature tensors are distilled through a multi-head distillation network to obtain the corresponding unimodal features, and the long-distance dependencies of the code are further extracted through BLSTM to obtain the final semantic auxiliary features. At the same time, the dynamic Sinkhorn algorithm is used to calculate the distribution distance between the graph structure feature space and the semantic auxiliary feature space, optimize the alignment between modalities, and use the global attention layer to further fuse the features of the optimized modalities to ensure their coordination in the feature space. Finally, the fused features and the Sinkhorn distance are input into the detector to output the detection results. The present invention effectively solves the problems of insufficient code semantic expression, heterogeneity of modality fusion, and feature redundancy by jointly optimizing the multi-modal information of the program and using the multi-head distillation network to enhance and filter redundant features of the sequence modality features, improving the detection efficiency of the model and the accuracy of vulnerability detection. Description of the Drawings

[0010] Figure 1 It is a schematic diagram of the framework structure of the present invention; Figure 2 It is a schematic diagram of the architecture of the multi-head feature distillation layer (MFDN) of the present invention. Detailed Embodiment

[0011] The implementation steps of the present method will be introduced in detail based on the drawings of the present invention. The datasets of this experiment come from CVEfixes and SARD, including synthetic programs, production programs, academic programs, and real-world vulnerability data.

[0012] Referring to Figure 1 、 Figure 2 A source code vulnerability detection method based on multi-modal joint distillation optimization mainly includes the following steps: Step A: Multimodal Feature Extraction; Extract multi-dimensional structural dependency information from the source code, fuse function calls, execution paths, and data transfer relationships to construct a program dependency network; Based on this network, extract function names, variable names, and auxiliary representation Token_type in the key code structure, and then combine with local code snippets to obtain four semantic modal feature tensors; At the same time, use the nodes and their dependency relationships in the subgraph to construct a heterogeneous adjacency matrix to generate a graph modal feature tensor; Aiming at the heterogeneity and complexity of multimodal features, the present invention designs an innovative deep processing framework to optimize feature fusion and information mining, so as to give full play to the complementarity between modalities. Step A further includes: A1: First, extract multi-dimensional structural dependency information from the source code, including function calls, execution paths, and data transfer relationships, and construct a comprehensive program dependency network; Based on the pre-designed vulnerability feature descriptions: arrays, pointers, operators, API calls, perform directional screening on this network, identify the key code structures reflecting potential vulnerability risks from it, and separate the key subgraphs from the overall network to obtain local and focused structural information; A2: After extracting the key subgraph, use a tree parser to extract function names, variable names, and auxiliary representation Token_type from it, and then combine with local code snippets to convert them into feature tensors of multiple semantic modalities through an advanced pre-trained semantic encoder; At the same time, construct a heterogeneous graph structure adjacency matrix based on the nodes and their dependency relationships in the subgraph, and use a graph attention network to generate a graph modal feature tensor; These multimodal features provide rich basic information for subsequent deep fusion and optimization.

[0013] Step B: Multi-Head Feature Distillation Layer; Input the four semantic modal feature tensors generated in Step A into the multi-head attention mechanism respectively to aggregate global context information and screen out key region features; Subsequently, the feed-forward network performs step-by-step non-linear transformation on the features at each time step to enhance the expression ability of the model; The multi-scale convolutional layer extracts features from different receptive fields to further improve the ability to capture local multi-scale context patterns, and at the same time compresses the feature dimension to reduce redundancy; Finally, the four deeply distilled semantic feature tensors are concatenated in the 0th dimension, which not only retains the independent information of each modality, but also models the global dependencies in the time dimension through BLSTM, thereby further enhancing the fusion and collaborative representation of cross-modal features and generating the final semantic auxiliary feature tensor.

[0014] Step B further includes: B1: First, the multi-head attention layer maps the feature tensors of the four semantic modalities into multiple subspaces. The vector representations of queries, keys, and values are calculated separately for each subspace. The attention scores are calculated through the scaled dot-product attention algorithm, which measures the correlation between the query vector and the key vector in the feature space. Subsequently, the scores are normalized using the Softmax function to convert them into a probability distribution form, generating attention weights. The high-weight regions correspond to more important features, which play a key role in aggregating context information when weighted summing with the value vectors, thereby dynamically screening important regions in the input semantic features and capturing global context information.

[0015] B2: The feed-forward network consists of two fully-connected layers. The first layer maps the input features from a low dimension to a high dimension and then introduces non-linearity through the ReLU activation function. The second layer maps the features back from a high dimension to a low dimension to ensure that the output dimension is the same as the input. The context information obtained in step B1 is input into the feed-forward network to perform an independent non-linear transformation on the features at each time step.

[0016] B3: The multi-scale convolutional layer takes the time-step feature representations enhanced by the feed-forward network as input and uses convolutional kernels of three different receptive fields [a×a], [b×b], [c×c] to capture different local feature detail patterns respectively. Each convolutional kernel slides on the input feature map to extract the local features within the corresponding receptive field, thereby generating feature representations of three scales. Subsequently, a 1×1 point convolution is used to perform weighted summation of the multi-scale features along the channel dimension to remove redundant information and compress the feature dimension, generating a more compact information representation, the unimodal features of each modality.

[0017] B4: Finally, the unimodal feature tensors of each modality obtained in step B3 are concatenated along the 0th dimension. This way not only preserves the independence between modalities but also lays the foundation for subsequent cross-modal interactions. The concatenated tensor is processed by the BLSTM to capture the global dependencies in the time dimension. This operation not only strengthens the co-representation between different modality features but also captures the implicit complementary information at the global level, finally forming a unified semantic auxiliary feature tensor.

[0018] Step C: Multimodal Feature Fusion and Optimization; In this step, first, a global attention layer is used to perform weighted fusion on the graph modality feature tensor and the semantic auxiliary feature tensor; through the multi-head attention mechanism, attention weights are dynamically assigned to features of different modalities to capture global cross-modal dependency information, and the stability of the fused features is ensured through a normalization layer; then, based on the Sinkhorn distance as a regularization term, the features of the two modalities are optimized to minimize their distribution differences and further enhance the alignment and complementarity of the features; the optimized two features will be refused in the next iteration and detected; finally, through forward propagation and a fully connected layer, class predictions for each sample are generated, and the classification loss and the Sinkhorn loss together constitute the total loss function, and the model parameters are updated through backpropagation by joint learning, thereby improving the accuracy of classification detection.

[0019] Step C further includes: C1: By performing weighted calculations on the feature matrix X of the graph modality output in Step A and the feature matrix Y of the semantic auxiliary features output in Step B, the complex relationships between modalities are dynamically captured to ensure that information can flow effectively from each modality; specifically, for each modality, its features are adjusted to the same dimension through a linear transformation, and the adjusted features are subject to multi-head attention calculations, enabling the model to focus on important parts of the features of different modalities and perform weighting according to their correlations; finally, the output is normalized through a normalization layer to obtain the feature tensor after fusion of different modalities; C2: Calculate the square of the Euclidean distance between the two modality feature matrices X and Y, and use it as the cost matrix C; at the same time, initialize the marginal distributions of the two modality feature matrices X and Y using a uniform distribution and initialize the dual variables as all-1 vectors; subsequently, use the alternating optimization algorithm Sinkhorn-Knopp to update the dual variables until they converge; C3: After the dual variables converge, transform the cost matrix C into a transport plan matrix according to the dual variables, and calculate the final Sinkhorn distance using the regularized Wasserstein distance formula based on the transport plan matrix; C4: Input the tensor after fusion of different modalities in Step C1 into the detector, which uses Cross-EntropyLoss as the cross-entropy loss function for binary classification tasks; at the same time, add the calculated Sinkhorn distance as a regularization term to the loss function, and the regularization coefficient is dynamically adjusted according to the loss value of model training and the Sinkhorn distance , represents overall scaling, controlling the influence degree of the classification loss in the regulation; when the modality distribution differences are large, automatically increases to strengthen alignment; when the classification error is large, Automatically reduce to preferentially enhance the discrimination ability; this process continuously adjusts the distributions of the two modal feature spaces through a joint optimization method to better align them and achieve better feature fusion; finally, the detection results output by the model are binary-classified based on the optimized feature representation.

[0020] Refer to Table 1: The experimental dataset is a mixed dataset of CVEfixes + SARD. Select classical deep learning-based methods VulDeePecker, SlicedLocator, SySeVR, Devign and the mVulD-DO method for comparative experiments. The dataset is divided into a training set, a validation set and a test set in a ratio of 8:1:1 to balance positive and negative samples. The table shows the comparative experimental results of the mVulD-DO method and other methods. From the results, the average ACC index of mVulD-DO is 32.14%, 9.72% and 24% higher than that of the sequence-based VulDeepecker, SySeVR and the graph-based Devign respectively. It is 5.07% higher than the average ACC index of the SlicedLocator method which is based on both sequence and graph dual channels. For the recall rate, F1 value and FPR, this method is also better than the comparative methods, indicating that this method can not only capture real vulnerabilities more comprehensively (higher recall rate and F1 value) in vulnerability detection, but also effectively reduce false positives (lower FPR), thus improving the overall detection performance and practicality.

[0021] Table 1 Comparative experimental data on the detection performance between the present invention and other vulnerability detection methods Refer to Table 2: To prove the effectiveness of function names, variable names, Token_Type, multi-head feature distillation, and Sinkhorn optimization on mVulD-DO, the present invention designs the following 5 groups of ablation experiments, using recall, accuracy, F1 value, and FPR as evaluation metrics. The F1 values of mVulD-DO are respectively 1.3%, 1.3%, and 3.35% higher than those of the methods without function names, variable names, and Token_Type as modalities, and are respectively 9.11% and 1.77% higher than those of the method without feature distillation and Sinkhorn optimization. mVulD-DO is also superior to these groups of ablation experiments in terms of ACC and FPR, indicating that this method has more advantages in overall classification performance and false positive rate control. The reason for the slightly lower recall may be that feature distillation and Sinkhorn optimization enhance the model's attention to high-confidence features to a certain extent, affecting the recall of some boundary samples. However, generally speaking, by combining program semantic information (function names, variable names, code slices, Token_type) and graph structure information (control dependence, data dependence, function call dependence), and using a multi-head distillation network to enhance feature and filter redundancy for sequence modal features, mVulD-DO has achieved optimal or near-optimal results in multiple metrics such as F1 value, ACC, and FPR, effectively solving the problems of insufficient code semantic expression, heterogeneity of modal fusion, and feature redundancy, and improving the detection efficiency of the model and the accuracy of vulnerability detection.

[0022] Table 2 Results of the ablation experiments of the present invention

Claims

1. A multi-modal joint distillation-based optimized vulnerability detection method - mVulD-DO, characterized in that It mainly includes the following steps: Step A: Multimodal feature extraction; Extract multi-dimensional structural dependency information from the source code, fuse function calls, execution paths, and data transfer relationships to construct a program dependency network; Based on this network, extract function names, variable names, and auxiliary representation Token_type in the key code structure, and combine with local code snippets to obtain four semantic modal feature tensors; At the same time, use the nodes and their dependency relationships in the subgraph to construct a heterogeneous adjacency matrix to generate a graph modal feature tensor; Step B: Multi-head feature distillation layer; Input the four semantic modal feature tensors generated in Step A into the multi-head attention mechanism respectively to aggregate global context information and filter out key region features; Subsequently, the feed-forward network performs step-by-step non-linear transformation on the features at each time step to enhance the expression ability of the model; The multi-scale convolutional layer extracts features from different receptive fields to further improve the ability to capture local multi-scale context patterns, and at the same time compresses the feature dimension to reduce redundancy; Finally, the four deeply distilled semantic feature tensors are concatenated in the 0th dimension, which not only retains the independent information of each modality, but also models the global dependencies in the time dimension through BLSTM, thereby further enhancing the fusion and co-representation of cross-modal features to generate the final semantic auxiliary feature tensor; Step C: Multimodal feature fusion and optimization; In this step, first, use the global attention layer to perform weighted fusion on the graph modal feature tensor and the semantic auxiliary feature tensor; Through the multi-head attention mechanism, the attention weights are dynamically assigned to the features of different modalities to capture the global dependencies across modalities, and the normalization layer is used to ensure the stability of the fused features; Next, based on the Sinkhorn distance as a regularization term, optimize the features of the two modalities, minimize their distribution differences, and further improve the alignment and complementarity of the features; The optimized two features will be re-fused and detected in the next iteration; Finally, generate the class prediction of each sample through forward propagation and the fully connected layer, and the classification loss and the Sinkhorn loss jointly constitute the total loss function, and the model parameters are updated through joint learning for backpropagation, thereby improving the accuracy of classification detection.

2. The vulnerability detection method based on multi-modal joint distillation optimization according to claim 1, wherein Step A further includes: A1: First, extract multi-dimensional structural dependency information from the source code, including function calls, execution paths, and data transfer relationships, to construct a comprehensive program dependency network; Based on the pre-designed vulnerability feature descriptions: arrays, pointers, operators, API calls, perform directional screening on this network, identify the key code structures reflecting potential vulnerability risks from it, and separate the key subgraphs from the overall network to obtain local and focused structural information; A2: After extracting the key subgraph, use the tree parser to extract function names, variable names, and auxiliary representation Token_type from it, and combine with local code snippets to convert them into feature tensors of multiple semantic modalities through an advanced pre-trained semantic encoder; At the same time, construct a heterogeneous graph structure adjacency matrix based on the nodes and their dependency relationships in the subgraph, and use the graph attention network to generate a graph modal feature tensor.

3. A vulnerability detection method based on multi-modal joint distillation optimization according to claim 1, characterized in that Step B further includes: B1: First, the multi-head attention layer maps the feature tensors of the four semantic modalities to multiple subspaces. The vector representations of queries, keys, and values are calculated separately in each subspace. The attention scores are calculated through the scaled dot-product attention algorithm, which measures the correlation between the query vector and the key vector in the feature space. Subsequently, the scores are normalized using the Softmax function to convert them into a probability distribution form, generating attention weights. The high-weight regions correspond to more important features, which play a key role in aggregating context information when weighted summing with the value vectors, thereby dynamically screening important regions in the input semantic features and capturing global context information. B2: The feed-forward network consists of two fully connected layers. The first layer maps the input features from a low dimension to a high dimension, and then introduces non-linearity through the ReLU activation function. The second layer maps the features from a high dimension back to a low dimension to ensure that the output dimension is consistent with the input. The context information obtained in Step B1 is input into the feed-forward network to perform independent non-linear transformations on the features at each time step. B3: The multi-scale convolutional layer takes the time-step feature representation enhanced by the feed-forward network as input, and uses convolutional kernels with three different receptive fields [a×a], [b×b], [c×c] to capture different local feature detail patterns respectively. Each convolutional kernel slides on the input feature map to extract local features within the corresponding receptive field, thereby generating feature representations of three scales. Subsequently, 1×1 pointwise convolution is used to perform weighted summation of the multi-scale features along the channel dimension, removing redundant information and compressing the feature dimension to generate a more compact information representation, the unimodal features of each modality. B4: Finally, the unimodal feature tensors of each modality obtained in Step B3 are concatenated along the 0th dimension. This way not only preserves the independence between modalities but also lays a foundation for subsequent cross-modal interactions. The concatenated tensor is processed by BLSTM to capture the global dependencies in the time dimension, and finally a unified semantic auxiliary feature tensor is formed.

4. A vulnerability detection method based on multimodal joint distillation optimization according to claim 1, characterized in that, Step C further includes: C1: By performing weighted calculations on the feature matrix X of the graph modality output in Step A and the feature matrix Y of the semantic auxiliary features output in Step B, the complex relationships between modalities are dynamically captured to ensure that information can flow effectively from each modality. Specifically, for each modality, its features are adjusted to the same dimension through a linear transformation, and the adjusted features will undergo multi-head attention calculations. The model can focus on the important parts in the features of different modalities and perform weighting according to their correlations. Finally, the output is normalized through a normalization layer to obtain the feature tensor after fusing different modalities. C2: Calculate the square of the Euclidean distance between the two modality feature matrices X and Y as the cost matrix C. At the same time, initialize the marginal distributions of the two modality feature matrices X and Y using a uniform distribution and initialize the dual variables as all-1 vectors. Subsequently, use the alternating optimization algorithm Sinkhorn-Knopp to update the dual variables until they converge. C3: After the dual variables converge, convert the cost matrix C into a transportation plan matrix according to the dual variables, and calculate the final Sinkhorn distance using the regularized Wasserstein distance formula based on the transportation plan matrix; C4: Input the tensor after fusing different modalities in step C1 into the detector, which uses Cross-Entropy Loss as the cross-entropy loss function for binary classification tasks; meanwhile, add the calculated Sinkhorn distance as a regularization term to the loss function, and the regularization coefficient is dynamically adjusted according to the loss value of model training and the sinkhorn distance. , represents overall scaling, controlling the influence degree of the classification loss in the regulation; when the modality distributions are quite different, automatically increases to strengthen alignment; when the classification error is large, automatically decreases to preferentially improve the discrimination ability; this process continuously adjusts the distributions of the two modality feature spaces through a joint optimization method to make them better aligned and achieve better feature fusion; finally, the detection result output by the model performs binary classification based on the optimized feature representation.

Images