A software vulnerability detection method based on cross-modal fusion and a storage medium

Abstract

The application discloses a software vulnerability detection method based on cross-modal fusion and a storage medium, and the method comprises the following steps: code bimodal representation, converting an input source code function into code sequence Token and program dependency graph PDG bimodal representation; code sequence representation learning, taking a pre-trained code large language model LLM as a sequence encoder, and extracting deep semantic features from the code sequence layer by layer; code graph representation learning, extracting structural features from the program dependency graph layer by layer; prompt-based local information interaction PIM, realizing deep bidirectional knowledge exchange between the two modes through a cross-attention mechanism; cross-attention adaptive global fusion CAM, after feature extraction is completed, adaptively adjusting the fusion weight of different modes in the final decision based on the input code features. The storage medium is realized based on the above method. The application has the advantages of simple principle, wide application range, high precision and the like.

Description

Technical Field

[0001] This invention mainly relates to the field of software development and design technology, specifically a software vulnerability detection method and storage medium based on cross-modal fusion. Background Technology

[0002] Software vulnerabilities are security flaws introduced into software during the design, coding, or configuration phases. These flaws can be exploited by malicious attackers to achieve malicious purposes such as unauthorized access, data breaches, and denial-of-service attacks, causing significant economic losses and even threatening social security. Currently, deep learning-based vulnerability detection methods have received widespread attention in academia and industry, and mainly include: Sequence-based code vulnerability detection methods treat source code as a sequence of natural language text. Lexical analysis (word segmentation) divides the code into token sequences, which are then input into a neural network model for feature extraction and vulnerability prediction. For example, Recurrent Neural Networks (RNN / LSTM) methods: Early representative work VulDeePecker pioneered the use of Long Short-Term Memory (LSTM) networks to model code fragments, successfully detecting various real-world vulnerabilities and demonstrating significantly higher detection accuracy compared to traditional static analysis tools. LSTM addresses the long-range gradient vanishing problem of ordinary RNNs through gating mechanisms (input gate, forget gate, output gate), but its serialization processing method still struggles to capture semantic dependencies across long-distance program structures. SySeVR employs a bidirectional LSTM (Bi-LSTM) combined with an attention mechanism to process code sequences. The forward LSTM captures left-to-right context, while the backward LSTM captures right-to-left context. The attention mechanism adaptively weights and aggregates global information by calculating relevance scores at different positions in the sequence, effectively improving the quality of semantic feature representation. Another example is the pre-trained code large language model approach: With the significant potential of large-scale pre-trained language models in code understanding tasks, researchers have begun applying pre-trained code models to vulnerability detection. LineVD fine-tunes the CodeBERT model on vulnerability datasets, leveraging its semantic knowledge gained from pre-training on code corpora to adapt it to downstream vulnerability detection tasks, achieving finer-grained vulnerability feature extraction. CodeBERT, based on a Transformer encoder architecture, employs joint pre-training on two tasks: Masked Language Modeling (MLM) and Replaced Token Detection (RTD). Trained on both natural language and programming language dual-modal data, it possesses strong code semantic understanding capabilities. UniXcoder is a unified cross-modal pre-trained model that controls model behavior through a masked attention matrix and prefix adapter, utilizing abstract syntax trees and code comments as cross-modal content to enhance code representation, achieving state-of-the-art performance on multiple code-related tasks. UniXcoder uses a 12-layer stacked Transformer architecture, pre-trained through joint optimization of proxy tasks such as masked language modeling and code-comment alignment. In the exploration of generative large language models, some researchers have driven ChatGPT to complete vulnerability identification tasks by using manually designed diverse prompt templates, verifying the feasibility of generative models in this field; Vul-RAG further demonstrated the potential of large language models in vulnerability detection scenarios through retrieval-enhanced generation technology.

[0003] Graph-based code vulnerability detection methods convert source code into a graph structure representation, use the code graph as model input, and leverage graph neural networks to learn graph topology and node / edge features to capture the structured dependencies in the code. Specifically: Code diagram representation types generally include: • Abstract Syntax Tree (AST): A tree-like structure generated by the compiler after lexical and syntactic analysis of the source code. Nodes represent syntactic units (expressions, statements, declarations, etc.), and edges represent parent-child relationships between syntactic units. AST fully preserves the syntactic structure information of the code.

[0004] • Control Flow Graph (CFG): A directed graph with basic blocks as nodes and control transfer relationships as edges, representing all possible execution paths during program execution. A basic block is a continuous sequence of instructions without branches. The CFG reveals the execution order and branching structure of the program.

[0005] • Data Dependence Graph (DDG): Represents the dependencies between variable definitions and usages in a program. If statement S1 defines variable v, and statement S2 uses that definition, then there exists a data dependency edge from S1 to S2. Data dependencies are crucial for tracing data propagation paths (such as taint analysis).

[0006] • Program Dependence Graph (PDG): It combines the Control Dependence Graph (CDG) and the Data Dependence Graph (DDG), with statements or basic blocks as nodes and edges representing both control and data dependencies, thus fully reflecting the program's dependency structure.

[0007] • Code Property Graph (CPG): Integrates information from AST, CFG, and PDG into a unified heterogeneous graph structure. Nodes retain syntactic features (variable types, operators, etc.) and contextual information, while edges are labeled with control flow, data flow, or syntactic relationships. CPG provides a comprehensive perspective on code structure and semantics.

[0008] Graph Neural Network (GNN) methods generally include: Devign transforms code into AST, CFG, DDG, and natural code sequence graphs, using Gated Graph Neural Networks (GGNNs) to extract structural knowledge from various code graphs, and further enhances feature extraction capabilities by combining them with Convolutional Neural Networks (CNNs). GGNNs recursively update the hidden states of nodes through Gated Recurrent Units (GRUs), but only aggregate information from adjacent nodes in each iteration. AMPLE addresses the failure of GNNs on large code graphs by proposing a rule-based graph simplification strategy—reducing the graph size by removing nodes and edges irrelevant to specific analyses, and then using an attention-based GNN to extract knowledge from the simplified graph structure. Reveal transforms code into code attribute graphs, using GRUs and GGNNs for feature extraction, focusing on detecting deep logical flaws in the code. Coca leverages contrastive learning to enhance GNN-based vulnerability detection by maximizing feature similarity between similar vulnerability samples and minimizing similarity with dissimilar samples, learning more discriminative code representations.

[0009] Graph Transformer: To address the limitations of classic GNNs in modeling global graph structures, Graph Transformer was introduced into code-based graph learning. Graph Transformer employs a Transformer-style architecture, with its core consisting of a structure-aware multi-head self-attention mechanism and a feedforward neural network (FFN). The structure-aware self-attention first aggregates features from neighboring nodes, then performs a weighted aggregation based on self-attention scores across all nodes. When updating the representation of each node, Graph Transformer implicitly captures global dependencies by simultaneously considering its direct neighbors and distant nodes in the graph through a global attention score. A residual connection is introduced between the structure-aware self-attention and the FFN to mitigate the potential gradient vanishing problem in deep Graph Transformers.

[0010] However, existing methods still face two core challenges: First, unimodal methods based on code sequences (represented by code-large language models) struggle to capture the structured semantic dependencies of code (such as control dependencies and data dependencies); second, unimodal methods based on code graphs (represented by graph neural networks) lack a deep semantic understanding of the syntactic granularity of code text and the contextual program logic. Empirical studies have shown that sequence models are better at detecting syntax-related vulnerability types (such as external control of format strings, CWE-134), while graph models are better at identifying vulnerability types with structural dependency characteristics (such as resources not properly closed or released, CWE-404).

[0011] To address the inherent limitations of the aforementioned unimodal methods, researchers have begun exploring multimodal learning strategies that simultaneously extract and fuse code sequence features and code graph structural features for detection. One such approach is the multimodal learning method, which aims to integrate the semantic understanding capabilities of LLMs (Local Level Machines) and the structural understanding capabilities of GNNs (Generative Neural Networks) to achieve more comprehensive vulnerability detection. The basic principle involves constructing pre-trained LLMs and GNNs as two independent branches, extracting features from the code sequence and code graph respectively, and then combining these two encoding representations through summation or concatenation before inputting them into a multilayer perceptron for vulnerability prediction.

[0012] However, existing cross-modal fusion schemes have two fundamental flaws: First, cross-modal interactions are shallow. During the feature extraction stage, each modal branch is isolated from the others, interacting only at a shallow level during node initialization. This loses the opportunity for continuous mutual guidance during deep feature extraction. Furthermore, when the compilation / parsing process transforms the code from text to a graph structure, fine-grained semantic context information is lost. The pre-trained LLM code is only used to initialize graph node embeddings—assigning an initial feature vector to each node. The LLM and GNN branches then enter completely independent deep feature extraction stages. During multi-layer Transformer encoding, the sequence representations of the LLM branches are gradually integrated with deeper layers. While GNNs can grasp the contextual semantics, they cannot perceive the code graph structure (such as data dependency edge pointing, control flow branch conditions, etc.) as semantic constraints. Furthermore, when aggregating adjacency information, the node representations of GNN branches cannot perceive the deep semantic intent and contextual relationships of the corresponding tokens. Complementary knowledge from the two modalities cannot be continuously exchanged and enhanced during the progressive feature extraction process. LLM branches may miss patterns with questionable structures but normal semantics; GNN branches may misjudge operations with legitimate graph structures but abnormal semantics. The shallowing of cross-modal interactions leads to low utilization of complementary information, a lack of cross-modal guidance for deep features, and limited detection accuracy.

[0013] Second, the decision fusion is static, employing a fixed-weight strategy to fuse representations of different modalities. This ignores the significant differences in the sensitivity of different vulnerability types to different modalities—the vulnerability patterns of syntax-sensitive vulnerabilities are mainly reflected in semantic features, while structure-dependent vulnerabilities highly rely on graph-based data flow analysis. The diversity of vulnerability types and differences in modal sensitivity are inherent properties of code security. → The detection signals of syntax-sensitive vulnerabilities mainly exist in the surface patterns of code text token sequences (e.g., format string parameters from external input). → The detection signals of structure-dependent vulnerabilities mainly exist in the program's dependency relationships and topological structure patterns (e.g., the pairing relationship of resource acquisition and release). → Fixed-weight fusion is essentially a single-modal importance estimation of the global dataset, and cannot be dynamically adjusted for the features of individual input samples. → On syntax-sensitive vulnerability samples, sequence branches should receive higher weights, but their weights are fixed at 0.2, amplifying the noise of structural branches. → On structure-dependent vulnerability samples, graph branches should receive higher weights, but their weights are fixed, and the semantic noise of sequence branches cannot be suppressed. → Static fixed weights lead to suboptimal fusion across different vulnerability types, with performance degradation or even inferiority to single-modal methods for some types.

[0014] In other words, existing technologies still have the following shortcomings: 1. The cross-modal feature extraction stage lacks a deep bidirectional interaction mechanism. Existing cross-modal vulnerability detection methods (such as Vul-LMGNN) construct the Code Large Language Model (LLM) and Graph Neural Network (GNN) as two independent branches for feature extraction. In the core stage of feature extraction, the LLM branch cannot perceive the guiding information of the graph structure (such as data dependency flow), and the GNN branch cannot perceive semantic context information (such as the semantic intent of variables), resulting in the knowledge of the two modalities not being mutually reinforcing or complementary. The lack of structural constraints in the LLM branch may cause it to miss semantically correct but structurally questionable patterns (such as unverified pointer usage); the lack of semantic context in the GNN branch may cause it to misjudge structurally legal but semantically abnormal operations (such as format string parameters of unknown origin) as safe.

[0015] This is because existing methods limit cross-modal information interaction to a single node initialization stage. After constructing the code attribute graph, Vul-LMGNN only initializes the node embeddings in the graph using a pre-trained code language model; subsequently, the LLM and GNN branches enter independent feature extraction processes. In this design, interaction occurs at a shallow stage of node embedding initialization, rather than the core feature extraction stage. From the technical essence of modal complementarity mechanisms, the complementary value of semantic and structural features should continuously play a role in the progressive process of multi-layer feature extraction—lower-level features need semantic guidance to distinguish different meanings of similar structures, while higher-level features need structural constraints to filter possible semantic biases. Placing interaction only in the initialization stage is equivalent to injecting cross-modal information once at the initial moment of feature extraction, and then allowing the two branches to evolve independently along their respective trajectories. Due to the significant differences in distribution characteristics between semantic and structural representation spaces (the semantic space focuses on symbol co-occurrence and contextual patterns, while the structural space focuses on topological relationships and adjacency transitivity), without continuous interaction, their feature representations gradually "drift," their complementarity gradually weakens, and ultimately, it becomes difficult to maintain cross-modal synergy in higher-level semantic representations.

[0016] 2. Cross-modal decision fusion employs a static fixed-weight strategy. Methods like Vul-LMGNN use a fixed-weight fusion strategy in the decision-making stage—assigning a weight of 0.8 to graph branches and 0.2 to code sequence branches via a tuning parameter λ=0.8. This "one-size-fits-all" fixed-weight design ignores a crucial fact: vulnerability patterns exhibit modal specificity. The detection signals for syntax-sensitive vulnerabilities are primarily reflected in semantic features (sequence branches should be assigned higher weights), while the detection signals for structure-dependent vulnerabilities are primarily reflected in graph structure features (graph branches should be assigned higher weights). Fixed weights cannot be dynamically adjusted based on the features of the input samples, resulting in unstable and suboptimal detection performance across different vulnerability types.

[0017] This is because the mathematical form of static fusion implicitly assumes that all input samples follow the same modality importance distribution. The fixed interpolation fusion mechanism of Vul-LMGNN implicitly assumes that the modality importance of all code samples follows the exact same prior distribution. From a statistical learning perspective, this assumption does not hold true in vulnerability detection scenarios—under the CWE classification system, different types of vulnerabilities exhibit vastly different distribution patterns in the multimodal feature space: for syntax-driven vulnerabilities (such as the format string vulnerability CWE-134), vulnerability features are concentrated on abnormal patterns on the surface of the code text (such as user-controllable data flowing directly to the formatting function), and the signal-to-noise ratio of sequence modality is much higher than that of graph structure modality; for structure-driven vulnerabilities (such as resource leakage CWE-404), vulnerability features are hidden in the combination of cross-function resource acquisition-release pairings and abnormal path patterns, and the signal-to-noise ratio of graph structure modality is much higher than that of sequence modality. Using a single fixed weight to match these two drastically different task distributions inevitably leads to the loss of predictive information and degradation of detection accuracy for at least one type of task.

[0018] 3. Performance degradation or even outperformance compared to single-modal methods is rare on certain vulnerability types; due to the combined effect of the two aforementioned defects, existing cross-modal methods not only fail to demonstrate the complementary advantages of multimodal fusion on some vulnerability types, but also experience performance degradation. Vul-LMGNN's detection accuracy on specific types such as CWE-125 (out-of-bounds read) and CWE-476 (null pointer dereference) is lower than that of the single-sequence method SVulD or the single-graph method DeepDFA. The fundamental reason is that the insufficient cross-modal fusion mechanism cannot establish effective inter-modal semantic associations, resulting in poor integration capabilities in complex vulnerability scenarios that require simultaneous reliance on semantic reasoning and structural tracing.

[0019] This is because the combined effect of shallow interaction and static fusion leads to incomplete multimodal integration and underutilization of complementary information. In systems with both shallow interaction and static fusion, these two types of defects produce a combined effect, causing the detection performance of specific vulnerability types to degrade compared to single-modal methods. Take null pointer dereferencing (CWE-476) as an example: effective detection requires LLM to identify the semantic pattern of "pointers not being verified before use," while GNN needs to track the data dependency path from the pointer initialization failure point to the usage point. Under the shallow interaction design, LLM cannot obtain continuous "guidance" from the graph structure regarding data dependency paths during the deep feature extraction stage, potentially leading to semantic anomaly signals being "diluted" by representations of irrelevant context; GNN cannot prune graph paths based on semantic information in the deep stage, causing critical paths to be submerged in graph structure noise. In the static fusion stage, the fixed modal weights cannot be dynamically adjusted according to the modal sensitivity of CWE-476 (semantics and structure are equally important), ultimately resulting in partial loss of complementary information from both modalities during the integration decision, with detection performance even worse than a good single-modal model.

[0020] 4. Traditional graph neural networks have a bottleneck in their expressive power when capturing long-distance dependencies in code graphs. In terms of code graph representation learning, the built-in message passing mechanism of classic GNNs (such as GGNN and GAT) can only capture first-order neighbor features in each iteration. In multi-hop scenarios, it is prone to over-smoothing and performance degradation, making it difficult to effectively model long-distance control and data dependencies that span multiple nodes in the code graph.

[0021] This is because the inductive bias of traditional GNNs limits their ability to model the global graph topology. The inductive bias of classic GNNs like GGNN is designed based on the locality assumption—they update node representations by recursively aggregating local neighborhood information, making the receptive domain of each node limited to the neighborhood range within a certain number of iterations. For long-distance dependencies in program dependency graphs (PDGs) that span multiple function call levels and complex control flow branches, classic GNNs require extremely deep layers to capture them. However, deep GNN layers are prone to oversmoothing problems (convergence of all node representations) and gradient vanishing, ultimately impairing the ability to extract discriminative features. Graph Transformers, through a structure-aware multi-head self-attention mechanism, compute the attention score of any pair of nodes in the graph, providing a global receptive domain for each node, effectively solving the problem of long-distance dependency modeling. Traditional GNN message passing mechanism is based on local inductive bias design → each node only aggregates the feature information of its first-order neighbors in a single iteration → for dependency paths that span multiple function call levels in the code graph (such as the convergence point of a taint source in function A, transiting through function B, and finally reaching function C), effective perception requires stacking a number of GNN layers proportional to the path length → deep GNN layers cause oversmoothing problems (all node representations tend to be consistent) and gradient vanishing (backpropagation signal decays layer by layer) → the model has difficulty distinguishing between key nodes and irrelevant nodes on long-distance dependency paths, and the high-level representations of all nodes are highly assimilated → traditional GNNs have insufficient ability to model long-distance dependencies in code, and the dependency path features of complex vulnerabilities are difficult to extract effectively. Summary of the Invention

[0022] The technical problem to be solved by this invention is to provide a software vulnerability detection method and storage medium based on cross-modal fusion that is simple in principle, has a wide range of applications, and is highly accurate, in response to the technical problems existing in the prior art.

[0023] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: A software vulnerability detection method based on cross-modal fusion, comprising: Bimodal code representation converts the input source code function into two modal representations: code sequence tokens and program dependency graphs (PDGs). Code sequence representation learning uses a pre-trained code large language model LLM as a sequence encoder to extract deep semantic features layer by layer from the code sequence; Code graph representation learning: using Graph Transformer as a graph encoder to extract structural features layer by layer from the program dependency graph; PIM (Prompt-Based Local Information Interaction): At the end of the LLM and GraphTransformer branches Each layer is injected with a learnable hint token, and a deep bidirectional knowledge exchange between the two modalities is achieved through a cross-attention mechanism. Cross-attention adaptive global fusion (CAM) dynamically calculates the importance scores of semantic and structural representations through a dedicated cross-attention mechanism after feature extraction, and adaptively adjusts the fusion weights of different modalities in the final decision based on the input code features.

[0024] As a further improvement of the present invention, the step of code bimodal representation includes: Step S101: Perform word segmentation on the source code function using a byte-pair encoding BPE-based sub-word segmentation algorithm; Step S102: Convert the word segmentation results into a token index sequence as a code sequence representation; add a special classification token ([CLS]) to the beginning of the sequence for subsequent extraction of global sequence semantic features; Step S103: Call the static analysis tool Joern (v4.0.95) to parse the source code functions and generate a program dependency graph (PDG); the nodes of the PDG represent program statements or expressions, and the edges represent control dependencies or data dependencies; Joern is deployed via Docker containerization and exports the graph data in DOT format. Step S104: Formalize the PDG: ,in For a set of nodes, each node It has the original code text attribute; Let be the set of edges. Indicates the edge type; Step S105: Use UniXcoder's tokenizer to encode the source code text of each graph node to obtain the initial embedding representation of the node.

[0025] As a further improvement of the present invention, the step of learning code sequence representation includes: Step S201: Input the token index sequence into the embedding layer of the UniXcoder pre-trained model to obtain the initial token embedding matrix; Step S202: Perform forward propagation using 12 stacked Transformers; the operations performed at each layer include: • Multi-head self-attention (MHSA): The representation of each token is calculated by weighted aggregation with the attention scores of all tokens in the sequence; • Layer normalization: Normalizes the self-attention output to stabilize the training process; • Feedforward Neural Network (FFN): A two-layer fully connected network with the GELU activation function in between; • Residual connections: Add residual connections to the outputs of MHSA and FFN respectively; Step S203: Record the output hidden state of each layer, especially the last one. The layer's output is used for cross-modal information interaction within the PIM module; Step S204: The hidden vector corresponding to the [CLS] position in the final layer output is used as the global sequence semantic representation.

[0026] As a further improvement of the present invention, the step of code graph representation learning includes: Step S301: Construct the input of the GraphTransformer based on the initial node embeddings; Step S302: Perform the following operations for each GraphTransformer layer: (a) Structure-aware multi-head self-attention: For each node Let its hidden state be ; For each attention head, compute the query. ,key ,value ( ); Attention Score: ; Aggregated output: ; (b) Residual connectivity and layer normalization: ; (c) Feedforward neural network: ; (d) Residual connectivity and layer normalization: ; Step S303: Record the output hidden state of each layer, especially the last one. The layer's output is used in the PIM module; Step S304: Perform average pooling on all hidden states of nodes in the final layer output to obtain the global graph structure representation vector.

[0027] As a further improvement of the present invention: the steps of the prompt-based local information interaction (PIM) include: Step S401: Prompt for Token definition and initialization; Step S402: Cross-attention fusion operation, at the end Execute at each layer; Step S403: Execute recursively layer by layer.

[0028] As a further improvement of the present invention: in step S401, three types of learnable cue tokens are defined and initialized to obey. random vectors: Text-specific cues are used to abstract and aggregate text semantic features from LLM branches; Graph-specific hints are used to abstract aggregate graph structure features from GraphTransformer branches; Modal interaction hints, as modality-independent query vectors, simultaneously focus on the aggregate representation of two modalities in cross-modal interactions.

[0029] As a further improvement of the present invention: in step S402, for the first... layer( The corresponding GraphTransformer layer is ),include: (2a) Hint aggregation—extracting compact representations of each modality: use As a query, Using multi-head cross-attention as both key and value, compact text modal features are aggregated:

[0030] use As a query, As key and value, the compact feature of the aggregation graph modality:

[0031] (2b) Cross-modal query – Modal interaction prompts simultaneously focus on two modalities: use As a query, cross-attention is applied to the two types of compact representations to obtain cross-modal semantic supplementation:

[0032]

[0033] (2c) Fusion Cue Generation – MLP generates cue vectors for each modality: Cross-modal information is transformed into injectable fusion cues using a two-layer MLP:

[0034] (2d) Hint Injection – Merge hints into the corresponding branch: In the next level of input in the LLM branch, Add it to the beginning of the Token sequence, concatenate it with the original Token sequence, and use it as the input for the next Transformer layer; In the next layer of input in the GraphTransformer branch, It is added as an additional key-value pair to the K and V matrices of the self-attention.

[0035] As a further improvement of the present invention, the step of cross-attention adaptive global fusion CAM includes: Step S501: Feature projection; The representations of the two modalities are mapped to the same query space using learnable projection matrices respectively:

[0036] in For a learnable parameter matrix, the dimension is... ; Step S502: Cross-attention calculation and dynamic importance scoring; Step S503: Adaptive fusion weight calculation; The final dynamic fusion weights are automatically calculated through a gating mechanism. :

[0037] in For learnable parameters, For bias terms, This indicates vector concatenation; The range of values ​​is ; Step S504: Dynamic weighted fusion;

[0038] Step S505: Vulnerability Prediction; fusion representation Input multilayer perceptron classifier: First fully connected layer: Linear ( ), activation function ReLU; Second fully connected layer (classification header): Linear( The activation function is Softmax.

[0039] As a further improvement of the present invention: in step S502, a cross-attention mechanism is used to calculate the importance score between the two modalities: (a) Semantic → Structural Attention: For Query, For Key and Value:

[0040] (b) Structure → Semantic Attention: For Query, For Key and Value: .

[0041] The present invention also provides a storage medium that can be read by a computer or processor, wherein the storage medium stores a computer program for performing any of the methods described above.

[0042] Compared with the prior art, the advantages of the present invention are as follows: 1. The software vulnerability detection method and storage medium based on cross-modal fusion of the present invention are simple in principle, have a wide range of applications, and high accuracy. The present invention is a cross-modal vulnerability detection method that realizes deep cross-modal interaction in the feature extraction stage and adaptive weight adjustment in the decision fusion stage. It has extremely important theoretical value and practical significance for improving the accuracy and coverage of automated vulnerability detection and enhancing the security assurance capability of software systems.

[0043] 2. The software vulnerability detection method and storage medium based on cross-modal fusion of the present invention designs a deep cross-modal bidirectional information interaction mechanism, which continuously exchanges knowledge during the multi-layer feature extraction process of LLM branch and GNN branch. The syntactic and semantic knowledge of each layer of LLM branch is injected into GNN branch to guide structural aggregation, while the structural dependency knowledge of each layer of GNN branch is injected into LLM branch to constrain semantic representation, thus solving the problem of shallow cross-modal interaction from the source of feature extraction.

[0044] 3. The software vulnerability detection method and storage medium based on cross-modal fusion of the present invention designs a cross-modal dynamic weight fusion mechanism based on adaptive input content. It automatically adjusts the contribution weight of different modal branches in the final decision according to the modal sensitivity characteristics of the input code sample. For syntax-sensitive vulnerabilities, it automatically increases the weight of sequence branches, and for structure-dependent vulnerabilities, it automatically increases the weight of graph branches. It solves the modal mismatch problem of static fusion from the decision fusion level.

[0045] 4. The software vulnerability detection method and storage medium based on cross-modal fusion of the present invention adopts GraphTransformer instead of traditional GGNN as graph structure encoder. It uses its structure-aware global self-attention mechanism to break through the locality limitation of traditional message passing. By calculating the attention score of any pair of nodes in the graph, it realizes global modeling of dependencies across arbitrary distances, thus alleviating the problems of oversmoothing and long-distance dependency capture.

[0046] 5. The software vulnerability detection method and storage medium based on cross-modal fusion of the present invention, through the synergistic effect of local fusion and global fusion, enables the full integration of multimodal complementary information at both the feature extraction and decision fusion levels, achieving consistent and significant accuracy improvement across various vulnerability types and avoiding performance degradation on specific types. Attached Figure Description

[0047] Figure 1 This is a schematic diagram of the present invention in a specific embodiment.

[0048] Figure 2 This is pseudocode for a single-layer forward propagation of the Graph Transformer in a specific embodiment of the present invention.

[0049] Figure 3 This is a schematic diagram of local information interaction based on prompts in a specific embodiment of the present invention.

[0050] Figure 4 This is pseudocode illustrating the operation of PIM at a single layer in a specific implementation of the present invention.

[0051] Figure 5 This is a schematic diagram of cross-attention adaptive global fusion in a specific embodiment of the present invention.

[0052] Figure 6 This is the pseudocode for CAM in a specific embodiment of the present invention.

[0053] Figure 7 This is a simplified code snippet for detecting a null pointer dereference vulnerability (CWE-476) in a specific embodiment of the present invention. Detailed Implementation

[0054] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. In the description of this application, the following terms and parameters should be understood as follows:

[0055]

[0056] The core of this invention lies in integrating cross-modal multi-perspective information simultaneously at the local feature extraction level and the global decision fusion level. Cross-modal knowledge fusion in the feature extraction stage is achieved through deep bidirectional information interaction based on learnable prompts, and dynamic modal balance in the decision-making stage is achieved through an adaptive weighting mechanism based on cross-attention.

[0057] like Figures 1-6As shown, this invention discloses a software vulnerability detection method based on cross-modal fusion, which includes: Bimodal code representation converts the input source code function into two modal representations: a code sequence (token sequence) and a program dependency graph (PDG). Code sequence representation learning uses a pre-trained code large language model (UniXcoder) as a sequence encoder to extract deep semantic features from code sequences layer by layer; Code graph representation learning: using Graph Transformer as a graph encoder to extract structural features layer by layer from the program dependency graph; Prompt-based Information Interaction Module (PIM): At the end of the LLM and GraphTransformer branches. Each layer is injected with a learnable hint token, and a deep bidirectional knowledge exchange between the two modalities is achieved through a cross-attention mechanism. Cross-attention Adaptive Fusion Module (CAM) dynamically calculates the importance scores of semantic and structural representations through a dedicated cross-attention mechanism after feature extraction, and adaptively adjusts the fusion weights of different modalities in the final decision based on the input code features.

[0058] When performing the code sequence representation learning step, UniXcoder uses 12 stacked Transformer layers. The learning process of each layer can be represented as follows:

[0059] in For learnable parameters, This is the current floor number. This represents the total number of layers in the LLM.

[0060] When performing code graph representation learning steps, the learning process of each layer can be represented as follows:

[0061] in This represents the total number of layers in the Graph Transformer. The Graph Transformer consists of a structure-aware multi-head self-attention mechanism and a feedforward neural network (FFN), with residual connections introduced between layers to enhance training stability.

[0062] In a specific application example, the steps of code bimodal representation involve the following inputs: the source code function to be detected (C / C++ format source code text); and the outputs: a code sequence representation (tokenized sequence) and a program dependency graph representation. Its main processes may include: Step S101: Perform word segmentation on the source code function using a sub-word segmentation algorithm based on Byte Pair Encoding (BPE). The BPE algorithm generates a sub-word vocabulary by iteratively merging high-frequency character pairs by statistically analyzing character merging frequencies, which can effectively handle common named entities and out-of-vocabulary words in the code.

[0063] Step S102: Convert the word segmentation results into a token index sequence as a code sequence representation. Add a special classification token ([CLS]) to the beginning of the sequence for subsequent extraction of global sequence semantic features.

[0064] Step S103: Use the static analysis tool Joern (v4.0.95) to parse the source code functions and generate a program dependency graph (PDG). Nodes in the PDG represent program statements or expressions, and edges represent control or data dependencies. Joern is deployed via Docker containers and exports the graph data in DOT format.

[0065] Step S104: Formalize the PDG: ,in For a set of nodes, each node It has the original code text attribute; Let be the set of edges. Indicates the edge type (control dependency edge or data dependency edge).

[0066] Step S105: Use UniXcoder's tokenizer to encode the source code text of each graph node to obtain the initial embedding representation of the node, which has a dimension of 768.

[0067] The key to this invention lies in using PDG instead of CPG as the graph representation. PDG directly encapsulates control and data dependencies, aligning perfectly with the core requirements of vulnerability detection and avoiding noise potentially introduced by the additional syntax structures in CPG. Joern's DOT format export capability ensures the complete preservation of the graph structure and label information.

[0068] In a specific application example, in the code sequence representation learning step, the input is: a tokenized code sequence (including the [CLS] special token); the output is: the sequence semantic feature representation of each layer. ,in Its process may include: Step S201: Input the token index sequence into the embedding layer of the UniXcoder pre-trained model to obtain the initial token embedding matrix.

[0069] Step S202: Perform forward propagation using 12 stacked Transformers. The operations performed at each layer include: • Multi-Head Self-Attention (MHSA): The representation of each token is weighted and aggregated with the attention scores of all tokens in the sequence.

[0070] • Layer Normalization: Normalizes the self-attention output to stabilize the training process.

[0071] • Feedforward Neural Network (FFN): A two-layer fully connected network with the GELU activation function in between.

[0072] • Residual Connection: Add residual connections to the outputs of MHSA and FFN respectively.

[0073] Step S203: Record the output hidden state of each layer, especially the last one. Layer output ( (i.e., layers 11 and 12), used for cross-modal information interaction of PIM modules.

[0074] Step S204: The hidden vector corresponding to the [CLS] position in the final layer output is used as the global sequence semantic representation.

[0075] In the above process, the key point of the present invention is that UniXcoder is pre-trained on a large code corpus by jointly optimizing multiple agent tasks such as masked language modeling, one-way language modeling, denoising autoencoder and code-annotation contrast learning, and has the ability to deeply understand code semantics.

[0076] In specific application examples, combined with Figure 2 In the code graph representation learning process, the input is: program dependency graph. Initial node embedding; Output: Graph structure feature representation of each layer. ,in Its main processes may include: Step S301: Construct the input to the GraphTransformer based on the initial node embeddings. Total number of layers. Hidden layer dimensions .

[0077] Step S302: Perform the following operations for each GraphTransformer layer: (a) Structure-aware multi-head self-attention: For each node Let its hidden state be .

[0078] • For each attention head, compute the query ,key ,value ( ).

[0079] • Attention score: .

[0080] • Aggregated output: .

[0081] (b) Residual connectivity and layer normalization:

[0082] (c) Feedforward neural network:

[0083] (d) Residual connectivity and layer normalization:

[0084] Step S303: Record the output hidden state of each layer, especially the last one. layer( The output of ) is used in the PIM module.

[0085] Step S304: Perform average pooling on all hidden states of nodes in the final layer output to obtain the global graph structure representation vector.

[0086] The key to this invention lies in the fact that the global self-attention of the Graph Transformer allows each node to see all other nodes in the graph during updates, breaking the limitation that a single message pass only covers first-order neighbors. The dual residual connections effectively alleviate the deep gradient vanishing problem.

[0087] In specific application examples, combined with Figure 3 and Figure 4 In the Prompt-Based Local Information Interaction (PIM) step, enter: LLM branch final Hidden state sequence of layer ; The last branch of GraphTransformer Hidden state of a layer Three types of learnable hint tokens: (length ), (length ), (length Output: The two branches enhanced with cross-modal knowledge. Layer output. Its main process may include: Step S401: Prompt for Token definition and initialization.

[0088] Define three types of learnable cue tokens, initialized to obey. random vectors: · Text-specific cues are used to abstract and aggregate text semantic features from LLM branches.

[0089] · Graph-specific hints are used to abstract aggregate graph structure features from GraphTransformer branches.

[0090] · Modal interaction hints, as modality-independent query vectors, simultaneously focus on the aggregate representation of two modalities in cross-modal interactions.

[0091] Step S402: Cross-attention fusion operation (at the end) Execution at each layer.

[0092] For the layer( The corresponding GraphTransformer layer is ): (2a) Hint aggregation—extracting compact representations of each modality: ·use As a query, Using multi-head cross-attention as both key and value, compact text modal features are aggregated:

[0093] ·use As a query, As key and value, the compact feature of the aggregation graph modality:

[0094] (2b) Cross-modal query – Modal interaction prompts simultaneously focus on two modalities: ·use As a query, cross-attention is applied to the two types of compact representations to obtain cross-modal semantic supplementation:

[0095]

[0096] (2c) Fusion Cue Generation – MLP generates cue vectors for each modality: • Transform cross-modal information into injectable fusion cues using a two-layer MLP (including ReLU activation):

[0097] (2d) Hint Injection – Merge hints into the corresponding branch: • In the next level of input in the LLM branch, It is added to the beginning of the Token sequence and concatenated with the original Token sequence to serve as the input for the next Transformer layer.

[0098] • In the next layer of input in the GraphTransformer branch, It is added as an additional key-value pair to the K and V matrices of the self-attention.

[0099] Step S403: Execute recursively layer by layer.

[0100] PIM at the end The above interactive operations are executed layer by layer, with each layer using independent prompt Token parameters and MLP parameters. In the first layer (the... Layer LLM / First The first layer (GraphTransformer) begins the interaction with the original branch features as input; in the second layer (the GraphTransformer), the interaction continues... Layer / First (Layer), using the features from the previous layer as input to further deepen the interaction.

[0101] The key to this invention lies in the following process: Advantages of the cueing mechanism: The semantic features of LLM and the structural features of GraphTransformer have different distributions. Direct cross-modal attention may introduce noise. The cue token acts as an intermediary, first compressing high-dimensional features into a compact representation, and then... Cross-modal exchange is performed, and then converted into a suitable prompt form for injection via MLP, maintaining the stability of the backbone and realizing implicit, lightweight, bidirectional cross-modal information exchange.

[0102] The key role of modal interaction prompts: As a modality-independent shared query, step 2b focuses on both types of compact representations simultaneously, extracting complementary information valuable to both modalities. (Remove) This would result in a 3.32% and 3.29% decrease in F1 scores, respectively.

[0103] PIM only applies to the last one. The reason for using layers: shallow features are more modality-specific, and premature interaction may interfere with fine-grained feature encoding. Deep features, after multiple layers of abstraction, are more task-relevant; at this stage, the benefits of exchanging complementary knowledge outweigh the risks. Hyperparameter experiments show... To achieve optimal performance.

[0104]

[0105] • Token quantity notification Impact on performance: To achieve optimal performance. Insufficient capacity causes F1 to drop; An increase in learnable parameters can lead to overfitting and a decrease in accuracy.

[0106]

[0107] In specific application examples, combined with Figure 5 In the Cross-Attention Adaptive Global Fusion (CAM) step, the input is: the global sequence semantic representation vector of the LLM branch. (The final layer [CLS] vector after PIM enhancement); the global graph structure representation vector of the GraphTransformer branch. (Average pooling vector of all nodes after PIM enhancement); Output: Fusion comprehensive vulnerability feature representation vector Vulnerability prediction results (binary classification: containing vulnerabilities / not containing vulnerabilities); combined with Figure 6 Its main processes can include: Step S501: Feature projection.

[0108] The representations of the two modalities are mapped to the same query space using learnable projection matrices respectively:

[0109] in For a learnable parameter matrix, the dimension is... .

[0110] Step S502: Cross-attention calculation – dynamic importance scoring.

[0111] The importance score between the two modalities is calculated using the cross-attention mechanism: (a) Semantic → Structural Attention: For Query, For Key and Value:

[0112] (b) Structure → Semantic Attention: For Query, For Key and Value:

[0113] Step S503: Adaptive fusion weight calculation.

[0114] The final dynamic fusion weights are automatically calculated through a gating mechanism. :

[0115] in For learnable parameters, For bias terms, This indicates vector concatenation. The range of values ​​is .

[0116] Step S504: Dynamic weighted fusion.

[0117]

[0118] Step S505: Vulnerability prediction.

[0119] fusion representation Input multilayer perceptron classifier: • First fully connected layer: Linear ( ), activation function ReLU Dropout layer: ratio 0.2 • Second fully connected layer (classification head): Linear( ), Activation function Softmax Output: The probabilities of the two classes. ,Pick It has been determined to contain a vulnerability.

[0120] The key to this invention lies in the following process: Adaptive weights The principle of dynamic adjustment: The computation depends on the outputs of two cross-attention pairs. and The concatenated vector, and and It is derived through the interaction of semantic and structural representations. When the semantic features of the input sample contain obvious vulnerability signals... Enhanced, driven It tends to favor semantic branches; conversely, when structural features contain obvious vulnerability signals... Bias graph branching. This process is entirely self-adjusting based on the features of the input content, without the need for predefined vulnerability type labels.

[0121] The fundamental difference between Vul-LMGNN and its fixed-weight mechanism is that Vul-LMGNN uses a single global parameter. This is applied uniformly to all samples. (CAM) Each input sample is calculated independently and changes with the input features, thus achieving personalized fusion for each sample.

[0122] It is understood that the present invention currently uses UniXcoder as a sequence encoder. In other embodiments, CodeBERT, GraphCodeBERT, CodeT5 / CodeT5+ can also be used. Any Transformer encoder that has been pre-trained on a large scale for code understanding tasks can be used as an alternative, as long as it supports sequential input and obtains reasonable code semantic representation. The rest of the structure of PIM and CAM remains unchanged.

[0123] It is understood that the present invention currently uses Graph Transformer as a graph encoder. In other embodiments, gated graph neural network (GGNN), graph attention network (GAT), heterogeneous graph transformer (HGT) can also be used. Any graph neural network that can extract structural features from program dependency graph or code attribute graph can be used as an alternative, only requiring adjustment of the dimension alignment of the interaction with graph branches in PIM.

[0124] It is understood that the present invention currently uses a program dependency graph. In other embodiments, a combination of code attribute graph (CPG), abstract syntax tree and control flow graph can also be used. Any graph representation that can capture program control and data dependency relationships can be used as an alternative to PDG, as long as its graph structure can effectively support the tracing of vulnerability dependency paths.

[0125] It is understood that the present invention currently uses cross-attention interaction based on three types of prompts. In other embodiments, direct cross-modal cross-attention and inter-modal feature splicing injection can also be used. Any mechanism that realizes bidirectional information exchange between LLM and graph encoder during the multi-layer forward propagation of encoder can be used as an alternative to PIM, as long as it can realize continuous cross-modal knowledge interaction in the feature extraction stage and does not significantly damage the original modality feature extraction capability.

[0126] It is understood that the present invention currently uses cross-attention adaptive weighted fusion. In other embodiments, adaptive fusion based on gating units or fusion based on attention pooling can also be used. Any fusion mechanism that dynamically adjusts the contribution weights of different modalities based on the input content can be used as an alternative to CAM, as long as it can achieve independent calculation of modal weights for each input sample rather than globally uniform modal weights.

[0127] The following simplified example illustrates the advantages of Clara over existing methods. In an example scenario: null pointer dereference vulnerability (CWE-476) detection. Assume there is... Figure 7 The code snippet.

[0128] Clues to sequence characteristics: After the badSource(data) call, the variable data is directly used in strlen(data) and strncat(dest,data,strlen(data)), lacking null pointer validity checks at the semantic level (such as if(data!=NULL)).

[0129] Clues to the graph structure: There is a data dependency edge in PDG: from data = badSource(data) (definition node) to strlen(data) (use node). This dependency path constitutes a vulnerability dependency chain of "definition → unprotected use".

[0130] The failure process of the existing method (Vul-LMGNN): • LLM branches can detect semantic anomalies, but cannot detect PDG data dependency paths when coded independently, making it easy to lose focus.

[0131] The GNN branch can trace data dependency paths, but it does not understand the semantic fact that badSource may return NULL.

[0132] Fixed weights in the decision-making stage This leads to graph branches becoming dominant, drowning out semantic anomaly signals, and ultimately resulting in missed detections.

[0133] The success process of this invention (Clara): ①PIM continuously performs bidirectional knowledge integration during the deep feature extraction stage: The LLM semantic pattern of "dangerous functions may return NULL" is implemented through... Injecting branches into the GNN enables the GNN to gain semantic sensitivity in tracking data dependency paths.

[0134] The definition and usage of GraphTransformer directly depend on the structural pattern. Injecting an LLM branch allows the deep features of the LLM to focus on this dependency.

[0135] ②CAM automatically adjusts modal weights based on input features: CWE-476 requires both semantic and structural input. The calculated value is approximately 0.45-0.55. The weights of the two branches are nearly balanced, and the complementary information is fully utilized. The model is correctly identified as containing a vulnerability.

[0136] Taking a specific typical application as an example: function-level vulnerability detection for large-scale C / C++ code repositories, such as a software company performing automated vulnerability scanning on a large C / C++ code repository, determining whether each function contains security vulnerabilities at the function level. The input is the source code text of a single function, and the output is the probability of containing a vulnerability. The detailed execution flow of this invention is as follows: Step 1: Code Preprocessing. The source code functions to be analyzed are de-commented and standardized while preserving their semantics. Single-line and multi-line comments are removed, and tab characters are standardized to spaces.

[0137] Step 2: Construction of bimodal representation.

[0138] • Sequence Modality: The preprocessed code is segmented using UniXcoder's BPE tokenizer, converted into a TokenID sequence, and [CLS]TokenID is inserted at the beginning, truncated to a maximum length of 512 tokens.

[0139] • Graph Modality: The `joern-parse` command in Joern v4.0.95 is invoked, and the `--reprpdg` parameter is used to generate the program dependency graph, which is then exported as a DOT format. The PDG is parsed to obtain nodes, edges, and source text. The node text is then encoded using the UniXcoder tokenizer to obtain the initial embeddings.

[0140] Step 3: Feature extraction.

[0141] • LLM branch: The TokenID sequence is input into the UniXcoder, propagated forward through 12 layers of Transformers, and the last one is retained. Hidden state sequences of layers (11th and 12th layers) are used for PIM.

[0142] • GraphTransformer branch: Initial node embeddings and edge list input to GraphTransformer, through Layered structure-aware self-attention + FFN forward propagation, retaining the last The hidden state of layer (layer 5 and 6) nodes is used for PIM.

[0143] Step 4: PIM local fusion.

[0144] • Perform two-way PIM information exchange at layers 5 / 11 and 6 / 12 respectively: -use Aggregate compact features separately.

[0145] -use Perform cross-modal queries to obtain complementary knowledge.

[0146] - Generate fusion hints via MLP .

[0147] -Will Injecting input into the lower level of the LLM branch, Inject the lower-level input into the GraphTransformer branch.

[0148] • All prompts for token length Embedded Dimensions .

[0149] Both MLPs are two-layer networks with ReLU activation in the middle, and the dimensions are... .

[0150] Step 5: Global feature extraction.

[0151] • LLM branch: Take the vector corresponding to the final layer (layer 12) [CLS] after PIM enhancement as the global sequence semantic representation. (Dimension 768).

[0152] • GraphTransformer branch: Perform average pooling on the hidden states of all nodes in the final layer (layer 6) after PIM enhancement to obtain a global graph structure representation. (Dimension 768).

[0153] Step 6: Global CAM fusion.

[0154] • Calculate cross attention to obtain .

[0155] • Calculate adaptive fusion weights .

[0156] • Dynamic fusion: .

[0157] • Input MLP classifier: Linear(768→384)→ReLU→Dropout(0.2)→Linear(384→2)→Softmax.

[0158] Step 7: Vulnerability assessment.

[0159] Output .like If the system is found to contain a vulnerability, the detection result and confidence level will be output.

[0160]

[0161] Hardware configuration: 4 NVIDIA RTX 4090 GPUs, each GPU consumes approximately 18GB of memory at its peak, and the complete training takes about 12 hours.

[0162] Training process: • Dataset partitioning: Partitioned according to submission timestamps, with a training set: validation set: test set ratio of 8:1:1.

[0163] • Optimization objective: Minimize cross-entropy loss, using the AdamW optimizer.

[0164] • Accelerated training using mixed precision.

[0165] • Save the optimal model on the validation set based on the F1 score.

[0166] Parameter adjustment principles: · Value adjustment: If the target is primarily a syntax-sensitive vulnerability, you can try... Enhance semantic propagation to graph branches; if structural dependency vulnerabilities are the primary concern... That's enough.

[0167] • Length adjustment suggestion: Recommended when the training set size is small. To reduce overfitting; when the scale is large Enhance interactive capacity.

[0168] • Number of GraphTransformer layers: This can be increased appropriately when the code structure is particularly complex. Up to 8 layers, while increasing the number of training iterations.

[0169] This invention systematically addresses the core challenges of existing cross-modal vulnerability detection methods—such as shallow cross-modal interaction, static fusion, and insufficient long-distance dependency modeling—by introducing two core innovative mechanisms: Local Integral Motion (PIM) and Global Integral Motion (CAM). These challenges arise from both feature extraction and decision fusion levels. In summary, the key innovation of this invention lies in: The deep bidirectional cross-modal information exchange mechanism (PIM) fundamentally improves the utilization rate of complementary knowledge in the feature extraction stage. PIM continuously exchanges cross-modal knowledge during the feature extraction stage (rather than just the initialization stage)—the LLM branch continuously receives guidance information on graph structure dependencies during deep encoding, enabling semantic feature encoding to more accurately focus on the structural context related to vulnerabilities (e.g., when handling pointer usage, the graph structure tells which definition constitutes a data dependency with the current usage, and semantic encoding prioritizes the semantic security of that definition-use pair); the Graph Transformer branch continuously receives LLM semantic knowledge injection during deep aggregation, making structural aggregation more semantically sensitive (e.g., for the same type of data dependency edge, semantic knowledge can distinguish between two semantic modes: "safe definition → safe use" and "dangerous definition → dangerous use"). The two modalities continuously reinforce each other during the progressive feature extraction process, avoiding the decay of complementary information during independent evolution. Ablation experiments verify that removing PIM leads to a 13.66% and 10.07% decrease in F1 scores on the DiverseVul and PrimeVul datasets, respectively, demonstrating that PIM contributes the most to the performance improvement.

[0170] Optimal modality balance through input adaptation is achieved using a cross-attention-based adaptive fusion mechanism (CAM). CAM independently calculates dynamic fusion weights α for each input sample. For syntax-sensitive vulnerability samples, anomalous patterns in semantic features (such as the co-occurrence of dangerous API calls and external inputs) drive α to shift towards semantic branches, giving semantic evidence higher decision weights. For structure-dependent vulnerability samples, anomalous topologies in graph structure features (such as mismatched patterns of resource acquisition and release) drive α to shift towards graph branches, giving structural evidence higher decision weights. This mechanism, from a statistical learning perspective, eliminates the erroneous assumptions about modality importance distribution made by fixed fusion weight schemes, enabling the model to make decisions on various types of vulnerabilities with near-optimal modality matching.

[0171] By replacing traditional GNNs with Graph Transformers, the bottleneck of long-distance dependency modeling in code is overcome. Graph Transformers' global self-attention mechanism allows each node to perceive the feature information of any other node in the graph, without relying on message passing to expand the receptive domain layer by layer as in traditional GNNs. When dealing with long-distance dependencies across multiple function call levels (such as the path from taint source to intermediate function to convergence point), Graph Transformers can capture global relationships spanning arbitrary distances in a single layer of attention, effectively mitigating oversmoothing and gradient problems.

[0172] The synergistic effect of local and global fusion produces an overall performance gain greater than the sum of its parts (1+1>2). PIM addresses the issue of "sufficiency of complementary knowledge acquisition" in the feature extraction stage, while CAM addresses the issue of "rationality of complementary knowledge utilization" in the decision-making stage. The two are seamlessly integrated in their workflow and complementary in their functions—PIM enhances the deep features of both modalities across modalities, while CAM dynamically adjusts the decision weights based on the enhanced features. The performance gain when used together is greater than the sum of the gains from using PIM and CAM alone, confirming the positive synergistic effect between the two modules.

[0173] The framework exhibits excellent modal encoder scalability and cross-language generalization capabilities. Clara's PIM and CAM modules are decoupled from specific LLM and GNN selections, ensuring compatibility with different types of encoders.

[0174] Typical applications of this invention include, but are not limited to: Open source software supply chain security audit: Perform batch function-level vulnerability detection on open source component libraries to discover potential zero-day or known vulnerabilities and support supply chain security assessment.

[0175] Continuous Integration / Continuous Deployment (CI / CD) Security Access Control: Integrate into the CI / CD pipeline to perform automated vulnerability detection during the code submission phase as a security check before release.

[0176] Security Code Review Assistance: Provides auxiliary clues for manual code review, highlighting high-risk functions for reviewers to focus on and further analyze.

[0177] Vulnerability database construction and annotation: Assists in the automated screening and annotation of large-scale vulnerability samples, and supports the construction of training data in security research.

[0178] Security training and education: As an automated detection tool for secure coding training, it provides real-time feedback on potential security flaws in the code.

[0179] Embedded system firmware security testing: Perform security assessments on C / C++ code components in the firmware of IoT devices to identify vulnerabilities that could lead to remote attacks.

[0180] The present invention also provides a storage medium that can be read by a computer or processor, wherein the storage medium stores a computer program for executing any of the above methods.

[0181] Those skilled in the art will understand that the above embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0182] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A software vulnerability detection method based on cross-modal fusion, characterized in that, include: Bimodal code representation converts the input source code function into two modal representations: code sequence tokens and program dependency graphs (PDGs). Code sequence representation learning uses a pre-trained code large language model LLM as a sequence encoder to extract deep semantic features layer by layer from the code sequence; Code graph representation learning: using Graph Transformer as a graph encoder to extract structural features layer by layer from the program dependency graph; PIM (Prompt-Based Local Information Interaction): At the end of the LLM and GraphTransformer branches Each layer is injected with a learnable hint token, and a deep bidirectional knowledge exchange between the two modalities is achieved through a cross-attention mechanism. Cross-attention adaptive global fusion (CAM) dynamically calculates the importance scores of semantic and structural representations through a dedicated cross-attention mechanism after feature extraction, and adaptively adjusts the fusion weights of different modalities in the final decision based on the input code features.

2. The software vulnerability detection method based on cross-modal fusion according to claim 1, characterized in that, The steps involved in representing bimodal code include: Step S101: Perform word segmentation on the source code function using a byte-pair encoding BPE-based sub-word segmentation algorithm; Step S102: Convert the word segmentation results into a token index sequence as a code sequence representation; add a special classification token ([CLS]) to the beginning of the sequence for subsequent extraction of global sequence semantic features; Step S103: Call the static analysis tool Joern (v4.0.95) to parse the source code functions and generate a program dependency graph (PDG); the nodes of the PDG represent program statements or expressions, and the edges represent control dependencies or data dependencies; Joern is deployed via Docker containerization and exports the graph data in DOT format. Step S104: Formalize the PDG: ,in For a set of nodes, each node It has the original code text attribute; Let be the set of edges. Indicates the edge type; Step S105: Use UniXcoder's tokenizer to encode the source code text of each graph node to obtain the initial embedding representation of the node.

3. The software vulnerability detection method based on cross-modal fusion according to claim 1, characterized in that, The steps in learning code sequence representations include: Step S201: Input the token index sequence into the embedding layer of the UniXcoder pre-trained model to obtain the initial token embedding matrix; Step S202: Perform forward propagation using 12 stacked Transformers; the operations performed at each layer include: • Multi-head self-attention (MHSA): The representation of each token is calculated by weighted aggregation with the attention scores of all tokens in the sequence; • Layer normalization: Normalizes the self-attention output to stabilize the training process; • Feedforward Neural Network (FFN): A two-layer fully connected network with the GELU activation function in between; • Residual connections: Add residual connections to the outputs of MHSA and FFN respectively; Step S203: Record the output hidden state of each layer, especially the last one. The layer's output is used for cross-modal information interaction within the PIM module; Step S204: The hidden vector corresponding to the [CLS] position in the final layer output is used as the global sequence semantic representation.

4. The software vulnerability detection method based on cross-modal fusion according to claim 1, characterized in that, The steps in code graph representation learning include: Step S301: Construct the input of the GraphTransformer based on the initial node embeddings; Step S302: Perform the following operations for each GraphTransformer layer: (a) Structure-aware multi-head self-attention: For each node Let its hidden state be ; For each attention head, compute the query. ,key ,value ( ); Attention Score: ; Aggregated output: ; (b) Residual connectivity and layer normalization: ; (c) Feedforward neural network: ; (d) Residual connectivity and layer normalization: ; Step S303: Record the output hidden state of each layer, especially the last one. The layer's output is used in the PIM module; Step S304: Perform average pooling on all hidden states of nodes in the final layer output to obtain the global graph structure representation vector.

5. The software vulnerability detection method based on cross-modal fusion according to claim 1, characterized in that, The steps in prompt-based local information interaction (PIM) include: Step S401: Prompt for Token definition and initialization; Step S402: Cross-attention fusion operation, at the end Execute at each layer; Step S403: Execute recursively layer by layer.

6. The software vulnerability detection method based on cross-modal fusion according to claim 5, characterized in that, In step S401, three types of learnable cue tokens are defined and initialized to obey. random vectors: Text-specific cues are used to abstract and aggregate text semantic features from LLM branches; Graph-specific hints are used to abstract aggregate graph structure features from GraphTransformer branches; Modal interaction hints, as modality-independent query vectors, simultaneously focus on the aggregate representation of two modalities in cross-modal interactions.

7. The software vulnerability detection method based on cross-modal fusion according to claim 5, characterized in that, In step S402, for the first layer( The corresponding GraphTransformer layer is ),include: (2a) Hint aggregation—extracting compact representations of each modality: use As a query, Using multi-head cross-attention as both key and value, compact text modal features are aggregated: use As a query, As key and value, the compact feature of the aggregation graph modality: (2b) Cross-modal query – Modal interaction prompts simultaneously focus on two modalities: use As a query, cross-attention is applied to the two types of compact representations to obtain cross-modal semantic supplementation: (2c) Fusion Cue Generation – MLP generates cue vectors for each modality: Cross-modal information is transformed into injectable fusion cues using a two-layer MLP: (2d) Hint Injection – Merge hints into the corresponding branch: In the next level of input in the LLM branch, Add it to the beginning of the Token sequence, concatenate it with the original Token sequence, and use it as the input for the next Transformer layer; In the next layer of input in the GraphTransformer branch, It is added as an additional key-value pair to the K and V matrices of the self-attention.

8. The software vulnerability detection method based on cross-modal fusion according to claim 1, characterized in that, The steps in cross-attention adaptive global fusion CAM include: Step S501: Feature projection; The representations of the two modalities are mapped to the same query space using learnable projection matrices respectively: in For a learnable parameter matrix, the dimension is... ; Step S502: Cross-attention calculation and dynamic importance scoring; Step S503: Adaptive fusion weight calculation; The final dynamic fusion weights are automatically calculated through a gating mechanism. : in For learnable parameters, For bias terms, This indicates vector concatenation; The range of values ​​is ; Step S504: Dynamic weighted fusion; Step S505: Vulnerability Prediction; fusion representation Input multilayer perceptron classifier: First fully connected layer: Linear ( ), activation function ReLU; Second fully connected layer (classification header): Linear( The activation function is Softmax.

9. The software vulnerability detection method based on cross-modal fusion according to claim 8, characterized in that, In step S502, a cross-attention mechanism is used to calculate the importance score between the two modalities: (a) Semantic → Structural Attention: For Query, For Key and Value: (b) Structure → Semantic Attention: For Query, For Key and Value: 。 10. A storage medium capable of being read by a computer or processor, characterized in that, The storage medium stores a computer program for performing the method described in any one of claims 1 to 9.

Images