A function-level vulnerability detection method and system based on cross-modal code representation fusion

Abstract

The application discloses a function-level vulnerability detection method and system based on cross-modal code representation fusion, and relates to the technical fields of software security and artificial intelligence. The method comprises the following steps: acquiring function source code to be detected; constructing a simplified code graph structure input, a source code Token sequence and an effective Token mask; extracting and aggregating program relationship neighborhood information of each node in the simplified code graph, updating the hidden state of each node according to the aggregated program relationship neighborhood information, and performing graph-level convergence to obtain a function-level structure representation; extracting context semantic encoding of the source code Token sequence, and converging the context semantic encoding according to the effective Token mask to obtain a function-level semantic representation; performing multi-head parameterized transformation on the function-level structure representation and the function-level semantic representation respectively to generate semantic parameterized enhanced representations; and splicing the three representations to obtain a cross-modal fusion representation input into a vulnerability classifier to obtain a vulnerability probability, compare the vulnerability probability with a preset classification threshold, and output a detection result.

Description

Technical Field

[0001] This invention relates to a function-level vulnerability detection method and system based on cross-modal code representation fusion, belonging to the fields of software security and artificial intelligence technology. Background Technology

[0002] Software vulnerabilities are critical factors affecting the security and reliability of software systems, potentially leading to security consequences such as information leakage, privilege escalation, denial-of-service attacks, and remote code execution. Function-level vulnerability detection, which examines the source code of individual functions, provides fine-grained vulnerability assessments while preserving the local program context. Therefore, it is widely used in software development, code auditing, and security testing processes.

[0003] Existing automated vulnerability detection technologies mainly include rule matching, static analysis, dynamic analysis, and symbolic execution. These methods typically rely on manually constructed security rules, test cases, or path constraints, and may suffer from insufficient rule coverage, incomplete test paths, path explosion, and high false positive or false negative rates when dealing with complex programs. Deep learning methods, capable of automatically learning vulnerability-related features from large amounts of code samples, have become an important technical approach for function-level vulnerability detection.

[0004] Existing deep learning vulnerability detection methods typically employ source code sequence representations or program graph representations. Source code sequence-based methods can learn identifiers, operators, application programming interface calls, and local context semantics, but struggle to explicitly represent variable definitions and usages, control paths, and data propagation relationships. Program graph-based methods can represent explicit program relationships such as abstract syntax, control flow, and data flow, but they underutilize the source code context semantics knowledge inherent in large-scale pre-trained code models. Relying solely on either code representation may result in incomplete representation of vulnerability-related information.

[0005] Some existing multi-representation vulnerability detection methods fuse different code representations through simple concatenation, fixed weighting, or fine-grained interactions between nodes and tokens. However, simple concatenation or fixed weighting is insufficient to fully model the interaction relationships between different modalities; there is usually no stable one-to-one correspondence between nodes and tokens, and they are easily affected by code graph simplification, differences in source code segmentation, and cross-granularity association noise. Therefore, a vulnerability detection method is needed that jointly utilizes explicit program structure information and source code context semantic information within the function-level representation space to improve the effectiveness of structure-semantic fusion and the stability of detection results. Summary of the Invention

[0006] The purpose of this invention is to provide a function-level vulnerability detection method and system based on cross-modal code representation fusion. By extracting function-level structural representation and semantic representation respectively, multi-head parameterized transformation is performed with structure as query and semantic as key to generate enhanced representation, and then concatenated into a fused representation for classification. This approach combines structural and semantic information, avoids cross-granularity alignment noise, and improves vulnerability detection performance and stability.

[0007] To achieve the above objectives, the present invention is implemented using the following technical solution.

[0008] On the one hand, this invention provides a function-level vulnerability detection method based on cross-modal code representation fusion, comprising:

[0009] Obtain the source code of the function to be tested;

[0010] Based on the source code of the function to be detected, structural input and semantic input are constructed respectively; wherein, the structural input is a simplified code graph generated based on the program structure of the source code of the function to be detected, and the simplified code graph includes graph structure and node features; the semantic input is a source code token sequence and a valid token mask generated based on the source code text of the same source code of the function to be detected.

[0011] Extract and aggregate the program relationship neighborhood information of each node in the simplified code graph, update the hidden state of each node according to the aggregated program relationship neighborhood information, and then perform graph-level aggregation on the updated hidden states of each node to obtain a function-level structure representation; wherein, the program relationship includes syntax relationship, control flow relationship and data flow relationship;

[0012] Extract the context semantic encoding of the source code token sequence, and aggregate the context semantic encoding according to the valid token mask to obtain a function-level semantic representation;

[0013] The function-level structural representation and function-level semantic representation are projected onto a unified latent space of the same dimension to obtain the projected structural representation and projected semantic representation.

[0014] Using the projected structural representation as the query-side input and the projected semantic representation as the key-side and value-side input, a multi-head parameterization transformation is performed to generate a semantically parameterized enhanced representation.

[0015] By concatenating the projected structural representation, the projected semantic representation, and the semantic parameterized enhanced representation, a cross-modal fusion representation is obtained.

[0016] The cross-modal fusion representation is input into the vulnerability classifier to obtain the vulnerability probability of the function to be detected. The vulnerability probability is compared with a preset classification threshold, and the detection result of whether the function to be detected has a vulnerability or not is output.

[0017] Furthermore, the process of generating the simplified code diagram includes:

[0018] The source code of the function to be tested is parsed, and the nodes corresponding to statements, variables, expressions, operators and function calls are extracted from the source code of the function to be tested. The abstract syntax relationship, control flow relationship and data dependency relationship between the nodes are also extracted to construct an initial code structure diagram.

[0019] The initial code structure graph is simplified to obtain a simplified initial code structure graph; wherein, the graph simplification operation includes: removing auxiliary nodes in the initial code structure graph, pruning or compressing local abstract syntax subtrees, and merging redundant nodes representing the same variable or the same code semantic unit;

[0020] The abstract syntax edges, control flow edges, and data dependency edges in the simplified initial code structure graph are retained, and natural sequence edges are added according to the order of code units in the source code of the function to be detected. Self-loop edges are also added to each node to obtain a simplified graph structure.

[0021] The code text corresponding to each node in the simplified graph structure is split into sub-word units, the sub-word units are mapped to word vectors, and the word vectors belonging to the same node are averaged and pooled to obtain fixed-dimensional node features for each node. The simplified graph structure and the fixed-dimensional node features constitute the simplified code graph.

[0022] Furthermore, the process of generating the source code token sequence and valid token mask includes:

[0023] The source code text of the same function is segmented using the tokenizer corresponding to the pre-trained code model to obtain an initial token sequence;

[0024] Add a preset sequence marker to the initial token sequence and calculate the number of tokens after adding the marker;

[0025] When the number of tokens exceeds the preset maximum sequence length, truncation is performed; when the number of tokens is less than the preset maximum sequence length, padding is performed to generate the source code token sequence.

[0026] The effective token mask is generated by setting the mask value of the valid token position in the source code token sequence to 1 and setting the mask value of the filling token position to 0.

[0027] Furthermore, the structural representation extraction module performs the extraction and aggregation of program relationship neighborhood information of each node in the simplified code graph, the updating of the hidden state of each node, and graph-level convergence. The structural representation extraction module includes a multi-type program relationship encoding unit, a node state update unit, a node feature sequence processing unit, a graph-level feature convergence unit, and a structural feature projection layer, specifically including:

[0028] The multi-type program relationship encoding unit calculates the information weights between adjacent nodes and aggregates neighborhood information based on the relationship types of edges in the simplified code graph.

[0029] The node state update unit updates the hidden state of each node based on the aggregated program relationship neighborhood information;

[0030] The node feature sequence processing unit performs local and long-distance feature processing on the updated hidden states of each node to obtain the effective node hidden states.

[0031] The graph-level feature aggregation unit aggregates the hidden states of the effective nodes to obtain a function-level graph representation;

[0032] The structural feature projection layer maps the function-level graph representation to a unified latent space, resulting in the projected function-level structural representation.

[0033] Furthermore, the multi-type program relation encoding unit calculates the expression of the neighborhood aggregation information of node v during the encoding at the l-th layer. as follows:

[0034] (1),

[0035] In the formula, r represents the index of the set R of program relation types, and u represents the set of neighboring nodes of the r-th program relation that are adjacent to node v. index, This represents the information weights of neighbor nodes u and v in layer l under the r-th type of program relation. This represents the trainable transformation matrix corresponding to the r-th type of program relation in the l-th layer. This represents the hidden state of neighbor node u at level l;

[0036] The expression used by the node state update unit to determine the updated hidden state of the node is as follows:

[0037] (2),

[0038] In the formula, This represents the hidden state of node v after the update at level l+1; Represents a non-linear activation function. This represents the trainable transformation matrix corresponding to the self-loop of a node in layer l. This represents the hidden state of node v after the update at level l.

[0039] Furthermore, the semantic representation extraction module extracts the context semantic encoding of the source code token sequence, and aggregates the context semantic encoding according to the effective token mask to obtain a function-level semantic representation. The semantic representation extraction module includes a token embedding layer, a position encoding layer, a multi-layer context semantic encoding unit, an effective token mask processing unit, a semantic feature pooling unit, and a semantic feature projection layer.

[0040] The Token embedding layer and the position encoding layer convert the source code Token sequence into a Token vector sequence with position information;

[0041] The multi-layer context semantic encoding unit generates the context hidden state of each token based on the token vector sequence;

[0042] The effective token mask processing unit filters out the context hiding states of the effective token positions from the context hiding states of each token based on the effective token mask, and filters out the context hiding states that fill the token positions.

[0043] The semantic feature pooling unit aggregates the context hiding state of the valid token position according to the valid token mask to obtain the function-level source code semantic representation;

[0044] The semantic feature projection layer maps the function-level source code semantic representation to a unified latent space, resulting in the projected function-level semantic representation.

[0045] Furthermore, the projected function-level semantic representation The expression is as follows:

[0046] (3),

[0047] In the formula, Represents a trainable semantic projection matrix. Indicates the bias term; s represents the function-level source code semantic representation. , Indicates the preset maximum sequence length. This represents the hidden context state of the j-th token. Indicates whether the j-th position is a valid token. , When j is the filling position, it means that the j-th position is the filling position. This indicates that the j-th position is a valid token; This represents a preset constant used to avoid the denominator being zero.

[0048] Furthermore, the cross-modal fusion representation The expression is as follows:

[0049] (4),

[0050] In the formula, This represents the structure after projection. This represents the semantic representation after projection;

[0051] This represents semantic parameterization-enhanced representation. , This represents the trainable weight matrix of the output projection layer. This represents the bias term of the output projection layer, and M represents the number of heads in the multi-head parameterization transformation.

[0052] This represents the output of the i-th parameterized header. , , Let represent the parameterized transformation function in the i-th parameterized header that takes the query vector, key vector, and value vector as input. , as well as These represent the query vector, key vector, and value vector of the i-th parameterized header, respectively.

[0053] , , as well as These represent the parameters used in the i-th parameterized header to... Mapping to query vectors, A trainable projection matrix that maps to key vectors and value vectors.

[0054] Furthermore, the vulnerability classifier includes a first linear layer, a nonlinear activation layer, a random deactivation layer, and an output layer;

[0055] The cross-modal fusion representation is input to the first linear layer, and after processing by the nonlinear activation layer and the random deactivation layer, the vulnerability prediction value is output through the output layer. The vulnerability prediction value is then converted into the vulnerability probability of the function to be detected through a probability mapping function. The expression for the vulnerability probability of the function to be detected is as follows:

[0056] (5),

[0057] In the formula, This indicates the probability of a vulnerability in the function to be detected.

[0058] Indicates the vulnerability prediction value. , This represents the weight matrix of the first linear layer. This represents the bias term of the first linear layer. This represents the weight matrix of the output layer. This represents the bias term of the output layer. Represents the random deactivation function. Represents a non-linear activation function;

[0059] The probability of the vulnerability Compared with the preset classification threshold When comparing, When the function to be tested is found to have a vulnerability, it is output. When the function to be tested is found to be free of vulnerabilities, the output will indicate that the function does not contain any vulnerabilities.

[0060] On the other hand, the present invention also provides a function-level vulnerability detection system based on cross-modal code representation fusion, comprising:

[0061] The acquisition module is configured to: acquire the source code of the function to be detected;

[0062] The input construction module is configured to: construct structural input and semantic input based on the source code of the function to be detected; wherein, the structural input is a simplified code graph generated based on the program structure of the source code of the function to be detected, and the simplified code graph includes graph structure and node features; the semantic input is a source code token sequence and a valid token mask generated based on the source code text of the same source code of the function to be detected.

[0063] The structure representation extraction module is configured to: extract and aggregate the program relationship neighborhood information of each node in the simplified code graph, update the hidden state of each node according to the aggregated neighborhood information, and then perform graph-level aggregation on the updated hidden states of each node to obtain a function-level structure representation; wherein, the program relationship includes syntax relationship, control flow relationship and data flow relationship;

[0064] The semantic representation extraction module is configured to: extract the context semantic encoding of the source code token sequence, and aggregate the context semantic encoding according to the valid token mask to obtain a function-level semantic representation;

[0065] The projection module is configured to project the function-level structural representation and the function-level semantic representation onto a unified latent space of the same dimension, respectively, to obtain the projected structural representation and the projected semantic representation.

[0066] The semantic enhancement module is configured to: use the projected structural representation as the query-side input, and the projected semantic representation as the key-side input and value-side input, perform a multi-head parameterization transformation to generate a semantic parameterized enhanced representation;

[0067] The cross-modal fusion module is configured to obtain a cross-modal fused representation by concatenating the projected structural representation, the projected semantic representation, and the semantically parameterized enhanced representation.

[0068] The classification output module is configured to: input the cross-modal fusion representation into the vulnerability classifier to obtain the vulnerability probability of the function to be detected, compare the vulnerability probability with a preset classification threshold, and output the detection result of whether the function to be detected has a vulnerability or not.

[0069] Compared with the prior art, the beneficial effects achieved by the present invention are as follows:

[0070] 1. While existing source code sequence-based methods can learn the meaning of identifiers and contextual semantics, they are difficult to explicitly represent variable definitions and usages, control paths, and data propagation relationships. Therefore, this invention simultaneously constructs two types of inputs for the same function source code: a structure diagram and a source code sequence. This enables the subsequent vulnerability detection model to obtain complementary vulnerability-related clues from both program structure and source code semantics, thereby avoiding information loss caused by a single code representation.

[0071] 2. Existing simple concatenation or fixed-weighted fusion methods are insufficient to fully model the interaction relationships between different modalities. This invention projects two types of representations onto a unified latent space, and then performs a multi-head parameterized transformation using the structural representation as the query and the semantic representation as both the key and value, generating a semantically parameterized enhanced representation. Compared to simple concatenation, this transformation can parameterize the function-level semantic representation through multi-head learnable projection and concatenate it with the original structural and semantic representations to jointly utilize both types of representations.

[0072] 3. This invention avoids establishing a one-to-one correspondence between nodes and tokens by completing the fusion in the function-level representation space. At the same time, it retains three types of representations: the projected structural representation, the projected semantic representation, and the enhanced representation obtained by multi-head parameterization transformation of the projected semantic representation. This enables the vulnerability classifier to make decisions by jointly utilizing program structure features, pre-trained semantic features, and semantic parameterization enhancement features. Attached Figure Description

[0073] Figure 1 The diagram shown is a flowchart of a function-level vulnerability detection method and system based on cross-modal code representation fusion provided by the present invention.

[0074] Figure 2The diagram shown is a schematic of the function-level asymmetric multi-head parameterization transformation module structure of a function-level vulnerability detection method and system based on cross-modal code representation fusion provided by the present invention.

[0075] Figure 3 The diagram shown is a schematic diagram of the vulnerability classifier layer structure of a function-level vulnerability detection method and system based on cross-modal code representation fusion provided by the present invention.

[0076] Figure 4 The diagram shown illustrates the changes in data dimensions across various model layers of a function-level vulnerability detection method and system based on cross-modal code representation fusion, as provided by this invention.

[0077] Figure 5 The diagram shows a schematic of the vulnerability detection model training and testing process of a function-level vulnerability detection method and system based on cross-modal code representation fusion provided by the present invention. Detailed Implementation

[0078] It should be noted that the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations on the technical solution of the present invention. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0079] The term "and / or" simply describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0080] Example 1

[0081] See Figure 1 This embodiment provides a function-level vulnerability detection method based on cross-modal code representation fusion. The technical concept of the method is to simultaneously construct a simplified code graph and a source code token sequence for the function source code to obtain function-level structural representation and function-level semantic representation respectively. In a unified latent space, a function-level asymmetric multi-head parameterization transformation is used to generate semantic parameterization enhanced representation, and the three types of representations are jointly used for vulnerability classification, thereby avoiding complex node-to-token item-by-item interaction and making full use of the complementarity of structural and semantic information.

[0082] The method specifically includes the following steps:

[0083] Step S1: Obtain the source code of the function to be tested.

[0084] During the model training phase, function source code samples with function-level vulnerability tags are obtained. In this embodiment, function source code samples are obtained from publicly available vulnerability datasets. Each sample contains the source code text of a complete function and its corresponding vulnerability tag, with the tag value being 0 or 1. The function source code sample set is divided into a training set, a validation set, and a test set according to a preset ratio. As a specific implementation method, the set is divided in an 8:1:1 ratio, with each training sample corresponding to a structured input. Semantic input and tags ,in There were no vulnerabilities at that time. There are vulnerabilities.

[0085] In the actual testing phase, the source code of the function to be tested without function-level vulnerability tags is obtained, and a structured input is constructed for the source code of the function to be tested. and semantic input .

[0086] Step S2: Based on the same function source code, construct structural input and semantic input respectively; wherein, the structural input is a simplified code graph generated based on the program structure of the same function source code, and the simplified code graph includes graph structure and node features; the semantic input is the source code token sequence and valid token mask generated based on the source code text of the same function source code.

[0087] See Figure 1 , Figure 2 and Figure 4 In this embodiment, the structural input is a simplified code graph generated based on the program structure of the function source code, which includes graph structure and node features. The specific construction method is as follows:

[0088] Step S2A1: Utilize a program parsing tool to parse the function source code. As a specific implementation method, this embodiment uses Joern to parse the function source code, extracting nodes corresponding to statements, variables, expressions, operators, and function calls. It also extracts the abstract syntactic relationships, control flow relationships, and data dependencies between these nodes to construct an initial code structure graph. Here, abstract syntactic relationships refer to the hierarchical syntactic structure of the function source code, control flow relationships refer to the execution path and order of the program in the function source code, and data dependencies refer to the generation and usage relationships of data in the program. Joern is an open-source white-box static analysis tool based on code property graphs, used for security detection and vulnerability discovery of multi-language code.

[0089] Step S2A2: Perform graph simplification on the initial code structure graph obtained in step S2A1 to obtain a simplified initial code structure graph. As a specific implementation, this embodiment first removes auxiliary nodes from the initial code structure graph, such as bracket nodes, semicolon nodes, and other nodes that do not carry key semantic information; then, it prunes or compresses local abstract syntax subtrees, for example, merging syntax subtrees containing only a single child node into the parent node; and it merges redundant nodes representing the same variable or the same code semantic unit, thus reducing the graph size.

[0090] Step S2A3: Perform relation edge reconstruction. To compensate for the lost code unit order information and node self-information during the graph simplification process in step S2A2, while retaining abstract syntax edges, control flow edges, and data dependency edges, natural sequence edges are added according to the code unit arrangement order in the function source code, and self-loop edges are added to each node to obtain a simplified graph structure.

[0091] Step S2A4: Decompose the code text corresponding to each node in the simplified graph structure into sub-word units, and map the sub-word units to word vectors. As a specific implementation, in this embodiment, Word2Vec is used to map the code text corresponding to each node in the simplified graph structure to 100-dimensional word vectors, and average pooling is performed on the word vectors belonging to the same node to obtain fixed-dimensional node features for each node. The simplified graph structure and fixed-dimensional node features constitute the simplified code graph.

[0092] In this embodiment, the semantic input is a sequence of source code tokens and a valid token mask generated from the source code text of the same function. The specific generation method is as follows:

[0093] Step S2B1: Use the tokenizer corresponding to the pre-trained code model to segment the source code text of the same function to obtain the initial token sequence. As a specific implementation, this embodiment uses the tokenizer corresponding to UniXcoder to perform word segmentation and obtain the initial token sequence.

[0094] UniXcoder is a unified cross-modal pre-trained model proposed by Microsoft, designed to support both code-related understanding and generation tasks. It combines information from code comments and Abstract Syntax Trees (ASTs), using special prefix adapters and masked attention matrices to control the model's behavior in three modes: Encoder-only, Decoder-only, and Encoder-Decoder.

[0095] Step S2B2: Add preset sequence markers to the initial token sequence and calculate the number of tokens after adding the markers. As a specific implementation, this embodiment adds classification markers and separator markers to the initial token sequence and calculates the number of tokens in the initial token sequence after adding the classification markers and separator markers.

[0096] Step S2B3: When the number of tokens exceeds the preset maximum sequence length, truncation is performed; when the number of tokens is less than the preset maximum sequence length, padding is performed to generate the source code token sequence. As a specific implementation, in this embodiment, the preset maximum sequence length is set to 512. When the number of tokens is less than 512, the initial token sequence is padded, and the mask value at the padding position is set to 0; when the number of tokens is greater than 512, the initial token sequence is truncated, ensuring that the number of tokens in the generated source code sequence remains at 512.

[0097] Step S2B4: Set the mask value of the valid token position in the source code token sequence to 1, and set the mask value of the filling token position to 0 to generate a valid token mask.

[0098] Step S2B5: Associate simplified code graphs belonging to the same function with source code token sequences. Specifically, based on the file path list recorded in the graph data file, read the source code files of the same function corresponding to the simplified code graphs, and establish a one-to-one correspondence between the simplified code graphs, source code token sequences, and valid token masks. During the model training phase, further associate the simplified code graphs, source code token sequences, and valid token masks with function-level vulnerability labels of the same function to form complete training samples. Samples for which valid simplified code graphs cannot be established, such as samples that fail to be parsed, are filtered out.

[0099] Step S3: Extract and aggregate the program relationship neighborhood information of each node in the simplified code graph, update the hidden state of each node according to the aggregated neighborhood information, and then perform graph-level aggregation on the updated hidden states of each node to obtain the function-level structure representation; where program relationships include syntax relationships, control flow relationships and data flow relationships.

[0100] In this embodiment, the structural representation extraction module performs the extraction and aggregation of program relationship neighborhood information of each node in the simplified code graph, the updating of the hidden state of each node, and graph-level convergence. The structural representation extraction module includes a multi-type program relationship encoding unit, a node state update unit, a node feature sequence processing unit, a graph-level feature convergence unit, and a structural feature projection layer, as shown in Table 1:

[0101] Table 1 shows the framework composition of the extraction module.

[0102]

[0103] Specifically, the implementation steps are as follows:

[0104] Step S3A1: The multi-type program relationship encoding unit calculates the information weights between adjacent nodes based on the relationship types of edges in the simplified code graph and aggregates neighborhood information. The initial shape of the node features is [N, 100], which becomes [N, 100] after multi-type program relationship modeling by two layers of edge-aware GraphTransformer. Here, N represents the number of nodes in the simplified code graph corresponding to a single function.

[0105] The expression for calculating the neighborhood aggregation information of node v during the l-th layer encoding. as follows:

[0106] (1),

[0107] In the formula, l represents the layer index of the graph coding. L represents the total number of graph encoding layers; r represents the index of the set R of program relation types, which includes syntax relations, control flow relations, data flow relations, etc.; u represents the set of neighboring nodes of the r-th program relation adjacent to node v. index, This represents the information weights of neighbor nodes u and v in layer l under the r-th type of program relation. This represents the trainable transformation matrix corresponding to the r-th type of program relation in the l-th layer. This represents the hidden state of neighbor node u at level l.

[0108] Step S3A2: The node state update unit updates the hidden state of each node based on the aggregated neighborhood information, while maintaining the shape of the node features as [N, 100]. The specific calculation expression is as follows:

[0109] (2),

[0110] In the formula, This represents the hidden state of node v after the update at level l+1; Represents a non-linear activation function. This represents the trainable transformation matrix corresponding to the self-loop of a node in layer l. This represents the hidden state of node v after the update at level l.

[0111] Each node collects information from neighboring nodes under various program relationships, and then updates it in combination with its own state. After L-layer encoding, the hidden state of each node integrates the information of all neighbors within the L-hop range.

[0112] Step S3A3: The node feature sequence processing unit performs local and long-distance feature processing on the updated hidden states of each node to obtain the effective node hidden states.

[0113] In this embodiment, the node feature sequence processing unit sequentially includes a graph transform layer, a RepLK residual block, a first average pooling layer, a ConvFFN residual block, and a second average pooling layer. The graph transform layer performs feature transformation on the updated hidden states of each node; the RepLK residual block extracts local and long-range features from the transformed node feature sequence; the first average pooling layer downsamples the extracted feature sequence; the ConvFFN residual block performs point-by-point feature transformation on the downsampled feature sequence; and the second average pooling layer downsamples the point-by-point transformed feature sequence again to obtain [B, 100, ...]. The node feature sequence of ]. This represents the length of the node sequence after two AvgPool1d operations.

[0114] The expression for the node feature sequence processing unit is as follows:

[0115] (6),

[0116] In the formula, This represents the result of processing the hidden states of all nodes by the node feature sequence processing unit. This represents the node feature sequence processing function, and V represents the set of all nodes in the simplified code graph.

[0117] Step S3A4: The graph-level feature aggregation unit aggregates the hidden states of effective nodes to obtain a 100-dimensional function-level graph representation g. The specific expression is as follows:

[0118] (7),

[0119] In the formula, g represents the function-level graph representation; This represents the graph-level feature convergence function.

[0120] Step S3A5: The structural feature projection layer maps the function-level graph representation to a unified latent space, resulting in a projected 256-dimensional function-level structural representation. The specific expression is as follows:

[0121] (8),

[0122] In the formula, This represents the projected function-level structure representation; Represents the transformation matrix of the structural projection layer. This represents the bias term for the structural projection layer.

[0123] After the above processing, the structural representation extraction module condenses the information scattered across the nodes in the graph into a fixed-length vector. This vector is the structural representation of the entire function code.

[0124] Step S4: Refer to Figure 1 , 2 and Figure 4 The context semantic encoding of the source code token sequence is extracted, and the context semantic encoding is aggregated according to the valid token mask to obtain the function-level semantic representation.

[0125] In this embodiment, the source code token sequence and the valid token mask are input into the semantic representation extraction module to obtain a function-level semantic representation. The semantic representation extraction module includes a token embedding layer, a positional encoding layer, a multi-layer contextual semantic encoding unit, a valid token mask processing unit, a semantic feature pooling unit, and a semantic feature projection layer.

[0126] The semantic representation extraction module includes a token embedding layer, a position encoding layer, a multi-layer context semantic encoding unit, an effective token mask processing unit, a semantic feature pooling unit, and a semantic feature projection layer;

[0127] The token embedding layer and the position encoding layer convert the source code token sequence into a token vector sequence with position information;

[0128] The multi-layer context semantic coding unit generates the context hidden state of each token based on the token vector sequence;

[0129] The valid token mask processing unit filters out the context hiding states of valid token positions from the context hiding states of each token based on the valid token mask, and filters out the context hiding states that fill token positions.

[0130] The semantic feature pooling unit aggregates the contextual hidden state of the valid token position according to the valid token mask to obtain the function-level source code semantic representation;

[0131] The semantic feature projection layer maps the function-level source code semantic representation to a unified latent space, resulting in the projected function-level semantic representation.

[0132] Specifically, the projected function-level semantic representation The expression is as follows:

[0133] (3),

[0134] In the formula, Represents a trainable semantic projection matrix. Indicates the bias term; s represents the function-level source code semantic representation. , Indicates the preset maximum sequence length. This represents the hidden context state of the j-th token. Indicates whether the j-th position is a valid token. , When j is the filling position, it means that the j-th position is the filling position. This indicates that the j-th position is a valid token; This represents a preset constant used to avoid the denominator being zero.

[0135] As a specific implementation, this embodiment inputs a source code token sequence of shape [B, 512] and a valid token mask into UniXcoder to obtain a token-level context hiding state of shape [B, 512, 768]. Masked average pooling is then performed according to the valid token mask to obtain a 768-dimensional function-level source code semantic representation, which is then mapped to a 256-dimensional function-level semantic representation via a semantic projection layer. .

[0136] Step S5: Project the function-level structural representation and the function-level semantic representation onto a unified latent space of the same dimension to obtain the projected structural representation and the projected semantic representation.

[0137] The function-level structure representation is obtained through steps S3 and S4, respectively. and function-level semantic representation Since the two reside in different feature spaces—structural representation derived from the encoding of program structure by graph neural networks and semantic representation derived from the encoding of source code text by pre-trained code models—it is necessary to project them into a unified latent space of the same dimension for subsequent cross-modal fusion.

[0138] The structural feature projection layer maps the function-level graph representation g to a unified latent space, resulting in a 256-dimensional structural representation after projection. The semantic feature projection layer maps the function-level source code semantic representation s to a unified latent space, resulting in a 256-dimensional semantic representation after projection. At this point, both have the same dimension and exist in the same vector space.

[0139] Step S6: Using the projected structural representation as the query-side input and the projected semantic representation as the key-side and value-side input, perform a multi-head parameterization transformation to generate a semantically parameterized enhanced representation.

[0140] The core idea of ​​multi-head parameterized transformation is to perform multiple sets of learnable projections and output projections on the projected semantic representation to form an additional semantically enhanced representation, which is then combined with the structural and semantic representations for classification.

[0141] Step S7: By concatenating the projected structural representation, the projected semantic representation, and the semantic parameterized enhanced representation, a cross-modal fusion representation is obtained.

[0142] Specifically, M parallel parameterized heads are set up for cross-modal fusion representation. The expression is as follows:

[0143] (4),

[0144] In the formula, This represents the structure after projection. This represents the semantic representation after projection;

[0145] This represents semantic parameterization-enhanced representation. , This represents the trainable weight matrix of the output projection layer. This represents the bias term of the output projection layer, and M represents the number of heads in the multi-head parameterization transformation.

[0146] This represents the output of the i-th parameterized header. , , Let represent the parameterized transformation function in the i-th parameterized header that takes the query vector, key vector, and value vector as input. , as well as These represent the query vector, key vector, and value vector of the i-th parameterized header, respectively.

[0147] , , as well as These represent the parameters used in the i-th parameterized header to... Mapping to query vectors, A trainable projection matrix that maps to key vectors and value vectors.

[0148] As a specific implementation method, this embodiment preferably uses M=4, where the dimension of each parameterized head is... The value is 64. Parameterized transformation function. Using the scaled dot product method, first calculate , then calculate ,in Let represent the attention weight matrix of the i-th parameterized head; after concatenating the outputs of the four parameterized heads and performing output projection and length dimension compression, a 256-dimensional semantic parameterized augmentation representation is obtained. .

[0149] The structural and semantic branches each output a 256-dimensional representation. and Multi-head parameterized transformation outputs a 256-dimensional representation. The three types of representations are concatenated to obtain a 768-dimensional representation. The relevant modules and preferred parameters are shown in Table 2:

[0150] Table 2 Relevant Modules and Optimal Parameters

[0151]

[0152] Step S8: Input the cross-modal fusion representation into the vulnerability classifier to obtain the vulnerability probability of the function to be detected. Compare the vulnerability probability with the preset classification threshold and output the detection result of whether the function to be detected has a vulnerability or not.

[0153] The vulnerability classifier consists of a first linear layer, a non-linear activation layer, a random deactivation layer, and an output layer;

[0154] The cross-modal fusion representation is input to the first linear layer, and after processing by a nonlinear activation layer and a random deactivation layer, the vulnerability prediction value is output through the output layer. The vulnerability prediction value is then converted into the vulnerability probability of the function to be detected through a probability mapping function. In this embodiment, The data is sequentially processed through a ReLU nonlinear activation layer and a Dropout layer with a random inactivation rate of 0.2, and then through an output layer, the 256-dimensional vulnerability prediction is mapped to a 1-dimensional vulnerability prediction. Finally, the vulnerability probability is obtained using the Sigmoid function. .

[0155] The expression for the vulnerability probability of the function to be detected is as follows:

[0156] (5),

[0157] In the formula, This indicates the probability of a vulnerability in the function to be detected.

[0158] Indicates the vulnerability prediction value. , This represents the weight matrix of the first linear layer. This represents the bias term of the first linear layer. This represents the weight matrix of the output layer. This represents the bias term of the output layer. Represents the random deactivation function. This represents a non-linear activation function, which is preferably the ReLU function in this embodiment;

[0159] The probability of the vulnerability Compared with the preset classification threshold When comparing, When the function to be tested is found to have a vulnerability, it is output. When the function to be tested is found to be free of vulnerabilities, the output will indicate that the function does not contain any vulnerabilities.

[0160] The neural network mentioned above needs to be trained before the model can be deployed.

[0161] First, a sample set of function source code with function-level vulnerability labels is obtained, and the sample set is divided into a training set and a validation set. For each function source code sample, a simplified code graph and a source code token sequence are constructed. Based on the file path, function identifier, or sample index of the same function, the correspondence between structural input, semantic input, and function-level vulnerability labels is completed, and samples with missing source files or for which a valid simplified code graph cannot be constructed are filtered out.

[0162] The corresponding training samples are input into the initial neural network in batches, and the following steps are performed sequentially: structural representation extraction, semantic representation extraction, multi-head parameterization transformation, cross-modal representation concatenation, and vulnerability classification, to obtain the vulnerability prediction value for each training sample. The training loss is calculated based on the vulnerability prediction value and the corresponding function-level vulnerability label. The trainable parameters in the neural network are updated through backpropagation, including those in the structural representation extraction module, semantic representation extraction module, projection matrices of the multi-head parameterization transformation, and parameters in the vulnerability classifier. The optimal model parameters are determined based on the vulnerability detection performance on the validation set, resulting in the trained neural network.

[0163] like Figure 5 As shown, during the training phase, the labeled function sample set undergoes bimodal data preprocessing and sample alignment, and is input into the initial structure-semantic fusion model for forward propagation and loss calculation. Model parameters are updated through backpropagation, and the optimal vulnerability detection model after training is saved based on the validation set evaluation results. During the testing phase, the source code of the function to be detected undergoes the same bimodal preprocessing as during training, and the optimal trained model is loaded to calculate the vulnerability probability. ,Will With preset threshold The comparison outputs the detection results, indicating whether or not a vulnerability exists.

[0164] Example 2

[0165] This embodiment, based on Embodiment 1, uses two function-level vulnerability datasets, Devign and Big-Vul, to verify the overall detection performance of the present invention. After function source code parsing, code graph simplification, and alignment of structural and semantic inputs, Devign retains 19,493 function samples, of which 8,845 are vulnerable and 10,648 are non-vulnerable, with vulnerable samples accounting for 45.38%; Big-Vul retains 148,035 function samples, of which 6,887 are vulnerable and 141,148 are non-vulnerable, with vulnerable samples accounting for 4.65%. Training, validation, and test sets are constructed according to predetermined data partitioning, and Accuracy, Precision, Recall, and F1 are used as evaluation metrics. Given the significant class imbalance in Big-Vul, in addition to Accuracy, Precision, Recall, and F1 are emphasized in combination to evaluate the model's ability to detect vulnerability samples.

[0166] The model uses the AdamW optimizer with a learning rate of 1×10⁻ 5 The weight decay coefficient was set to 1×10⁻², the batch size to 16, and the maximum number of training epochs to 50. Early stopping was performed based on the validation set F1 score. The UniXcoder was fully fine-tuned on Devign using BCEWithLogitsLoss; on Big-Vul, only the last two layers of the UniXcoder were fine-tuned using Focal Loss. Set to 2.0, class weight Set to (1.0, 15.0).

[0167] To verify the overall detection capability of this invention, seven representative methods—Devign, LineVul, UniXcoder, AMPLE, MAGNET, MGVD, and VulCNN—were selected for comparison. These methods cover three technical approaches: graph structure representation, pre-trained semantic representation, and multi-representation fusion. UniXcoder, AMPLE, and the complete method of this invention were tested using the same data partitioning, preprocessing procedures, and test sets. The publicly available results of the other methods were used as references for comparison. Since Big-Vul does not provide a single official data partition, data cleaning, function extraction, and duplicate sample handling may differ between studies; therefore, the publicly available results are not considered for rigorous comparison with the same setup. Table 3 presents the results of each method on two datasets, with all indicators expressed as percentages.

[0168] Table 3 Comparison of results from Devign and Big-Vul datasets

[0169]

[0170] As shown in Table 3, on the Devign dataset, the accuracy, precision, recall, and F1 scores of the complete method of this invention are 69.13%, 64.37%, 75.05%, and 69.30%, respectively, with the highest F1 score among the listed results. Compared with UniXcoder under the same settings, the complete method of this invention improves recall by 8.57 percentage points and F1 by 2.52 percentage points, indicating that the introduction of structural information can detect more real vulnerability samples. Compared with AMPLE under the same settings, precision improves by 6.08 percentage points and F1 by 2.01 percentage points, indicating that pre-trained semantic representation can improve false positive control when relying solely on structural representation. Compared with the reference results of MAGNET, MGVD, and VulCNN, the complete method of this invention also achieves a higher F1 score, indicating that structural representation and semantic representation are complementary.

[0171] On the Big-Vul dataset, the accuracy, precision, recall, and F1 score of the complete method of this invention are 98.81%, 89.75%, 84.01%, and 86.79%, respectively. Compared with UniXcoder under the same settings, the F1 score is improved by 1.98 percentage points, the precision is improved by 7.97 percentage points, and the recall is reduced by 4.07 percentage points. This indicates that structural branching and function-level asymmetric cross-modal fusion, while reducing false positives and improving overall detection performance, strike a balance between precision and recall in terms of coverage of some vulnerability samples. Compared with AMPLE under the same settings, all four metrics of the complete method of this invention are significantly improved; it also has high overall detection performance compared with publicly available reference results such as VulCNN.

[0172] The results from the two datasets in Table 3 show that, compared with UniXcoder and AMPLE under the same experimental settings, the complete method of this invention achieves higher F1 scores; it also demonstrates strong competitiveness compared with the publicly available results of other representative methods. These results indicate that by projecting the function-level structural representation learned from the simplified code graph and the function-level semantic representation learned from the pre-trained code model onto a unified latent space, and generating semantically parameterized enhanced representations through function-level asymmetric multi-head parameterization transformation, it is possible to jointly utilize program structure information and source code context semantic information, thereby improving the overall performance of function-level vulnerability detection.

[0173] Furthermore, to verify the independent roles of structural representation, semantic representation, and function-level asymmetric cross-modal fusion, four model settings were constructed: Text-only, Graph-only, Concat, and the complete method of this invention. Text-only retains only the semantic branch and classifier; Graph-only retains only the structural branch and classifier; Concat directly concatenates the structural and semantic representations before classification; the complete method of this invention generates a semantically parameterized augmented representation in a unified latent space and concatenates the structural, semantic, and semantically parameterized augmented representations before classification. Results on the Devign dataset are shown in Table 4.

[0174] Table 4 Results on the Devign dataset

[0175]

[0176] The results on the Big-Vul dataset are shown in Table 5.

[0177] Table 5 Results on the Big-Vul dataset

[0178]

[0179] As shown in Table 4, both structural monomodality and semantic monomodality can provide vulnerability detection information; structural monomodality has a higher recall, indicating that explicit control flow, data flow, and variable dependencies help detect more vulnerability samples; simple concatenation improves F1, indicating that the two code representations are complementary; the F1 of the complete method of this invention is further improved to 69.30%, indicating that function-level asymmetric cross-modal fusion can more effectively combine the two types of representations than simple concatenation.

[0180] As shown in Table 5, semantic monomodality has strong performance on large-scale datasets, while structural monomodality has lower performance when used alone. However, after the fusion of structural and semantic representations, both Concat and the complete method of this invention outperform semantic monomodality. The complete method of this invention achieves a precision of 89.75% and an F1 score of 86.79%, indicating that structural information can effectively supplement the pre-trained semantic representation after cross-modal fusion.

[0181] Further experiments on fusion granularity show that the F1 scores of function-level vector fusion on Devign and Big-Vul are 69.30% and 86.79%, respectively, which are higher than the 68.25% and 86.18% of node-to-token level fine-grained fusion. This indicates that the function-level fusion used in this invention can reduce cross-granularity correlation noise and obtain more stable overall detection performance.

[0182] This embodiment proposes a function-level vulnerability detection system based on cross-modal code representation fusion, building upon Embodiment 1. The system includes:

[0183] The acquisition module is configured to: acquire the source code of the function to be detected;

[0184] The input construction module is configured to: construct structural input and semantic input based on the source code of the function to be detected; wherein, the structural input is a simplified code graph generated based on the program structure of the source code of the function to be detected, and the simplified code graph includes graph structure and node features; the semantic input is a source code token sequence and a valid token mask generated based on the source code text of the same source code of the function to be detected.

[0185] The structure representation extraction module is configured to: extract and aggregate the program relationship neighborhood information of each node in the simplified code graph, update the hidden state of each node according to the aggregated neighborhood information, and then perform graph-level aggregation on the updated hidden states of each node to obtain the function-level structure representation; wherein, the program relationship includes syntax relationship, control flow relationship and data flow relationship;

[0186] The semantic representation extraction module is configured to: extract the context semantic encoding of the source code token sequence, and aggregate the context semantic encoding according to the valid token mask to obtain the function-level semantic representation;

[0187] The projection module is configured to project the function-level structural representation and the function-level semantic representation onto a unified latent space of the same dimension, respectively, to obtain the projected structural representation and the projected semantic representation.

[0188] The semantic enhancement module is configured to: take the projected structural representation as the query-side input, and the projected semantic representation as the key-side input and value-side input, perform multi-head parameterization transformation, and generate a semantic parameterized enhanced representation;

[0189] The cross-modal fusion module is configured to obtain a cross-modal fused representation by concatenating the projected structural representation, the projected semantic representation, and the semantically parameterized enhanced representation.

[0190] The classification output module is configured to: input the cross-modal fusion representation into the vulnerability classifier, obtain the vulnerability probability of the function to be detected, compare the vulnerability probability with a preset classification threshold, and output the detection result of whether the function to be detected has a vulnerability or not.

[0191] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0192] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0193] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0194] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0195] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A function-level vulnerability detection method based on cross-modal code representation fusion, characterized in that, include: Obtain the source code of the function to be tested; Based on the source code of the function to be detected, structural input and semantic input are constructed respectively; wherein, the structural input is a simplified code graph generated based on the program structure of the source code of the function to be detected, and the simplified code graph includes graph structure and node features; the semantic input is a source code token sequence and a valid token mask generated based on the source code text of the same source code of the function to be detected. Extract and aggregate the program relationship neighborhood information of each node in the simplified code graph, update the hidden state of each node according to the aggregated program relationship neighborhood information, and then perform graph-level aggregation on the updated hidden states of each node to obtain a function-level structure representation; wherein, the program relationship includes syntax relationship, control flow relationship and data flow relationship; Extract the context semantic encoding of the source code token sequence, and aggregate the context semantic encoding according to the valid token mask to obtain a function-level semantic representation; The function-level structural representation and function-level semantic representation are projected onto a unified latent space of the same dimension to obtain the projected structural representation and projected semantic representation. Using the projected structural representation as the query-side input and the projected semantic representation as the key-side and value-side input, a multi-head parameterization transformation is performed to generate a semantically parameterized enhanced representation. By concatenating the projected structural representation, the projected semantic representation, and the semantic parameterized enhanced representation, a cross-modal fusion representation is obtained. The cross-modal fusion representation is input into the vulnerability classifier to obtain the vulnerability probability of the function to be detected. The vulnerability probability is compared with a preset classification threshold, and the detection result of whether the function to be detected has a vulnerability or not is output.

2. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 1, characterized in that, The process of generating the simplified code diagram includes: The source code of the function to be tested is parsed, and the nodes corresponding to statements, variables, expressions, operators and function calls are extracted from the source code of the function to be tested. The abstract syntax relationship, control flow relationship and data dependency relationship between the nodes are also extracted to construct an initial code structure diagram. The initial code structure graph is simplified to obtain a simplified initial code structure graph; wherein, the graph simplification operation includes: removing auxiliary nodes in the initial code structure graph, pruning or compressing local abstract syntax subtrees, and merging redundant nodes representing the same variable or the same code semantic unit; The abstract syntax edges, control flow edges, and data dependency edges in the simplified initial code structure graph are retained, and natural sequence edges are added according to the order of code units in the source code of the function to be detected. Self-loop edges are also added to each node to obtain a simplified graph structure. The code text corresponding to each node in the simplified graph structure is split into sub-word units, the sub-word units are mapped to word vectors, and the word vectors belonging to the same node are averaged and pooled to obtain fixed-dimensional node features for each node. The simplified graph structure and the fixed-dimensional node features constitute the simplified code graph.

3. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 1, characterized in that, The process of generating the source code token sequence and valid token mask includes: The source code text of the same function is segmented using the tokenizer corresponding to the pre-trained code model to obtain an initial token sequence; Add a preset sequence marker to the initial token sequence and calculate the number of tokens after adding the marker; When the number of tokens exceeds the preset maximum sequence length, truncation is performed; when the number of tokens is less than the preset maximum sequence length, padding is performed to generate the source code token sequence. The effective token mask is generated by setting the mask value of the valid token position in the source code token sequence to 1 and setting the mask value of the filling token position to 0.

4. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 1, characterized in that, The structural representation extraction module performs the extraction and aggregation of program relationship neighborhood information of each node in the simplified code graph, the updating of the hidden state of each node, and graph-level aggregation. The structural representation extraction module includes a multi-type program relationship encoding unit, a node state update unit, a node feature sequence processing unit, a graph-level feature aggregation unit, and a structural feature projection layer, specifically including: The multi-type program relationship encoding unit calculates the information weights between adjacent nodes and aggregates neighborhood information based on the relationship types of edges in the simplified code graph. The node state update unit updates the hidden state of each node based on the aggregated program relationship neighborhood information; The node feature sequence processing unit performs local and long-distance feature processing on the updated hidden states of each node to obtain the effective node hidden states. The graph-level feature aggregation unit aggregates the hidden states of the effective nodes to obtain a function-level graph representation; The structural feature projection layer maps the function-level graph representation to a unified latent space, resulting in the projected function-level structural representation.

5. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 4, characterized in that, The multi-type program relation encoding unit calculates the expression of the neighborhood aggregation information of node v during the encoding of the l-th layer. as follows: （1）， In the formula, r represents the index of the set R of program relation types, and u represents the set of neighboring nodes of the r-th program relation that are adjacent to node v. index, This represents the information weights of neighbor nodes u and v in layer l under the r-th type of program relation. This represents the trainable transformation matrix corresponding to the r-th type of program relation in the l-th layer. This represents the hidden state of neighbor node u at level l; The expression used by the node state update unit to determine the updated hidden state of the node is as follows: （2）， In the formula, This represents the hidden state of node v after the update at level l+1; Represents a non-linear activation function. This represents the trainable transformation matrix corresponding to the self-loop of a node in layer l. This represents the hidden state of node v after the update at level l.

6. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 1, characterized in that, The semantic representation extraction module extracts the context semantic encoding of the source code token sequence and aggregates the context semantic encoding according to the effective token mask to obtain a function-level semantic representation. The semantic representation extraction module includes a token embedding layer, a position encoding layer, a multi-layer context semantic encoding unit, an effective token mask processing unit, a semantic feature pooling unit, and a semantic feature projection layer. The Token embedding layer and the position encoding layer convert the source code Token sequence into a Token vector sequence with position information; The multi-layer context semantic coding unit generates the context hidden state of each token based on the token vector sequence; The effective token mask processing unit filters out the context hiding states of the effective token positions from the context hiding states of each token based on the effective token mask, and filters out the context hiding states that fill the token positions. The semantic feature pooling unit aggregates the context hiding state of the valid token position according to the valid token mask to obtain the function-level source code semantic representation; The semantic feature projection layer maps the function-level source code semantic representation to a unified latent space, resulting in the projected function-level semantic representation.

7. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 6, characterized in that, The projected function-level semantic representation The expression is as follows: （3）， In the formula, Represents a trainable semantic projection matrix. Indicates the bias term; s represents the function-level source code semantic representation. , Indicates the preset maximum sequence length. This represents the hidden context state of the j-th token. Indicates whether the j-th position is a valid token. , When j is the filling position, it means that the j-th position is the filling position. This indicates that the j-th position is a valid token; This represents a preset constant used to avoid the denominator being zero.

8. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 1, characterized in that, The cross-modal fusion representation The expression is as follows: （4）， In the formula, This represents the structure after projection. This represents the semantic representation after projection; This represents semantic parameterization-enhanced representation. , This represents the trainable weight matrix of the output projection layer. This represents the bias term of the output projection layer, and M represents the number of heads in the multi-head parameterization transformation. This represents the output of the i-th parameterized header. , , Let represent the parameterized transformation function in the i-th parameterized header that takes the query vector, key vector, and value vector as input. , as well as These represent the query vector, key vector, and value vector of the i-th parameterized header, respectively. , , as well as These represent the parameters used in the i-th parameterized header to... Mapping to query vectors, A trainable projection matrix that maps to key vectors and value vectors.

9. The function-level vulnerability detection method based on cross-modal code representation fusion according to claim 8, characterized in that, The vulnerability classifier includes a first linear layer, a non-linear activation layer, a random deactivation layer, and an output layer; The cross-modal fusion representation is input to the first linear layer, and after processing by the nonlinear activation layer and the random deactivation layer, the vulnerability prediction value is output through the output layer. The vulnerability prediction value is then converted into the vulnerability probability of the function to be detected through a probability mapping function. The expression for the vulnerability probability of the function to be detected is as follows: （5）， In the formula, This indicates the probability of a vulnerability in the function to be detected. This represents the predicted vulnerability value. , This represents the weight matrix of the first linear layer. This represents the bias term of the first linear layer. This represents the weight matrix of the output layer. This represents the bias term of the output layer. Represents the random deactivation function. Represents a non-linear activation function; The probability of the vulnerability Compared with the preset classification threshold When comparing, At that time, the output of the function to be tested showed a vulnerability; when When the function to be tested is found to be free of vulnerabilities, the output will indicate that the function does not contain any vulnerabilities.

10. A function-level vulnerability detection system based on cross-modal code representation fusion, characterized in that, include: The acquisition module is configured to: acquire the source code of the function to be detected; The input construction module is configured to: construct structural input and semantic input based on the source code of the function to be detected; wherein, the structural input is a simplified code graph generated based on the program structure of the source code of the function to be detected, and the simplified code graph includes graph structure and node features; the semantic input is a source code token sequence and a valid token mask generated based on the source code text of the same source code of the function to be detected. The structure representation extraction module is configured to: extract and aggregate the program relationship neighborhood information of each node in the simplified code graph, update the hidden state of each node according to the aggregated neighborhood information, and then perform graph-level aggregation on the updated hidden states of each node to obtain a function-level structure representation; wherein, the program relationship includes syntax relationship, control flow relationship and data flow relationship; The semantic representation extraction module is configured to: extract the context semantic encoding of the source code token sequence, and aggregate the context semantic encoding according to the valid token mask to obtain a function-level semantic representation; The projection module is configured to project the function-level structural representation and the function-level semantic representation onto a unified latent space of the same dimension, respectively, to obtain the projected structural representation and the projected semantic representation. The semantic enhancement module is configured to: use the projected structural representation as the query-side input, and the projected semantic representation as the key-side input and value-side input, perform a multi-head parameterization transformation to generate a semantic parameterized enhanced representation; The cross-modal fusion module is configured to obtain a cross-modal fused representation by concatenating the projected structural representation, the projected semantic representation, and the semantically parameterized enhanced representation. The classification output module is configured to: input the cross-modal fusion representation into the vulnerability classifier to obtain the vulnerability probability of the function to be detected, compare the vulnerability probability with a preset classification threshold, and output the detection result of whether the function to be detected has a vulnerability or not.

Images