Large language model auxiliary vulnerability detection method and system based on abstract syntax tree decomposition and annotation enhancement

Abstract

The invention discloses a vulnerability detection method and system based on abstract syntax tree decomposition and large language model assistance. The vulnerability detection method and system are used for solving the problem that an existing pre-training model is insufficient in detection accuracy under complex code logic and multiple execution paths. The method comprises the following steps: firstly, analyzing a code snippet into an abstract syntax tree, splitting the abstract syntax tree into a plurality of sub-trees through an improved decomposition algorithm, and combining each sub-tree with a natural language annotation generated by a large language model to form an abstract sub-tree with the annotation; then, a semantic aggregator based on Transform is used for modeling the relation between the sub-trees, features are fused to a target vulnerability vector, and finally, vulnerabilities are predicted through a classifier. Based on the technical scheme, the vulnerability detection accuracy is effectively improved, and the performance of the vulnerability detection model is greatly improved.

Description

Technical Field

[0001] This invention relates to the fields of software engineering and network security technology, and in particular to a source code vulnerability detection method and system that combines abstract syntax tree structure decomposition and large language model to generate annotations. It is applicable to scenarios such as automated security detection, static code analysis, and software vulnerability discovery. Background Technology

[0002] In today's highly digitalized era, software systems are widely used in critical industries such as government, finance, healthcare, industrial control, intelligent manufacturing, and the Internet of Things, handling massive data processing, core business logic, and critical system control. Once software vulnerabilities are discovered and exploited by attackers, it can lead to data leaks, business interruptions, economic losses, and even national security risks. According to CrowdStrike's 2024 Threat Hunting Report, the number of major cybersecurity incidents caused by vulnerability exploitation increased significantly in the past year, with vulnerability exploitation becoming a major entry point for advanced persistent attacks and ransomware attacks. Data from the official CVE (Common Vulnerabilities and Exposures) statistics platform shows that a total of 40,300 CVE security vulnerabilities were disclosed in 2024, a 38% increase compared to 2023, setting a new record. Of these, high-risk vulnerabilities accounted for over 62%. These vulnerabilities are widely distributed across various software types, including basic libraries, device drivers, embedded systems, and web services. In the financial industry, vulnerabilities hidden in source code can lead to illicit fund transfers and the leakage of sensitive information; in medical systems, software defects can affect the confidentiality of patient data or even interfere with equipment operation; in the fields of intelligent manufacturing and industrial control, attackers can exploit vulnerabilities to infiltrate PLC systems, causing production process disruptions or even personal safety accidents. The prevalence, concealment, and high risk of vulnerabilities have become one of the major security threats to modern information society.

[0003] Despite the progress made in automated vulnerability detection technology in recent years, existing technologies still face three core challenges in real-world software environments: The first potential challenge is that for complex source code structures, models struggle to accurately parse the semantic logic of expressions. Real-world vulnerabilities are often hidden within multiple operators, complex nested expressions, or pointer operations, such as conditional statements containing symbols like ||, &&, ->, and *. Because traditional detection methods (especially token-sequence-based models) lack deep modeling capabilities for syntactic structures, such complex logic is easily misjudged or missed. The second potential challenge is that existing detection methods struggle to understand the actual execution path of the source code. Many high-risk vulnerabilities are only triggered under specific combinations of conditions and execution sequences; for example, some vulnerabilities are only exposed when two or more nested if conditions are met and variables are not initialized. Linear modeling methods cannot capture these non-linear control flows, making it difficult for models to identify such path-dependent security risks. The third potential challenge is that existing neural network-based vulnerability detection methods typically require a large number of high-quality labeled samples during the training phase, resulting in strong data dependence and high construction costs. Most deep learning models, such as those based on Transformers or graph neural networks, require parameter optimization using tens of thousands of samples. However, practically available labeled vulnerability datasets are very limited, and labels often exhibit subjectivity and inconsistency. Even constructing synthetic samples faces the problem of significant discrepancies between sample distribution and real-world vulnerabilities. Furthermore, model training and deployment require substantial computational resources and GPU memory consumption, making it difficult for ordinary enterprises and research teams to maintain the training cycle and inference overhead of high-performance models over long periods. This high dependence on data scale and computational resources severely limits the application, promotion, and generalization capabilities of models in real-world industrial environments. In summary, most current mainstream vulnerability detection methods still suffer from weak structural analysis capabilities, insufficient path modeling, and high dependence on large-scale training data. These problems are particularly prominent in real-world software development and auditing environments. Especially when faced with source code that is syntactically complex, has variable execution paths, and contains non-standard or missing comments, existing methods exhibit significant bottlenecks in detection accuracy, generalization ability, and deployment costs, failing to meet the urgent needs of key industries for high-precision, high-efficiency, and automated vulnerability detection. Therefore, there is an urgent need to propose a novel vulnerability detection method that integrates structural decomposition and semantic enhancement mechanisms. This method should have the following capabilities: on the one hand, it can effectively extract structurally sound and semantically complete subgraph units from the source code to explicitly model the syntax hierarchy and control flow structure of the code; on the other hand, it can enhance the model's understanding of code functionality and semantics by introducing a large language model to generate high-quality natural language annotations, thereby achieving accurate identification of potential vulnerabilities. Summary of the Invention

[0004] This invention discloses a software vulnerability detection method and system, VulTrLM, based on abstract syntax tree decomposition and large language model annotation enhancement. This method parses the source code into an abstract syntax tree (AST) and, while maintaining the control flow structure, uses an improved decomposition algorithm to divide the AST into multiple semantic subtrees. Each subtree automatically generates high-quality annotations through a large language model, constructing an annotation-enhanced subtree, further achieving refined modeling of vulnerability semantic information. Based on a pre-trained model, this method introduces annotation information generated by a large language model as a semantic supplement on top of structural encoding, effectively overcoming two major technical bottlenecks in existing technologies: (1) inability to accurately parse complex logical expressions containing multiple operators and pointers; and (2) difficulty in capturing vulnerability activation sequences hidden in multi-branch execution paths within the code. Furthermore, to enhance the modeling ability of execution sequences, this invention also designs a Transformer-based semantic aggregator to integrate the semantic features of multiple annotation-enhanced subtrees and fuses them with the source code representation through a cross-attention mechanism, thereby generating a unified vulnerability representation vector for downstream classifiers to determine whether the target code has security vulnerabilities. Empirical analysis shows that the method described in this invention achieves superior performance on multiple publicly available vulnerability code benchmark datasets. It exhibits good adaptability to different types of pre-trained models, and has broad application value and practical prospects.

[0005] The technical solution provided by this invention is as follows: On one hand, a software vulnerability detection method and system based on abstract syntax tree decomposition and large language model annotation enhancement includes:

[0006] Step 1: Constructing the source code abstract syntax tree;

[0007] Obtain the source code fragment to be tested, and use a lightweight syntax parsing tool (such as Tree-sitter) to convert the source code into the corresponding AST to preserve its syntax structure and control flow information.

[0008] Step 2: Structured decomposition of the abstract syntax tree;

[0009] A depth-first traversal is performed on the constructed AST. Based on the set depth and node number thresholds, an improved decomposition algorithm is used to divide the AST into multiple semantically complete subtrees, while preserving the parent-child relationship and execution order between subtrees.

[0010] Step 3: Annotate and enhance subtree CAS generation;

[0011] For each AST subtree obtained from decomposition, a large language model (such as Qwen2.5-Coder) is called to automatically generate natural language annotations describing the function and logic of the subtree, and the annotations are attached as annotation nodes to the root node of the subtree to form an annotation-enhanced subtree.

[0012] Step 4: Unified encoding representation of code and comment subtrees;

[0013] The source code and the CAS set are input into a pre-trained model (such as UniXcoder) to extract their respective high-dimensional semantic embedding vectors. The CAS set is format-converted using a tree-structured encoding function to adapt to the encoder's input requirements.

[0014] Step 5: Subtree semantic feature aggregation;

[0015] The semantic representations of all CAS are input into a structure-aware Transformer aggregator. An attention mask matrix is ​​constructed based on the control relationships (parent-child relationship, sibling relationship) between subtrees to achieve structure-sensitive semantic information fusion and generate a unified global subtree semantic vector.

[0016] Step 6: Multimodal feature fusion and cross-attention alignment:

[0017] By employing a cross-attention mechanism, the source code representation and the aggregated CAS global semantic representation are fused to generate a unified vulnerability representation vector. Fine-grained control flow information and semantic context are preserved during the fusion process.

[0018] Step 7: Vulnerability Detection and Classification:

[0019] The fused vulnerability representation vector is input into a multilayer perceptron classifier, and the Sigmoid function outputs a probability score indicating whether the target code fragment contains a security vulnerability, thus completing the final determination. This achieves secure access to and storage of plaintext attribute information.

[0020] Furthermore, in a preferred embodiment of the present invention, the structural decomposition step of the AST includes the following:

[0021] First, for each source code sample, the lightweight syntax parsing library Tree-sitter is used to convert it into a standard Abstract Syntax Tree (AST). This AST abstracts away the specific syntactic details of the code, retaining only structural information, and is an effective representation of program control flow, nested structures, and scope relationships. However, when the code logic is complex, the depth and width of the AST increase dramatically, bringing huge overhead to the subsequent modeling process and even interfering with the stability and accuracy of semantic modeling.

[0022] To this end, the present invention sets a set of structural constraint parameters, defined as the maximum depth threshold. With the maximum number of nodes threshold During the AST traversal, the system considers each candidate node... Calculate the actual depth of its corresponding subtree With the number of nodes .

[0023] If the conditions are met and If the condition is met, then the subtree is added to the result set as a semantic unit; if any condition is not met, then... or This triggers the decomposition operation, calling the sub-function. The subtree corresponding to this node is further divided to generate a more fine-grained set of subtrees.

[0024] Secondly, to fully preserve the semantic context relationships between subtrees, this invention further introduces a subtree index structure and a parent-child identification mechanism. Specifically, the system assigns a unique identifier to each decomposed subtree. And record the number of its direct parent node. This forms a subtree triplet. All subtrees are arranged in the order they were executed in the original code, ensuring that the original execution flow and control dependencies can be restored during the model processing phase;

[0025] Furthermore, preferably, the node selection strategy only decomposes and determines syntactic nodes with control structure meanings, such as "If", "For", "While", "DoWhile", "Switch", "Try", and "FuncDef". This design helps the model focus on potentially vulnerability-sensitive areas, such as conditional statements, loop structures, and function bodies, thereby improving the accuracy of vulnerability pattern localization and contextual interpretation capabilities.

[0026] On the other hand, one method that uses cue words to guide large language models in generating functional annotations for abstract subtrees is:

[0027] By employing carefully designed prompt templates, a large language model is guided to extract key semantic information from a structured syntax tree and generate accurate, concise, and context-sensitive natural language descriptions. This method not only avoids problems such as semantic generalization and structural misalignment in traditional annotation generation but also improves the target orientation of annotations and the model's ability to understand control statements.

[0028] Specifically, the prompts used in this invention comprise two core components:

[0029] The first is the System Prompt, which sets the model behavior, such as "You are a C++ expert, and your task is to generate concise and accurate functional comments for the code structure"; the second is the User Prompt, which embeds the subtree structure to be annotated and includes a clear explanation of the purpose of generating the comments, such as "Please provide a precise and concise comment to explain the function and purpose of the following code structure".

[0030] To improve the semantic focus of annotations, this invention further proposes to select only the core structure of the subtree as input content, omit redundant syntactic details, and at the same time explicitly emphasize the need for functional explanation.

[0031] Optionally, this method employs a low temperature setting (temperature = 0) to ensure the consistency and determinism of the generated annotations and avoid semantic drift. Finally, the generated annotation nodes are used as the root nodes of the subtrees, forming an abstract annotation subtree structure.

[0032] On the other hand, a unified encoding method for source code and its set of abstract annotation subtrees includes:

[0033] First, the original source code snippets are input into a pre-trained language model, such as UniXcoder, to obtain the corresponding semantic representation of the code. This model can generate dynamic high-dimensional vector representations for each token based on contextual information, thereby preserving the semantic relationships between variables, structures, and control flow.

[0034] Secondly, for each abstract annotation subtree, its structure is first formatted. This formatting process transforms the subtree into a nested sequence with boundary markers, explicitly identifying the type of each syntax node and its corresponding natural language annotation content.

[0035] The formatted sequence described above is also fed into the same pre-trained model as the source code for encoding. In this way, the model can embed the source code and subtree annotations into a unified semantic space.

[0036] To integrate information from multiple abstract annotation subtrees, the system further employs an aggregation module to uniformly model the embeddings of each subtree. Preferably, this aggregation module uses a Transformer architecture to capture the structural relationships and semantic dependencies between subtrees. After aggregation, a global subtree semantic representation vector is obtained. This vector maintains the same dimensionality as the source code representation to facilitate subsequent fusion operations.

[0037] Further, optionally, a semantic method for aggregating annotated abstract subtrees is specifically embodied as follows:

[0038] This method addresses the multiple annotated enhanced subtrees obtained from the decomposition of each input function, assuming their set is .

[0039]

[0040] in, Indicates the first A tree.

[0041] Each sapling After unified encoding, the corresponding embedding vector is generated. Indicate its semantic features;

[0042] Combine the embedding vectors of all subtrees to form the subtree semantic matrix.

[0043]

[0044] To model the structural dependencies between subtrees, a parent-child relationship mask matrix is ​​introduced.

[0045]

[0046] Among them, the subtree Accessible subtree ,but Otherwise .

[0047] Under the constraint of the mask matrix, the attention weights are calculated using a multi-head attention mechanism as follows:

[0048]

[0049] in, , , These are query, key, and value matrices, respectively. These are trainable parameters. The context representations of each subtree obtained after aggregation are then combined again to form the output matrix. ;

[0050] Finally, max pooling is used to extract the global semantic representation:

[0051]

[0052] The vector This is an aggregated representation of the annotated subtree set, which is used by the subsequent feature fusion module.

[0053] Furthermore, after obtaining the representation of the source code and the global abstract subtree representation, feature fusion is performed through a cross-attention mechanism, specifically as follows:

[0054] First, let the source code be represented as... ;in, Indicates the length of the code sequence. For the embedding dimension; the global abstract subtree is represented as This representation is obtained through a semantic aggregator, which condenses the semantic information of all subtrees. To achieve fusion, [the following is used:] As a query vector As keys and values.

[0055] Calculate the projection matrix separately:

[0056]

[0057] in, This is a trainable parameter matrix.

[0058] Then, cross-attention is calculated:

[0059]

[0060] This formula is used to obtain the degree of responsiveness of each location in the source code to the semantics of the subtree;

[0061] Then, a multi-head attention mechanism is used to enhance the fusion capability of different semantic subspaces. The outputs of multiple attention heads are concatenated to obtain the fused representation. ;

[0062] Finally, for Perform average pooling or linear transformation to obtain the final fused feature vector. .

[0063] Furthermore, after completing feature fusion, the vulnerability detection step begins, which is specifically manifested as follows:

[0064] First, fuse the feature vectors The input is fed into a fully connected layer for dimension mapping; the mapped result is then passed through a non-linear activation function, such as... To enhance the nonlinear expressive power of features;

[0065] Subsequently, a random deactivation operation was introduced, proportionally... Randomly masking the output of some neurons reduces the risk of model overfitting.

[0066] Finally, use the Sigmoid function.

[0067]

[0068] Mapping the output to the interval [0,1] yields the probability that the code snippet contains a vulnerability. .

[0069] when (threshold) If the value is 0.5 (or adjusted according to the task), the code is considered to have a vulnerability; otherwise, it is considered to have no vulnerability.

[0070] On the other hand, a vulnerability detection method and system based on abstract syntax tree decomposition and large language model assistance includes:

[0071] Abstract Syntax Tree Decomposition Module: Used to parse the input source code into an Abstract Syntax Tree (AST), based on a preset maximum depth. With the number of nodes The AST is decomposed into a structure to generate multiple semantic subtrees that maintain parent-child relationships and execution order;

[0072] Annotation Abstract Subtree Building Module: Used to call the large language model to generate functional natural language annotations for each semantic subtree, and attach the annotations to the root node of the subtree to form CAS, so as to enrich the semantic information of the subtree;

[0073] Unified encoding module: used to encode the source code and CAS set separately, mapping them to a unified semantic embedding space to ensure that the two types of information can be aligned and fused in the same dimension;

[0074] Semantic aggregation module: Under the constraint of parent-child relationship mask matrix, it performs structure-sensitive aggregation of semantic representations of multiple CAS through multi-head attention mechanism to generate global subtree semantic vectors;

[0075] Feature fusion module: Used to perform cross-attention calculation with global subtree semantic vector as query and source code semantic vector as key and value, to achieve dynamic alignment and complementary fusion of the two types of information, and obtain a unified fused feature representation;

[0076] Vulnerability detection module: It is used to input fused features into the classifier, output vulnerability probability values ​​through non-linear activation and Sigmoid function, and determine whether there are security vulnerabilities in the code based on preset thresholds.

[0077] In another aspect, a computer-readable storage medium stores one or more computer programs, which are invoked by a processor for execution:

[0078] The steps of the above-mentioned vulnerability detection method and system based on abstract syntax tree decomposition and large language model assistance.

[0079] One of the above technical solutions has the following advantages and beneficial effects:

[0080] This method can effectively improve detection accuracy and recall while ensuring the integrity of structural semantics, and is applicable to various code scenarios and pre-trained model frameworks.

[0081] Compared with existing methods, the advantages of the technical solution proposed in this invention are as follows:

[0082] 1. This invention preserves the hierarchical structure and control flow relationship of the code through AST decomposition, and combines it with semantic annotations generated by a large language model to effectively enhance the model's semantic understanding of multiple operators, pointer operations and nested conditions, and significantly reduce missed detections caused by insufficient logical parsing.

[0083] 2. This invention introduces parent-child node masks and a structure-aware Transformer aggregation mechanism on the basis of AST decomposition, enabling the model to accurately capture the vulnerability triggering conditions in nonlinear execution paths such as conditional branches and loops, and reducing misjudgments caused by ignoring the execution order.

[0084] 3. This method achieves significant F1 score improvements on various mainstream pre-trained models and different vulnerability datasets, verifying its generalization and stability in multi-source code environments, and is suitable for vulnerability detection tasks in software projects of different sizes and types. Attached Figure Description

[0085] Figure 1 This is a structural diagram of the abstract syntax tree in this invention;

[0086] Figure 2 This is a flowchart illustrating the method described in an embodiment of the present invention;

[0087] Figure 3 This is a flowchart of the subtree decomposition algorithm described in an embodiment of the present invention;

[0088] Figure 4 This is an example diagram of subtree decomposition as described in an embodiment of the present invention;

[0089] Figure 5 This refers to the large language model prompt word graph used in the annotation generation described in this embodiment of the invention;

[0090] Figure 6 This is a flowchart of the subtree sequence transformation algorithm described in an embodiment of the present invention;

[0091] Figure 7 This is a flowchart illustrating the semantic aggregator described in an embodiment of the present invention;

[0092] Figure 8 This is a schematic diagram of the parent-child matrix described in an embodiment of the present invention;

[0093] Figure 9 These are experimental comparison images of embodiments of the present invention after removing key components;

[0094] Figure 10 This is an experimental comparison diagram of the influence of the coding layer number parameter on the embodiments of the present invention;

[0095] Figure 11 This is an experimental comparison diagram of the influence of the attention head number parameter in the embodiments of the present invention. Detailed Implementation

[0096] The present invention will now be described in further detail with reference to the accompanying drawings. Before describing the system architecture of the solution, the concept of an "abstract syntax tree" will be explained first.

[0097] An Abstract Syntax Tree (ABST) is a tree-like data structure used to represent the syntactic structure of source code. Its nodes represent syntactic elements in the source code (such as expressions, statements, operators, identifiers, etc.), while edges represent the hierarchical and subordinate relationships between these syntactic elements. An ABST looks like... Figure 1 As shown. Compared to the textual representation of source code, an abstract syntax tree (AST) can abstract away specific syntactic details (such as parentheses, delimiters, etc.), presenting the logical structure and execution order of a program in a more intuitive and structured way. In this invention, the AST used must meet the following conditions:

[0098] (1) Structural integrity. It should be able to completely cover all the syntax elements of the source code to be tested, and ensure that any statement, expression or control structure can be mapped to the corresponding node in the AST.

[0099] (2) Semantic fidelity. The parent-child relationship between nodes should accurately reflect the syntactic dependencies and control flow order of the code, ensuring that the semantics of the code are expressed unambiguously in the tree structure.

[0100] (3) Decomposability. The structure of the AST should support decomposition based on a preset depth threshold and a node number threshold, so that complex code structures can be split into multiple subtrees with independent semantics for subsequent semantic enhancement processing.

[0101] (4) Cross-language adaptability. Preferably, it supports parsing multiple programming languages ​​(such as C / C++) and maintains the consistency of structure definition in different language environments to meet the universality requirements of this invention in different software projects.

[0102] Before describing the solution in Example 1, we first introduce the system architecture of the solution. The system architecture diagram of the solution is as follows: Figure 2 As shown, it comprises five main components: an abstract syntax tree decomposition module, a large language model auxiliary annotation abstract subtree construction module, a unified representation module, a feature fusion module, and a vulnerability detection module.

[0103] This embodiment provides one aspect of a vulnerability detection method based on abstract syntax tree decomposition and large language model assistance, including:

[0104] Step 1: Construction and decomposition of the abstract syntax tree; the specific process of construction and decomposition of the abstract syntax tree is as follows:

[0105] Step 1.1: Use a syntax parser to parse the target source code into an abstract syntax tree; the syntax parser is a syntax analysis tool that supports the target programming language (such as C / C++), such as Tree-sitter, to perform lexical analysis and syntax analysis on the source code, and output an abstract syntax tree structure that conforms to the language specification; this structure uses nodes to represent grammatical components and uses hierarchical relationships to describe the grammatical organization of the program, ensuring that the grammatical structure and semantic information can be completely preserved in subsequent processing.

[0106] Step 1.2: Set the maximum subtree depth threshold With the maximum number of nodes threshold The thresholds are set according to the model's processing capacity and code complexity. The maximum depth threshold is used to limit the degree of subtree expansion, and the maximum number of nodes threshold is used to limit the number of syntax nodes contained in a single subtree, so as to prevent excessively deep or large subtrees from causing excessive computational complexity. Preferably, in a specific implementation, it can be Set to 8, Set to 40. This parameter can also be adaptively adjusted through experimental presets or based on the statistical results of the dataset.

[0107] Step 1.3: Traverse the nodes of the abstract syntax tree. If the depth or number of nodes of the subtree corresponding to a node exceeds a preset threshold, then decompose the subtree of that node.

[0108] The subtree decomposition algorithm is as follows: Figure 3 The above;

[0109] Step 1.4: Assign a unique identifier to each decomposed subtree and record the parent-child relationship and the original execution order. Figure 4 This demonstrates the application effect of the AST decomposition algorithm on an example code snippet: (e.g.) Figure 4 As shown in (a), the input code is first parsed into an abstract syntax tree (AST). The tree's nodes contain function definitions, declarations, conditional statements, loop statements, and assignment expressions. The algorithm traverses the entire AST, determining whether subtree splitting is necessary based on the set maximum depth. For structures not exceeding a threshold (such as the if statement in lines 5–6), the entire subtree is retained; for structures exceeding the threshold (such as the for loop in lines 7–8), it is further split into several smaller subtrees.

[0110] In Figure 4(b), the root node of each subtree is marked with a blue box, while the unsplit grammar parts are marked with dashed boxes. Figure 4 (c) shows the order of the decomposed subtrees and the parent-child relationship table. Each subtree is assigned a unique ID and the parent ID is recorded, thus preserving the original syntax hierarchy while maintaining the execution order.

[0111] Step 2: Construction of the annotation abstract subtree; the specific process of constructing the annotation abstract subtree is as follows:

[0112] Step 2.1: Preserve the original identifier information in the subtree; in the subtree structure, do not generalize or replace identifiers such as variable names, function names, and structure names, but retain their original forms completely. This provides stronger semantic references in the subsequent annotation generation process, enabling the large language model to interpret in conjunction with the real context and avoid semantic loss or ambiguity caused by symbolization processing.

[0113] Step 2.2: Based on the preset prompt template, the subtree is input into the large language model to generate functional descriptive annotations; in this step, a unified prompt template is first constructed according to the grammatical structure of the subtree. This template guides the large language model to focus on the control logic and functional semantics of the subtree, rather than being limited to grammatical symbols.

[0114] Subsequently, the node information of the subtree is input into the large language model, and the natural language description corresponding to the function of the subtree is output to ensure the accuracy and consistency of the annotation content.

[0115] Among them, the specific prompt word templates are as follows: Figure 5 As shown. The entire data is saved as a JSON file.

[0116] Step 2.3: Form the annotation abstract subtree:

[0117] The annotation is attached as a special node to the root node of the subtree.

[0118] Step 3: Generate a unified encoding representation;

[0119] Step 3.1: Input the source code into the pre-trained model to obtain the code representation vector;

[0120] Based on its learning ability on large-scale code corpora, the pre-trained model can generate context-sensitive dense vector representations. These representations not only contain syntactic information but also capture implicit semantics such as function calls, logical operations, and variable dependencies, providing high-quality encoded input for subsequent feature fusion.

[0121] Step 3.2: Convert the annotated abstract subtrees into structured sequences and input them into the pre-trained model to obtain subtree representation vectors;

[0122] In this step, the annotation abstract subtree is first serialized, and boundary markers are used to explicitly distinguish different nodes and annotation content, thereby maintaining the hierarchical relationship of the tree structure.

[0123] Next, the sequence is input into a pre-trained model identical to the source code to obtain subtree representation vectors that match the syntactic semantics;

[0124] In this way, the model can learn both the grammatical patterns of the subtrees and the high-level semantic information provided by the annotations.

[0125] The specific serialization processing steps are as follows: Figure 5 :

[0126] Step 4: Semantic aggregation processing;

[0127] The specific process of semantic aggregation is as follows:

[0128] Step 4.1: Construct an attention mask matrix based on parent-child and sibling relationships;

[0129] First, the subtree vectors from step three are reduced in dimensionality using the dimension projection module;

[0130] Next, by establishing a matrix ,in Subtree sapling There must be a father-son or brother relationship; otherwise, it is... .

[0131] Step 4.2: Aggregate subtree semantics based on the attention mask matrix under a multi-head attention mechanism;

[0132] Figure 7 This provides a complete demonstration of the workflow of our semantic aggregator;

[0133] The semantic aggregator contains four layers of Transformer encoders.

[0134] Using a mask matrix to constrain multi-head attention computation, subtree representation Interaction and information dissemination occur within the scope of structural relevance.

[0135] Step 4.3: Pool the aggregated subtree vectors to obtain the global subtree representation.

[0136] The specific mask matrix representation is as follows: Figure 8 As shown;

[0137] The global representation is obtained through max pooling. This is used to characterize the semantic features of the entire subtree set.

[0138] Step 5: Feature fusion and alignment;

[0139] The specific process of feature fusion and alignment is as follows:

[0140] Step 5.1: Perform cross-attention calculation using the global subtree representation as the query vector and the source code representation as the key-value vector;

[0141] Let the global subtree be represented as The source code is represented as Then calculate:

[0142] .

[0143] Step 5.2: Concatenate and pool the results of the multi-head cross attention to obtain a unified feature representation vector.

[0144] Based on the formula:

[0145]

[0146] The fusion representation is obtained Then, a unified feature vector is obtained through pooling. .

[0147] Step 6: Vulnerability prediction output;

[0148] The specific process of the vulnerability prediction output is as follows:

[0149] Step 6.1: Input the unified feature representation vector into the fully connected classification layer;

[0150] Let the unified feature be represented as Then calculate the result of the linear transformation. .

[0151] Step 6.2: Perform nonlinear activation and regularization on the output of the classification layer;

[0152] pass Activate and combine with Dropout for regularization to obtain intermediate results. .

[0153] Step 6.3: Output the probability value of the existence of the vulnerability, and generate a binary classification prediction result based on the preset threshold.

[0154] Calculate probability using the Sigmoid function ,like (threshold) If the value is usually 0.5, the output will be "Vulnerability exists"; otherwise, the output will be "No vulnerability".

[0155] Example 2

[0156] This invention provides a code vulnerability detection method based on abstract syntax tree decomposition and comment fusion, including:

[0157] By applying steps 1-6 above, a unified representation vector integrating source code semantic information and annotation abstract subtree information is obtained, and vulnerability detection is performed on the target source code based on a classifier. This method enables more accurate identification and classification of potential vulnerabilities in complex logic code fragments and multi-branch execution paths.

[0158] It should be noted that the detection method provided in this embodiment utilizes the results of generating CAS and then using a pre-trained model for assistance to predict vulnerabilities in the source code to be detected. It should be understood that this embodiment does not limit the specific pre-trained model or classification network structure; any model or equivalent structure capable of implementing the vulnerability detection function can adopt the technical approach of this invention and fall within the protection scope of this invention.

[0159] Example 3

[0160] This invention provides a code vulnerability detection system based on abstract syntax tree decomposition and comment fusion, comprising:

[0161] Abstract Syntax Tree Decomposition Module: Used to parse the input source code into AST, and perform structured decomposition of AST based on preset maximum depth threshold and node number threshold to obtain multiple semantic subtrees that maintain parent-child relationships and execution order;

[0162] Annotation Abstract Subtree Building Module: Used to call the large language model to generate functional natural language annotations for each semantic subtree, and attach the annotations as special nodes to the root node of the subtree to form CAS, so as to enhance the semantic expressiveness of the subtree;

[0163] Unified encoding module: used to encode the source code and CAS set respectively to obtain two types of semantic embedding representations, and map them to a unified embedding space to ensure the feasibility of subsequent feature alignment and fusion;

[0164] Semantic aggregation module: Under the constraints of parent-child and sibling relationships, it constructs an attention mask matrix and aggregates the semantic vectors of each annotation abstract subtree based on a multi-head attention mechanism to obtain a global subtree semantic representation;

[0165] Feature fusion module: Used to perform cross-attention calculation with the global subtree semantic vector as the query vector and the source code semantic vector as the key and value, to achieve dynamic alignment and complementary fusion of the two types of information, and obtain a unified fused feature representation;

[0166] Vulnerability detection module: It is used to input the fused feature representation into the classifier, output the probability value of the existence of the vulnerability after nonlinear activation and sigmoid function, and generate the detection result based on the preset threshold, thereby determining whether there is a security vulnerability in the target source code.

[0167] The specific implementation process of each module can be referred to the steps in the method section above, and will not be repeated here. It should be understood that the above module division is only a logical functional division, and different division methods can be used in actual implementation. For example, multiple modules can be combined or integrated into another subsystem, or some functions can be omitted or replaced. At the same time, the above modules can be implemented by hardware or by software functional units, both of which fall within the protection scope of this invention.

[0168] Example 4

[0169] The embodiments of the technical solution of the present invention also provide a computer-readable storage medium, characterized in that:

[0170] The system contains a computer program that, when invoked by a processor, implements the steps of the aforementioned code vulnerability detection method based on abstract syntax tree decomposition and annotation fusion, or the functions of the aforementioned code vulnerability detection system based on abstract syntax tree decomposition and annotation fusion.

[0171] Example 5

[0172] The embodiments of the present invention also provide a computer device, comprising at least:

[0173] One or more processors;

[0174] A memory that stores one or more computer programs;

[0175] The processor invokes the computer program to achieve the following:

[0176] The steps of the above-mentioned code vulnerability detection method based on abstract syntax tree decomposition and annotation fusion, or the functions of the above-mentioned code vulnerability detection system based on abstract syntax tree decomposition and annotation fusion.

[0177] Specifically, when the processor calls the computer program to implement the above-mentioned code vulnerability detection method based on abstract syntax tree decomposition and comment fusion, the following steps are executed:

[0178] Step 1: Parse the input source code into an abstract syntax tree, and decompose it based on the preset maximum depth and number of nodes to generate multiple semantic subtrees that maintain parent-child relationships and execution order;

[0179] Step 2: For the semantic subtree, call the large language model to generate functional natural language annotations, and attach the annotations to the root node of the subtree to form an annotation abstract subtree;

[0180] Step 3: Encode the source code and comment abstract subtrees separately, and map them to a unified semantic embedding space to obtain code representation vectors and subtree representation vectors;

[0181] Step 4: Construct attention mask matrices for parent-child and sibling relationships, and aggregate the semantics of each subtree based on a multi-head attention mechanism to generate a global subtree semantic representation;

[0182] Step 5: Using the global subtree semantic representation as the query vector and the source code semantic vector as the key and value, perform cross-attention calculation to obtain a unified fusion feature representation;

[0183] Step 6: Input the fused feature representation into the classifier, pass it through non-linear activation and the Sigmoid function to output the vulnerability probability value, and determine whether there is a vulnerability in the source code based on a preset threshold.

[0184] When the processor calls the computer program to implement the above-mentioned code vulnerability detection system based on abstract syntax tree decomposition and comment fusion, it specifically executes the following:

[0185] Based on the functions of the aforementioned modular design of the abstract syntax tree decomposition module, annotation abstract subtree construction module, unified encoding module, semantic aggregation module, feature fusion module, and vulnerability detection module, the source code is parsed, represented, fused, and vulnerability predicted, thereby achieving automated vulnerability detection of the target code.

[0186] Example 6

[0187] The present invention also provides a computer-readable storage medium storing a computer program, which is invoked by a processor to implement the following:

[0188] The steps of the above-mentioned code vulnerability detection method based on abstract syntax tree decomposition and annotation fusion, or the functions of the above-mentioned code vulnerability detection system based on abstract syntax tree decomposition and annotation fusion.

[0189] Specifically, when the processor calls the computer program to implement the above-mentioned code vulnerability detection method based on abstract syntax tree decomposition and comment fusion, the following steps are executed:

[0190] Step 1: Parse the input source code into an abstract syntax tree, and decompose it based on the preset maximum depth and number of nodes to generate multiple semantic subtrees that maintain parent-child relationships and execution order;

[0191] Step 2: For the semantic subtree, call the large language model to generate functional natural language annotations, and attach the annotations to the root node of the subtree to form an annotation abstract subtree;

[0192] Step 3: Encode the source code and comment abstract subtrees separately, and map them to a unified semantic embedding space to obtain code representation vectors and subtree representation vectors;

[0193] Step 4: Construct attention mask matrices for parent-child and sibling relationships, and aggregate the semantics of each subtree based on a multi-head attention mechanism to generate a global subtree semantic representation;

[0194] Step 5: Using the global subtree semantic representation as the query vector and the source code semantic vector as the key and value, perform cross-attention calculation to obtain a unified fusion feature representation;

[0195] Step 6: Input the fused feature representation into the classifier, pass it through non-linear activation and the Sigmoid function to output the vulnerability probability value, and determine whether there is a vulnerability in the source code based on a preset threshold.

[0196] When the corresponding computer program is invoked by the processor to perform the functions of the vulnerability detection system based on abstract syntax tree decomposition and annotation fusion, the specific execution is as follows: according to steps 1-6 above, a unified representation of the fused source code and annotation semantics is generated, and vulnerability prediction and detection are achieved based on the classification results, thereby improving the accuracy of vulnerability identification under complex logic. For the specific implementation process of each step, please refer to the description in the foregoing method embodiments.

[0197] The readable storage medium is a computer-readable storage medium, which can be an internal storage unit of the device described in any of the foregoing embodiments, such as a hard disk or memory; or it can be an external storage device, such as a plug-in hard disk, a smart media card (SMC), a secure digital card (SD), a flash card, etc. Further, the computer-readable storage medium can include both internal storage units and external storage devices. The medium is used to store the computer program and other programs and data required for system operation, and can also be used to temporarily store data that has been output or is to be output.

[0198] Experiment and Results Analysis

[0199] To verify the effectiveness of the VulTrLM vulnerability detection method proposed in this invention and its adaptability under different conditions, a systematic experimental study was conducted in the embodiments of this invention. The purpose of the experiment is to illustrate the following points: (1) the adaptability of the method proposed in this invention to different pre-trained models; (2) the impact of different hyperparameter configurations on model performance; and (3) to demonstrate the contribution of each component module to the overall performance through ablation experiments.

[0200] In the experiments, the datasets used in this invention are derived from publicly available software vulnerability-related data, including but not limited to benchmark datasets such as Devign, Reveal, and SVulD, which are widely used in vulnerability detection tasks. These datasets cover a variety of mainstream programming languages ​​and diverse vulnerability types, ensuring the generality and objectivity of the experimental conclusions. To ensure the reproducibility and objectivity of the experiments, the hardware environment used in this invention includes: an Intel Xeon Silver4214R CPU, 16GB RAM, an NVIDIA RTX 3090 GPU, an Ubuntu 20.04 LTS operating system, and the use of the PyTorch 2.2.0 deep learning framework for fine-tuning the pre-trained model and generating language annotations.

[0201] (1) Model adaptability

[0202] To further verify the applicability and effectiveness of the VulTrLM vulnerability detection method of this invention in different pre-trained models, the embodiments of this invention compare and examine the performance of various pre-trained models as the basic backbone network. Specifically, this invention replaces the original UniXcoder with CodeBERT and CodeT5, and performs a horizontal comparison of experimental results on the same dataset to verify the universality and robustness of the method of this invention.

[0203] Figure 9 Experimental results show that regardless of the pre-trained model used, the performance of all models is significantly improved when combined with the fusion strategy of "abstract syntax tree substructure and natural language annotation" proposed in this invention. On three different datasets, CodeBERT, CodeT5, and UniXcoder achieved average F1 score improvements of 2.35%, 7.47%, and 6.98%, respectively. UniXcoder, in particular, performed best when combined with CAS, which is highly compatible with its pre-trained cross-modal alignment capability (covering AST information) and can better synergize with the subtree decomposition strategy proposed in this invention.

[0204] Even so, experiments show that CodeBERT and CodeT5, after introducing CAS, have outperformed most existing methods in overall detection performance. This fully demonstrates that the substructure and annotation combination proposed in this invention not only captures potential code semantic relationships, but also unifies implicit semantics with explicit logic, significantly improving the model's ability to infer vulnerabilities in complex code fragments.

[0205] (2) The influence of parameters on the model

[0206] To further verify the stability and effectiveness of the VulTrLM vulnerability detection method of this invention under different parameter configurations, the embodiments of this invention systematically examined the impact of two types of core hyperparameters on model performance: (1) the number of Transformer encoding layers; and (2) the number of attention heads. Through comparative experiments on three different vulnerability detection datasets, this invention can reveal the modulating effect of hyperparameter selection on model performance.

[0207] Appendix Figure 10 and attached Figure 11 The impact of relevant parameters is illustrated. Circles represent accuracy, crosses represent precision, triangles represent recall, and rectangles represent F1 scores.

[0208] Figure 10 This invention first examines the impact of varying the number of encoding layers on model performance. Experimental results show that as the number of encoding layers gradually increases, model performance exhibits a "first increase, then decrease" trend. With four encoding layers, the proposed method achieves the best F1 scores of 67.62%, 53.01%, and 29.78% on the FFmpeg+Qemu, Reveal, and SVulD datasets, respectively. This phenomenon indicates that moderately increasing network depth helps improve the aggregation ability of semantic features, but excessive layers can lead to overfitting, thus weakening the detection performance.

[0209] Figure 11 The impact of the number of attention heads was demonstrated, and further, this invention examined the effect of the number of attention heads on model performance. During the experiments, the number of attention heads was gradually increased from 2 to 32. The results show that when the number of attention heads was set to 4, the proposed method achieved the best F1 score on all datasets, and this trend remained consistent across different datasets. As the number of attention heads continued to increase, the model performance actually showed a downward trend. This result indicates that a moderate multi-head attention mechanism can effectively improve the model's feature capture ability, but too many attention heads can introduce redundant information, thus affecting overall performance.

[0210] Therefore, it can be seen that the VulTrLM proposed in this invention has a significant impact on model performance in terms of hyperparameter configuration.

[0211] Experimental results confirm that the method of this invention achieves optimal performance when the number of Transformer layers and attention heads is 4. This conclusion provides clear guidance for parameter settings in practical deployments, thus ensuring the applicability and stability of the method.

[0212] (3) Comparison of ablation experiments

[0213] To further verify the independent contributions of the Annotated Abstract Subtree (CAS) and Parent-Child Mask (PCM) proposed in this invention to the overall framework, an ablation experiment was designed in this embodiment to compare and examine the performance changes after removing the relevant modules. This experiment first replaces the CAS with a complete abstract syntax tree to evaluate the actual contribution of the CAS. Experimental results show that this replacement operation leads to a significant performance decrease:

[0214] The F1 score decreased from 67.36% to 64.28%, precision decreased from 66.28% to 63.84%, and recall decreased from 68.49% to 65.37%. These results clearly demonstrate that CAS plays a crucial role in enhancing the model's understanding of source code semantics, and that subtree decomposition has significant advantages over directly using the complete AST.

[0215] Furthermore, this invention removes all hierarchical relationships between subtrees and parent nodes, resulting in complete semantic isolation between subtrees. Experimental results show that the F1 score decreased from 67.36% to 64.58%, the accuracy slightly decreased from 65.58% to 65.32%, while the recall significantly decreased from 68.49% to 61.56%. This result fully demonstrates that PCM plays a crucial role in semantic feature aggregation. By guiding the attention mechanism to focus on structurally relevant subtrees, it effectively maintains the hierarchical structure, thereby ensuring the correct understanding of the execution sequence.

[0216] In summary, both the CAS and PCM mechanisms proposed in this invention play irreplaceable roles in improving vulnerability detection performance. CAS significantly improves the accuracy of code semantic understanding, while PCM effectively maintains the contextual relationships and execution logic between subtrees. The synergistic effect of the two significantly enhances the model's ability to identify complex vulnerability patterns.

[0217] As shown above, existing vulnerability detection methods based on pre-trained models suffer from insufficient semantic understanding and low detection accuracy when handling complex logical structures and multi-branch execution paths. This invention proposes a vulnerability detection method based on abstract syntax tree decomposition and large language model annotation enhancement. Under conditions of non-uniform distribution and diverse vulnerability types, this method introduces annotated abstract subtree structures and utilizes semantic aggregation mechanisms to effectively model the logic and execution order of the source code, thereby improving the model's ability to express the semantics of complex code. Experiments show that this invention achieves significantly better detection results than existing methods on multiple publicly available vulnerability datasets, effectively enhancing the model's adaptability and robustness, and reducing the risk of performance degradation due to model limitations. Furthermore, this invention possesses good versatility, is applicable to various pre-trained model architectures, and balances scalability and stability while maintaining high detection accuracy.

[0218] Based on the above understanding, the technical solution of this invention, or the contribution made to the prior art, can essentially be embodied in the form of a software product. The software product of this invention can be stored in a computer-readable storage medium containing several instructions to cause a computer device (which may be a personal computer, server, network security detection device, or other terminal with computing capabilities) to execute all or part of the steps described in the various embodiments of this invention, thereby realizing the vulnerability detection method based on VulTrLM proposed in this invention.

[0219] The aforementioned computer-readable storage media may include, but are not limited to: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, or other media capable of storing program code.

[0220] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that various modifications or equivalent substitutions can be made to the specific embodiments of the present invention without departing from the spirit and scope of protection of the present invention. These modifications or equivalent substitutions should all fall within the scope of protection of the present invention and are defined by the claims of the present invention.

Claims

1. A vulnerability detection method based on Abstract Syntax Tree (AST) decomposition and large language model assistance, characterized in that, Includes the following steps: Step 1: Parse the abstract syntax tree; The code snippet to be tested is subjected to syntax analysis, and a corresponding abstract syntax tree is constructed using a parsing tool. In order to obtain a structured representation of the code; Step 2: Subtree decomposition; An improved decomposition algorithm is used to divide the abstract syntax tree into multiple subtrees to refine the expression of different code logic units, based on a set depth threshold. and node number threshold ,Will Decompose into a set of subtrees that satisfy the conditions. ; Step 4: Generate natural language annotations; Use a large language model for each subtree Generate semantic annotations This, combined with other elements, forms Commented Abstract Subtrees (CAS). ; Step 5: Unify the encoding of source code and subtree structure; Use a pre-trained model on the source code With all Embedding is performed to generate vector representations. and ; Step 6: CAS semantic relationship modeling; CAS vector representation models the relationships between annotated abstract subtrees and extracts semantic features using a Transformer-based semantic aggregator. Among them, constructing a structure-aware mask matrix This is used to indicate the parent-child, sibling, and self-relationships among the CASes; then, a masked Transformer semantic aggregator is used to further... Aggregation is performed to obtain the global CAS vector representation. ; Step 7: Feature fusion; right and Perform cross-attention fusion to obtain the fused representation. ; Step 8: Vulnerability Vector Construction and Detection; Based on fusion representation Identifying vulnerability patterns in code enables vulnerability detection; among which, Input a multilayer perceptron and output a predicted probability of whether a vulnerability exists. .

2. The method according to claim 1, characterized in that, The decomposition process of the AST involves multi-level recursive analysis of the source code structure. Its core lies in automatically extracting several semantic subtrees of appropriate granularity from the AST structure, as follows: First, a depth-first traversal strategy is used to perform a complete traversal of the abstract syntax tree, identifying all possible structural nodes, including but not limited to function definitions (FuncDef), conditional statements (If), loop statements (For, While, DoWhile), and exception handling statements (Try). For each candidate node, the maximum depth of its corresponding subtree is calculated. and number of nodes and with the preset depth threshold and node number threshold Compare, if satisfied If so, then the node is taken as a valid decomposition point and extracted as a subtree; Secondly, to preserve control dependency information and contextual semantics in the program, priority is assigned to the control flow structure during node selection, prioritizing the extraction of subtrees containing complete branch logic. For example, if-else structures, nested judgments, and multi-level loops are preferentially retained as complete subtrees rather than being broken down, thereby maximizing the preservation of code execution paths and logical integrity. Furthermore, for subtrees containing too many statements or with excessively deep nesting levels, if their structural complexity exceeds a threshold limit, the same judgment and decomposition process is recursively performed on their child nodes to improve embeddability. The value is 8. The value is 40, and the maximum number of subtrees obtained by single function decomposition is 64. Any excess is truncated in the order of appearance. Finally, in order to reconstruct the hierarchical relationship of the original syntax tree in the subsequent semantic modeling stage, each subtree is assigned a globally unique ID during the decomposition process, and its corresponding direct parent node identifier is recorded. , constitute a set of relations This set of relationships allows for the recovery of structural dependency paths between subtrees during the aggregation phase, thereby ensuring the structural consistency and interpretability of attention computation in the semantic aggregator.

3. The method according to claim 1 or 2, characterized in that, The construction process of the annotation abstract subtree is as follows: A large language model is used to generate functional-level semantic descriptions for each decomposed AST subtree, thereby enhancing the semantic expressiveness of the subtrees and assisting downstream representation learning. Specifically, each subtree All of these are fed into the large language model as input structures. The model uses conversational prompts to generate semantic annotations. The prompts are constructed in English format and include two parts: system role settings and user task requests. The large language model generates natural language annotation text based on syntactic features such as subtree control structure, function calls, and variable references. The annotation is formatted as an annotation node and attached to the root node of the atomic tree, thus forming a semantically enhanced annotation abstract subtree, i.e. ; To improve the quality of annotation generation, it is suggested that the temperature parameter be set to 0 during the build process to ensure that the generated content is deterministic and consistent. The annotation content is post-processed to remove redundant symbols and special markers, while retaining the semantic core of the natural language description. Through this mechanism, the system externalizes the semantic information that is difficult to capture directly in the AST structure, thereby enhancing the semantic recognition capability and cross-modal alignment performance of the substructure.

4. The method according to claim 1, characterized in that, The serialization of each CAS to adapt the pre-trained model is as follows: Each annotation abstract subtree The input is serialized into an embedded sequence according to the following rules; Add boundary markers to the comment content<comment, left> and<comment, right> ; Add formatting tags to non-leaf nodes<NodeType, left> and<NodeType, right> In order to preserve its hierarchical structure; The traversal order is a preorder recursive traversal, generating a linear representation. Input the pre-trained model.

5. The method according to claim 1, characterized in that, The Transformer semantic aggregator is as follows: The constructed m×m mask matrix M, where m is the number of the current function subtrees, and the matrix elements M ij Values ​​should be assigned according to the following rules: - If i = j, then M ij =1, meaning self-attention; - If subtree j is either the parent node or a child node of subtree i, then M ij =1; - If subtree i and subtree j share a common parent node, then M ij =1; - Other cases M ij =−∞, used to filter out irrelevant attention channels.

6. The method according to claim 1, characterized in that, The network architecture of the semantic aggregator is as follows: The Transformer aggregator employs... The layer stacking structure (preferably 4 layers) includes a multi-head attention mechanism, residual connections, layer normalization, and a feedforward network in each layer, ultimately outputting an aggregated vector. Where max represents the max pooling operation along the feature dimension; the final stage of semantic aggregation outputs a global subtree embedding vector H with dimension 128 after max pooling. t .

7. The method according to claim 1, characterized in that, The and The feature fusion process is as follows: The fusion step is implemented using the following cross-attention mechanism: in, For a trainable parameter matrix, For the embedding dimension, the Attention outputs are concatenated and then average pooled to generate the final fused vector. .

8. The method according to claim 1, characterized in that, The classifier is used to classify the fused semantic representation vector. The structure for determining whether a security vulnerability exists in source code includes the following components: First, the input layer receives the fusion vector generated by the cross-attention mechanism. This vector contains high-order features of source code semantic embedding and annotation abstract subtree aggregation semantics, representing a comprehensive representation of the source code fragment in both structural and semantic dimensions; Secondly, The input is fed into a set of fully connected linear transformation layers, mapped to the hidden space, and the model's discriminative ability is improved through dual nonlinear transformations. This process includes the following steps: Step 1 Linear mapping: in, For trainable weight matrix, For bias terms; Step 2: Activation Function and Dropout Handling: Where tanh is the hyperbolic tangent function, used to enhance nonlinear fitting capabilities, and Dropout sets the probability. (Preferably 0.1) Randomly mask the output of some neurons to avoid overfitting; Step 3: The final output layer is normalized using the Sigmoid activation function to generate a scalar output representing the vulnerability probability. The calculation formula is as follows: in, The Sigmoid function is used to map prediction results to a probability space. Finally, the output value This indicates the possibility of a security vulnerability in the input source code, where This indicates a high probability of a vulnerability. This indicates that there are no vulnerabilities; the above structure can be updated with gradients during training using the backpropagation algorithm to minimize the cross-entropy loss function and improve discrimination performance.

9. The method or system according to claim 1, characterized in that: It is applicable to a variety of programming languages, including but not limited to C, C++, Java, and Python, and can be deployed in code auditing platforms, static analysis tools, or CI / CD integrated environments to assist in automated vulnerability detection and security auditing.

10. A computer device, characterized in that, At least including: One or more processors; A memory, wherein a set of executable computer program instructions is stored; The processor invokes the instructions to perform the method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The medium stores a computer program that, when executed by one or more processors, causes the processors to perform the method as described in any one of claims 1 to 8.

Images