Program defect detection method based on semantic structure double-flow collaboration and anti-noise fusion

Abstract

The invention is suitable for the technical field of software defect detection and vulnerability detection, and provides a program defect detection method based on semantic structure double-flow collaboration and anti-noise fusion. According to the method, a logical reasoning path (data flow, control dependency and key semantics) containing a data flow chain, a control dependency relationship and key variable semantics is generated, key logic features related to vulnerabilities are quantitatively extracted through a hierarchical bottleneck fusion mechanism, cross-flow information alignment and aggregation are achieved, and therefore the detection capacity for complex logic defects is improved. The problems that an existing deep learning model only pays attention to a code lexical structure and lacks deep logical reasoning ability, and a traditional structure chart is high in construction cost, complex in engineering, prone to structural information redundancy and difficult to reproduce stably within limited time are solved.

Description

Technical Field

[0001] This invention belongs to the field of software defect detection and vulnerability detection technology, and particularly relates to a program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion. Background Technology

[0002] Software defect detection and vulnerability detection are important applications in software engineering and cybersecurity. Their goal is to predict the defect category of a code segment based on its content. The existence of software defects or vulnerabilities can not only lead to huge economic losses but also potentially trigger catastrophic security incidents. Therefore, how to automatically and accurately identify potential defective modules in the early stages of the software development lifecycle—that is, software defect detection—has become a critical issue that urgently needs to be addressed in the field of software engineering.

[0003] Deep learning-based defect detection in source code is an important and efficient technical approach. Source code inherently possesses a sequential structure, so research often employs pre-trained language models to encode code sequences and uses classifiers to predict defects. Other studies represent code as abstract syntax trees, control flow / data flow graphs, or code attribute graphs, combining these with graph neural networks to transmit information between nodes and capture long-distance dependencies. However, existing research indicates that using only code sequences as input may still struggle to stably understand the semantics of statements containing complex logic, pointer operations, and multiple operators, and it may also fail to fully capture the execution sequence and structural information of the code. While graph-based methods enhance structure awareness, the complex graph construction process often incurs significant computational and time overhead, and graph neural networks are prone to "oversmoothing" during deep propagation, leading to convergent node representations and difficulty in effectively capturing fine-grained deep semantics, thus affecting defect detection performance. Therefore, efforts have begun to explore introducing auxiliary information beyond the source code to compensate for the insufficient modeling of execution semantics and structural information by pre-trained models.

[0004] As large language models (MLMs) improve their code understanding capabilities, researchers are exploring ways to leverage external semantic information generated by MLMs to enhance vulnerability detection. This includes generating annotations and summaries and fusing them with source code features to improve the model's ability to capture vulnerability patterns. Other studies have designed specialized fusion mechanisms that allow natural language interpretations to participate in attention calculations and form complementary semantics with code representations. However, related research has also observed that when MLMs are used directly for zero-shot vulnerability assessment, the output may be unstable, tending to classify most samples as vulnerable. Furthermore, the free text output of MLMs may contain redundant or difficult-to-normalize content, leading to noise amplification and inconsistent distribution in downstream fusion models. Therefore, how to stably leverage the capabilities of large language models within engineering-feasible costs and reproducible evaluation conditions is crucial for further research and engineering implementation in this area.

[0005] Due to the outstanding performance of Large Language Models (LLMs) in the field of code understanding, some scholars have begun to try to introduce the semantic information generated by them into vulnerability detection tasks. However, research on the deep integration of code logic and LLM-generated content is still in its early stages. Wen et al. proposed a "structured natural language annotation tree" by attaching annotations generated by the large model to the abstract syntax tree nodes of the code, attempting to integrate annotation semantics and original code information. However, this approach failed to utilize substantial logical structure information and lacked noise filtering and feature compression mechanisms.

[0006] Based on existing technologies, the main shortcomings of current research include:

[0007] Directly using large language models to identify vulnerabilities is costly and has insufficient output stability. In recent years, more and more methods have guided large language models to give vulnerability classification conclusions directly during the inference stage through hint engineering, retrieval enhancement, instruction fine-tuning or multi-agent inference. However, the output of this approach is sensitive to hint templates, sample selection and retrieval content, and is prone to result fluctuations and high false positives, which bring consistency and controllability risks to engineering implementation.

[0008] Acquiring structural information is costly and difficult to reuse stably in large-scale and multi-environment scenarios. Existing methods rely on tools such as Joern to construct structural views such as CPG / CFG / PDG and use graph neural networks to propagate information between nodes. Although structural dependencies can be explicitly modeled, they usually suffer from problems such as heavy engineering dependencies, time-consuming graph construction and cleaning, large graph size and many irrelevant nodes. In scenarios where the dataset samples are mostly function fragments, insufficient context may also lead to incomplete structural views, which further increases the preprocessing cost and affects the stability of cross-project migration.

[0009] Graph structures and graph neural network approaches are highly sensitive to graph construction toolchains and training randomness, resulting in unstable performance, large score fluctuations, and difficulty in reproducing experiments. Under the conditions of function-level samples, class imbalance, and sparse effective logical evidence, the training process of GNNs is more sensitive to factors such as random initialization, sampling order, and batch composition. Significant performance fluctuations or training instability are likely to occur under different random seeds, which makes it difficult for subsequent researchers to stably reproduce the metrics reported in the literature under different environments.

[0010] Cross-modal interaction mechanisms are mostly one-way or shallow fusion, lacking the compression and screening of key logic. Existing methods often complete the fusion at a single-layer cross-attention or simple splicing layer, making it difficult to retain only a small number of logical clues most relevant to vulnerability identification in scenarios with long code and sparse logical relationships, thus resulting in limited or even degraded benefits after fusion. Summary of the Invention

[0011] The purpose of this invention is to provide a program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion, aiming to solve the problems mentioned in the background art.

[0012] To achieve the above-mentioned objectives, the present invention provides the following technical solution:

[0013] A program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion includes the following steps:

[0014] (1) Data processing is performed using a dual-stream input construction method consisting of a source code sequence and a symbolic logic description.

[0015] (2) The source code sequence is used as the first input stream and the symbolic logic description is used as the second input stream to form a sample-level dual-stream input pair. The pre-trained language model is used to encode the source code stream and the structure encoding stream respectively to obtain two context representation sequences.

[0016] (3) Based on the hierarchical bottleneck fusion module, feature interaction is performed. The bottleneck query sequence is initialized with the hidden state of the source code sequence, and the information bottleneck is realized by truncating and compressing layer by layer. Information is aggregated from the source code stream and the structure supplement stream alternately to obtain the fused global representation vector.

[0017] (4) Based on the fused global representation vector, input the linear classifier to perform binary classification output and obtain the prediction result of whether the code has defects.

[0018] Compared with the prior art, the specific beneficial effects of the present invention are as follows:

[0019] (1) Good versatility: For the binary classification scenario of function-level program defects, it adopts a dual-stream input form of source code + structured logic description. Without changing the original code text, it makes the execution logic clues in the code that are difficult to be stably captured by the sequence model alone explicit, thus better adapting to the representation difficulties brought about by complex logic in actual vulnerability detection;

[0020] (2) Structural evidence can be obtained without relying on explicit graph construction: Existing graph methods usually rely on toolchains to build CPG / CFG and propagate information with GNN, which is heavily dependent on engineering and the results are easily affected by the quality of graph construction. This invention positions the large language model as a "structural element extractor", only outputs verifiable structural facts, and ensures the stability of structural flow through identifier consistency verification and unified fallback mechanism, so as to provide logical clues related to defect identification for the model without building a graph;

[0021] (3) Good noise resistance and robustness: The layered bottleneck fusion mechanism realizes the cross-flow aggregation of a small amount of key information, suppresses redundancy and strengthens complementarity. Under the dual-flow setting of source code flow and structural description flow, if only simple splicing or single cross attention is used, two problems are likely to occur: First, the structural description flow has a large amount of information and contains redundancy, and direct fusion will dilute the key logic; Second, there are differences between the two representation spaces, and insufficient fusion will make it difficult to effectively utilize the structural information. This invention adopts layered bottleneck fusion for cross-flow interaction: the hidden state of the source code sequence is used as the initial query. By compressing the query length layer by layer and interacting with the two features alternately, the progressive bidirectional filtering and aggregation of structural supplementary information is realized. Thus, under the premise of controllable computation, the bidirectional fusion of the two branches is promoted, the part of the structural flow that contributes to the vulnerability judgment is strengthened, and the interference caused by irrelevant content is suppressed.

[0022] (4) More robust training: The introduction of structure fitting auxiliary loss promotes the absorption of structure supplementation and automatically reduces the weight of low-quality supplementation. When relying solely on classification loss, the model may treat the structure flow as noise or learn unstable fusion patterns in the early stages of training, resulting in limited or even fluctuating fusion benefits. To this end, this invention introduces structure fitting auxiliary loss in addition to the main classification objective: the global representation of the structure branch is predicted by the source code branch representation, and the loss is automatically set to zero through a masking mechanism when the structure description is used as a fallback. This promotes the consistent absorption of structural semantics by the source code representation on the one hand, and avoids negative transfer of training caused by low-quality or missing structure supplementation on the other hand, making the overall training process more stable. Moreover, when the structure supplementation is unreliable, it can still fall back to rely on the source code branch to complete the discrimination.

[0023] (5) Low engineering cost and strong reproducibility: Compared with the scheme that requires complex graph construction and graph neural network training, the present invention uses a unified text sequence to carry structural information and uses an end-to-end trainable dual-stream coding and fusion framework to complete defect prediction, reducing external dependence and preprocessing complexity; at the same time, it avoids performance fluctuations caused by unstable graph construction, which is more conducive to stable reproduction and deployment in different environments. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the architecture of a parallel dual-stream encoder provided in an embodiment of the present invention;

[0025] Figure 2 This is a schematic diagram of cross-modal fusion of the hierarchical bottleneck fusion module provided in an embodiment of the present invention. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0027] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0028] Example 1: A structural diagram of a program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion, including the following steps:

[0029] (1) Data representation and augmentation:

[0030] To overcome the limitations of relying solely on source code text sequences, this invention constructs a dual-modal data representation that includes surface-level code and deep structure, capturing potential defect features of the program from different dimensions:

[0031] Source code preprocessing and serialization: For the original C language function code snippets, standard preprocessing is first performed, including removing irrelevant comments and standardizing formatting. Then, a pre-trained model's tokenizer converts the code into a TokenID sequence. This part mainly carries the program's lexical information, syntactic structure, and local contextual semantics.

[0032] Structured Knowledge Extraction and Filtering: Utilizing a large language model as a knowledge extractor, a structured logical description is generated for each piece of code and output strictly in JSON format. To ensure the accuracy and validity of the information, the following two key steps are included:

[0033] 1) Multidimensional Structure Extraction: The LLM instruction extracts information from three key dimensions:

[0034] Data flow chain: Explicitly describes the propagation path of key variables and captures potential data contamination;

[0035] Control Dependency: Extract the conditional logic of key control operations to detect defects such as missing boundary checks;

[0036] Key semantics: Semantic annotation of ambiguously named variables in the code;

[0037] 2) Literal Binding Filtering: To address the potential "illusion" phenomenon in large models—generating variables that do not exist in the code—this embodiment of the invention designs an identifier consistency verification mechanism. First, it extracts real identifiers from the source code to construct a set; then, it extracts the identifier set from the structure description generated by the LLM for literal matching; if the structure description does not contain a resolvable identifier, or its identifier set has no intersection with the source code identifier set, then the structure description is determined to be unverifiable, and the structure field of the sample is uniformly reverted to a preset catch-all text to ensure the stability and consistency of the data distribution of the auxiliary modality;

[0038] To adapt to the input format of the Transformer model, the cleaned structured data no longer retains the original JSON hierarchy. Instead, it is transformed into a compact natural language sequence through a template-based linearization strategy. Specifically, the retained entries are concatenated by category and separated by semicolons to form a text stream such as "Data Flow: A->B; C->D. ControlDependencies: X controls Y." For samples from which no effective structure can be extracted, pre-defined catch-all text is filled in to maintain the integrity of the modal input. Using a tokenizer that is the same as that of the source code encoder, the linearized structured text is mapped to a token ID sequence. This step ensures the isomorphism and semantic alignment of the structure stream and the code stream in the numerical space.

[0039] (2) Based on the Transformer architecture, this embodiment of the invention designs a parameter-sharing parallel dual-stream encoder to process the data input of the two modalities mentioned above, such as... Figure 1 As shown, this embodiment of the invention utilizes a pre-trained language model as a universal encoder for source code and structural coding streams. The reason for choosing this type of model is that it has been pre-trained on massive code corpora and possesses powerful cross-modal semantic understanding and context modeling capabilities.

[0040] Code Encoding Flow: This branch takes a preprocessed source code sequence as input, utilizes the powerful context modeling capabilities of the pre-trained model, and captures the syntactic and semantic dependencies within the code through a multi-layer self-attention mechanism. Finally, it obtains a source code sequence representation containing complete contextual information, which highly condenses the literal semantics and contextual environment of the code.

[0041] Structural Encoding Flow: This branch takes into linearized structural description sequences, carrying deep logical topology refined by LLM. It is also processed by a shared encoder to obtain a global representation of the structure. This vector supplements the causal logic and key variable intents implicit in the source code in the semantic space, making up for the limitations of a single code view.

[0042] (3) Cross-modal fusion based on hierarchical bottleneck fusion:

[0043] After obtaining two independent modal features, simple feature concatenation often introduces redundancy due to the inconsistency in information density between the modalities. To achieve efficient and low-noise feature fusion, a hierarchical bottleneck fusion module is introduced, such as... Figure 2 As shown:

[0044] The hierarchical bottleneck fusion module no longer allows full pairwise interactions between code features and structural features. Instead, it initializes a set of "bottleneck query vectors" using the sequence representation of the source code stream. In each layer of interaction, the query sequence is truncated and compressed (for example, retaining the first 8 tokens as the bottleneck query and then further compressing it to the first 4 tokens). This allows the query sequence to aggregate key cross-stream information layer by layer and suppress redundant noise. Each layer of interaction includes two types of operations:

[0045] 1) Cross-attention of query vector → (Code, Struct): The bottleneck query vector is used as the Query, and cross-attention operation is performed on the "source code flow" and the "structural supplement flow" respectively. Through the adaptive allocation of attention weights, the bottleneck vector actively retrieves and aggregates the feature fragments most relevant to the defect detection task from the two modalities.

[0046] 2) (Code, Struct) → Reverse cross-attention of query vector: The two original sequences are used as queries and the bottleneck query is used as memory to update information through cross-attention, so that the two representations are aligned under the mediation of the bottleneck.

[0047] After multiple layers of interaction, the module outputs a fusion vector composed of three parts: the first token representation of the bottleneck query, the first token representation of the updated source code sequence, and the first token representation of the updated structural supplement sequence. This fusion vector retains both the semantic context of the source code and the information of the explicit structural facts, and reduces the interference of structural supplement noise on the final decision through the bottleneck compression mechanism.

[0048] (4) Structural knowledge alignment and joint loss optimization:

[0049] To further enhance generalization ability, this embodiment of the invention proposes a multi-task joint loss function. A multi-task joint optimization strategy is employed during training, and the total loss function consists of the following two weighted sums:

[0050] Defect classification main loss: Based on the final mixed features output by the hierarchical bottleneck fusion module, the defect prediction probability is output through the fully connected classification layer. The difference between the predicted value and the true label is calculated using the cross-entropy loss function weighted for positive class samples, which directly guides the model to learn the defect classification boundary.

[0051] Structure alignment auxiliary loss: This loss is implemented through a nonlinear projection module composed of a multilayer perceptron. The fitter receives code flow features and attempts to map them to the structural semantic space. By calculating the mean square error between the predicted pseudo-structural features and the real structural features, the code encoder is forced to not only focus on the surface text during the feature extraction stage, but also to internalize and infer the deep structural logic of the program.

[0052] Considering that some training samples may use catch-all text, the corresponding structural features are not of reference value. Therefore, a masking mechanism is applied when calculating the alignment loss, that is, these invalid samples are automatically ignored when calculating the mean squared error to prevent the learning of incorrect mapping relationships.

[0053] The method provided in this invention, while ensuring the integrity of the original semantic information of the source code, introduces factual structured logical supplementary information and utilizes a hierarchical bottleneck interaction mechanism to achieve effective fusion of the two information streams.

[0054] Example 2 illustrates the specific form and processing of each field using a single sample fragment, and briefly describes the main operation flow and data format of the method provided in this embodiment of the invention:

[0055] (1) Data Representation: Each sample in the dataset is represented by a function-level source code as the smallest unit, and a binary classification label is given (target=1 indicates the presence of a defect, target=0 indicates the absence of a defect). For each sample, an additional structured logical supplement is generated and linearized into short text. Table 1 is a prompt example, instructing deepseek-v3 to generate structured logical supplement information in JSON format. Table 2 gives the source code fragments and the extracted content of the structured logical supplement (linearized text) of the example sample:

[0056] Table 1. An example prompt instructing deepseek-v3 to generate structured logical supplementary information.

[0057]

[0058] Table 2 Example sample source code and generated structured logic supplement

[0059]

[0060] (2) Model structure:

[0061] The model structure of the method provided in this embodiment of the invention can be summarized as: dual-stream shared encoder + hierarchical bottleneck cross-stream interactive fusion + classifier. The dual-stream shared encoder obtains sequence-level representations from the source code and structural supplements, respectively. The hierarchical bottleneck fusion module uses the source code sequence representation as the initial query, performs 8→4 layer-by-layer truncation and compression on the query in the two-layer interaction, and realizes cross-stream information exchange through bidirectional cross-attention. The classifier discriminates the fusion vector and outputs the defect probability.

[0062] (3) Evaluation Indicators:

[0063] The method in this embodiment of the invention is a function-level binary classification task. During evaluation, metrics such as Accuracy, Precision, Recall, and F1 can be used for comprehensive measurement. Accuracy measures the overall correct prediction rate; Precision measures the proportion of true defects among samples predicted as defects; Recall measures the proportion of true defect samples detected; F1 is the harmonic mean of Precision and Recall, used to balance false positives and false negatives. The definitions of these metrics are shown in Table 3, where TP refers to the number of positive samples correctly predicting defective codes as defective; TN refers to the number of negative samples correctly predicting non-defective codes as non-defective; FP refers to the number of false positive samples incorrectly predicting non-defective codes as defective; and FN refers to the number of false negative samples incorrectly predicting defective codes as non-defective.

[0064] Table 3 Definitions of each indicator

[0065]

[0066] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion, characterized in that, Includes the following steps: (1) Data processing is performed using a dual-stream input construction method consisting of a source code sequence and a symbolic logic description. (2) The source code sequence is used as the first input stream and the symbolic logic description is used as the second input stream to form a sample-level dual-stream input pair. The pre-trained language model is used to encode the source code stream and the structure encoding stream respectively to obtain two context representation sequences. (3) Based on the hierarchical bottleneck fusion module, feature interaction is performed. The bottleneck query sequence is initialized with the hidden state of the source code sequence, and the information bottleneck is realized by truncating and compressing layer by layer. Information is aggregated from the source code stream and the structure supplement stream alternately to obtain the fused global representation vector. (4) Based on the fused global representation vector, input the linear classifier to perform binary classification output and obtain the prediction result of whether the code has defects.

2. The program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion according to claim 1, characterized in that, The data processing steps employing a dual-stream input construction method consisting of a source code sequence and a symbolic logic description specifically include: Source code preprocessing and serialization: Standard preprocessing is performed on the original C language function code snippets, and then the code is converted into a token ID sequence using the token segmenter of the pre-trained model; Structured knowledge extraction and filtering: Using a large language model as a knowledge extractor, a structured logical description is generated for each piece of code and output in JSON format.

3. The program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion according to claim 2, characterized in that, The structured knowledge extraction and filtering steps specifically include: Multidimensional structure extraction: Instruction LLM extracts information from three key dimensions, including data flow chain, control dependency, and key semantics; Literal binding filtering: First, extract real identifiers from the source code to construct a set; then extract the identifier set from the structure description generated by LLM for literal matching. If the structure description does not contain a resolvable identifier, or its identifier set has no intersection with the source code identifier set, the structure description is determined to be unverifiable, and the structure field of the sample is uniformly reverted to the preset catch-all text.

4. The program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion according to claim 2, characterized in that, The source code sequence is used as the first input stream, and the symbolic logic description is used as the second input stream, forming a sample-level two-stream input pair. A pre-trained language model is used to encode the source code stream and the structure encoding stream separately, resulting in two context representation sequences, specifically including: A pre-trained language model is used as a universal encoder for both source code streams and structured encoding streams, where: The code encoding stream input is a preprocessed source code sequence. Utilizing the context modeling capabilities of a pre-trained language model, a multi-layer self-attention mechanism is used to capture the syntactic and semantic dependencies within the code, resulting in a source code sequence representation containing complete contextual information. The linearized structural description sequence is input into the structural encoding stream to obtain a global representation of the structure.

5. The program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion according to claim 4, characterized in that, In the step of feature interaction based on the hierarchical bottleneck fusion module, the bottleneck query sequence is initialized with the hidden state of the source code sequence, and the information bottleneck is realized by truncating and compressing it layer by layer. Information is alternately aggregated from the source code stream and the structure supplement stream to obtain the fused global representation vector. In this step, the hierarchical bottleneck fusion module initializes a set of "bottleneck query vectors" with the sequence representation of the source code stream, and truncates and compresses the query sequence in each layer of interaction, so that the query sequence aggregates key cross-stream information layer by layer and suppresses redundant noise. Each layer of interaction includes two types of operations: The bottleneck query vector, as a query, performs cross-attention operations on the "source code stream" and the "structure supplement stream" respectively. Through the adaptive allocation of attention weights, the bottleneck vector actively retrieves and aggregates the feature fragments most relevant to the defect detection task from the two modalities. The two original sequences are used as queries, and the bottleneck query is used as memory to update information through cross attention, so that the two representations are aligned under the mediation of the bottleneck. After multiple layers of interaction, the output is a fusion vector composed of three parts: the first token representation of the bottleneck query, the first token representation of the updated source code sequence, and the first token representation of the updated structure supplement sequence.

6. The program defect detection method based on semantic structure dual-stream collaboration and noise resistance fusion according to claim 1, characterized in that, The program defect detection method further includes introducing a multi-task joint loss function, including: Defect classification main loss: Based on the output fusion vector, the defect prediction probability is output through a fully connected classification layer. The difference between the predicted value and the true label is calculated using a cross-entropy loss function weighted for positive class samples, which directly guides the model to learn the defect classification boundary. Structure alignment auxiliary loss: implemented through a nonlinear projection module composed of a multilayer sensing mechanism.

Images