A code vulnerability arbitration detection method fusing attention mechanism and counterfactual reasoning

Abstract

The application discloses a code vulnerability arbitration detection method fusing an attention mechanism and counterfactual reasoning, and comprises the following steps: obtaining source code to be detected; inputting the source code to be detected into a lightweight special-purpose vulnerability detection model and a large language model respectively to obtain first vulnerability judgment results and second vulnerability judgment results; obtaining an attention weight distribution of the lightweight special-purpose vulnerability detection model according to a consistency comparison result of the first vulnerability judgment results and the second vulnerability judgment results; locating a key code area in which the lightweight special-purpose vulnerability detection model makes a decision according to the attention weight distribution; obtaining a counterfactual code variant generated by the large language model according to the key code area; and inputting the counterfactual code variant and the original source code into the lightweight special-purpose vulnerability detection model respectively to obtain an arbitration detection result. The application can automatically and deeply analyze the root cause of the disagreement when the model judgment is inconsistent, and make a more reliable final judgment through a structured debate process.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary field of software engineering and artificial intelligence, and in particular relates to a code vulnerability arbitration detection method that integrates attention mechanism and counterfactual reasoning. Background Technology

[0002] With the increasing scale of software and the widespread use of open-source code, the risks posed by software security vulnerabilities are becoming increasingly severe. Automated vulnerability detection has become a crucial aspect of software security assurance. Currently, mainstream methods are mainly based on deep learning models, which can be divided into two categories: one is traditional models that are pre-trained and fine-tuned for code understanding tasks. These models are usually based on the Transformer architecture, learning a general representation of code through self-supervised tasks such as masked language modeling on large-scale code corpora, and then fine-tuning them for vulnerability detection tasks. These models have relatively few parameters, fast inference speed, and perform well on specific vulnerability datasets, but their decision-making process is like a "black box," lacking interpretability, and their generalization ability is greatly affected by the training data. The other category is large language models (LLMs, such as the GPT series), which have powerful code semantic understanding and generation capabilities and can provide natural language descriptions, but they suffer from the "illusion" problem, which may not be accurate enough in judging code logic in specific domains, and fine-tuning them directly for specific projects is costly.

[0003] To combine the advantages of both, existing technologies propose a multi-model collaboration framework: first, the fine-tuned model and the LLM (Local Modeling Module) perform independent detection; when the results are inconsistent, the result of the fine-tuned model is fed back to the LLM for secondary judgment; finally, the description generated by the LLM enhances the training data. This method improves performance to some extent, but its "alignment" process is unidirectional and superficial. It merely uses inconsistent results as triggering conditions and simply uses the output of one model to prompt the other, without delving into the root causes of the discrepancies between the two models—that is, what features and logic of the code each model is based on to make its judgment. This collaboration, lacking deep interaction and verifiable arbitration, still suffers from insufficient reliability and persuasiveness in its final decision when facing complex and ambiguous vulnerability patterns. Summary of the Invention

[0004] To address the aforementioned technical issues, this invention proposes a code vulnerability arbitration detection method that integrates attention mechanisms and counterfactual reasoning. This method can automatically and deeply analyze the root causes of disagreements when model judgments are inconsistent, make a more reliable and credible final judgment through a structured "debate" process, and generate a detailed decision-making basis report.

[0005] To achieve the above objectives, this invention provides a code vulnerability arbitration detection method that integrates attention mechanisms and counterfactual reasoning, comprising: Obtain the source code to be tested; The source code to be detected is input into a dedicated vulnerability detection model and a large language model respectively to obtain the first vulnerability judgment result and the second vulnerability judgment result. Based on the consistency comparison results between the first vulnerability judgment result and the second vulnerability judgment result, the attention weight distribution of the dedicated vulnerability detection model is obtained; Based on the attention weight distribution, the key code regions for decision-making by the dedicated vulnerability detection model are located. Based on the key code region, obtain the counterfactual code variants generated by the large language model; The dedicated vulnerability detection model is then used to input the counterfactual code variant and the source code to be detected, respectively, to obtain the arbitration detection results.

[0006] Optionally, the dedicated vulnerability detection model includes: a pre-training module, a global semantic feature module, a local multi-scale feature module, and a feature fusion module; The pre-training module is used to obtain global semantic representation and embedding representation based on the source code to be detected; The global semantic feature module is used to obtain global semantic features based on the global semantic representation; The local multi-scale feature module is used to obtain local multi-scale features based on the embedded representation; The feature fusion module is used to fuse the global semantic features and local multi-scale features to obtain fused features, which are used for classification decisions and finally form the first vulnerability judgment result.

[0007] Optionally, the embedding layer is used to convert structured hints containing key decision-making regions into fixed-dimensional vector representations; The multi-head self-attention layer takes the output of the embedding layer as input, calculates the dependencies between positions in the input sequence, and outputs a self-attention representation. The feedforward neural network layer takes the output of the multi-head self-attention layer as input, performs a nonlinear transformation on the representation of each position, and outputs a feedforward representation. The normalization layer normalizes the outputs of the multi-head self-attention layer and the feedforward neural network layer, respectively, and combines them with the residual connection to form a basic block of the Transformer decoder; multiple basic blocks are stacked, and the output of the last layer serves as the final hidden layer representation of the model. The final hidden layer representation is generated by the language model head into a structured analysis text containing vulnerability type, risk level and reasoning basis, and the second vulnerability judgment result is extracted from the structured analysis text according to preset rules.

[0008] Optionally, based on the consistency comparison between the first vulnerability assessment result and the second vulnerability assessment result, the attention weight distribution of the dedicated vulnerability detection model is obtained, including: The first vulnerability assessment result is compared with the second vulnerability assessment result to obtain a consistency comparison result; Based on the consistency comparison results, determine whether to trigger the arbitration process; Based on the determination result of triggering the arbitration process, the source code to be detected is input into the dedicated vulnerability detection model for forward propagation; Based on the forward propagation process, the attention weight matrix of the last layer Transformer encoder of the dedicated vulnerability detection model is extracted; Based on the attention weight matrix, the attention weight distribution of the dedicated vulnerability detection model is obtained.

[0009] Optionally, the attention weight distribution of the dedicated vulnerability detection model can be obtained based on the attention weight matrix. include: ; in, For the first The weights of each attention head are given, where h is the index of the h-th attention head, and H is the total number of attention heads.

[0010] Optionally, based on the attention weight distribution, the key code regions for decision-making by the dedicated vulnerability detection model include: Based on the attention weight distribution, code locations whose attention weight values ​​exceed a preset threshold are selected; Based on the selected code locations, identify the key lines of code; Based on the key code lines, obtain the decision-making key code region containing context information.

[0011] Optionally, obtaining counterfactual code variants generated by the large language model based on the key code region includes: Based on the key code regions, construct a structured hint that includes the highlighted key code regions; Based on the structured prompts, obtain the analysis text for the key code regions output by the large language model; Based on the analyzed text, a counterfactual code variant generated by the large language model is obtained. The counterfactual code variant is the code with minimal semantic modification to the key code region while maintaining the overall functionality of the code.

[0012] Optionally, the arbitration detection results are obtained by inputting the counterfactual code variant and the original source code into the dedicated vulnerability detection model respectively, including: The original source code and the counterfactual code variant are respectively input into the dedicated vulnerability detection model to obtain the original prediction result and the variant prediction result; Based on the comparison between the original prediction result and the variant prediction result, determine whether the prediction result has been reversed; The arbitration detection result is obtained based on the determination of whether the prediction result has been reversed.

[0013] Optionally, obtaining the arbitration detection result based on the determination of whether the prediction result has been reversed includes: If the prediction result is reversed, the second vulnerability judgment result shall be adopted as the final judgment result; If the prediction result does not reverse, the first vulnerability judgment result shall be adopted as the final judgment result.

[0014] Optionally, after inputting the original source code and the counterfactual code variant into the dedicated vulnerability detection model to obtain the original prediction result and the variant prediction result, the method further includes: Based on the confidence values ​​of the original prediction results and the variant prediction results, determine whether both are close to the decision threshold; Based on the judgment result, if the confidence values ​​of both the original prediction result and the variant prediction result are close to the decision threshold, a high uncertainty label is added.

[0015] Compared with the prior art, the present invention has the following advantages and technical effects: Through the above-mentioned technical solution, this invention achieves multi-level innovation from model architecture to decision logic, and has substantial features and significant progress compared with the prior art, specifically reflected in the following aspects: 1. Deep interpretability: This invention transforms black-box decision-making into an understandable and discussable focus by extracting and visualizing the attention weights of a dedicated model, enabling LLM analysis and subsequent arbitration processes to be based on the model's internal evidence.

[0016] 2. Scientific Arbitration Logic: This invention innovatively introduces "counterfactual variants" as a verification tool. Arbitration decisions no longer rely on simple comparisons or one-way hints from model outputs, but are accomplished through a controlled "hypothesis-test" experiment, making the final conclusion more persuasive and robust.

[0017] 3. Automation and Intelligence: The entire arbitration process is fully automated, requiring no human intervention. The system guides the two models through a structured "debate," ultimately reaching a consensus or clarifying uncertainties automatically based on verifiable evidence, significantly improving analytical efficiency.

[0018] 4. Enhanced detection reliability: This mechanism can effectively handle fuzzy discrepancies between models and select more reasonable judgments in most cases, thereby reducing the false positive rate and false negative rate of vulnerability detection as a whole and improving the reliability and generalization ability of the results.

[0019] 5. High output value: The detailed alignment report generated by this invention not only provides the final conclusion, but also clearly shows the differences in thinking between models, key code areas for decision-making, counterfactual verification process and arbitration reasons, which greatly facilitates security engineers in reviewing results, root cause analysis and knowledge accumulation, and has high auxiliary decision-making and auditing value. Attached Figure Description

[0020] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart of a code vulnerability arbitration detection method that integrates attention mechanism and counterfactual reasoning according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the attention-guided interpretation generation process according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the counterfactual reasoning and arbitration process according to an embodiment of the present invention. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0022] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0023] This embodiment proposes a code vulnerability arbitration detection method that integrates attention mechanisms and counterfactual reasoning, such as... Figure 1 As shown, the specific steps include: Obtain the source code to be tested; The source code to be detected is input into a dedicated vulnerability detection model and a large language model respectively to obtain the first vulnerability judgment result and the second vulnerability judgment result. Based on the consistency comparison results between the first vulnerability judgment result and the second vulnerability judgment result, the attention weight distribution of the dedicated vulnerability detection model is obtained; Based on the attention weight distribution, the key code regions for decision-making by the dedicated vulnerability detection model are located. Based on the key code region, obtain the counterfactual code variants generated by the large language model; The counterfactual code variant and the source code to be detected (original source code) are respectively input into the dedicated vulnerability detection model (lightweight dedicated vulnerability detection model) to obtain the arbitration detection result.

[0024] Specifically, S1: Use a lightweight dedicated vulnerability detection model (Model A) and a large language model (Model B) to independently determine the existence of vulnerabilities in the source code; S2: If the judgment results of model A and model B are inconsistent, extract the attention weight distribution of model A and locate its key decision code area. S3: Based on the analysis of the key code region by Model B, a counterfactual code variant is generated; S4: Input the original source code and the counterfactual variant into model A respectively, and arbitrate and adopt the final judgment based on whether the prediction result is reversed. S5: Generate a structured report containing the aforementioned disagreements, key areas, variations, arbitration process, and conclusions.

[0025] Furthermore, the lightweight dedicated vulnerability detection model includes: a pre-training module, a global semantic feature module, a local multi-scale feature module, and a feature fusion module; The pre-training module is used to obtain global semantic representation and embedding representation based on the source code to be detected; The global semantic feature module is used to obtain global semantic features based on the global semantic representation; The local multi-scale feature module is used to obtain local multi-scale features based on the embedded representation; The feature fusion module is used to fuse the global semantic features and local multi-scale features to obtain fused features, which are used for classification decisions and finally form the first vulnerability judgment result.

[0026] Specifically, S1: Collaborative Detection and Disagreement Triggering. A lightweight dedicated vulnerability detection model (denoted as Model A) and a large language model (denoted as Model B, such as ChatGPT) are used to independently determine the existence of vulnerabilities in the source code fragments to be detected. If the preliminary judgments of the two models are consistent, the consistent result is directly output; if the judgments are inconsistent, an arbitration process is triggered.

[0027] The method for constructing model A includes: 1) Source code segmentation and encoding: The source code is processed by a parser to preserve the syntactic structure. An improved byte encoding algorithm combining predefined programming language keywords is used to segment the source code, generate a token sequence and map it to a vector representation.

[0028] 2) Pre-trained representation learning: Based on the Transformer architecture, a masked language modeling task is performed on a large amount of unlabeled source code data to learn a deep representation of the syntax and semantics of the code.

[0029] 3) Vulnerability detection fine-tuning: On top of the pre-trained model, the global-local dual-path feature fusion classifier proposed in this invention is added for supervised fine-tuning, and trained on vulnerability detection labeled data to obtain the final detection model.

[0030] Model A is characterized by its small parameter size and high inference efficiency, but its decision logic can be partially explained through the attention mechanism.

[0031] Furthermore, the large language model includes: an embedding layer, a multi-head self-attention layer, a feedforward neural network layer, and a normalization layer; The embedding layer is used to convert structured hints containing key decision-making regions into vector representations of fixed dimensions; The multi-head self-attention layer takes the output of the embedding layer as input, calculates the dependencies between positions in the input sequence, and outputs a self-attention representation. The feedforward neural network layer takes the output of the multi-head self-attention layer as input, performs a nonlinear transformation on the representation of each position, and outputs a feedforward representation. The normalization layer normalizes the outputs of the multi-head self-attention layer and the feedforward neural network layer, respectively, and combines them with the residual connection to form a basic block of the Transformer decoder; multiple basic blocks are stacked, and the output of the last layer serves as the final hidden layer representation of the model. The final hidden layer representation is generated by the language model head into a structured analysis text containing vulnerability type, risk level and reasoning basis, and the second vulnerability judgment result is extracted from the structured analysis text according to preset rules.

[0032] Specifically, Model B, as an externally pre-trained language model, has a core architecture based on Transformer and possesses powerful code understanding capabilities. However, its ability to identify specific vulnerabilities may be affected by the "illusion" problem. In this invention, Model B is not fine-tuned but serves only as an analysis tool. It receives the attention interpretation from Model A through a natural language prompt mechanism and generates analytical text and counterfactual variants based on this interpretation. The input-output flow of Model B is as follows: 1) Input: Description of the "decision key region" of Model A (including highlighted code snippets and context); 2) Processing: Model B uses its internal language understanding capabilities to analyze the potential vulnerabilities in the code snippets; 3) Output: Generate structured analysis text and counterfactual code variants.

[0033] Furthermore, based on the consistency comparison between the first vulnerability assessment result and the second vulnerability assessment result, the attention weight distribution of the lightweight dedicated vulnerability detection model is obtained, including: The first vulnerability assessment result is compared with the second vulnerability assessment result to obtain a consistency comparison result; Based on the consistency comparison results, determine whether to trigger the arbitration process; Based on the determination result of triggering the arbitration process, the source code to be detected is input into the lightweight dedicated vulnerability detection model for forward propagation; Based on the forward propagation process, the attention weight matrix of the last layer Transformer encoder of the lightweight dedicated vulnerability detection model is extracted; Based on the attention weight matrix, the attention weight distribution of the lightweight dedicated vulnerability detection model is obtained.

[0034] Specifically, S2: Attention-Guided Interpretation Generation. The weights of Model A are frozen, and the source code to be tested is input into Model A again for forward propagation. The attention weight distribution of the specific token used for final classification against all other tokens in the source code is extracted from the last layer of the Transformer encoder in Model A. Based on this distribution, the top K code tokens with the highest attention weights and their corresponding lines of code are identified and marked as "decision-critical regions" for Model A. Subsequently, the system automatically constructs a natural language prompt containing highlighted (or textual) information about these "decision-critical regions" and sends it to Model B. This prompt instructs Model B to act as a security analyst, interpreting why Model A might make its current judgment based on the presented highlighted code patterns, and analyzing the significance of these code patterns in the actual vulnerability context.

[0035] Furthermore, based on the attention weight matrix, the attention weight distribution of the lightweight dedicated vulnerability detection model is obtained. include: ; in, For the first The weights of each attention head are given, where h is the index of the h-th attention head, and H is the total number of attention heads.

[0036] Furthermore, based on the attention weight distribution, the key code regions for decision-making by the lightweight dedicated vulnerability detection model include: Based on the attention weight distribution, code locations whose attention weight values ​​exceed a preset threshold are selected; Based on the selected code locations, identify the key lines of code; Based on the key code lines, obtain the decision-making key code region containing context information.

[0037] Furthermore, based on the key code region, obtaining the counterfactual code variants generated by the large language model includes: Based on the key code regions, construct a structured hint that includes the highlighted key code regions; Based on the structured prompts, obtain the analysis text for the key code regions output by the large language model; Based on the analyzed text, a counterfactual code variant generated by the large language model is obtained. The counterfactual code variant is the code with minimal semantic modification to the key code region while maintaining the overall functionality of the code.

[0038] Specifically, S3: Counterfactual Reasoning and Variant Generation. After completing the interpretation in S2, Model B generates a "counterfactual code variant" based on its inference of Model A's decision logic. The principle for generating this variant is: while maintaining the overall functionality and syntactic correctness of the source code as much as possible, only minimal semantic modifications are made to the "decision-critical areas" identified in S2, in an attempt to subvert the decision features upon which Model A relies. For example, if Model A is deemed safe due to boundary checks, the variant can remove or weaken those checks; if Model A is deemed vulnerable due to a dangerous function call, the variant can replace that function with a functionally equivalent safe function. This variant is a key tool for verifying hypotheses.

[0039] Furthermore, by inputting the counterfactual code variant and the original source code into the lightweight dedicated vulnerability detection model respectively, the arbitration detection results obtained include: The original source code and the counterfactual code variant are respectively input into the lightweight dedicated vulnerability detection model to obtain the original prediction result and the variant prediction result; Based on the comparison between the original prediction result and the variant prediction result, determine whether the prediction result has been reversed; The arbitration detection result is obtained based on the determination of whether the prediction result has been reversed.

[0040] Furthermore, based on the determination of whether the prediction result has been reversed, obtaining the arbitration detection result includes: If the prediction result is reversed, the second vulnerability judgment result shall be adopted as the final judgment result; If the prediction result does not reverse, the first vulnerability judgment result shall be adopted as the final judgment result.

[0041] Furthermore, after inputting the original source code and the counterfactual code variant into the lightweight dedicated vulnerability detection model to obtain the original prediction result and the variant prediction result, the process further includes: Based on the confidence values ​​of the original prediction results and the variant prediction results, determine whether both are close to the decision threshold; Based on the judgment result, if the confidence values ​​of both the original prediction result and the variant prediction result are close to the decision threshold, a high uncertainty label is added.

[0042] Specifically, S4: Arbitration and Final Judgment Generation. The original source code and the counterfactual code variant generated by Model B are input into Model A in parallel for a second inference, yielding two new predictions. Based on the changes in Model A's response to these two inputs, the arbitration logic is executed: If Model A's predictions for the original code and its counterfactual variants are reversed (e.g., the original predicts "secure" while the variant predicts "vulnerable," or vice versa), then Model B has successfully identified the vulnerable feature logic upon which Model A's decision relies. In this case, the system adopts Model B's initial judgment as the final result.

[0043] If Model A's predictions for both remain unchanged, it indicates that Model A's decision-making logic may be based on a more robust and deeper code pattern, or that Model B's inference is inaccurate. In this case, the system adopts Model A's initial judgment as the final result. As an optimization, an additional confidence threshold can be set. When Model A's predicted probability values ​​for both are very close to the decision boundary, a "high uncertainty" flag is output, suggesting manual review.

[0044] S5: Alignment Report Generation. Regardless of the final decision adopted, the system automatically summarizes and generates a structured alignment report. This report includes: the initial disagreement, a visual summary of Model A's attention, Model B's explanation of the attention pattern, the generated counterfactual variant code, the arbitration test results (Model A's predictions for the original code and variants), and the final arbitration reasons and conclusions.

[0045] The following is a detailed description of this embodiment with reference to the accompanying drawings: like Figure 1 As shown, the method of the present invention begins with source code input, and vulnerability detection is performed in parallel by a lightweight detection model A and a large language model B.

[0046] Model A is constructed using the following steps: First, the source code is segmented and encoded while preserving its grammatical structure. This involves using a Clang parser to process the source code, remove comments, and generate an initial token sequence. A byte-pair encoding algorithm is employed, and a predefined vocabulary is introduced for tokenization. Each token is mapped to a 768-dimensional embedding vector, and a maximum sequence length is set (512 for pre-training and 1024 for fine-tuning), with insufficient lengths padded on the right side using special padding tokens. Then, the RoBERTa-base architecture is used as the backbone network, and pre-training is performed on a large amount of unlabeled source code data using a masked language modeling task. 15% of the tokens in the input sequence are randomly masked, and the model predicts the masked original tokens based on the context, minimizing the cross-entropy loss. After pre-training, a global-local feature fusion classification layer proposed in this invention is added to the pre-trained model for supervised fine-tuning to adapt it to the specific task of vulnerability detection.

[0047] The lightweight classification architecture proposed in this invention employs a global-local dual-path feature fusion classifier, as detailed below: 1. Input representation: The pre-trained Transformer encoder outputs the following two types of features: Hidden state of the sequence start marker [CLS]: , as a global semantic representation; Complete tag embedding matrix: ,in The sequence length is given.

[0048] 2. Global semantic feature extraction branch: Will Input a two-layer perceptron and extract global semantic features: ; ; in, Number of classification categories (in vulnerability detection tasks) ), , , , These are trainable parameters.

[0049] 3. Local multi-scale feature extraction branch: Embedding matrix Input multi-width one-dimensional convolutional neural network: use Different widths ( Parallel convolutions are performed using convolutional kernels, with each kernel corresponding to a certain number of output channels. For the first Each convolutional kernel has weights. bias Output feature map The calculation is as follows: ; in, This represents a one-dimensional convolution operation. It is a non-linear activation function (such as ReLU). is the length of the sequence after convolution.

[0050] For each feature map Perform global max pooling operation: for each channel ( Extract the maximum value independently to obtain the local feature vector. : ; in, Representation of feature map No. Each channel at time step The activation value.

[0051] All The pooling outputs of each convolutional kernel are concatenated along the feature dimension to form a joint representation vector that integrates multi-scale features: , in, The concatenated feature dimensions.

[0052] The joint representation vector is mapped to the category space through a fully connected layer: ; in, This is the weight matrix. This is the bias vector.

[0053] 4. Feature fusion and classification decision: Output the global branch With local branch output Perform weighted fusion: ; in, For learnable scalar fusion weight parameters, It is initialized to 0.5 and adaptively adjusted through backpropagation during training.

[0054] Softmax normalization is applied to the fused vector to obtain the class prediction probability: ; 5. Model Training: Using the cross-entropy loss function, the pre-trained encoder parameters, global branch parameters, local branch parameters, and fusion weights are jointly optimized through backpropagation: ; in, The total number of training samples, This is an indicator function (its value is 1 when the condition inside the parentheses is true, and 0 otherwise). For the first The true label of each sample For the model to sample Category The predicted probability.

[0055] The large-scale language model B uses the existing pre-trained model ChatGPT3.5 (based on OpenAI's GPT-3.5 architecture) as its foundation without any fine-tuning. It only uses prompting engineering as an analysis tool, thus avoiding the complexity of model training and solving the "illusion" problem (model B does not directly output vulnerability judgments, but generates analysis text for the arbitration system to use).

[0056] Model B is constructed using the following steps: 1. Model Construction and Structural Composition: Model B's architecture is based on the Transformer decoder structure, and its core components include: Embedding layer: Converts the input text into a fixed-dimensional vector representation using word embeddings and positional encoding.

[0057] Multi-head self-attention layer: calculates the dependencies between positions in the input sequence, and its mathematical expression is: , in, , ; Feed-Forward Network: Performs a non-linear transformation on the representation at each location, using the ReLU activation function. ; Layer normalization: Stabilizes the training process and ensures that the output distribution of each layer is consistent.

[0058] 2. Model Functions and Workflow: Model B is not fine-tuned; it only uses Prompt Engineering as an analysis tool. Its input-output flow is as follows: Input: Receive a description of the "decision-critical region" of model A (including highlighted code snippets and context), in structured format. "Please analyze the following code snippet for vulnerabilities, focusing on the [highlighted code] and its context. Code snippet: [code content]."

[0059] Processing: Model B utilizes its pre-trained language understanding capabilities to analyze the vulnerability probability of code snippets through a self-attention mechanism, generating structured analysis text containing vulnerability types, risk levels, and reasoning basis.

[0060] Output: Generates a two-part response: Structured analysis text: such as "Vulnerability type: Buffer overflow; Risk level: High; Reasoning basis: [Analysis content]"; Counterfactual code variant: A modified code example generated based on the analysis results, used to verify the effectiveness of vulnerability fixes.

[0061] 3. Model representation: The input-output mapping of model B can be represented as: ; in, For input prompts, To output analytical text and counterfactual variants, These are pre-trained parameters (fixed). This is the functional expression for model B.

[0062] The results were then compared, and the comparison process was based on the vulnerability assessment results of Model A. The vulnerability judgment results of structured text extraction from Model B are compared with those of Model B. The consistency verification process is as follows: 1. Input extraction and preprocessing: Model A Output: Vulnerability Assessment (0 indicates no vulnerability, 1 indicates a vulnerability), and an explanation of attention. (Includes highlighted code snippets and context).

[0063] Model B output: Structured analysis text (including vulnerability type, risk level, and reasoning basis), from which to extract... : ; 2. Consistency judgment logic: Comparison via consistency condition function Judgment result: ; Consistent scenarios ( ): Directly output the judgment result of model A, and the arbitration ends.

[0064] Inconsistent scenarios ( : Triggers S2 (attention-guided interpretation generation) and S3 (counterfactual reasoning and variant generation).

[0065] 3. Specific comparison examples are shown in Table 1: Table 1 3. Example Analysis: Consistent Example: The code contains a clear buffer overflow vulnerability, and both Model A and Model B identify it as vulnerable. If the comparison is consistent, the arbitration will directly adopt the result of Model A.

[0066] Inconsistent example: Model A outputs no flaws due to misjudgment of the attention region (focusing only on arr while ignoring boundary checks). However, Model B explicitly identified the array out-of-bounds vulnerability by analyzing the text. If the comparison is inconsistent, it will trigger the subsequent arbitration process.

[0067] 4. Comparison and Arbitration Process Integration: The comparison results directly determine the arbitration path: When they are in agreement, the arbitration output is the judgment of model A (to avoid redundant calculations); When there is inconsistency, interpret the attention of model A. As input, it is passed to S2 (attention-guided interpretation generation), which drives model B to generate deep counterfactual reasoning.

[0068] After entering the arbitration process (corresponding) Figure 1 Arbitration process section): First, extract the attention weights of model A and locate the key regions (corresponding to...). Figure 2 ): Extract the attention weight distribution of model A and locate key lines of code. Model A adopts the existing Transformer encoder architecture (implemented based on the HuggingFace Transformers library). Its structure, functions, and the process of attention weight extraction and key region localization are as follows: 1. Structure and function of Model A: Model A is a lightweight, dedicated vulnerability detection model. Its network structure consists of three parts: Embedding layer: This layer contains the source code token sequence. ( Mapping the number of lines of code to a vector representation ,in For the embedding dimension, standard word embedding and positional encoding are used.

[0069] N-layer Transformer encoder: Each layer contains: Multi-head self-attention: Calculates the dependencies between positions in the input sequence. The core formula is: ; in, For learnable weight matrix, As the key dimension, For the number of attention heads.

[0070] Feedforward Neural Network (FFN): Performs a non-linear transformation on the representation at each position, as shown in the formula: Classification layer: Based on the output of the last Transformer layer, the vulnerability assessment result is output through the fully connected layer. .

[0071] Existing technology description: The architecture of model A is based on the Hugging Face Transformers library (v4.30.0), initialized with pre-trained weights, and without any fine-tuning.

[0072] 2. Attention weight extraction process: From the last layer of the Transformer encoder in model A (layer index) Extracting attention weight distribution: Extracting the attention weight matrix ,in Indicates the first The code location for the first Attention weights for each position.

[0073] right Pay attention ( The average is calculated to obtain the normalized weight matrix. : ; in For the first The weight of each attention head.

[0074] 3. Key Area Location Process: Based on normalized weight matrix Locating the key line of code: Calculate the average attention weights for each code location. ( ), calculate its average weight: ; Threshold setting: The threshold determined using patented experiments. (Based on the weight distribution statistics of model A on the test set).

[0075] Locating key lines: The set of key lines of code is defined as follows: ; Generate a structured description: The key region contains the key line and its context (defined in the patent as 3 lines before and after the key line), and the output is: ; in For the critical line of code, This is a context code snippet.

[0076] The system then constructs natural language prompts, sending highlighted key regions to Model B, which generates the analyzed text accordingly. Model B uses existing technology ChatGPT 3.5 (based on OpenAI's GPT-3.5 architecture, Transformer decoder structure) without any fine-tuning, relying solely on Prompt Engineering as an external analysis tool. Its generation process follows a standardized workflow, detailed below: 1. Network algorithm model and structural composition: Model Foundation: Model B uses the pre-trained model GPT-3.5-turbo (based on the Transformer decoder architecture) provided by OpenAI. Its core structure includes: Embedding layer: Converts the input text into a vector representation (using word embeddings and positional encoding).

[0077] N-layer Transformer decoder: Each layer contains a multi-head self-attention mechanism and a feedforward neural network (standard Transformer structure).

[0078] Output layer: The probability distribution is generated using the Softmax function, and the text sequence is output.

[0079] Technical Note: This model is based on existing open-source technology (OpenAI API v1.0). Its parameters and structure are fixed, and it does not involve any custom training or fine-tuning. It relies entirely on pre-trained weights for text generation.

[0080] 2. Specific process for generating analysis text: The analysis text generation process for Model B strictly follows the prompting engineering specifications and is executed in three steps: (1) Input formatting: Explaining the attention of model A Convert to structured hints: prompt="Please analyze the following code snippet for vulnerabilities, focusing on key ones. and its context. Code snippet: context + + context"; in: The key line of code (string) for locating Model A.

[0081] : The context code snippet (string) of the key line.

[0082] (2) API calls and text generation: Send a prompt to Model B via the OpenAI API to generate a response. The input-output mapping of model B is as follows: ; in: This is the generating function for model B (based on the GPT-3.5 architecture). These are pre-trained parameters (fixed). Provide input suggestions.

[0083] (3) Structured output parsing: From the response Extract and analyze the text; the parsing logic is as follows: ; Where: [Type]: Vulnerability type (e.g., "buffer overflow"), [Level]: Risk level (e.g., "high"), [Basis]: Reasoning basis (natural language description extracted from the response).

[0084] Existing technology description: Model B is entirely based on OpenAI's ChatGPT 3.5 (GPT-3.5-turbo version), and its architecture, parameters, and generation logic are all publicly available pre-trained models (not custom-developed). The patent only uses its API interface for hint engineering and does not modify any internal structure or training process.

[0085] Next, Model B analyzes and generates counterfactual variants (corresponding to...) Figure 1 and Figure 3 Model B, based on its analysis text (including vulnerability type, risk level, and reasoning basis), generates counterfactual code variants through prompting engineering. The specific process, function formulas, and technical implementation are as follows: Network algorithm model and structure composition: Model Foundation: Model B adopts existing technology OpenAI's GPT-3.5-turbo (based on the Transformer decoder architecture), and its structure includes: Embedding layer: Converts the input text into a vector representation (word embedding + position encoding).

[0086] N-layer Transformer decoder: Each layer contains a multi-head self-attention mechanism and a feedforward neural network (standard Transformer structure).

[0087] Output layer: Probability distribution of code sequences generated using Softmax.

[0088] Technical Note: This model is entirely based on the pre-trained API (OpenAI API v1.0), without any fine-tuning or custom training. It is called through prompts only, which meets the definition of "external pre-trained language model" in the patent.

[0089] The specific process of generating counterfactual variants: The generation process strictly follows the prompt-based engineering specifications and is executed in three steps: 1. Input prompt construction: Based on text analysis ( = "Vulnerability type: [type]; Risk level: [level]; Reasoning basis: [reason]") Constructing structured hints: prompt="Generate a counterfactual code variant to fix the [type] vulnerability. Basis: [reason]. Original code snippet: [code]" Where: [type]: vulnerability type (e.g., "buffer overflow"), [level]: risk level (e.g., "high"), [reason]: reasoning basis (e.g., "strcpy function did not check input length"), [code]: highlighted code snippet provided by model A.

[0090] 2. API calls and variant generation: A prompt is sent to Model B via the OpenAI API, generating a response. The input-output mapping is as follows: ; in: This is the text generation function for model B (based on the GPT-3.5 architecture). These are pre-trained parameters (fixed).

[0091] 3. Counterfactual variant extraction: The structured code variant is parsed from the response. The parsing logic is as follows: ; in The code snippet used to fix the vulnerability (e.g., changing strcpy(buffer, input) to strncpy(buffer, input, sizeof(buffer))).

[0092] Existing technology description: Model B relies entirely on the OpenAI GPT-3.5-turbo API (pre-trained model), and its architecture, parameters, and generation logic are all publicly available existing technologies. The patent only prompts the engineering team to call the API without modifying any internal structure or training process, which conforms to the technical definition of "externally pre-trained language model".

[0093] Then, model A re-predicts the original code and variants (corresponding to...) Figure 1 and Figure 3 ): Inputting the original code and the counterfactual variant back into model A yields two new predictions. and .

[0094] Finally, arbitration is conducted and a report is generated (corresponding to...) Figure 1 and Figure 3 Arbitration logic: If and If the result is reversed, the initial judgment of Model B is adopted; if the result is not reversed, the initial judgment of Model A is adopted. During this process, if the confidence levels of Model A's predictions for both are close to the decision threshold, a "high uncertainty" label can be added. Finally, the system integrates all intermediate data and arbitration conclusions to generate a complete structured alignment report (corresponding to...). Figure 1 ).

[0095] Compared with the prior art, the present invention has the following non-obvious technical features and innovations: 1. This invention is the first to implement an inter-model alignment mechanism based on internal evidence: Existing multi-model collaboration methods only compare the final output results of the models, lacking in-depth exploration of the decision-making basis. This invention innovatively extracts the attention weights of a dedicated model (Model A), transforming the decision focus within the model into a visualized "key decision region," providing a concrete evidentiary basis for the analysis of a large language model (Model B), and realizing the transformation from "black-box alignment" to "white-box alignment."

[0096] 2. A closed-loop scientific arbitration mechanism of "evidence-hypothesis-verification" has been constructed: This invention creatively links three steps—attention-guided interpretation generation, counterfactual variant generation, and prediction verification—to form a complete scientific verification process. When model judgments are inconsistent, the system automatically executes the verification loop of "extracting evidence (attention weights) → proposing hypotheses (counterfactual variants) → experimental verification (predicting changes)," ensuring that arbitration decisions are based on repeatable and verifiable experiments, rather than subjective inferences.

[0097] 3. A learnable global-local feature adaptive fusion mechanism is proposed: The dual-path feature fusion classifier designed in this invention achieves adaptive importance adjustment of global semantic and local syntactic features through learnable fusion weights α. This mechanism is not a simple feature concatenation, but rather enables the model to autonomously learn how to optimally combine the two types of features in different code contexts through end-to-end training, thereby producing a synergistic enhancement effect of "1+1>2".

[0098] 4. A qualitative leap from "result alignment" to "logical alignment": Existing technologies only pursue consistency in model output results, while this invention achieves alignment between the two models at the decision-making logic level through in-depth analysis of the root causes of disagreements and verifiable arbitration. This "logical alignment" not only improves the credibility of the final conclusion but also provides security experts with clear decision-making basis, significantly enhancing the interpretability and practicality of vulnerability detection.

[0099] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A code vulnerability arbitration detection method that integrates attention mechanisms and counterfactual reasoning, characterized in that, include: Obtain the source code to be tested; The source code to be detected is input into a dedicated vulnerability detection model and a large language model respectively to obtain the first vulnerability judgment result and the second vulnerability judgment result. Based on the consistency comparison results between the first vulnerability judgment result and the second vulnerability judgment result, the attention weight distribution of the dedicated vulnerability detection model is obtained; Based on the attention weight distribution, the key code regions for decision-making by the dedicated vulnerability detection model are located. Based on the key code region, obtain the counterfactual code variants generated by the large language model; The dedicated vulnerability detection model is then used to input the counterfactual code variant and the source code to be detected, respectively, to obtain the arbitration detection results.

2. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning as described in claim 1, characterized in that, The dedicated vulnerability detection model includes: a pre-training module, a global semantic feature module, a local multi-scale feature module, and a feature fusion module; The pre-training module is used to obtain global semantic representation and embedding representation based on the source code to be detected; The global semantic feature module is used to obtain global semantic features based on the global semantic representation; The local multi-scale feature module is used to obtain local multi-scale features based on the embedded representation; The feature fusion module is used to fuse the global semantic features and local multi-scale features to obtain fused features, which are used for classification decisions and finally form the first vulnerability judgment result.

3. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 1, characterized in that, The large language model includes: an embedding layer, a multi-head self-attention layer, a feedforward neural network layer, and a normalization layer; The embedding layer is used to convert structured hints containing key decision-making regions into vector representations of fixed dimensions; The multi-head self-attention layer takes the output of the embedding layer as input, calculates the dependencies between positions in the input sequence, and outputs a self-attention representation. The feedforward neural network layer takes the output of the multi-head self-attention layer as input, performs a nonlinear transformation on the representation of each position, and outputs a feedforward representation. The normalization layer normalizes the outputs of the multi-head self-attention layer and the feedforward neural network layer, respectively, and combines them with the residual connection to form a basic block of the Transformer decoder; multiple basic blocks are stacked, and the output of the last layer serves as the final hidden layer representation of the model. The final hidden layer representation is generated by the language model head into a structured analysis text containing vulnerability type, risk level and reasoning basis, and the second vulnerability judgment result is extracted from the structured analysis text according to preset rules.

4. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 1, characterized in that, Based on the consistency comparison between the first vulnerability assessment result and the second vulnerability assessment result, the attention weight distribution of the dedicated vulnerability detection model is obtained as follows: The first vulnerability assessment result is compared with the second vulnerability assessment result to obtain a consistency comparison result; Based on the consistency comparison results, determine whether to trigger the arbitration process; Based on the determination result of triggering the arbitration process, the source code to be detected is input into the dedicated vulnerability detection model for forward propagation; Based on the forward propagation process, the attention weight matrix of the last layer Transformer encoder of the dedicated vulnerability detection model is extracted; Based on the attention weight matrix, the attention weight distribution of the dedicated vulnerability detection model is obtained.

5. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 4, characterized in that, Based on the attention weight matrix, obtain the attention weight distribution of the dedicated vulnerability detection model. include: ； in, For the first The weights of each attention head are given, where h is the index of the h-th attention head, and H is the total number of attention heads.

6. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 1, characterized in that, Based on the attention weight distribution, the key code regions for decision-making by the dedicated vulnerability detection model include: Based on the attention weight distribution, code locations whose attention weight values ​​exceed a preset threshold are selected; Based on the selected code locations, identify the key lines of code; Based on the key code lines, obtain the decision-making key code region containing context information.

7. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 1, characterized in that, Based on the key code region, obtaining the counterfactual code variants generated by the large language model includes: Based on the key code regions, construct a structured hint that includes the highlighted key code regions; Based on the structured prompts, obtain the analysis text for the key code regions output by the large language model; Based on the analyzed text, a counterfactual code variant generated by the large language model is obtained. The counterfactual code variant is the code with minimal semantic modification to the key code region while maintaining the overall functionality of the code.

8. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 1, characterized in that, The arbitration detection results are obtained by inputting the counterfactual code variant and the original source code into the dedicated vulnerability detection model, respectively, and obtaining the following: The original source code and the counterfactual code variant are respectively input into the dedicated vulnerability detection model to obtain the original prediction result and the variant prediction result; Based on the comparison between the original prediction result and the variant prediction result, determine whether the prediction result has been reversed; The arbitration detection result is obtained based on the determination of whether the prediction result has been reversed.

9. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 8, characterized in that, Based on the determination of whether the prediction result has been reversed, the arbitration detection result is obtained as follows: If the prediction result is reversed, the second vulnerability judgment result shall be adopted as the final judgment result; If the prediction result does not reverse, the first vulnerability judgment result shall be adopted as the final judgment result.

10. The code vulnerability arbitration detection method integrating attention mechanism and counterfactual reasoning according to claim 8, characterized in that, After inputting the original source code and the counterfactual code variant into the dedicated vulnerability detection model to obtain the original prediction result and the variant prediction result, the process further includes: Based on the confidence values ​​of the original prediction results and the variant prediction results, determine whether both are close to the decision threshold; Based on the judgment result, if the confidence values ​​of both the original prediction result and the variant prediction result are close to the decision threshold, a high uncertainty label is added.

Images