A vulnerability localization method based on a large model

Abstract

A vulnerability localization method based on a large model is characterized by employing the LIVL tool for vulnerability localization, with the following steps: S1, using an information aggregator and syntax highlighter to enhance vulnerability localization; S2, using an out-of-domain detection algorithm to bridge the gap between in-domain and out-of-domain vulnerability localization; S3, making decisions based on in-domain and out-of-domain data. This invention can improve the accuracy of vulnerability detection, enhance out-of-domain data processing capabilities, improve model generalization ability, and reduce the possibility of false positives and false negatives.

Description

Technical Field

[0001] This invention provides a vulnerability localization method based on a large model, belonging to the field of computer program security technology. Background Technology

[0002] Software vulnerabilities are potential flaws that attackers or malicious actors can exploit to perform malicious operations, thereby compromising the security of systems and applications. Due to the significant impact of software vulnerabilities on software security, and their potential for substantial economic losses, the rapid and effective location and remediation of security vulnerabilities is crucial. Vulnerability localization techniques, by precisely identifying the areas of source code that require modification, help developers efficiently debug and fix vulnerabilities. In recent years, the number of disclosed vulnerabilities has been steadily increasing, attracting widespread attention from academia and industry. These threats urgently require advanced automated tools and technologies to effectively locate and remediate vulnerable code, reducing human intervention.

[0003] Existing fault localization methods, such as spectrum-based and mutation-based techniques, primarily determine fault locations by analyzing the execution trajectories of a large number of passed and failed test cases. However, a sufficient number of high-quality test cases are often unavailable. Furthermore, for observable vulnerabilities, obtaining failed tests capable of triggering the vulnerability (i.e., exploitation) is also a challenge. To avoid relying on test cases, another direction for fault localization is based on machine learning and deep learning techniques. Learning-based localization techniques have gained increasing attention in recent years by leveraging the capabilities of deep learning in automatic feature extraction and software semantic interpretation. Specifically, learning-based localization methods predict the likelihood of vulnerabilities in code segments by utilizing features generated by models trained on defect datasets.

[0004] Currently, the most effective vulnerability localization technique is based on large models. By leveraging the capabilities of large models in automatic feature extraction and deep semantic interpretation of code, large model-based localization techniques have received increasing attention in recent years.

[0005] LLMAO is a fault localization method that combines a large language model (LLM) with a bidirectional attention mechanism, aiming to automatically locate vulnerable lines in code without requiring test coverage information. Existing vulnerability localization methods based on large models have several problems: First, the sensitivity of security data leads to limited labeled data related to vulnerabilities, affecting the model's generalization ability in real-world scenarios, and fine-tuning large models is costly and prone to catastrophic amnesia; second, existing methods lack sufficient understanding of complex dependencies in programs, making it difficult to accurately capture the key details of vulnerabilities, and failing to effectively distinguish the contribution of different statement tags to the vulnerability; finally, the model sometimes generates seemingly plausible but actually erroneous results, increasing the burden of manual verification and reducing the tool's practicality. Summary of the Invention

[0006] The purpose of this invention is to propose a novel vulnerability localization method that addresses key issues in existing methods, such as insufficient training data, inadequate understanding of program semantics, and hallucinatory output. This invention enhances the accuracy of vulnerability localization and its ability to handle unknown data through three core modules: an information aggregator, a syntax highlighter, and an out-of-domain (OOD) detector. The highlighter and aggregator are used to gain a deeper understanding of program semantics and dependencies in the code, improving the performance of domain-specific fine-tuning models. Simultaneously, the OOD detector identifies out-of-domain data, reducing false positives and hallucinatory output, thereby achieving more efficient vulnerability localization and remediation in various scenarios.

[0007] A vulnerability localization method based on a large model is proposed, which employs LIVL, a tool for vulnerability localization. LIVL includes two main modules: an enhanced vulnerability localization Aggregator and a Highlighter, as well as an algorithm module for detecting out-of-distribution data.

[0008] Includes the following steps:

[0009] S1. Enhance vulnerability localization using information aggregators and syntax highlighters. First, LIVL extracts hidden states containing semantic information from a large language model, inputting tokenized source code. Then, an information aggregator integrates information from all code elements in a single line of code, while a syntax highlighter highlights key code elements to determine if the line contains a vulnerability.

[0010] S2. An out-of-distribution detection algorithm is used to bridge the gap between in-domain and out-of-domain vulnerability localization. During the inference phase, LIVL employs an out-of-distribution detector to assess whether a vulnerability falls within the known range of the fine-tuning model, i.e., whether it is an in-distribution vulnerability. This detector is evaluated using the KNN method to analyze the enhanced row-level representations obtained in the previously fine-tuned training dataset. Based on this evaluation result, LIVL guides the decision between using the enhanced fine-tuned model or a pre-trained large language model for prediction.

[0011] S3. Decisions are made on in-distribution and out-of-distribution data. If the vulnerability function falls within a consistent distribution range, LIVL will let the fine-tuning model handle it and ultimately generate a vulnerability probability for each row. Otherwise, for out-of-distribution data, LIVL will directly use a large language model combined with thought chains for reasoning.

[0012] Further:

[0013] The specific method for aggregating vulnerability features using an information aggregator in step S1 is as follows:

[0014] First, an aggregator module is used to aggregate code features at the line level, thereby obtaining a deep semantic representation of each line of code. The aggregator consists of two main components: an intra-statement encoder and an inter-statement encoder. The intra-statement encoder captures the dependencies between elements within a line of code and integrates this information into an enhanced line-level representation, while the inter-statement encoder further processes the contextual dependencies between lines, ensuring that cross-line semantic information is effectively captured.

[0015] The aggregation process of LIVL can be simplified into the following three key steps:

[0016] (1) Code segmentation and hidden state extraction: LIVL first segments the code in function S, and then inputs the segmented tokens into a large language model (LLM) to generate hidden states. To extract the most representative code features, LIVL selects the second-to-last layer of hidden states. The goal is to extract the hidden states of all tokens in each line of code from the LLM, process them uniformly, retain a maximum of 64 tokens, and pad the insufficient parts to ensure that the feature length of each line of code is consistent.

[0017] (2) Line-level and Cross-line Feature Aggregation: After obtaining the initial features of each line of code, LIVL begins feature aggregation using an aggregator. First, an intra-statement encoder is used to process the dependencies between elements within each line of code, generating an enhanced line-level representation. Then, an inter-statement encoder is used to capture cross-line dependencies, especially potential vulnerability issues that may arise across lines. This process ensures the integrity of the code semantics and captures potential vulnerability factors both within and across lines.

[0018] (3) Context Aggregation and Line-Level Representation Generation: Finally, LIVL further aggregates cross-line contextual information through a standard Transformer layer, using a bidirectional attention mechanism to simultaneously capture forward and backward dependencies in the code, overcoming the limitations of causal attention in LLM. After encoding, LIVL performs average pooling on the features of each line of code to generate the final line-level representation. These line-level representations provide deep semantic information for each line of code, ultimately used to detect potential vulnerabilities in the code.

[0019] The specific method for obtaining key vulnerability information using the syntax highlighter in step S1 is as follows:

[0020] By filtering and highlighting tokens that are directly related to the vulnerability, interference can be reduced and the accuracy of vulnerability detection can be improved.

[0021] The syntax highlighter consists of two steps: decisive feature filtering and decisive token filtering.

[0022] (1) Deterministic Feature Screening:

[0023] In this phase, LIVL employs an adaptive domain mask specifically designed to filter out decisive token features and eliminate code elements irrelevant to the vulnerability. Specifically, LIVL applies an adaptive mask to the hidden state of each token. This learnable mask filters out invalid features while retaining those decisive features relevant to the vulnerability. Following feature filtering within the token, LIVL further filters among tokens using an attention mechanism. This step operates across the token set to select the most decisive token.

[0024] To remove redundant and irrelevant values, a learnable adaptive mask is designed to identify and prune these invalid features. For each token's hidden state, this mask is adaptively learned, using binary elements to filter out unnecessary information. The mask is multiplied element-wise with the token's feature representation; zero values ​​in the mask filter out spurious features in the token embedding, while non-zero values ​​highlight key vulnerability-related features.

[0025] (2) Decisive Token Screening:

[0026] LIVL processes the feature representation of each token obtained in the previous step. First, the hidden states of these tokens are used as key vectors. Then, the learned mask vectors are used as query vectors to calculate the correlation between each token and the vulnerability. In this way, LIVL generates an attention score to measure the connection between each token and vulnerability-related knowledge. This score represents the magnitude of each token's contribution to vulnerability detection. For each vulnerability type, LIVL sets only one specific mask, which represents the domain characteristics of that type of vulnerability. These scores are ultimately used to calculate the contribution of each token to the vulnerability. LIVL obtains the highlighted token feature representation by multiplying these scores by the token's feature representation, i.e., those tokens directly related to the vulnerability.

[0027] The specific method for detecting out-of-domain data in step S2 is as follows:

[0028] During the testing phase, LIVL uses the K-Nearest Neighbors (KNN) algorithm to determine whether a given code snippet belongs to out-of-domain data. The specific process is as follows: LIVL first selects the most likely lines of code to cause vulnerabilities from the test data and extracts their features. To achieve this determination, LIVL first calculates the distance between the normalized feature vector of the code line and the "in-domain data" in the training set. Then, LIVL sorts these vectors in ascending order of distance and, based on a pre-defined K value, finds the K nearest neighbor vectors to the test code snippet. If the furthest distance among these K nearest neighbor vectors exceeds a preset threshold, then the code snippet is classified as out-of-domain data. LIVL's OOD detection process consists of the following steps: First, during the training phase, the distribution of in-domain data is determined. Then, feature averaging and normalization are performed to allow direct comparison of the feature vectors of the training and test data. Then, during the testing phase, the KNN algorithm is used to calculate the distance between the test sample and the training data, and finally, a threshold is used to determine whether the test sample is classified as out-of-domain data.

[0029] The specific method for locating the target based on the detection results in step S3 is as follows:

[0030] (1) Domain differentiation:

[0031] By calculating the distance between a vulnerability sample and the training set data, it is determined whether the sample belongs to out-of-domain data. If the distance of a sample exceeds a preset threshold, it means that the sample exceeds the knowledge boundary of the model. In this case, LIVL feeds these detected out-of-domain samples, i.e., instances that the fine-tuning model has not seen, into a pre-trained large language model (LLM) for further processing and vulnerability localization.

[0032] (2) In-domain location: For in-domain data, LIVL directly uses the enhanced fine-tuning model to locate vulnerabilities. Specifically, LIVL generates a probability value containing the possibility of a vulnerability for each line of code, and then sorts these probability values ​​in descending order, selecting the top N lines as the locations where vulnerabilities are most likely to exist.

[0033] (3) External location: When the model encounters external data, LIVL uses the Chained Thinking CoT hint technology to guide the LLM to think about and analyze the code, thereby locating the vulnerability.

[0034] In terms of implementation, firstly, activate the vulnerability identification capabilities of the LLM, allowing it to focus on code features related to vulnerabilities. Next, design a more granular hint for the LLM to help it further refine the vulnerability localization process.

[0035] The beneficial effects of the technical solution of this invention are as follows:

[0036] 1. Improve the accuracy of vulnerability detection;

[0037] LIVL comprehensively captures semantic information within and across lines of code through aggregators and syntax highlighters, and filters out key tokens directly related to vulnerabilities, significantly improving the accuracy of vulnerability localization. It effectively filters redundant information using an adaptive masking mechanism to avoid false positives, while ensuring the model focuses on the core factors contributing to the vulnerability.

[0038] 2. Enhance the ability to process data outside the domain;

[0039] LIVL effectively distinguishes between in-domain and out-of-domain data through its OOD detection algorithm. For out-of-domain samples, LIVL utilizes a pre-trained large-scale language model for analysis, combined with chain-of-thought technology, significantly improving the model's ability to handle unknown vulnerability types. This mechanism reduces false positives and false negatives, increasing the detection coverage for novel vulnerabilities.

[0040] 3. Improved model generalization ability;

[0041] By combining pre-trained large-scale language models with fine-tuned models, LIVL exhibits greater adaptability and generalization capabilities when dealing with vulnerabilities in different programming languages ​​and application scenarios. This enables LIVL to be widely applied to cross-domain vulnerability detection tasks, rather than being limited to known distributions in the training data.

[0042] 4. Reduce the possibility of false alarms and misjudgments;

[0043] Traditional vulnerability detection models are easily influenced by irrelevant code snippets or frequently used functions, leading to a high false positive rate. LIVL, however, uses syntax highlighters and out-of-distribution detection algorithms to focus on critical code elements, avoiding misclassifying irrelevant code as vulnerabilities and significantly reducing the false positive rate. Attached Figure Description

[0044] Figure 1 This is a schematic diagram of the solution architecture of the present invention. Detailed Implementation

[0045] This invention provides a vulnerability localization method based on a large model, which employs LIVL, a tool for vulnerability localization. The core workflow framework is as follows: Figure 1 As shown, LIVL comprises two main modules: an enhanced vulnerability location accelerator and a highlighter, as well as an algorithm for detecting out-of-distribution data. The vulnerability location process of this invention can be mainly divided into the following three steps:

[0046] S1. Enhance vulnerability localization using information aggregators and syntax highlighters. First, LIVL extracts hidden states containing semantic information from a large language model, inputting tokenized source code. Then, an information aggregator integrates information from all code elements in a single line of code, while a syntax highlighter highlights key code elements to determine if the line contains a vulnerability.

[0047] S2. An out-of-distribution detection algorithm is used to bridge the gap between in-domain and out-of-domain vulnerability localization. During the inference phase, LIVL employs an out-of-distribution detector to assess whether a vulnerability falls within the known range of the fine-tuning model, i.e., whether it is an in-distribution vulnerability. This detector is evaluated using the KNN method to analyze the enhanced row-level representations obtained in the previously fine-tuned training dataset. Based on this evaluation result, LIVL guides the decision between using the enhanced fine-tuned model or a pre-trained large language model for prediction.

[0048] S3. Decisions are made on in-distribution and out-of-distribution data. If the vulnerability function falls within a consistent distribution range, LIVL will let the fine-tuning model handle it and ultimately generate vulnerability probabilities for each row. Otherwise, for out-of-distribution data, LIVL will directly use the large language model and combine it with chain-of-thought for reasoning.

[0049] This method demonstrates in detail how to improve the accuracy of vulnerability localization by integrating code information and distributing internal and external assessments.

[0050] The specific method for aggregating vulnerability features using an information aggregator in step S1 is as follows:

[0051] First, this invention uses an aggregator module to aggregate code features at the line level, thereby obtaining a deep semantic representation of each line of code. The aggregator consists of two main components: an intra-statement encoder and an inter-statement encoder. The intra-statement encoder captures the dependencies between elements within a line of code and integrates this information into an enhanced line-level representation, while the inter-statement encoder further processes the contextual dependencies between lines, ensuring that cross-line semantic information is effectively captured.

[0052] The aggregation process of LIVL can be simplified into the following three key steps:

[0053] (1) Code segmentation and hidden state extraction: LIVL first segments the code in function S, and then inputs the segmented tokens into a large language model (LLM) to generate hidden states. To extract the most representative code features, LIVL selects the second-to-last layer of hidden states. The goal of this step is to extract the hidden states of all tokens in each line of code from the LLM, process them uniformly, retain a maximum of 64 tokens, and pad the insufficient parts to ensure that the feature length of each line of code is consistent.

[0054] (2) Line-level and Cross-line Feature Aggregation: After obtaining the initial features of each line of code, LIVL begins feature aggregation using an aggregator. First, an intra-statement encoder is used to process the dependencies between elements within each line of code, generating an enhanced line-level representation. Then, an inter-statement encoder is used to capture cross-line dependencies, especially potential vulnerability issues that may arise across lines. This process ensures the integrity of the code semantics and captures potential vulnerability factors both within and across lines.

[0055] (3) Context Aggregation and Line-Level Representation Generation: Finally, LIVL further aggregates cross-line contextual information through a standard Transformer layer, using a bidirectional attention mechanism to simultaneously capture forward and backward dependencies in the code, overcoming the limitations of causal attention in LLM. After encoding, LIVL performs average pooling on the features of each line of code to generate the final line-level representation. These line-level representations provide deep semantic information for each line of code, ultimately used to detect potential vulnerabilities in the code.

[0056] These three steps are closely integrated, realizing a complete process from code segmentation to feature aggregation, and then to line-level representation generation.

[0057] The specific method for obtaining key vulnerability information using the syntax highlighter in step S1 is as follows:

[0058] While LIVL's aggregation module can effectively integrate global semantic information at the line-of-code level, it struggles to retain all details without losing crucial information when faced with a large number of tokens per line of code. The sheer number of tokens in the code is enormous, and not every token is critical for vulnerability detection; many tokens are actually irrelevant and may even interfere with the model's analysis. Therefore, LIVL further processes tokens by filtering and highlighting those directly related to vulnerabilities to reduce interference and improve the accuracy of vulnerability detection.

[0059] For example, in the code snippet `sprintf(buffer,"Number:%d",number);`, the actual reason for the potential vulnerability is that the size of `number` exceeds the memory allocation space of `buffer`. However, during the learning process, the model might make an incorrect judgment due to the high frequency of the `sprintf` function, assuming that any line of code using the `sprintf` function might contain a vulnerability. This illustrates that the model is easily influenced by irrelevant factors during the learning process, misjudging unrelated code snippets (such as `sprintf` itself) as vulnerabilities. Therefore, to avoid this situation, LIVL needs to further filter tokens, suppressing code elements unrelated to vulnerabilities and retaining only those tokens closely related to vulnerabilities for analysis.

[0060] The syntax highlighter consists of two steps: decisive feature filtering and decisive token filtering.

[0061] Decisive Feature Filtering: In this stage, LIVL employs an adaptive domain mask specifically designed to filter out decisive token features, further eliminating code elements irrelevant to the vulnerability. Specifically, LIVL applies an adaptive mask to the hidden state of each token. This learnable mask filters out invalid features while retaining the decisive features relevant to the vulnerability. Following feature filtering within the token, LIVL further filters among tokens using an attention mechanism. This step operates across the token set to select the most decisive token.

[0062] Due to the inherent complexity of code—for example, the same identifier may have multiple meanings—LIVL models are prone to generating spurious features during the learning process, resulting in output vectors containing a large number of redundant and irrelevant values. This redundant information affects the model's judgment, thus requiring a specific mechanism to remove it. LIVL's solution is to design a learnable adaptive mask to identify and prune these invalid features. For the hidden state of each token, this mask adaptively learns, using binary elements (0 and non-zero values) to filter out unnecessary information. In this way, LIVL ensures that invalid features are removed while important, decisive features are preserved.

[0063] This mask is created by element-wise multiplying with the token feature representation. Zero values ​​in the mask effectively filter out spurious features in the token embedding, while non-zero values ​​highlight key vulnerability-related features. Through this adaptive masking mechanism, LIVL can not only effectively remove redundant information but also retain features crucial for vulnerability detection, thereby improving the model's ability and accuracy in identifying vulnerabilities.

[0064] Decisive Token Filtering: In this step, LIVL processes the feature representation of each token obtained in the previous step. First, the hidden states of these tokens are used as key vectors. Then, the learned mask vectors are used as query vectors to calculate the correlation between each token and the vulnerability. In this way, LIVL generates an attention score to measure the connection between each token and vulnerability-related knowledge. This score represents the magnitude of each token's contribution to vulnerability detection. For each vulnerability type, LIVL sets only one specific mask, which represents the domain characteristics of that type of vulnerability. The masking in this process is crucial because it ensures that the model can automatically identify and suppress code tokens unrelated to the vulnerability, allowing the model to focus more on code snippets that may lead to the vulnerability.

[0065] These scores are ultimately used to calculate the contribution of each token to the vulnerability. LIVL multiplies these scores with the token's feature representation to obtain the highlighted token feature representation, i.e., those tokens directly related to the vulnerability. Through this process, the model can focus its attention on the most relevant code elements, thereby improving the accuracy of vulnerability detection. This method not only avoids the model missing important information when processing large amounts of code, but also ensures that the model can accurately identify the key code segments where the vulnerability occurs.

[0066] The specific method for detecting out-of-domain data in step S2 is as follows:

[0067] The primary goal of out-of-domain data detection is to identify code snippets that fall outside the known distribution range, preventing false positives from fine-tuned models when faced with these unknown inputs. LIVL's OOD detection method differs from traditional methods. Traditional methods typically rely solely on the model's final output features, while LIVL comprehensively utilizes the output of a large language model and aggregated row-level features to detect OOD samples. Because LLM's output is pre-trained on a large code corpus, it captures a broader range of code semantic information, while the aggregated features contain more characteristics relevant to specific vulnerabilities. Therefore, LIVL can capture information from the code at multiple levels, better identifying out-of-domain samples.

[0068] During the training phase, LIVL first determines the distribution of "in-domain data" using the training data, which will be used as a benchmark in subsequent OOD detection. For each statement representation generated by LLM, LIVL generates a vector representing that statement by calculating the average of each token feature. This averaging process not only preserves maximum information but also ensures a consistent shape with the aggregated row-level features, facilitating subsequent comparisons. Next, LIVL normalizes each feature vector, and the normalized vector is used to calculate the distance between the training and test data. In this way, the model can more effectively identify the distribution of the training data, thereby distinguishing which code snippets are in-domain data and which are out-of-domain data in subsequent detection.

[0069] During the testing phase, LIVL uses the K-Nearest Neighbors (KNN) algorithm to determine whether a given code snippet belongs to out-of-domain data. Specifically, LIVL first selects the line of code most likely to cause a vulnerability from the test data and extracts its features. This step is crucial because models typically have a certain level of confidence when detecting vulnerabilities, and the line of code most likely to cause a vulnerability forms the basis for the model's most confident judgment. If the feature vector of that line of code is identified as out-of-domain data, it can be inferred that the function as a whole represents an unknown code type that the model has not seen before.

[0070] To achieve this judgment, LIVL first calculates the distance between the normalized feature vector of the code line and the "in-domain data" in the training set. Then, LIVL sorts these vectors by distance from smallest to largest and finds the K nearest neighbor vectors to the test code snippet based on a pre-defined K value. If the furthest distance among these K nearest neighbor vectors exceeds a preset threshold (λ), the code snippet is classified as out-of-domain data. This means the model considers the code snippet to be outside the distribution range it has seen during training, belonging to a code type or structure the model has not yet encountered, thus avoiding false positives for these unknown inputs. The threshold λ here does not depend on the inherent distribution characteristics of the in-domain data but is set empirically or by other methods. When the distance between a test code snippet and the in-domain data exceeds this threshold, LIVL classifies it as out-of-domain data. In this way, LIVL can effectively detect code snippets that do not belong to the training distribution range, avoiding overfitting or false positives when the model encounters out-of-domain data, thereby improving the accuracy and robustness of vulnerability detection.

[0071] In summary, the LIVL OOD detection process consists of the following main steps: First, during the training phase, the distribution of data within the domain is determined. Then, feature averaging and normalization are performed to allow direct comparison of the feature vectors of the training and test data. Next, during the testing phase, the K-nearest neighbor algorithm is used to calculate the distance between the test sample and the training data. Finally, a threshold is used to determine whether the test sample should be classified as out-of-domain data. This method fully leverages LLM's broad semantic understanding capabilities and precise aggregation of row-level features, ensuring that the model can accurately identify unknown code fragments, thereby avoiding incorrect judgments by the model regarding unknown inputs.

[0072] The specific method for locating the target based on the detection results in step S3 is as follows:

[0073] This section will explain in detail how to apply the large model to vulnerability localization tasks both within and outside the domain based on the detection results from the previous step, thereby achieving more accurate alignment.

[0074] (1) Domain Differentiation: The fine-tuned model of this invention is enhanced through aggregators and highlighters, but due to the limited size of the training dataset, this model faces certain limitations in practical applications. Although the fine-tuned model demonstrates a certain generalization ability, its knowledge is still limited, confined to the known distribution in the training set. Therefore, the OOD (Out-of-Domain Detection) mechanism of this invention is designed based on this. This invention determines whether a vulnerability sample belongs to out-of-domain data by calculating the distance between the vulnerability sample and the data in the training set. If the distance of a sample exceeds a preset threshold, it indicates that the sample exceeds the knowledge boundary of the model. In this case, LIVL will hand over these detected out-of-domain samples, i.e., instances not seen by the fine-tuned model, to a pre-trained large language model (LLM) for further processing and vulnerability localization.

[0075] For vulnerability types unfamiliar to fine-tuning models, accurate localization is difficult due to their limited generalization capabilities. While these pre-trained LLMs are not experts in any specific domain, as general-purpose models, they possess extensive code knowledge and can handle vulnerability samples outside the training set distribution to some extent. Therefore, directly relying on pre-trained LLMs to handle data exceeding the capabilities of fine-tuning models is a reasonable and effective solution. Because pre-trained LLMs have already undergone extensive learning on a large amount of code, they can apply zero-shot learning capabilities to analyze new types of vulnerabilities.

[0076] (2) In-domain location: For in-domain data, since it falls within the model's knowledge range, LIVL can directly use the enhanced, fine-tuned model to locate vulnerabilities. Specifically, LIVL generates a probability value containing the likelihood of a vulnerability for each line of code, then sorts these probability values ​​in descending order, selecting the top N lines as the most likely locations of vulnerabilities. This probability-based sorting method allows the model to focus on the lines of code most likely to have problems, thereby improving the accuracy of vulnerability location. This method is highly efficient because it can quickly filter out the lines of code with the highest vulnerability risk and prioritize their analysis, saving time and computational resources.

[0077] (3) Out-of-Domain Localization: When the model encounters out-of-domain data, this invention relies on the powerful zero-shot learning and code understanding capabilities demonstrated by current large-scale language models to complete the vulnerability localization task. Although these pre-trained models have not undergone domain-specific fine-tuning, they still possess a certain degree of versatility when handling code beyond the training scope of the fine-tuned models. To leverage this capability of LLM, LIVL employs Chain-of-Thought (CoT) hinting technology. Through this technology, this invention can guide LLM to think about and analyze the code step by step, thereby locating vulnerabilities.

[0078] In terms of specific implementation, firstly, this invention provides the LLM with a system prompt: "You are an expert in the vulnerability field and can understand the deep semantic information of the program." This step aims to activate the LLM's vulnerability identification capabilities, allowing it to focus on code features related to vulnerabilities. Next, this invention designs a more granular prompt for the LLM to help it further refine the vulnerability localization process. The prompt is roughly as follows: "Analyze the security vulnerabilities in the provided code snippet and respond in the form of a JSON object. This object should contain two keys: 'Functionality' (describing the function of the code) and 'Vulnerable Positions' (a list containing five JSON objects). Each object in 'Vulnerable Positions' should contain 'lineNumber' (showing the line where the potential problem is located) and 'reason' (explaining in detail why this part of the code is considered vulnerable). Think step by step."

[0079] Through this chain-thinking approach, LLM can systematically analyze vulnerabilities in complex code, providing detailed information on vulnerability locations and causes. This method not only fully leverages LLM's strengths in understanding code semantics but also ensures that it provides a reasonable analysis process when facing new types of vulnerabilities, helping developers understand and fix problems.

[0080] LIVL effectively distinguishes between vulnerability samples from within and outside a vulnerability domain through a domain differentiation mechanism, and employs different processing strategies for different types of data. For within-domain data, LIVL uses a finely tuned augmented model for efficient vulnerability localization; for outside-domain data, LIVL relies on the zero-shot learning capability of a pre-trained large-scale language model, guiding the model step-by-step through chain-like hints to analyze vulnerabilities. This two-pronged strategy ensures that the model exhibits efficient and accurate detection and localization capabilities when handling both known and unknown vulnerabilities. This process not only improves the coverage of vulnerability detection but also significantly reduces false positives and false negatives caused by outside-domain samples.

[0081] In terms of detecting data outside the domain, the method used in this invention is based on the nearest neighbor algorithm. Algorithms such as Mahalanobis distance can achieve similar results, but the analysis results are not as accurate as those of the nearest neighbor algorithm.

Claims

1. A vulnerability localization method based on a large model, characterized in that, The tool LIVL for vulnerability localization was adopted. LIVL includes two main modules: an enhanced vulnerability localization information aggregator and syntax highlighter, and an algorithm module for detecting out-of-distribution data. The method includes the following steps: S1. Enhance vulnerability localization using information aggregator and syntax highlighter; First, LIVL extracts hidden states containing semantic information from the large language model by inputting the segmented source code; then, the information aggregator is used to integrate all code element information in a single line of code, and the syntax highlighter is responsible for highlighting key code elements to determine whether the line contains a vulnerability. S2. Out-of-distribution detection algorithms are used to bridge the gap between in-domain and out-of-domain vulnerability localization. During the inference phase, LIVL designs an out-of-distribution detector to evaluate whether a vulnerability falls within the known range of the fine-tuning model, i.e., whether it is an in-distribution vulnerability. This detector is evaluated by analyzing the enhanced row-level representations obtained in the fine-tuning training dataset using the KNN method. Based on the evaluation results, LIVL will guide whether to use the enhanced fine-tuning model or the pre-trained large language model for prediction. S3. Make decisions on in-distribution and out-of-distribution data; if the vulnerability function falls within a consistent distribution range, LIVL will hand it over to the fine-tuning model and ultimately generate vulnerability probabilities for each row; otherwise, for out-of-distribution data, LIVL will directly use the large language model and combine it with thought chains for reasoning.

2. The vulnerability localization method based on a large model according to claim 1, characterized in that, The specific method for aggregating vulnerability features using an information aggregator in step S1 is as follows: First, the aggregator module is used to aggregate code features at the line level to obtain a deep semantic representation of each line of code. The aggregator consists of two main components: an intra-statement encoder and an inter-statement encoder. The intra-statement encoder is used to capture the dependencies between elements within a line of code and integrate this information into an enhanced line-level representation, while the inter-statement encoder further processes the contextual dependencies between lines to ensure that cross-line semantic information is effectively captured. The aggregation process of LIVL can be simplified into the following three key steps: (1) Code segmentation and hidden state extraction: LIVL first segments the code in the vulnerability function, and then inputs the segmented tokens into a large language model LLM to generate hidden states; in order to extract the most representative code features, LIVL selects the second to last hidden state; the goal is to extract the hidden states of all tokens in each line of code from the LLM, process them uniformly, retain a maximum of 64 tokens, and fill in the insufficient parts to ensure that the feature length of each line of code is consistent; (2) Line-level and cross-line feature aggregation: After obtaining the initial features of each line of code, LIVL begins to aggregate features through an aggregator; first, the intra-statement encoder is used to process the dependencies between elements within each line of code to generate an enhanced line-level representation; then, the inter-statement encoder is used to capture cross-line dependencies. (3) Context aggregation and line-level representation generation: Finally, LIVL further aggregates cross-line context information through a standard Transformer layer and uses a bidirectional attention mechanism to capture forward and backward dependencies in the code at the same time, overcoming the limitations of causal attention in LLM. After the encoding is completed, LIVL performs average pooling on the features of each line of code to generate the final line-level representation. These line-level representations provide deep semantic information for each line of code, which is ultimately used to detect potential vulnerabilities in the code.

3. The vulnerability localization method based on a large model according to claim 1, characterized in that, The specific method for the syntax highlighter to obtain key vulnerability information in step S1 is as follows: by filtering and highlighting those tokens that are directly related to the vulnerability, interference can be reduced and the accuracy of vulnerability detection can be improved. The syntax highlighter consists of two steps: decisive feature filtering and decisive token filtering. (1) Deterministic feature screening: In this phase, LIVL designs an adaptive domain mask specifically for filtering out decisive token features and eliminating code elements irrelevant to the vulnerability. Specifically, LIVL applies an adaptive mask to the hidden state of each token. This mask is a learnable mask that filters out invalid features while retaining those decisive features relevant to the vulnerability. After feature filtering within the token, LIVL further filters among the tokens using an attention mechanism. This step operates on the token set to select the most decisive token. To remove redundant and irrelevant values, a learnable adaptive mask is designed to identify and prune these invalid features. For the hidden state of each token, this mask is adaptively learned, using binary elements to filter out unnecessary information. The mask is multiplied element-wise with the token feature representation. Zero values ​​in the mask filter out false features in the token embedding, while non-zero values ​​highlight key vulnerability-related features. (2) Decision token selection: LIVL processes the feature representation of each token obtained in the previous step. First, the hidden states of these tokens are used as key vectors. Then, the learned mask vectors are used as query vectors to calculate the correlation between each token and the vulnerability. In this way, LIVL generates an attention score to measure the connection between each token and vulnerability-related knowledge. This score represents the contribution of each token to vulnerability detection. For each vulnerability type, LIVL sets only one specific mask, which represents the domain characteristics of that type of vulnerability. These scores are ultimately used to calculate the contribution of each token to the vulnerability. LIVL multiplies these scores with the token feature representations to obtain the highlighted token feature representations, i.e., those tokens that are directly related to the vulnerability.

4. The vulnerability localization method based on a large model according to claim 1, characterized in that, The specific method for detecting out-of-domain data in step S2 is as follows: During the testing phase, LIVL uses the K-Nearest Neighbors (KNN) algorithm to determine whether a given code snippet belongs to out-of-domain data. Specifically, LIVL first selects the code lines most likely to cause vulnerabilities from the test data and extracts their features. To achieve this judgment, LIVL first calculates the distance between the normalized feature vector of the code line and the "in-domain data" in the training set; then, LIVL sorts these vectors in ascending order of distance and, based on a pre-set K value, finds the K nearest neighbor vectors that are closest to the test code snippet; if the farthest distance among these K nearest neighbor vectors exceeds a preset threshold, ... If so, the code snippet will be classified as out-of-domain data. The LIVL OOD detection process consists of the following steps: First, the distribution of in-domain data is determined during the training phase. Then, feature averaging and normalization are performed to make the feature vectors of the training data and the test data directly comparable. Then, during the testing phase, the K-nearest neighbor algorithm is used to calculate the distance between the test sample and the training data. Finally, a threshold is used to determine whether the test sample is classified as out-of-domain data.

5. The vulnerability localization method based on a large model according to claim 1, characterized in that, The specific method for locating the target based on the detection results in step S3 is as follows: (1) Domain differentiation: By calculating the distance between the vulnerability sample and the training set data, it is determined whether it belongs to the outside domain data. If the distance of a sample exceeds a preset threshold, it means that the sample exceeds the knowledge boundary of the model. In this case, LIVL will hand over these detected outside domain samples, that is, instances that the fine-tuning model has not seen, to the pre-trained large language model LLM for further processing and vulnerability localization. (2) Intra-domain location: For intra-domain data, LIVL directly uses the enhanced fine-tuning model to locate vulnerabilities; specifically, LIVL generates a probability value containing the possibility of a vulnerability for each line of code, and then sorts these probability values ​​in descending order, selecting the first N lines as the most likely locations where vulnerabilities exist. (3) External location: When the model encounters external data, LIVL uses the Chained Thinking CoT hint technology to guide the LLM to think about and analyze the code, thereby locating the vulnerability; First, activate the vulnerability identification capabilities of the LLM, allowing it to focus on code features related to vulnerabilities; then, design a more granular hint for the LLM to help it further refine the vulnerability localization process.

Images