Source code vulnerability detection method and device based on instruction perception, electronic equipment and storage medium

Abstract

The invention discloses a source code vulnerability detection method and device based on instruction perception, electronic equipment and a storage medium, and belongs to the technical field of vulnerability analysis, the method comprises the following steps: obtaining a to-be-detected source code, and splicing a preset task instruction prefix with the source code to generate a task perception input sequence; inputting the task perception input sequence into a preset vulnerability feature extraction model to generate a source code semantic feature vector; based on the source code semantic feature vector, historical vulnerability vectors with the similarity meeting a preset condition are retrieved from a preset vulnerability knowledge base, and a corresponding vulnerability mode label and a historical vulnerability analysis report are obtained; and constructing a detection cue word context according to the task perception input sequence, the vulnerability mode tag and a historical vulnerability analysis report, and inputting the detection cue word context into a preset generative large language model to generate a source code vulnerability detection result. By implementing the method, the problem that code vulnerability detection is not accurate enough in the prior art can be solved.

Description

Technical Field

[0001] This invention relates to the field of vulnerability analysis technology, specifically to a source code vulnerability detection method, device, electronic device, and storage medium based on instruction awareness. Background Technology

[0002] Source code vulnerability detection is a core component of ensuring software supply chain security and preventing cyberattacks. With the explosive growth in the scale of software systems, manual code auditing is no longer sufficient to efficiently identify potential security risks. Automated vulnerability detection technology can quickly identify known vulnerability patterns in large-scale code corpora, thereby reducing remediation costs early in the software development lifecycle and preventing significant economic losses and privacy breaches caused by malicious exploitation of vulnerabilities. This is of paramount importance for building a robust cybersecurity protection system.

[0003] However, existing code vulnerability detection methods still face the problem of insufficient accuracy in application. Current detection processes lack guidance on how the code under test is processed under specific detection intent when acquiring reference information. When faced with different security audit objectives, the detection system cannot dynamically adjust the focus of code feature extraction according to specific detection instructions. This results in historical data recalled during the retrieval phase showing similarities only at the code appearance or general functional level, but being completely misaligned with the underlying security logic and the intended test. This lack of task awareness in retrieval results leads to subsequent generation stages acquiring a large amount of interfering information unrelated to the current detection intent. This not only fails to provide effective evidence for vulnerability determination but also limits the accurate analysis of vulnerability features in complex code contexts. Summary of the Invention

[0004] This invention provides a source code vulnerability detection method, apparatus, electronic device, and storage medium based on instruction awareness, which can solve the problem of inaccurate code vulnerability detection in the prior art.

[0005] One embodiment of the present invention provides a source code vulnerability detection method based on instruction awareness, comprising: Obtain the source code to be tested; The preset task instruction prefix and the source code to be detected are concatenated to generate a task-aware input sequence; wherein, the task instruction prefix is ​​natural language text used to guide the generation of semantic feature vectors of the source code; The task-aware input sequence is input into a preset vulnerability feature extraction model so that the vulnerability feature extraction model generates source code semantic feature vectors based on the task-aware input sequence. Based on the semantic feature vector of the source code, historical vulnerability vectors that meet the preset conditions of similarity with the semantic feature vector of the source code are retrieved from the preset vulnerability knowledge base and used as candidate vulnerability vectors. Based on each candidate vulnerability vector, vulnerability pattern tags and historical vulnerability analysis reports corresponding to each candidate vulnerability vector are extracted from a pre-set vulnerability knowledge base. Based on the task-aware input sequence, the vulnerability pattern labels corresponding to each candidate vulnerability vector, and the historical vulnerability analysis reports corresponding to each candidate vulnerability vector, a detection prompt context is constructed. The detection prompt context is then input into a preset generative large language model, so that the generative large language model generates source code vulnerability detection results based on the detection prompt context.

[0006] Furthermore, the preset task instruction prefix and the source code to be detected are concatenated to generate a task-aware input sequence, including: The preset task instruction prefix is ​​used as the guiding text and placed before the source code to be detected. A preset separator is inserted between the task instruction prefix and the source code to be detected. The text is merged according to the order of task instruction prefix, separator, and source code to be inspected to generate a task-aware input sequence.

[0007] Furthermore, the vulnerability feature extraction model is trained in the following way: Obtain a vulnerability training sample set; wherein, the vulnerability training sample set contains several vulnerability training samples; each vulnerability training sample includes query code, corresponding positive samples, and corresponding hard negative samples; For each vulnerability training sample, the preset task instruction prefix is ​​concatenated with the current query code to generate the current task-aware query sequence; the preset task instruction prefix is ​​concatenated with the current positive sample to generate the current task-aware positive sample sequence; the preset task instruction prefix is ​​concatenated with the current hard-to-bear sample to generate the current task-aware hard-to-bear sample sequence; the current task-aware query sequence, the current task-aware positive sample sequence, and the current task-aware hard-to-bear sample sequence are used as training task-aware sequence samples. Construct a training task perception sequence set based on the perception sequence samples of each training task; The training task perception sequence set is divided into several batches of training samples according to the preset batch size. Repeat the loop process until the preset number of training iterations is reached to generate a vulnerability feature extraction model; The cyclic process includes: Each training task-aware sequence sample in the current batch of training samples is sequentially input into the current vulnerability feature extraction model, and the source code semantic feature vector corresponding to each training task-aware sequence sample in the current batch of training samples is output; wherein, the source code semantic feature vector includes: the query vector corresponding to the task-aware query sequence, the positive sample vector corresponding to the task-aware positive sample sequence, and the hard-negative sample vector corresponding to the task-aware hard-negative sample sequence; the initial training samples of the current batch are the training samples of the first batch in the training task-aware sequence set. For each training task perception sequence sample in the current batch of training samples, the task perception query sequence and the task perception difficult sample sequence of the current training task perception sequence sample are input into the preset teacher model, so that the teacher model performs a vulnerability logic similarity score on the task perception query sequence and the task perception difficult sample sequence of the current training task perception sequence sample, and generates the penalty weight corresponding to the current training task perception sequence sample. Based on the query vector, positive sample vector, hard negative sample vector, and corresponding penalty weights of the perceptual sequence samples of each training task in the current batch of training samples, the loss function value is calculated and generated through a preset weighted loss function. The optimizer optimizes the current vulnerability feature extraction model based on the loss function value, and generates an optimized vulnerability feature extraction model. Determine whether the current number of training iterations has reached the preset number of training iterations. If yes, use the optimized vulnerability feature extraction model as the final vulnerability feature extraction model and select the next batch of training samples as the training samples for the current batch. If no, update the optimized vulnerability feature extraction model to the current vulnerability feature extraction model.

[0008] Furthermore, the hard-negative samples for any vulnerability training sample are obtained in the following way: Obtain a candidate code library; wherein each candidate code in the candidate code library has a corresponding vulnerability pattern label; the vulnerability pattern label is used to characterize the vulnerability type to which the code belongs; Based on the vulnerability pattern tags corresponding to each candidate code, target negative candidate codes with different vulnerability pattern tags from the candidate code library are selected. The query code is input into a preset general code embedding model, so that the general code embedding model generates a base semantic vector of the query code based on the query code; Each target negative candidate code is sequentially input into a preset general code embedding model, so that the general code embedding model generates a contrast semantic vector of each target negative candidate code based on each target negative candidate code; Calculate the similarity between the baseline semantic vector and the comparative semantic vector of each target negative candidate code, and generate a similarity score for each target negative candidate code; The similarity scores of the negative candidate codes for each target are sorted in descending order to generate the ranking results; Select the target negative candidate code ranked in the preset position from the sorting results as the hard negative sample.

[0009] Furthermore, positive samples for any vulnerability training sample are obtained in the following way: Based on the vulnerability pattern tags corresponding to each candidate code, target positive candidate codes with the same vulnerability pattern tags as the query code are selected from the candidate code library; Each target positive candidate code is sequentially input into a preset general code embedding model, so that the general code embedding model generates a comparative semantic vector of each target positive candidate code based on each target positive candidate code; Calculate the similarity between the baseline semantic vector and the comparative semantic vector of each target positive candidate code, and generate a similarity score for each target positive candidate code; From the positive candidate codes of each target, select the positive candidate code with the highest similarity score and a similarity of less than 1 as the positive sample.

[0010] Furthermore, the pre-designed teacher model is trained in the following ways: Obtain the teacher model training set; wherein, the teacher model training set includes multiple sets of vulnerability comparison chains, each set of vulnerability comparison chains includes a first code segment with a vulnerability, a second code segment after fixing the first code segment, and a corresponding vulnerability logic analysis report; Based on a pre-defined teacher base model, a differential analysis is performed on the first code snippet and the second code snippet to generate a vulnerability feature logic difference. Using the vulnerability feature logic difference and the vulnerability logic analysis report as a supervision signal, the pre-defined teacher base model is subjected to supervised fine-tuning to obtain a teacher model. The teacher model is used to output a penalty weight representing the logical similarity between the task-aware query sequence and the task-aware hard-to-bear sample sequence.

[0011] Furthermore, the pre-defined generative large language model is trained in the following way: Obtain generative training samples; each generative training sample includes: simulated detection instructions, code instance to be tested, retrieved similar code snippets, corresponding historical vulnerability analysis reports, and standard expert detection conclusions; The simulated detection commands, code instances to be tested, similar code snippets, and historical vulnerability analysis reports are templated and concatenated to construct an input prompt word sequence; The input prompt word sequence is used as input, and the standard expert detection conclusion is used as the target output. The preset large language model is fine-tuned through supervised learning to generate a generative large language model. The generative large language model is used to generate detection results containing vulnerability type location and vulnerability remediation suggestions based on the input task-aware input sequence.

[0012] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0013] An embodiment of the present invention provides a source code vulnerability detection device based on instruction awareness, comprising: a data acquisition module, a feature extraction module, a vector retrieval module, and a vulnerability detection module; The data acquisition module is used to acquire the source code to be detected; The feature extraction module is used to concatenate a preset task instruction prefix and the source code to be detected to generate a task-aware input sequence; wherein, the task instruction prefix is ​​natural language text used to guide the generation of source code semantic feature vectors; the task-aware input sequence is input into a preset vulnerability feature extraction model so that the vulnerability feature extraction model generates source code semantic feature vectors based on the task-aware input sequence. The vector retrieval module is used to retrieve historical vulnerability vectors that meet preset conditions in similarity to the source code semantic feature vectors from a preset vulnerability knowledge base, based on the source code semantic feature vectors, as candidate vulnerability vectors; and to extract vulnerability pattern tags and historical vulnerability analysis reports corresponding to each candidate vulnerability vector from the preset vulnerability knowledge base based on each candidate vulnerability vector. The vulnerability detection module is used to construct a detection prompt context based on the task-aware input sequence, the vulnerability pattern labels corresponding to each candidate vulnerability vector, and the historical vulnerability analysis report corresponding to each candidate vulnerability vector; and input the detection prompt context into a preset generative large language model so that the generative large language model generates source code vulnerability detection results based on the detection prompt context.

[0014] Based on the above method embodiments, the present invention provides corresponding electronic device embodiments.

[0015] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements any of the instruction-aware source code vulnerability detection methods described in the above-described method embodiments.

[0016] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments.

[0017] One embodiment of the present invention provides a storage medium storing a computer program thereon, wherein, when the computer program is running, it controls the device where the storage medium is located to execute any of the instruction-aware source code vulnerability detection methods described in the above-described method embodiments.

[0018] Compared with the prior art, the present invention has the following beneficial effects: This invention provides a source code vulnerability detection method, apparatus, electronic device, and storage medium based on instruction awareness. The method acquires the source code to be detected and concatenates a preset task instruction prefix with the source code to generate a task-aware input sequence. The task-aware input sequence is input into a preset vulnerability feature extraction model to generate a source code semantic feature vector. Based on the source code semantic feature vector, historical vulnerability vectors with similarity meeting preset conditions are retrieved from a preset vulnerability knowledge base to obtain corresponding vulnerability pattern tags and historical vulnerability analysis reports. A detection prompt context is constructed based on the task-aware input sequence, vulnerability pattern tags, and historical vulnerability analysis reports, and input into a preset generative large language model to generate source code vulnerability detection results.

[0019] This invention generates a task-aware input sequence by pre-concatenating a preset task instruction prefix before the source code to be detected. This guides the vulnerability feature extraction model to dynamically adjust the focus of feature extraction and generate task-aware semantic feature vectors, thereby ensuring accurate retrieval of historical vulnerability vectors and analysis reports that are logically aligned with specific detection intentions from the vulnerability knowledge base. Subsequently, the task-aware input sequence and the retrieved accurate evidence are used to construct a context of detection prompts, effectively eliminating interference from irrelevant information. This enables the generative large language model to generate accurate source code vulnerability detection results based on a highly correlated security logic background. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a source code vulnerability detection method based on instruction awareness, provided in an embodiment of the present invention.

[0021] Figure 2 This is a schematic diagram of the structure of a source code vulnerability detection device based on instruction awareness provided in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, to address the problem of inaccurate code vulnerability detection in existing technologies, an embodiment of the present invention provides a source code vulnerability detection method based on instruction awareness, comprising at least the following steps: Step S1: Obtain the source code to be tested.

[0024] Specifically, in this embodiment of the invention, the first step is to acquire the source code to be detected. The source code to be detected refers to computer program text that requires security analysis to identify the presence of specific vulnerability patterns. As the input for subsequent deep learning model analysis, the source code to be detected is logically equivalent to the query code in the model inference process. The source code to be detected can originate from code files on local storage devices, commit records in code version control repositories, or real-time code input streams in integrated development environments. Formally, the source code to be detected is a sequence of strings containing complete or partial control flow and data flow logic. The source code to be detected can be a function fragment containing specific functional implementations or a complete source file. After being acquired, the source code to be detected is stored in memory for subsequent processing. This step does not limit the specific programming language of the source code to be detected; it can be code text written in various programming languages ​​such as C, C++, Java, or Python. By acquiring the source code to be detected, the specific analysis object of the vulnerability detection task can be established, providing the necessary raw data foundation for subsequent instruction-aware feature vectorization and vulnerability pattern matching.

[0025] Step S2: Concatenate the preset task instruction prefix and the source code to be detected to generate a task-aware input sequence; wherein, the task instruction prefix is ​​natural language text used to guide the generation of semantic feature vectors of the source code.

[0026] In a preferred embodiment, a preset task instruction prefix and the source code to be detected are concatenated to generate a task-aware input sequence, including: The preset task instruction prefix is ​​used as the guiding text and placed before the source code to be detected. A preset separator is inserted between the task instruction prefix and the source code to be detected. The text is merged according to the order of task instruction prefix, separator, and source code to be inspected to generate a task-aware input sequence.

[0027] Specifically, the task-aware input sequence is a mixed text sequence containing natural language instructions and programming language code. The task instruction prefix is ​​natural language text used to guide the generation of semantic feature vectors from the source code. The content of the task instruction prefix is ​​set to align with the instruction-following characteristics of large language models, specifically configured as "Please find the code most relevant to the following code vulnerability patterns:" or text with the same semantic guidance function. By adding the task instruction prefix before the source code to be detected, the subsequent processing model is explicitly informed that the current coding task is not general code semantic understanding, but rather focuses on feature extraction and matching of vulnerability patterns.

[0028] It should be noted that the preset delimiter is configured as a specific character or string sequence used to explicitly define the boundary between the natural language domain and the programming language domain in the text structure. In practical applications, the specific form of the delimiter can be chosen based on the vocabulary characteristics of the preset generative large language model. One implementation involves configuring the delimiter as a standard newline character (such as "\n" or "\r\n"). By inserting a newline character, the task instruction prefix and the source code to be detected are physically isolated on the text line, conforming to the natural reading logic of source code text. This guides the model to identify the first line as a comment or instruction and subsequent lines as code text during processing.

[0029] Another implementation is that the delimiter is configured as a predefined special semantic token, such as " <sep>", "[SEP]" or "###". These special markers have independent embedding vectors in the model's word vector space, which can forcibly truncate the contextual dependencies of the preceding text and force the model to reallocate attention weights, thereby preventing the semantics of the task instruction prefix from excessively penetrating into the syntax structure of the code at the logical level.

[0030] Through the aforementioned clear physical definition, the separator not only serves as a connector for text splicing, but more importantly, it constructs a clear structured boundary within the input sequence, ensuring that the subsequent feature extraction model can accurately distinguish between instructions and code.

[0031] In one specific embodiment, a preset task instruction prefix and the source code to be detected are concatenated to generate a task-aware input sequence, which specifically includes the following steps: First, a preset task instruction prefix is ​​placed as guiding text before the source code to be detected, and a preset separator is inserted between the task instruction prefix and the source code. This separator semantically distinguishes the instruction portion from the data portion, preventing subsequent models from confusing the task instruction prefix with the source code.

[0032] Subsequently, the text is merged strictly in the order of the task instruction prefix, the separator identifier, and the source code to be detected, thereby constructing the complete task-aware input sequence.

[0033] By implementing the above steps of sequentially concatenating the task instruction prefix with the source code to be detected, the model can be effectively guided by natural language instructions to achieve accurate alignment of vulnerability patterns in the vector space. This prompts the model to reduce its focus on surface features such as variable naming and code structure when generating vectors, and instead focus on the vulnerability pattern features inside the code. This significantly improves the accuracy and relevance of the final generated source code semantic feature vector in vulnerability detection tasks.

[0034] Step S3: Input the task-aware input sequence into the preset vulnerability feature extraction model so that the vulnerability feature extraction model generates source code semantic feature vectors based on the task-aware input sequence.

[0035] In a preferred embodiment, the vulnerability feature extraction model is trained in the following manner: Obtain a vulnerability training sample set; wherein, the vulnerability training sample set contains several vulnerability training samples; each vulnerability training sample includes query code, corresponding positive samples, and corresponding hard negative samples; For each vulnerability training sample, the preset task instruction prefix is ​​concatenated with the current query code to generate the current task-aware query sequence; the preset task instruction prefix is ​​concatenated with the current positive sample to generate the current task-aware positive sample sequence; the preset task instruction prefix is ​​concatenated with the current hard-to-bear sample to generate the current task-aware hard-to-bear sample sequence; the current task-aware query sequence, the current task-aware positive sample sequence, and the current task-aware hard-to-bear sample sequence are used as training task-aware sequence samples. Construct a training task perception sequence set based on the perception sequence samples of each training task; The training task perception sequence set is divided into several batches of training samples according to the preset batch size. Repeat the loop process until the preset number of training iterations is reached to generate a vulnerability feature extraction model; The cyclic process includes: Each training task-aware sequence sample in the current batch of training samples is sequentially input into the current vulnerability feature extraction model, and the source code semantic feature vector corresponding to each training task-aware sequence sample in the current batch of training samples is output; wherein, the source code semantic feature vector includes: the query vector corresponding to the task-aware query sequence, the positive sample vector corresponding to the task-aware positive sample sequence, and the hard-negative sample vector corresponding to the task-aware hard-negative sample sequence; the initial training samples of the current batch are the training samples of the first batch in the training task-aware sequence set. For each training task perception sequence sample in the current batch of training samples, the task perception query sequence and the task perception difficult sample sequence of the current training task perception sequence sample are input into the preset teacher model, so that the teacher model performs a vulnerability logic similarity score on the task perception query sequence and the task perception difficult sample sequence of the current training task perception sequence sample, and generates the penalty weight corresponding to the current training task perception sequence sample. Based on the query vector, positive sample vector, hard negative sample vector, and corresponding penalty weights of the perceptual sequence samples of each training task in the current batch of training samples, the loss function value is calculated and generated through a preset weighted loss function. The optimizer optimizes the current vulnerability feature extraction model based on the loss function value, and generates an optimized vulnerability feature extraction model. Determine whether the current number of training iterations has reached the preset number of training iterations. If yes, use the optimized vulnerability feature extraction model as the final vulnerability feature extraction model and select the next batch of training samples as the training samples for the current batch. If no, update the optimized vulnerability feature extraction model to the current vulnerability feature extraction model.

[0036] In a preferred embodiment, the hard-negative sample of any vulnerability training sample is obtained by: Obtain a candidate code library; wherein each candidate code in the candidate code library has a corresponding vulnerability pattern label; the vulnerability pattern label is used to characterize the vulnerability type to which the code belongs; Based on the vulnerability pattern tags corresponding to each candidate code, target negative candidate codes with different vulnerability pattern tags from the candidate code library are selected. The query code is input into a preset general code embedding model, so that the general code embedding model generates a base semantic vector of the query code based on the query code; Each target negative candidate code is sequentially input into a preset general code embedding model, so that the general code embedding model generates a contrast semantic vector of each target negative candidate code based on each target negative candidate code; Calculate the similarity between the baseline semantic vector and the comparative semantic vector of each target negative candidate code, and generate a similarity score for each target negative candidate code; The similarity scores of the negative candidate codes for each target are sorted in descending order to generate the ranking results; Select the target negative candidate code ranked in the preset position from the sorting results as the hard negative sample.

[0037] In a preferred embodiment, positive samples of any vulnerability training sample are obtained in the following manner: Based on the vulnerability pattern tags corresponding to each candidate code, target positive candidate codes with the same vulnerability pattern tags as the query code are selected from the candidate code library; Each target positive candidate code is sequentially input into a preset general code embedding model, so that the general code embedding model generates a comparative semantic vector of each target positive candidate code based on each target positive candidate code; Calculate the similarity between the baseline semantic vector and the comparative semantic vector of each target positive candidate code, and generate a similarity score for each target positive candidate code; From the positive candidate codes of each target, select the positive candidate code with the highest similarity score and a similarity of less than 1 as the positive sample.

[0038] In a preferred embodiment, a pre-defined teacher model is trained in the following manner: Obtain the teacher model training set; wherein, the teacher model training set includes multiple sets of vulnerability comparison chains, each set of vulnerability comparison chains includes a first code segment with a vulnerability, a second code segment after fixing the first code segment, and a corresponding vulnerability logic analysis report; Based on a pre-defined teacher base model, a differential analysis is performed on the first code snippet and the second code snippet to generate a vulnerability feature logic difference. Using the vulnerability feature logic difference and the vulnerability logic analysis report as a supervision signal, the pre-defined teacher base model is subjected to supervised fine-tuning to obtain a teacher model. The teacher model is used to output a penalty weight representing the logical similarity between the task-aware query sequence and the task-aware hard-to-bear sample sequence.

[0039] Specifically, the pre-defined vulnerability feature extraction model is configured as a deep neural network architecture, containing multiple layers of self-attention mechanisms and feedforward neural networks. When the vulnerability feature extraction model receives a task-aware input sequence containing natural language instructions and source code text, it uses the attention mechanism to calculate global dependencies on the task-aware input sequence. Based on the semantic guidance of the task instruction prefix, it dynamically adjusts the attention weights for different lexical units and their contextual structures in the source code text. The vulnerability feature extraction model suppresses the feature responses of variable naming and general grammatical structures in the source code text that are unrelated to the task instruction prefix, while enhancing the feature responses of control flow and data flow logic related to vulnerability patterns. After multiple layers of feature abstraction and aggregation, the vulnerability feature extraction model maps the variable-length task-aware input sequence into a fixed-dimensional real-number vector, i.e., generates a source code semantic feature vector. The geometric position of the source code semantic feature vector in the high-dimensional vector space can characterize the deep semantic attributes of the source code text under a specific vulnerability pattern.

[0040] In a preferred embodiment, the vulnerability feature extraction model is trained in the following manner: First, a vulnerability training sample set is constructed. The data source for the vulnerability training sample set is selected from authoritative vulnerability databases. Each piece of data in the vulnerability training sample set contains a clear vulnerable code snippet, a vulnerability pattern label, and the corresponding fix code. Based on the vulnerability training sample set, several vulnerability training samples are constructed. Each vulnerability training sample consists of three parts: query code as the anchor, positive samples with the same vulnerability pattern label as the query code, and hard-to-bear samples with different vulnerability pattern labels but similar text features to the query code.

[0041] Next, a sequence construction operation is performed on each vulnerability training sample. The preset task instruction prefix is ​​concatenated with the current query code to generate the current task-aware query sequence; the preset task instruction prefix is ​​concatenated with the current positive sample to generate the current task-aware positive sample sequence; and the preset task instruction prefix is ​​concatenated with the current hard-to-bear sample to generate the current task-aware hard-to-bear sample sequence. The current task-aware query sequence, the current task-aware positive sample sequence, and the current task-aware hard-to-bear sample sequence are combined as training task-aware sequence samples. Based on the generated training task-aware sequence samples, a training task-aware sequence set is constructed, and the training task-aware sequence set is divided into several batches of training samples according to a preset batch size.

[0042] Subsequently, the iterative training process is repeated until the preset number of training iterations is reached. Specifically, the iterative process includes: inputting each training task-aware sequence sample from the current batch of training samples into the current vulnerability feature extraction model for forward propagation, and outputting the source code semantic feature vector corresponding to each training task-aware sequence sample in the current batch of training samples. The source code semantic feature vector specifically includes: a query vector obtained by encoding the task-aware query sequence, a positive sample vector obtained by encoding the task-aware positive sample sequence, and a hard-negative sample vector obtained by encoding the task-aware hard-negative sample sequence.

[0043] After obtaining the vectors, an operation is performed to dynamically evaluate the sample quality by introducing a pre-defined teacher model. Specifically, for each training task-aware sequence sample in the current batch of training samples, the task-aware query sequence and the task-aware hard-to-bear sample sequence of the current training task-aware sequence sample are input as paired input data into the pre-defined teacher model.

[0044] The pre-defined teacher model is configured as an analysis model with deep code logic reasoning capabilities, generated through supervised fine-tuning based on a pre-defined teacher base model. Thanks to specialized training utilizing the logical differences in vulnerability features and vulnerability logic analysis reports, the pre-defined teacher model can ignore the differences in variable naming conventions and syntactic structures on the surface of the code, focusing on the consistency between the task-aware query sequence and the task-aware hard-to-bear sample sequence in vulnerability triggering logic and control flow paths, and output a standardized vulnerability logic similarity score accordingly.

[0045] Subsequently, based on the output vulnerability logic similarity score, penalty weights are calculated for the perceptual sequence samples of the current training task. The calculation of penalty weights follows an inverse proportional mapping logic: When the vulnerability logic similarity score is high, it indicates that although the hard-to-bear sample is labeled as a negative sample, its internal vulnerability logic has a high potential similarity to the query code, belonging to a false negative. In this case, a lower penalty weight (e.g., a value close to 0) is assigned, thereby significantly reducing the impact of this sample on the model's gradient update in the loss function calculation and preventing the model from learning the wrong classification boundary. When the vulnerability logic similarity score is low, it indicates that the hard-to-bear sample is indeed completely different from the query code in deep logic, belonging to a high-quality hard-to-bear sample. In this case, a higher penalty weight (e.g., a value close to 1) is assigned, thereby strengthening the model's ability to distinguish this sample during training and forcing the model to learn more discriminative semantic features.

[0046] Based on the query vector, positive sample vector, hard-to-negative sample vector, and corresponding penalty weights generated by the teacher model for each training task's perception sequence samples in the current batch of training samples, a loss function value is calculated using a preset weighted loss function. The mathematical expression of the weighted loss function is as follows: in, This represents the value of the loss function; Represents the query vector; Represents a positive sample vector; Indicates the first A difficult negative sample vector; Indicates corresponding to the first Penalty weights for each difficult-to-bear sample; This represents the temperature coefficient, used to adjust the smoothness of the similarity distribution; This represents the function for calculating cosine similarity.

[0047] The optimizer uses backpropagation to update the parameters of the current vulnerability feature extraction model based on the calculated loss function value, generating an optimized vulnerability feature extraction model. It then checks if the preset number of training iterations has been reached. If so, the optimized vulnerability feature extraction model is used as the final model, and the next batch of training samples is selected as the current batch for potential subsequent validation or use in the next round. If not, the optimized model is updated to the current model, and training continues for the next batch.

[0048] In a preferred embodiment, the hard-negative sample of any vulnerability training sample is obtained by: First, a candidate codebase is obtained. Each candidate code in the database has a clearly labeled vulnerability pattern tag, which represents the fundamental vulnerability type to which the code belongs. Based on the vulnerability pattern tags of each candidate code, target negative candidate code with different vulnerability pattern tags from the query code is selected from the candidate codebase. This selection process ensures that the selected code is significantly different from the query code in terms of actual security.

[0049] To identify hard-to-replicate samples with similar surface textures but different logics, a query code is input into a pre-defined general code embedding model, which generates a baseline semantic vector for the query code. Simultaneously, each target negative candidate code is sequentially input into the general code embedding model, which generates a contrastive semantic vector for each target negative candidate code. The cosine similarity between the baseline semantic vector and the contrastive semantic vectors of each target negative candidate code is calculated, generating a similarity score for each target negative candidate code. The similarity scores of each target negative candidate code are sorted in descending order to generate a ranking result. Target negative candidate codes ranking at a predetermined position are selected from the ranking result as hard-to-replicate samples.

[0050] It should be noted that the preset position is defined as an integer truncation threshold (Top-K) used to define the sampling range of hard-to-bear samples. The preset position is numerically configured to be an integer between 10 and 50, with a preferred value of 50.

[0051] The technical logic behind setting the preset ranking lies in constructing a semantic confusion zone. In the ranking results, the higher the ranking of the target negative candidate code, the greater its similarity to the query code in terms of lexical structure, variable naming, and the arrangement of code statements. Since the target negative candidate code has been identified as having vulnerability pattern labels completely different from the query code during the screening stage, the target negative candidate codes ranked higher than the preset ranking constitute adversarial samples with highly similar textual features but fundamentally different vulnerability semantics—that is, difficult-to-pass samples.

[0052] The specific selection process is as follows: First, the sorting result is read. The sorting result is an ordered list containing all target negative candidate codes and arranged in descending order of similarity score. Next, the sampling cutoff index is determined according to the configured preset position (e.g., K=50). Subsequently, starting from the beginning of the sorting result, all target negative candidate codes are sequentially extracted up to the sampling cutoff index position. Finally, the set of these extracted target negative candidate codes is determined as the hard-to-pass sample set.

[0053] The difficult negative samples selected in this way have the characteristics of highly similar text features but completely different vulnerability patterns, which can force the vulnerability feature extraction model to not rely solely on shallow lexical or syntactic features for differentiation during training.

[0054] In a preferred embodiment, positive samples of any vulnerability training sample are obtained in the following manner: Based on the vulnerability pattern labels corresponding to each candidate code, target positive candidate codes with the same vulnerability pattern labels as the query code are selected from the candidate code library. Using the same general code embedding model as in the hard-to-pass sample acquisition step, each target positive candidate code is sequentially input into the general code embedding model, allowing the model to generate a comparative semantic vector for each target positive candidate code. The similarity between the baseline semantic vector and the comparative semantic vectors of each target positive candidate code is calculated, generating a similarity score for each target positive candidate code. From all target positive candidate codes, the target positive candidate code with the highest similarity score and a similarity score less than 1 is selected as the positive sample. Selecting target positive candidate codes with a similarity score less than 1 is to exclude completely identical code copies, aiming to select highly similar code that differs from the query code in code structure or expression but belongs to the same type of vulnerability, which helps enhance the model's generalization ability to different manifestations of the same vulnerability pattern.

[0055] By implementing the above training method based on teacher model weighting and hard example mining, the pseudo-similarity problem of general code models in the security field can be effectively solved, enabling the generated source code semantic feature vectors to accurately capture the vulnerability pattern semantics of the code, and significantly improving the accuracy and robustness of source code vulnerability detection.

[0056] It should be noted that the fundamental technical purpose of setting the "similarity less than 1" screening condition in the specific operation of selecting positive samples is to construct effective comparative feature differences and prevent the model training from falling into ineffective repetitive memorization.

[0057] Specifically, a similarity score of 1 typically means that the target positive candidate code and the query code are completely identical in text content, representing an exact copy of the code. If such code is input as a positive sample into a vulnerability feature extraction model, the model only needs to compare two completely identical vectors during training, without any deep feature abstraction or logical reasoning. This lack of diversity in training samples fails to stimulate the learning ability of the vulnerability feature extraction model, making it difficult for the model to capture the changing patterns of the same vulnerability under different code structures.

[0058] Conversely, this method forcibly selects the target positive candidate code with the highest similarity score but less than 1. This strategy ensures that while the selected positive samples are consistent with the query code in terms of vulnerability type, there are objective textual differences in specific variable naming, statement structure, or code style. This characteristic of "different text but the same semantics" forces the vulnerability feature extraction model to actively ignore surface noise features such as variable names and indentation when performing vector convergence, and instead use deep learning algorithms to mine the core logical features that transcend textual appearance and can stably represent the cause of the vulnerability. Therefore, selecting samples with a similarity score of less than 1 is a key means to improve the generalization ability of the vulnerability feature extraction model and ensure that the vulnerability feature extraction model can identify unknown variant vulnerabilities.

[0059] In one specific embodiment, to accurately convert the vulnerability logic similarity score output by the preset teacher model into penalty weights for the weighted loss function, a dynamic mapping mechanism based on an inverse proportional linear function is employed. The core logic of this mapping mechanism lies in dynamically adjusting the contribution of different difficult-to-bear samples in the model gradient update process based on the teacher model's assessment of the sample's potential risk.

[0060] Specifically, the teacher model performs a logical comparison between the task-aware hard-to-bear sample sequence and the task-aware query sequence in the current training task-aware sequence samples, outputting a vulnerability logic similarity score normalized to between 0 and 1. Subsequently, a corresponding penalty weight is calculated using a preset mapping formula. The specific expression of the mapping formula is as follows: in, Indicates the first The penalty weights corresponding to each difficult-to-bear sample; This indicates that the teacher model is for the first... The similarity score of the vulnerability logic output of a difficult sample; This represents the base weight constant (usually set to 1.0), used to establish the baseline value for the weights; This represents the adjustment coefficient (e.g., set to 0.8), used to control the degree to which the vulnerability logic similarity score reduces the final weight.

[0061] It is understandable that when the teacher model determines that a difficult-to-handle sample is highly similar to the query code in terms of vulnerability logic (i.e., When it approaches 1.0, the calculated value is... This will significantly reduce the penalty weight. This means that although the sample is labeled as a negative sample, it is very likely a false negative. Reducing the penalty weight can effectively suppress the erroneous gradient signal generated by the false negative, preventing the vulnerability feature extraction model from forcibly pushing away code with the same vulnerability pattern.

[0062] Conversely, when the teacher model determines that a difficult sample differs significantly from the query code in terms of vulnerability logic (i.e., When it approaches 0, the calculated value is... This will be maintained at a high level. This means that the sample is a high-quality, hard-to-bear sample, capable of providing clear classification boundaries. Maintaining a high penalty weight forces the vulnerability feature extraction model to pay full attention to the sample, thereby learning more refined feature discrimination capabilities.

[0063] Step S4: Based on the source code semantic feature vector, retrieve historical vulnerability vectors from the preset vulnerability knowledge base that meet the preset conditions for similarity with the source code semantic feature vector, and use them as candidate vulnerability vectors.

[0064] Specifically, the pre-built vulnerability knowledge base is a structured database pre-constructed and stored on non-volatile storage media. This base stores a massive amount of historical vulnerability vectors, each corresponding to a known historical vulnerability code snippet. Since both the source code semantic feature vectors and historical vulnerability vectors are generated by the same pre-built vulnerability feature extraction model, they are mapped to the same high-dimensional semantic vector space. This allows the geometric distance in the high-dimensional semantic vector space to directly and objectively reflect the semantic correlation between code segments at the levels of vulnerability patterns, control flow logic, and data flow dependencies.

[0065] To accurately quantify the semantic similarity between source code semantic feature vectors and any historical vulnerability vector, a cosine similarity algorithm is used to calculate a similarity score. The cosine similarity algorithm eliminates the influence of differences in code text length on feature matching, focusing on the directional consistency of vectors in the feature space. The specific formula for calculating the similarity score is as follows: in, This represents the calculated similarity score, which ranges from -1 to 1. This represents the semantic feature vector of the source code output by the preset vulnerability feature extraction model; This represents any historical vulnerability vector stored in a pre-defined vulnerability knowledge base; This represents the vector dot product operation; The magnitude of the semantic feature vector of the source code; This represents the magnitude of the historical vulnerability vector.

[0066] After calculating the similarity scores for all historical vulnerability vectors in the pre-defined vulnerability knowledge base, the vectors are sorted in descending order of similarity score to generate a sorted list. Subsequently, the sorted list is truncated and filtered according to pre-defined conditions. These conditions are configured to select the top pre-defined number of historical vulnerability vectors in the sorted list. These selected vectors are then identified as candidate vulnerability vectors. Each candidate vulnerability vector is associated with a complete metadata index in the pre-defined vulnerability knowledge base, including the original code snippet, vulnerability pattern tags, and historical vulnerability analysis reports written by experts or generated by models.

[0067] By implementing the above-mentioned vector retrieval steps based on deep semantics, we can accurately recall reference cases from massive historical data that are essentially similar to the source code to be detected in terms of vulnerability causes. This effectively eliminates pseudo-related code that is literally similar but logically unrelated, providing high-quality contextual knowledge support for the subsequent generation of accurate and interpretable detection reports by generative large language models.

[0068] In one specific embodiment, the preset vulnerability knowledge base is pre-generated through an offline construction process aimed at establishing a vector space benchmark for subsequent online retrieval. The construction process of the preset vulnerability knowledge base specifically includes the following steps: First, historical vulnerability data collection was performed. Data sources included the National Vulnerability Database (NVD), the Software Assurance Reference Dataset (SARD), and publicly available code version control repositories. Each raw data record collected included a historical vulnerability code snippet, the corresponding Common Weakness Enumeration (CWE) number, Common Vulnerability and Exposure (CVE) identifiers, and a historical vulnerability analysis report written by security experts or generated by large models.

[0069] Next, the historical vulnerability vectorization calculation is performed. Using the vulnerability feature extraction model trained in step S3 as an encoder, each of the collected historical vulnerability code fragments is sequentially input into the vulnerability feature extraction model. Before input, the task instruction prefix (e.g., "Please find the code most relevant to the following code vulnerability pattern:") is appended before the historical vulnerability code fragment to maintain semantic guidance conditions consistent with online detection. The vulnerability feature extraction model performs forward reasoning on each appended sequence, outputting the corresponding high-dimensional real-valued vector, i.e., generating the historical vulnerability vector. Because the exact same model parameters as in the online detection stage are used, the historical vulnerability vector and the subsequent source code semantic feature vectors to be generated are mapped to the same semantic metric space, thus ensuring the mathematical validity of the cosine similarity calculation.

[0070] Finally, vector indexing and persistent storage operations are performed. All the calculated historical vulnerability vectors are imported into a pre-defined vector search engine (e.g., an index structure built based on an inverted file index (IVF) or a hierarchical navigable small-world graph (HNSW) algorithm) to construct a high-dimensional vector index file to support millisecond-level approximate nearest neighbor search. Simultaneously, a key-value pair mapping database is constructed, using the unique index identifier of each historical vulnerability vector as the key, and the corresponding historical vulnerability code snippet, vulnerability pattern label (CWE number), and historical vulnerability analysis report as the value for associated storage.

[0071] This storage-computation separation architecture ensures that after retrieving highly similar vectors, rich metadata information can be immediately extracted through index identifiers to construct subsequent detection prompt word context.

[0072] In a preferred embodiment, an operation is performed to obtain a preset general code embedding model. The preset general code embedding model is configured as a metric for calculating the basic semantic distance between code segments in the feature space. To ensure that the preset general code embedding model can accurately capture the lexical and syntactic features of code across different programming languages, the preset general code embedding model specifically adopts a pre-trained model built based on a bidirectional Transformer Encoder architecture.

[0073] This embodiment explicitly selects the mainstream CodeBERT model or UniXcoder model as the preset general code embedding model. Before being acquired, the preset general code embedding model has completed a general pre-training process on a large-scale mixed code corpus (such as the CodeSearchNet dataset) covering more than six mainstream programming languages ​​(including Python, Java, JavaScript, PHP, Ruby, and Go).

[0074] The training process of the pre-defined general code embedding model mainly relies on the following two self-supervised learning tasks: The first is Masked Language Modeling (MLM). During training, some words in the input code sequence are randomly masked, forcing the model to use the contextual syntactic structure information to predict the masked words, thereby learning the local syntactic features of the code. The second is Replaced Token Detection (RTD) or contrastive learning. During training, some words are replaced with reasonable but incorrect words, forcing the model to identify which words have been tampered with, or contrastive learning to narrow the distance between natural language annotations and corresponding code snippets in the vector space, thereby learning the global semantic features of the code.

[0075] The specific steps to obtain the preset general code embedding model include: First, accessing an open-source model hosting platform (such as Hugging Face Hub) and locating the official model repository released by Microsoft, either CodeBERT-base or UniXcoder-base. Next, downloading the corresponding model configuration file (config.json) and pre-trained weight file (pytorch_model.bin). Finally, loading the downloaded files into the memory of the computing device completes the initialization of the preset general code embedding model. Since the preset general code embedding model has undergone the aforementioned large-scale multilingual pre-training, it possesses the ability to directly convert code text into high-dimensional dense vectors containing rich semantic information without task-specific fine-tuning. Therefore, it can be directly used in subsequent steps for preliminary screening and similarity ranking of difficult-to-bear samples.

[0076] Step S5: Based on each candidate vulnerability vector, extract the vulnerability pattern tags and historical vulnerability analysis reports corresponding to each candidate vulnerability vector from the preset vulnerability knowledge base. Specifically, the pre-defined vulnerability knowledge base employs a structured data storage method. This base not only stores high-dimensional vector data for similarity calculations but also persistently associates each historical vulnerability vector with its original metadata. This association is achieved through key-value pair mapping or an inverted index structure, ensuring that each historical vulnerability vector has a unique corresponding detailed description. Based on the candidate vulnerability vectors determined in step S4, the vulnerability pattern tags corresponding to each candidate vulnerability vector are located and extracted using their index identifiers in the pre-defined vulnerability knowledge base. Vulnerability pattern tags are predefined classification identifiers used to standardize the representation of the fundamental type of code vulnerability. The specific forms of vulnerability pattern tags include, but are not limited to, Common Weakness Enumeration (CWE) numbers. The purpose of vulnerability pattern tags is to clearly indicate what type of security flaw the historical code segment represented by the candidate vulnerability vector specifically contains, thus providing a definite category guide for subsequent analysis.

[0077] Simultaneously, for each candidate vulnerability vector, a corresponding historical vulnerability analysis report is extracted from a pre-defined vulnerability knowledge base. This historical vulnerability analysis report is a pre-generated natural language description of the historical vulnerability code snippet. Specifically, the report includes public vulnerability and vulnerability exposure (CVE) identifiers, official vulnerability description text, a structured analysis summary of the vulnerability triggering logic and potential security hazards, and the corresponding patched code snippet. The historical vulnerability analysis report contains both analysis documents written by security experts and pre-generated structured summaries based on fixed prompt templates using a large language model.

[0078] By extracting this detailed textual information, abstract mathematical similarity can be transformed into concrete semantic context rich in expert knowledge. This ensures that subsequent generative large language models can obtain sufficient and accurate background knowledge support about similar historical cases when performing vulnerability detection, effectively avoiding model illusions and improving the professionalism of detection conclusions.

[0079] Step S6: Construct a detection prompt context based on the task-aware input sequence, the vulnerability pattern labels corresponding to each candidate vulnerability vector, and the historical vulnerability analysis report corresponding to each candidate vulnerability vector; input the detection prompt context into a preset generative large language model so that the generative large language model generates source code vulnerability detection results based on the detection prompt context.

[0080] In a preferred embodiment, a pre-defined generative large language model is trained in the following manner: Obtain generative training samples; each generative training sample includes: simulated detection instructions, code instance to be tested, retrieved similar code snippets, corresponding historical vulnerability analysis reports, and standard expert detection conclusions; The simulated detection commands, code instances to be tested, similar code snippets, and historical vulnerability analysis reports are templated and concatenated to construct an input prompt word sequence; The input prompt word sequence is used as input, and the standard expert detection conclusion is used as the target output. The preset large language model is fine-tuned through supervised learning to generate a generative large language model. The generative large language model is used to generate detection results containing vulnerability type location and vulnerability remediation suggestions based on the input task-aware input sequence.

[0081] Specifically, the detection prompt context is constructed as a semantically rich text sequence containing background information on the source code to be detected and relevant reference knowledge. The process of constructing the detection prompt context includes: using the task-aware input sequence generated in step S2 as the core query part; using the vulnerability pattern labels corresponding to each candidate vulnerability vector extracted in step S5 as reference category prompt information; and using the historical vulnerability analysis reports corresponding to each candidate vulnerability vector as reference case knowledge. Through the above concatenation operations, the detection prompt context is constructed to not only contain the code currently to be analyzed, but also several historical cases that are logically highly similar to the current code and their corresponding expert-level analysis conclusions.

[0082] Subsequently, the constructed detection prompt context is input into a pre-defined generative large language model. This pre-defined generative large language model is a deep neural network built on a decoder-only architecture, possessing long text context processing capabilities and natural language generation capabilities. Upon receiving the detection prompt context, the pre-defined generative large language model performs deep logical reasoning on the source code to be detected, based on the instructions within the context and the analytical approach provided in the historical vulnerability analysis report. Finally, the pre-defined generative large language model outputs the source code vulnerability detection results. These results not only include a binary judgment of whether a vulnerability exists, but also the specific vulnerability type location, the specific line number where the vulnerability was triggered, a natural language explanation of the vulnerability's cause, and targeted code remediation suggestions.

[0083] In a preferred embodiment, a pre-defined generative large language model is trained in the following manner: First, the process of obtaining generative training samples is performed. Each generative training sample contains five core components: simulated detection instructions, code instances to be tested, retrieved similar code snippets, corresponding historical vulnerability analysis reports, and standard expert detection conclusions. The simulated detection instructions are consistent with the task instructions used in actual inference; the code instances to be tested are known code selected from historical data; the similar code snippets and historical vulnerability analysis reports are reference contexts retrieved for the code instances to be tested using the aforementioned vector retrieval steps; and the standard expert detection conclusions are accurate and standardized analysis results written by security experts for the code instances to be tested, serving as ground truth labels for model training.

[0084] Next, the operation of constructing the input prompt word sequence is performed. The simulated detection instructions, the code instance to be tested, similar code snippets, and historical vulnerability analysis reports are concatenated according to a predefined template format to construct the input prompt word sequence. Template-based concatenation ensures the structural consistency of the input data, clearly separating the code section, reference section, and instruction section using specific delimiters, preventing the model from confusing information from different sources during training.

[0085] Finally, supervised fine-tuning (SFT) is performed on the pre-defined generative base model. The pre-defined generative base model is a general-purpose large language model (e.g., a base version of GPT-4 or Qwen-72B) pre-trained on a large-scale general corpus. The constructed input prompt word sequence is used as the input data for the pre-defined generative base model, and standard expert detection conclusions are used as the target output data. The probability distribution difference between the generated results and the target output data of the pre-defined generative base model is calculated using the cross-entropy loss function, and the parameters of the pre-defined generative base model are updated using the gradient descent algorithm, thereby generating an optimized generative large language model. The fine-tuned generative large language model is configured to perceive the input sequence based on the input task and, in conjunction with retrieval information from a pre-defined vulnerability knowledge base, generate detection results including vulnerability type location and vulnerability remediation suggestions.

[0086] By implementing the above-mentioned training steps based on retrieval enhancement, the risk of illusion can be effectively reduced when generative large language models deal with professional code security issues. By introducing highly relevant historical cases as references, the model is forced to reason based on factual evidence, thereby significantly improving the accuracy, interpretability, and practical value of source code vulnerability detection results and remediation suggestions.

[0087] It should be noted that, in a preferred embodiment, the operation of obtaining a preset teacher base model and a preset generation base model is performed. To ensure the accuracy of subsequent differential analysis and the text quality of the final report, this embodiment explicitly adopts a mainstream open-source large language model as the base.

[0088] For the pre-defined teacher base model, given that it requires extremely strong code logic reasoning capabilities to perform differentiated analysis tasks, the pre-defined teacher base model is specifically configured to be selected from any of the following model groups: the Llama-3-70B-Instruct model released by Meta, the DeepSeek-V3 model released by DeepSeek, or the Qwen-2.5-Coder-32B model released by Alibaba Cloud. All of these models are deep neural networks pre-trained on mixed corpora containing trillions of lines of code and text, possessing an intrinsic understanding of complex control flow and data flow.

[0089] For the preset generative base model, given that the preset generative base model needs to have the ability to process long text context to accommodate the context of detection prompt words, and has the ability to follow instructions to generate standardized detection reports, the preset generative base model is specifically configured to be selected from any of the following model groups: Llama-3-70B model released by Meta, Qwen-2.5-72B model released by Alibaba Cloud, or Mixtral-8x22B hybrid expert model released by Mistral AI.

[0090] The specific process of obtaining the above models includes: First, accessing a third-party open-source model hosting platform (specifically Hugging Face Hub or the ModelScope community) through a network communication module, and using the official name string of the model (such as "meta-llama / Meta-Llama-3-70B-Instruct" or "deepseek-ai / DeepSeek-V3") for precise retrieval to locate the officially released model repository.

[0091] Next, download the configuration files and weight files required to build the model from the model repository. Specifically, the downloaded files include: the architecture configuration file (config.json) for defining the model network layers and hyperparameters, the tokenizer files (tokenizer.json and tokenizer_config.json) for defining the vocabulary and word segmentation rules, and the weight file (model.safetensors or pytorch_model.bin) that stores the pre-trained parameter matrix.

[0092] Subsequently, a pre-defined loading function is invoked using a deep learning inference framework (such as PyTorch or vLLM) to read and map the downloaded architecture configuration file and weight file into the high-performance graphics processing unit (GPU) memory of the computing device. During the loading process, depending on the available GPU memory, the model parameters are loaded in full-precision (FP32), half-precision (BF16), or quantized format (INT8 / INT4), thereby instantiating the pre-defined teacher base model and the pre-defined generative base model. By directly reusing these high-performance open-source models that have been widely validated in the industry, this method ensures that it possesses industry-leading fundamental performance metrics in both the logical inference and text generation stages.

[0093] Based on the above method embodiments, the present invention provides corresponding apparatus embodiments.

[0094] like Figure 2 As shown, an embodiment of the present invention provides a source code vulnerability detection device based on instruction awareness, including: a data acquisition module, a feature extraction module, a vector retrieval module, and a vulnerability detection module; The data acquisition module is used to acquire the source code to be detected; The feature extraction module is used to concatenate a preset task instruction prefix and the source code to be detected to generate a task-aware input sequence; wherein, the task instruction prefix is ​​natural language text used to guide the generation of source code semantic feature vectors; the task-aware input sequence is input into a preset vulnerability feature extraction model so that the vulnerability feature extraction model generates source code semantic feature vectors based on the task-aware input sequence. The vector retrieval module is used to retrieve historical vulnerability vectors that meet preset conditions in similarity to the source code semantic feature vectors from a preset vulnerability knowledge base, based on the source code semantic feature vectors, as candidate vulnerability vectors; and to extract vulnerability pattern tags and historical vulnerability analysis reports corresponding to each candidate vulnerability vector from the preset vulnerability knowledge base based on each candidate vulnerability vector. The vulnerability detection module is used to construct a detection prompt context based on the task-aware input sequence, the vulnerability pattern labels corresponding to each candidate vulnerability vector, and the historical vulnerability analysis report corresponding to each candidate vulnerability vector; and input the detection prompt context into a preset generative large language model so that the generative large language model generates source code vulnerability detection results based on the detection prompt context.

[0095] It should be noted that the embodiments of the apparatus described above correspond to the embodiments of the present invention described above, and can implement the source code vulnerability detection method based on instruction awareness described above in any one of the present invention. Furthermore, the embodiments of the apparatus described above are merely illustrative. The modules described as separate components may or may not be physically separate, and the components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Additionally, in the accompanying drawings of the apparatus embodiments provided by the present invention, the connection relationship between modules indicates that they have a communication connection, which can be specifically implemented as one or more communication buses or signal lines. Those skilled in the art can understand and implement this without creative effort.

[0096] Based on the above-described method embodiments of the present invention, a corresponding embodiment of an electronic device is provided.

[0097] An embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements the instruction-aware source code vulnerability detection method according to any one of the present invention, or the processor executes the computer program to implement the functions of each module in the above-described device embodiments.

[0098] For example, the computer program may be divided into one or more modules, which are stored in the memory and executed by the processor to perform the present invention. The one or more modules may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of the computer program in the terminal device.

[0099] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0100] The processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor is the control center of the terminal device, connecting all parts of the terminal device via various interfaces and lines.

[0101] The memory can be used to store the computer programs and / or modules. The processor implements various functions of the terminal device by running or executing the computer programs and / or modules stored in the memory and by calling data stored in the memory. The memory may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function, etc.; the data storage area may store data created based on the use of the mobile phone, etc. In addition, the memory may include high-speed random access memory, and may also include non-volatile memory, such as hard disk, memory, plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0102] Based on the above method embodiments, the present invention provides corresponding storage medium embodiments; Another embodiment of the present invention provides a storage medium including a stored computer program, wherein, when the computer program is running, the device where the storage medium is located executes any of the above-described instruction-aware source code vulnerability detection methods of the present invention.

[0103] The aforementioned storage medium is a computer-readable storage medium, and the computer program includes computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0104] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0105] It should be noted that the models "CodeBERT" and "DeepSeek-R1" mentioned in this embodiment are merely specific examples to verify the effectiveness of the technical solution of this invention, and are not intended to limit the invention. In practical applications, the preset general code embedding model can be replaced with GraphCodeBERT, UniXcoder, StarCoder, or other pre-trained models with code representation capabilities; the preset teacher model and generative large language model can be replaced with Llama series, Qwen series, GPT series, or other large language models with instruction following and logical reasoning capabilities. As long as the instruction-aware contrastive learning framework and weighted loss function design proposed in this invention are used, they all fall within the protection scope of this invention.

[0106] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications are also considered to be within the scope of protection of the present invention.< / sep>

Claims

1. A source code vulnerability detection method based on instruction awareness, characterized in that, include: Obtain the source code to be tested; The preset task instruction prefix and the source code to be detected are concatenated to generate a task-aware input sequence; wherein, the task instruction prefix is ​​natural language text used to guide the generation of semantic feature vectors of the source code; The task-aware input sequence is input into a preset vulnerability feature extraction model so that the vulnerability feature extraction model generates source code semantic feature vectors based on the task-aware input sequence. Based on the semantic feature vector of the source code, historical vulnerability vectors that meet the preset conditions of similarity with the semantic feature vector of the source code are retrieved from the preset vulnerability knowledge base and used as candidate vulnerability vectors. Based on each candidate vulnerability vector, vulnerability pattern tags and historical vulnerability analysis reports corresponding to each candidate vulnerability vector are extracted from a pre-set vulnerability knowledge base. Based on the task-aware input sequence, the vulnerability pattern labels corresponding to each candidate vulnerability vector, and the historical vulnerability analysis reports corresponding to each candidate vulnerability vector, a detection prompt context is constructed. The detection prompt context is then input into a preset generative large language model, so that the generative large language model generates source code vulnerability detection results based on the detection prompt context.

2. The source code vulnerability detection method based on instruction awareness as described in claim 1, characterized in that, The preset task instruction prefix and the source code to be detected are concatenated to generate a task-aware input sequence, including: The preset task instruction prefix is ​​used as the guiding text and placed before the source code to be detected. A preset separator is inserted between the task instruction prefix and the source code to be detected. The text is merged according to the order of task instruction prefix, separator, and source code to be inspected to generate a task-aware input sequence.

3. The source code vulnerability detection method based on instruction awareness as described in claim 2, characterized in that, The vulnerability feature extraction model is trained using the following methods: Obtain a vulnerability training sample set; wherein, the vulnerability training sample set contains several vulnerability training samples; each vulnerability training sample includes query code, corresponding positive samples, and corresponding hard negative samples; For each vulnerability training sample, the preset task instruction prefix is ​​concatenated with the current query code to generate the current task-aware query sequence; the preset task instruction prefix is ​​concatenated with the current positive sample to generate the current task-aware positive sample sequence; the preset task instruction prefix is ​​concatenated with the current hard-to-bear sample to generate the current task-aware hard-to-bear sample sequence; the current task-aware query sequence, the current task-aware positive sample sequence, and the current task-aware hard-to-bear sample sequence are used as training task-aware sequence samples. Construct a training task perception sequence set based on the perception sequence samples of each training task; The training task perception sequence set is divided into several batches of training samples according to the preset batch size. Repeat the loop process until the preset number of training iterations is reached to generate a vulnerability feature extraction model; The cyclic process includes: Each training task-aware sequence sample in the current batch of training samples is sequentially input into the current vulnerability feature extraction model, and the source code semantic feature vector corresponding to each training task-aware sequence sample in the current batch of training samples is output; wherein, the source code semantic feature vector includes: the query vector corresponding to the task-aware query sequence, the positive sample vector corresponding to the task-aware positive sample sequence, and the hard-negative sample vector corresponding to the task-aware hard-negative sample sequence; the initial training samples of the current batch are the training samples of the first batch in the training task-aware sequence set. For each training task perception sequence sample in the current batch of training samples, the task perception query sequence and the task perception difficult sample sequence of the current training task perception sequence sample are input into the preset teacher model, so that the teacher model performs a vulnerability logic similarity score on the task perception query sequence and the task perception difficult sample sequence of the current training task perception sequence sample, and generates the penalty weight corresponding to the current training task perception sequence sample. Based on the query vector, positive sample vector, hard negative sample vector, and corresponding penalty weights of the perceptual sequence samples of each training task in the current batch of training samples, the loss function value is calculated and generated through a preset weighted loss function. The optimizer optimizes the current vulnerability feature extraction model based on the loss function value, and generates an optimized vulnerability feature extraction model. Determine whether the current number of training iterations has reached the preset number of training iterations. If yes, use the optimized vulnerability feature extraction model as the final vulnerability feature extraction model and select the next batch of training samples as the training samples for the current batch. If no, update the optimized vulnerability feature extraction model to the current vulnerability feature extraction model.

4. The source code vulnerability detection method based on instruction awareness as described in claim 3, characterized in that, Obtain the hard negative sample of any vulnerability training sample using the following method: Obtain a candidate code library; wherein each candidate code in the candidate code library has a corresponding vulnerability pattern label; the vulnerability pattern label is used to characterize the vulnerability type to which the code belongs; Based on the vulnerability pattern tags corresponding to each candidate code, target negative candidate codes with different vulnerability pattern tags from the candidate code library are selected. The query code is input into a preset general code embedding model, so that the general code embedding model generates a base semantic vector of the query code based on the query code; Each target negative candidate code is sequentially input into a preset general code embedding model, so that the general code embedding model generates a contrast semantic vector of each target negative candidate code based on each target negative candidate code; Calculate the similarity between the baseline semantic vector and the comparative semantic vector of each target negative candidate code, and generate a similarity score for each target negative candidate code; The similarity scores of the negative candidate codes for each target are sorted in descending order to generate the ranking results; Select the target negative candidate code ranked in the preset position from the sorting results as the hard negative sample.

5. The source code vulnerability detection method based on instruction awareness as described in claim 4, characterized in that, Positive samples for any vulnerability training sample can be obtained in the following way: Based on the vulnerability pattern tags corresponding to each candidate code, target positive candidate codes with the same vulnerability pattern tags as the query code are selected from the candidate code library; Each target positive candidate code is sequentially input into a preset general code embedding model, so that the general code embedding model generates a comparative semantic vector of each target positive candidate code based on each target positive candidate code; Calculate the similarity between the baseline semantic vector and the comparative semantic vector of each target positive candidate code, and generate a similarity score for each target positive candidate code; From the positive candidate codes of each target, select the positive candidate code with the highest similarity score and a similarity of less than 1 as the positive sample.

6. The source code vulnerability detection method based on instruction awareness as described in claim 5, characterized in that, The pre-designed teacher model is trained using the following methods: Obtain the teacher model training set; wherein, the teacher model training set includes multiple sets of vulnerability comparison chains, each set of vulnerability comparison chains includes a first code segment with a vulnerability, a second code segment after fixing the first code segment, and a corresponding vulnerability logic analysis report; Based on a pre-set teacher foundation model, a differential analysis is performed on the first code snippet and the second code snippet to generate logical differences in vulnerability features; Using the logical differences in vulnerability features and the vulnerability logic analysis report as supervision signals, a pre-set teacher base model is subjected to supervised fine-tuning to obtain a teacher model; wherein, the teacher model is used to output a penalty weight representing the logical similarity between the task-aware query sequence and the task-aware difficult sample sequence.

7. The source code vulnerability detection method based on instruction awareness as described in claim 6, characterized in that, The pre-defined generative large language model is trained using the following methods: Obtain generative training samples; each generative training sample includes: simulated detection instructions, code instance to be tested, retrieved similar code snippets, corresponding historical vulnerability analysis reports, and standard expert detection conclusions; The simulated detection commands, code instances to be tested, similar code snippets, and historical vulnerability analysis reports are templated and concatenated to construct an input prompt word sequence; The input prompt word sequence is used as input, and the standard expert detection conclusion is used as the target output. The preset large language model is fine-tuned through supervised learning to generate a generative large language model. The generative large language model is used to generate detection results containing vulnerability type location and vulnerability remediation suggestions based on the input task-aware input sequence.

8. A source code vulnerability detection device based on instruction awareness, characterized in that, include: Data acquisition module, feature extraction module, vector retrieval module, and vulnerability detection module; The data acquisition module is used to acquire the source code to be detected; The feature extraction module is used to concatenate a preset task instruction prefix and the source code to be detected to generate a task-aware input sequence; wherein, the task instruction prefix is ​​natural language text used to guide the generation of source code semantic feature vectors; the task-aware input sequence is input into a preset vulnerability feature extraction model so that the vulnerability feature extraction model generates source code semantic feature vectors based on the task-aware input sequence. The vector retrieval module is used to retrieve historical vulnerability vectors that meet preset conditions in similarity to the source code semantic feature vectors from a preset vulnerability knowledge base, based on the source code semantic feature vectors, as candidate vulnerability vectors; and to extract vulnerability pattern tags and historical vulnerability analysis reports corresponding to each candidate vulnerability vector from the preset vulnerability knowledge base based on each candidate vulnerability vector. The vulnerability detection module is used to construct a detection prompt context based on the task-aware input sequence, the vulnerability pattern labels corresponding to each candidate vulnerability vector, and the historical vulnerability analysis report corresponding to each candidate vulnerability vector; and input the detection prompt context into a preset generative large language model so that the generative large language model generates source code vulnerability detection results based on the detection prompt context.

9. An electronic device, characterized in that, The method includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor, when executing the computer program, implements the instruction-aware source code vulnerability detection method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that, The storage medium includes a stored computer program, wherein, when the computer program is running, it controls the device where the storage medium is located to execute the instruction-aware source code vulnerability detection method as described in any one of claims 1 to 7.

Images