A code vulnerability detection method and device based on internal features of a model

Abstract

A code vulnerability detection method and device based on internal features of a model, the device comprising a processor configured to: obtain a comparative analysis data set constructed based on vulnerability repair code, the comparative analysis data set containing vulnerable code fragments and non-vulnerable code fragments for comparison; input the code fragments in the comparative analysis data set into a code language model, and obtain activation values of each neuron in the code language model on the vulnerable code fragments and the non-vulnerable code fragments, respectively; based on the distribution difference of the activation values of each neuron on the vulnerable code fragments and the non-vulnerable code fragments, locate neurons having a response preference for the vulnerable code fragments, and take them as vulnerability selective neurons; obtain a code to be tested, extract activation values of the code to be tested at corresponding positions of the vulnerability selective neurons to form a feature vector, input the feature vector into a vulnerability detection classifier, and obtain a vulnerability state of the code to be tested.

Description

Technical Field

[0001] This invention relates to the fields of software engineering and artificial intelligence technology, and in particular to a method and device for detecting code vulnerabilities based on internal features of a model. Background Technology

[0002] Code language models (CLMs), such as CodeGen, Code Llama, Qwen2.5-Coder, and DeepSeek-Coder, have been applied to automated software engineering tasks such as code generation, code summarization, and program repair. These CLMs are based on the Transformer architecture and generate source code word-by-word in an autoregressive manner by pre-training on large-scale code corpora. In code generation tasks, the CLM receives a prompt as input, which can be partially written code, natural language documentation, or a combination of both, and the model progressively generates subsequent code based on this prompt.

[0003] However, code language models pose a risk of introducing security vulnerabilities when generating code. For example, when given a prompt to load a user-uploaded YAML configuration file, the code language model might generate code that calls the insecure API `yaml.load`. This insecure API can deserialize arbitrary Python objects, allowing attackers to achieve remote code execution by embedding malicious payloads in the YAML file. This type of flaw corresponds to CWE-502 (Deserialization of Untrusted Data) in the MITRE General Weakness Enumeration (CWE) category.

[0004] Existing end-to-end security assessment methods, such as CWE-based hint testing and benchmarking, can quantify the security rate of code generated by code language models. However, these methods typically treat the code language model as a black box, failing to reveal the reasons behind insecure predictions generated within the model from the perspectives of data flow or feature extraction. Furthermore, existing methods cannot extract features of neural network components within the code language model that are associated with specific vulnerability patterns, making it difficult to leverage internal feature information to guide vulnerability detection during the generation phase.

[0005] In the field of explainable artificial intelligence, neuron analysis is a method to explain the reasoning mechanism of neural network models by studying their internal components. In code language models based on the Transformer architecture, each neuron corresponds to one dimension in the hidden layer of a multilayer perceptron (MLP), generating a scalar activation value at each lexical location. Existing research has shown that neuron-level analysis can be used to understand how knowledge is encoded in code language models, locate neurons corresponding to specific knowledge, and modify the behavior of code language models by editing neurons. In the field of software engineering, existing research has attempted to use neuron-level editing to modify model feature weights.

[0006] However, when applying neuronal analysis to assess the internal mechanisms of secure code generation in code language models and to detect vulnerabilities, a key technical challenge is the context-dependent ambiguity problem: a single neuron may activate semantically different concepts depending on the input context. This ambiguity makes it easy to generate a large number of spurious associations when directly performing statistical analysis on all activation values, making it difficult to map specific activation statistics into stable neuronal features that can be used to detect vulnerabilities.

[0007] In the fields of secure code generation and vulnerability detection, various improvement schemes have been proposed in existing research. Existing post-training optimization methods partially mitigate security risks through fine-tuning, alignment, and hint engineering; however, these methods treat the code language model as a whole for parameter adjustment, lacking precise localization of vulnerability features within the model. While reinforcement learning fine-tuning methods can guide the code language model to generate more secure code, they are costly to train and cannot extract the internal causes of insecure outputs. Existing model editing techniques can directly repair specific knowledge within the code language model; however, these techniques focus on updating factual knowledge and have not yet introduced isolation and comparison mechanisms for security vulnerabilities, failing to achieve systematic, neuron-level vulnerability feature extraction. Overall, current technologies cannot simultaneously achieve precise localization of vulnerability features within the code language model, resulting in a lack of specificity in vulnerability detection feature extraction and limited detection accuracy.

[0008] For example, CN119046944A discloses a deep learning-based code security vulnerability detection method, including the following steps: Step 1: Collect a large amount of code data, including normal code and code containing security vulnerabilities; Step 2: Preprocess the collected code data, including code parsing, feature extraction, and labeling; Step 3: Construct a deep learning model to learn feature representations from the code data; Step 4: Train the deep learning model using the preprocessed code data until the model reaches a predetermined performance index. This technical solution cannot solve the objective technical problems faced by this invention. Specifically, firstly, this technical solution cannot overcome the interference of feature extraction caused by contextual ambiguity. In code language models, the same code term may be safe or trigger a vulnerability in different contexts. This technical solution only uses a binary dataset of normal code and vulnerable code for training, lacking a targeted data comparison and isolation mechanism. This leads to the code language model being unable to effectively exclude the activation differences of the same term in different contexts when extracting features, easily misidentifying normal code terms as vulnerability features, thereby establishing false associations. Secondly, the black-box detection mechanism of this technical solution determines that it does not have the ability to accurately locate the cause of the vulnerability. Vulnerabilities generated by code language models are often caused by the abnormal activation of a few vulnerable neurons within the model. This technical solution only extracts external features of the code to fit the macroscopic mapping relationship between input and output, and lacks a mechanism to deeply evaluate the activation state of specific neurons within the model. Therefore, the feature extraction is not targeted enough when facing hidden code vulnerabilities.

[0009] For example, CN109190379A discloses a vulnerability detection method and apparatus for a deep learning system. The method includes: selecting and activating several neurons from the deep learning system under test, guided by maximizing neuron coverage; constructing an optimization function based on the tensor expressions of the several neurons and the tensor expression of the prediction difference of the deep learning system under test; and obtaining several perturbations by maximizing the optimization function; if any of the several perturbations satisfies a preset condition, obtaining a detection sample based on the perturbation, and detecting vulnerabilities in the deep learning system under test using the detection sample. The neuron computation logic of this technical solution is only used to guide the generation of adversarial use cases and does not establish a feature mapping mechanism between neuron activation states and source code vulnerability semantics. Therefore, it cannot solve the problem of secure code generation and detection across different domains.

[0010] For example, US20230342811A1 discloses an ad fraud detection device and method. The ad fraud detection device includes a processor and a memory storing instructions executable by the processor. The processor receives user data of users whose content is targeted by online advertisements, extracts features related to ad fraud from the user data, obtains first predicted ad fraud data from a first ad fraud detection model based on a neural network using the extracted features as input, obtains second predicted ad fraud data from a second ad fraud detection model based on an autoencoder using the extracted features as input, obtains third predicted ad fraud data from a third ad fraud detection model based on logistic regression using the extracted features as input, and determines whether a user is a fraudulent ad user based on the first, second, and third predicted ad fraud data. This prior art document, in the data preprocessing stage, only extracts macroscopic behavioral features of users and does not disclose the actions of constructing multidimensional comparison datasets with different contextual attributes for the same underlying elements. This coarse-grained feature extraction method leads to poor technical results: the model cannot separate the ambiguity of the same underlying elements in different semantic environments, and is prone to misclassifying regular features as risk features (high false positives); the huge redundant noise in the macro feature space will mask the small vulnerability features that do not really play a decisive role (high false negatives); due to the lack of anchor points in the micro feature dimension, the system is difficult to achieve the interpretability of deep attribution, blocking the technical path of accurate source tracing.

[0011] Based on the above-mentioned technical deficiencies, this invention aims to provide a code vulnerability detection method and device based on internal model features, breaking the black-box limitations of code language models. By accurately isolating and locating specific vulnerable features that have a causal relationship with vulnerability generation at the micro-neuron level, it significantly improves the accuracy and interpretability of code vulnerability detection mechanisms.

[0012] Furthermore, on the one hand, there are differences in understanding among those skilled in the art; on the other hand, the applicant studied a large number of documents and patents when making this invention, but due to space limitations, not all details and contents were listed in detail. However, this does not mean that the present invention does not possess the features of these prior art. On the contrary, the present invention already possesses all the features of the prior art, and the applicant reserves the right to add relevant prior art to the background art. Summary of the Invention

[0013] Existing technologies have significant limitations in code language model security assessment and code vulnerability detection. Current security assessment methods typically treat code language models as end-to-end black-box systems, failing to delve into the underlying neural network mechanisms that generate insecure code, resulting in a lack of targeted feature extraction. Existing neuron analysis methods face the challenge of context-dependent ambiguity when directly statistically analyzing activation values. Due to the lack of fine-grained contrast isolation mechanisms for code fragment security attributes, individual neurons in code language models are easily affected by the multiple semantic influences of the same code term in different contexts (e.g., activation confusion between random number seeds and key length constants). This ambiguity leads to the misidentification of numerous neurons that have only accidental or syntactic associations with code vulnerability generation, resulting in significant feature redundancy and spurious associations. This, in turn, limits the accurate localization of vulnerable representations within the code language model and the improvement of subsequent vulnerability detection classifier accuracy.

[0014] To address the shortcomings of existing technologies, this invention provides a code vulnerability detection device based on internal model features. The device includes a processor configured to: acquire a comparative analysis dataset constructed based on vulnerability patching code, the dataset containing vulnerable code fragments and non-vulnerable code fragments for comparison; input the code fragments from the comparative analysis dataset into a code language model, and acquire the activation values ​​of each neuron in the model on both vulnerable and non-vulnerable code fragments; based on the distribution differences of the activation values ​​of each neuron on the vulnerable and non-vulnerable code fragments, locate neurons that have a preference for responding to vulnerable code fragments and designate them as vulnerability-selective neurons; acquire the code to be tested, extract the activation values ​​at the corresponding positions of the vulnerability-selective neurons to form a feature vector, and input the feature vector into a vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

[0015] To address the shortcomings of CN119830300A, a comparative dataset construction unit was developed to build a comparative analysis dataset containing vulnerable, safe, benign, and neutral code snippets. The unit obtains benign code snippets based on similarity retrieval and establishes a fine-grained comparative isolation mechanism targeting the security attributes of these code snippets. This fine-grained isolation mechanism eliminates interference from activation differences of the same code terms in different contexts.

[0016] The technical solution of this invention first obtains activation values ​​and locates vulnerability-selective neurons, and then extracts internal features to form feature vectors. This technical solution breaks the limitation of treating code language models as black boxes, enabling precise location of vulnerability-selective neurons and extraction of feature vectors. This allows the detection process to directly rely on the computational mechanisms within the code language model, solving the technical problem that existing end-to-end assessments struggle to reveal the underlying causes of insecure code generation. According to a preferred embodiment, non-fragile code fragments include at least one of secure code fragments, benign code fragments, and neutral code fragments. The step of the processor constructing a comparative analysis dataset based on vulnerability patching code includes: obtaining vulnerability patching code pairs containing vulnerable code and patched code from the vulnerability patching code; performing character-level difference comparisons on the vulnerability patching code pairs to identify modified code blocks; extracting code lines from the modified code blocks to obtain vulnerable code fragments and secure code fragments; performing code retrieval based on the vulnerable code fragments to obtain benign code fragments; and marking unmodified code contexts other than vulnerable, secure, and benign code fragments as neutral code fragments.

[0017] The steps in constructing this dataset ensure accurate isolation of the security attributes of code snippets. These operations reduce the ambiguity caused by identical code terms in different contexts, overcoming the technical shortcomings of directly performing statistical analysis on all activation values, which easily leads to spurious associations and feature redundancy.

[0018] According to a preferred embodiment, the step of the processor constructing a comparative analysis dataset based on vulnerability patching code includes: in the same code repository, searching the code outside the modified code block based on lexical similarity and / or semantic similarity to find lines of code similar to the vulnerable code fragment, and selecting the lines of code with the highest similarity as benign code fragments among the non-vulnerable code fragments.

[0019] This step retrieves benign code snippets based on similarity, selectively extracting reference data with identical elements but no security risks. This operation eliminates interference from activation differences of identical code terms in different contexts by constructing comparative data, reducing the probability of misclassifying regular features as risky features and ensuring the accuracy of feature extraction.

[0020] According to a preferred embodiment, the step of the processor obtaining the activation values ​​of each neuron in the code language model on fragile and non-fragile code segments includes: initializing a highest activation value buffer for each neuron in the code language model, the highest activation value buffer being configured to store a preset number of activation values; performing lexicalization on code segments in the contrastive analysis dataset and assigning contrastive type labels to the generated lexical tags; constructing code prefixes for the code segments and inputting them into the code language model to perform forward propagation to obtain the neuron activation values ​​corresponding to the lexical positions.

[0021] This step configures the highest activation value buffer for the neuron and performs forward propagation, thus mapping the input code to the scalar activation values ​​within the code language model. This processing method records the neuron activation state corresponding to a specific lexical position, providing a quantitative data foundation for calculating the distribution differences of activation values ​​and screening for vulnerable selective neurons.

[0022] According to a preferred embodiment, the step of the processor obtaining the activation values ​​of each neuron in the code language model on fragile and non-fragile code segments further includes: configuring a highest activation value buffer for the target neuron in the code language model, and updating the highest activation value buffer using a streaming update strategy; wherein, for the target neuron, it is determined whether the current activation value meets a preset replacement condition; wherein, the preset replacement condition includes at least that the current activation value is greater than or equal to the minimum value in the highest activation value buffer; if the preset replacement condition is met, the minimum value in the highest activation value buffer is updated using the current activation value, the corresponding code prefix, and the comparison type label.

[0023] This invention uses a streaming update strategy to update the highest activation value buffer, avoiding the memory consumption caused by storing all activation values ​​of the entire code segment, reducing the hardware memory usage during forward propagation, and improving the running efficiency of neuron detection computation.

[0024] According to a preferred embodiment, the step of the processor locating neurons that have a response preference to fragile code segments based on the distribution difference of activation values ​​of each neuron on fragile code segments and non-fragile code segments, and designating them as vulnerability-selective neurons, includes: obtaining a sorted list of target neurons that meet preset activation conditions, wherein the activation values ​​in the sorted list are ranked based on their numerical values; calculating a selection score for the target neurons according to the code segment type corresponding to the activation values ​​in the sorted list, wherein the selection score is used to measure the degree of concentration of activation values ​​on fragile code segments; and selecting vulnerability-selective neurons from the neurons of the code language model based on the selection score.

[0025] This invention calculates the selective score of the target neuron based on the sorted list and code fragment type, forming a quantification index for neuron response preference. It converts the internal activation state into a numerical value that measures the degree of concentration of activation values ​​on vulnerable code fragments, thus solving the technical problem of the difficulty in accurately locating and extracting vulnerable feature representations within code language models.

[0026] According to a preferred embodiment, the step of the processor calculating the selectivity score of the target neuron based on the code segment type corresponding to the activation value in the sorted list includes: obtaining the sorted list of the target neurons; determining the relevance weight of the ranking based on the code segment type to which the activation value in each ranking of the target neuron belongs; wherein the relevance weight corresponding to the fragile code segment has a positive contribution to the selectivity score, and the relevance weight corresponding to the non-fragile code segment has a negative contribution or zero contribution to the selectivity score; and calculating the selectivity score of the target neuron based on the positional loss weight and relevance weight associated with the ranking.

[0027] This invention quantifies the influence of various code segments on activation value ranking based on relevance weights and positional loss weights. This can determine the positive or negative contribution of fragile and non-fragile code segments to the selection score, objectively assess the response preferences of neurons, and ensure that the selected fragile feature representations are targeted.

[0028] According to a preferred embodiment, the processor is configured to: perform lexicalization on the code under test and extract the scalar activation values ​​output by all vulnerability-selective neurons at each lexical position in the code under test to construct a lexical-level feature vector; perform feature aggregation on multiple lexical-level feature vectors belonging to the same code unit based on the code unit boundary of the code under test to obtain a code unit-level feature vector; and input the lexical-level feature vector or the code unit-level feature vector into a vulnerability detection classifier to output the vulnerability status of at least one code unit.

[0029] This invention extracts scalar activation values ​​from vulnerability-selective neurons to construct feature vectors and performs feature aggregation processing. This enables the conversion of activation features into code unit-level features, eliminates interference information in the feature space, and allows the vulnerability detection classifier to perform judgments using vulnerability feature representations within the code language model, thereby improving prediction accuracy.

[0030] This invention provides a code vulnerability detection method based on internal model features from a second aspect. The method includes: acquiring a comparative analysis dataset constructed based on vulnerability patching code, the comparative analysis dataset containing vulnerable code fragments and non-vulnerable code fragments for comparison; inputting the code fragments in the comparative analysis dataset into a code language model, and obtaining the activation values ​​of each neuron in the code language model on the vulnerable code fragments and non-vulnerable code fragments respectively; based on the distribution difference of the activation values ​​of each neuron on the vulnerable code fragments and non-vulnerable code fragments, locating neurons that have a response preference to vulnerable code fragments, and using them as vulnerability-selective neurons; acquiring the code to be tested, extracting the activation values ​​of the code to be tested at the corresponding positions of the vulnerability-selective neurons to form a feature vector, and inputting the feature vector into a vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

[0031] The method of this invention establishes a feature mapping relationship between neuron activation states and code vulnerabilities. This invention overcomes the technical deficiency of end-to-end security assessments in failing to reveal internal model features. By acquiring activation values ​​and extracting vulnerability features to construct feature vectors, vulnerability detection is based on the scalar activation states of neurons and the resulting feature vectors, thus improving detection accuracy.

[0032] This invention provides a code vulnerability detection device based on internal model features from a third aspect. The code vulnerability detection device includes a comparison dataset construction unit, a neuron detection unit, a neuron localization unit, and a vulnerability detection unit. The comparison dataset construction unit acquires a comparison analysis dataset constructed based on vulnerability-fixing code. The comparison analysis dataset contains vulnerable code fragments and non-vulnerable code fragments for comparison. The neuron detection unit inputs the code fragments from the comparison analysis dataset into a code language model and obtains the activation values ​​of each neuron in the code language model on both vulnerable and non-vulnerable code fragments. The neuron localization unit locates neurons with a response preference to vulnerable code fragments based on the distribution differences of the activation values ​​of each neuron on the vulnerable and non-vulnerable code fragments, and uses these neurons as vulnerability-selective neurons. The vulnerability detection unit acquires the code to be tested, extracts the activation values ​​at the corresponding positions of the vulnerability-selective neurons to form a feature vector, and inputs the feature vector into a vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

[0033] This technical solution adopts a device structure based on logical modules, dividing the system into a comparison dataset construction unit, a neuron detection unit, a neuron localization unit, and a vulnerability detection unit. This structure reduces the system coupling between processing steps, allowing each unit to be implemented through an independent computing module or through a software module on the same computing device. This configuration establishes a standardized and modular data flow mechanism, creating an orderly data interaction process between the functional modules and clarifying the transmission relationships of the comparison analysis dataset, neuron activation values, vulnerability-selective neuron index set, and feature vectors at different processing stages.

[0034] Existing technologies treat code language models as end-to-end black-box systems, lacking the ability to reveal the underlying neural network mechanisms that generate insecure code. The technical solution of this invention extracts the scalar activation values ​​of vulnerability-selective neurons within the code language model to construct feature vectors. This operation enables the vulnerability detection classifier to utilize specific vulnerability features within the model to perform judgments, establishing a mapping relationship between activation states and vulnerability semantics, thereby improving the accuracy and interpretability of vulnerability detection mechanisms.

[0035] Existing neuron analysis methods face context-dependent ambiguity interference when statistically analyzing all activation values. This can easily lead to misidentification of the same code term with multiple semantic meanings in different contexts as vulnerable features, resulting in false associations. The technical solution of this invention introduces a comparative analysis dataset constructed from safe, benign, and neutral code snippets, and uses similarity retrieval to selectively obtain benign code snippets. This approach eliminates activation differences of the same underlying elements in different contexts, avoids the probability of conventional features being misjudged as risky features, and ensures the targeted extraction of vulnerability features.

[0036] Existing post-training optimization or feature extraction methods typically treat the model as a whole for adjustment, lacking a precise mechanism for locating vulnerability representations within the code language model. The technical solution of this invention calculates a selective score based on a ranking list of activation values ​​of the target neurons and the relevance weights of code fragment types. This computational logic transforms the internal activation states into a numerical indicator measuring the concentration of activation values ​​on vulnerable code fragments, quantifying the response preferences of each neuron and solving the technical problem of locating and extracting specific vulnerability representations within the model. Attached Figure Description

[0037] Figure 1 This is a flowchart illustrating the code vulnerability detection method based on internal model features provided by the present invention; Figure 2 This is a schematic diagram of the benign code fragment retrieval process and four comparison types provided by the present invention; Figure 3 This is a schematic diagram of the neuron detection process provided by the present invention; Figure 4 This is a schematic diagram of the neuron localization process provided by the present invention; Figure 5 This is a vulnerability detection architecture diagram based on vulnerability-selective neuron features provided by the present invention; Figure 6 This is an application example diagram of the code vulnerability detection method based on internal model features provided by the present invention; Figure 7 This is a schematic diagram of the structure of the computer device provided by the present invention; Figure 8 This is a schematic diagram of the logic module of the code vulnerability detection device based on internal model features provided by the present invention; Figure 9 It is the Spearman coefficient of the distribution difference of vulnerable selective neurons under the four types of code transformations provided in this invention.

[0038] List of reference numerals 100: Code vulnerability detection device; 110: Comparison dataset construction unit; 120: Neuron detection unit; 130: Neuron localization unit; 140: Vulnerability detection unit; 200: Computer equipment; 210: Processor; 211: Central processing unit; 212: Graphics processing unit; 220: Memory; 221: Stored program area; 222: Stored data area; 230: System bus. Detailed Implementation

[0039] The following is a detailed explanation with reference to the accompanying drawings.

[0040] To facilitate understanding of the technical solutions provided by this invention, the core terms and concepts used in this invention will be explained first.

[0041] Code Language Model: A large-scale language model based on the Transformer architecture, trained on a large-scale code corpus, capable of generating source code in an autoregressive manner. The code language model receives prompts as input and transforms them into a sequence x=[x1,...,x...]. |x| The hidden states are calculated step by step, and the probability distribution of the next word is predicted.

[0042] Neuron: In the code language model of the Transformer architecture, a neuron Corresponding to the The d-th dimension of the MLP hidden layer. The code language model has L layers, each with D neurons, for a total of L×D neurons. Each neuron generates a scalar activation value at each word position when processing the input. For a Transformer architecture code language model with L layers and D hidden dimensions in each MLP layer, the code snippet x=[x1,...,x...] |x| In the The hidden state of a layer is represented as: .

[0043] In the above formula, This is the hidden vector at word position u. Neuron Defined as the first The activation value of the d-th coordinate of the layer activation vector at the word position u is: .

[0044] A code language model with L layers and a width D has a total of L×D neurons.

[0045] Code vulnerabilities: Defects in program logic that can be exploited to compromise the confidentiality, integrity, or availability of a system. Industry classification standards such as the MITRE General Weakness Enumeration (CWE) categorize vulnerabilities into standardized classes, such as CWE-078 (operating system command injection) and CWE-502 (untrusted data deserialization).

[0046] Vulnerability-selective neurons: Neural network units in a code language model that are selectively activated when insecure code is generated. The activation patterns of these neurons are aligned with the construction of insecure code, and inhibiting them can reduce the generation of insecure code.

[0047] Vulnerability patching code refers to original code files or sets of code that are derived from known vulnerability datasets or code repositories, fully record a specific code vulnerability and its patching process, and include the surrounding unmodified code context.

[0048] Vulnerability patch code pair: refers to a code combination consisting of vulnerable code containing a specific code vulnerability and its corresponding patch code; the patch code is confirmed to fix the specific code vulnerability and does not contain irrelevant modifications.

[0049] The Comparative Analysis Dataset is a specialized dataset built upon vulnerability patch code pairs, designed to address the context-dependent ambiguity problem in neural network analysis. The dataset rigorously categorizes code into four comparison types: vulnerable code snippets, safe code snippets, benign code snippets, and neutral code snippets.

[0050] Selectivity score: A quantitative metric used to measure the concentration of the highest activation values ​​of a specific neuron on vulnerable code segments. The calculation is based on the distribution of each comparison type label in the list of highest activation values: vulnerable code segments ranked higher contribute positive scores, while safe code segments ranked higher contribute negative scores.

[0051] Buffer: A fixed-capacity storage space allocated for each neuron in the code language model during the neuron probing step. It is used to dynamically record the top k highest activation values ​​generated by the neuron across all lexical positions, along with their corresponding code prefixes and contrast type labels.

[0052] Example 1 Source code generated by code language models often contains security vulnerabilities. Existing security assessment methods primarily treat code language models as black-box systems, only quantifying the security rate at the output end, failing to reveal the reasoning mechanisms that generate insecure decisions within the code language model. This makes it difficult to pinpoint the correlation between the model's internal neural network components and specific code vulnerability patterns. Neuron analysis methods face the challenge of context-dependent ambiguity when studying the internal mechanisms of secure code generation. Because individual neurons produce semantically different activation values ​​in different code contexts, activation statistics are difficult to map into stable, interpretable vulnerability representations, easily leading to numerous spurious associations. Furthermore, existing model fine-tuning or alignment techniques mostly focus on overall adjustments to the code language model, lacking precise localization and feature isolation mechanisms for vulnerability representations within the model. This prevents existing vulnerability detection methods from implementing targeted feature extraction based on the model's internal causes, limiting the predictive accuracy of vulnerability detection classifiers.

[0053] This invention provides a code vulnerability detection architecture based on feature isolation and streaming updates.

[0054] Existing technologies, such as prior art document CN119830300A, disclose filtering features using adaptive domain masks. However, prior art document CN119830300A treats the code language model as a whole, which has the objective technical problem of failing to isolate the ambiguity interference generated by the same code terms in different contextual semantics. For example, the same constant used as a random seed in one context and as a key length in another context will produce different security semantics.

[0055] The comparison dataset construction unit 110 constructs a comparison analysis dataset containing vulnerable code snippets, safe code snippets, benign code snippets, and neutral code snippets. The comparison dataset construction unit 110 obtains benign code snippets based on similarity retrieval and establishes a fine-grained comparison isolation mechanism targeting the security attributes of code snippets. This fine-grained comparison isolation mechanism eliminates interference from activation differences of the same code terms in different contexts.

[0056] Considering that storing the full activation values ​​during forward propagation of the code language model consumes a significant amount of video memory, processor 210 employs a heterogeneous computing architecture comprising a central processing unit 211 and a graphics processing unit 212. The highest activation value buffer is deployed in the global video memory of the graphics processing unit 212 and is implemented using a min-heap data structure at the underlying level. Processor 210 uses a streaming update strategy to update the highest activation value buffer. Activation tensors are transferred and updated between the computing unit and memory 220 via the system bus 230. Processor 210 updates the highest activation value buffer using the current activation values, reducing the hardware video memory usage during forward propagation.

[0057] Neuron localization unit 130 calculates the selectivity score of the target neuron based on ranking relevance weights and positional loss weights. Neuron localization unit 130 introduces a minimal smoothing term into the denominator of the calculation formula to prevent division-by-zero anomalies. Based on the selectivity score, neuron localization unit 130 filters out vulnerable selective neurons from the neurons of the code language model, thus achieving the localization of vulnerable representations within the model.

[0058] The vulnerability detection unit 140 extracts the activation values ​​at corresponding positions of the vulnerability-selective neurons in the code under test to form a feature vector. The vulnerability detection unit 140 performs feature aggregation processing on multiple word-level feature vectors based on the code unit boundaries of the code under test. The vulnerability detection classifier uses the vulnerability features within the code language model to represent the vulnerability status of the output code.

[0059] The comparison dataset construction unit 110 performs similarity retrieval in the unmodified code context outside the modified code block to obtain benign code fragments, thus isolating contextual ambiguity interference. The neuron detection unit 120 is connected to the vulnerability detection unit 140 via a shared system bus 230. When extracting feature vectors, the vulnerability detection unit 140 calls the code language model configured in the neuron detection unit 120 via the system bus 230. While keeping the underlying pre-trained parameters of the code language model frozen, it extracts the scalar activation values ​​of vulnerability-selective neurons. When calculating the selectivity score, the neuron localization unit 130, to avoid arithmetic anomalies, introduces a pre-defined minimal smoothing term into the formula to maintain the continuity of the calculation process and prevent the absolute value weight accumulation of the denominator from being zero due to the zero relevance of benign or neutral code fragments. To address the problems of existing security assessment methods treating code language models as black boxes and failing to locate internal vulnerable representation units, as well as the problem of numerous false associations caused by context-dependent ambiguity when directly performing neuron analysis, this invention aims to solve the technical problem of constructing a device and method that can effectively isolate contextual interference and accurately identify selective neurons that are vulnerable within the code language model. This would resolve the technical shortcomings of existing vulnerability detection methods, such as insufficient feature extraction targeting and limited detection accuracy.

[0060] To address the above deficiencies, this invention provides a code vulnerability detection method and device based on model internal features. Preferably, the code vulnerability detection device 100 based on model internal features of this invention includes at least a processor 210 and may also include a memory 220. The processor 210 is configured to execute the code vulnerability detection method based on model internal features of this invention. Preferably, the processor 210 can be an integrated circuit chip with signal processing capabilities. In practical applications, the processor 210 can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or register logic devices, or discrete hardware components.

[0061] Processor 210 is not limited to a single processing core; it may contain one or more processing cores. Processor 210 is connected via system bus 230 (system bus 230 is connected via communication module ( Figure 7 (Not shown) Receive and send information) Call computer program instructions in memory 220 to perform the aforementioned steps of constructing the comparative analysis dataset and detecting neurons.

[0062] Preferably, the system bus 230 is used to connect physical hardware such as the central processing unit (CPU) 211, the graphics processing unit (GPU) 212, and the memory 220. In a heterogeneous computing architecture, the peripheral component interconnect expansion bus is used to realize the data transfer of model parameters and activation tensors between the memory and the video memory of the graphics processing unit (GPU) 212. The system bus 230 may also include memory buses, high-speed interconnect buses between graphics processing units, and logically divided data buses, address buses, and control buses.

[0063] Preferably, to optimize the forward propagation efficiency of the code language model and the detection efficiency of massive neuron activation values, the processor 210 adopts a heterogeneous computing architecture including a central processing unit (CPU) 211 and a graphics processing unit (GPU) 212. The central processing unit 211 is used to perform logic control and data preprocessing tasks; the graphics processing unit 212 is used to accelerate matrix operations, thereby supporting the streaming update operation of the highest activation value buffer at the hardware level.

[0064] The overall process of the code vulnerability detection method based on model internal features of the present invention includes four main steps: comparative analysis dataset construction, neuron detection, neuron localization, and vulnerability detection.

[0065] S100: Comparative Analysis Dataset Construction: Processor 210 acquires a comparative analysis dataset built based on vulnerability patch code, such as... Figure 1 As shown.

[0066] The comparative analysis dataset contains vulnerable code snippets as well as non-fragile code snippets used for comparison. Non-fragile code snippets include at least one of safe, benign, and neutral code snippets.

[0067] The goal of this step is to build an analysis dataset containing four contrast types, starting from vulnerability patch code pairs, to support precise contrastive neuron analysis.

[0068] In this embodiment, the manually verified vulnerability datasets SVEN and SafeCoder are selected as the base datasets as exemplary experimental subjects. It should be noted that the method of this invention is also applicable to other datasets containing vulnerability patch code pairs. SVEN contains 803 vulnerability patch code pairs, covering 9 CWE categories, and each code pair has been manually verified. SafeCoder contains 465 code pairs, spanning 6 programming languages ​​and 23 CWE categories, and each sample has been verified using a static analyzer. Both datasets provide CWE labels, along with the vulnerable code and patch code, and confirm that the patches effectively fix the vulnerabilities and do not contain irrelevant modifications.

[0069] The steps for processor 210 to build a comparative analysis dataset based on vulnerability fix code include the following operations.

[0070] S101: Processor 210 compares the execution differences of the vulnerability fix code to identify modified code blocks.

[0071] Preferably, processor 210 obtains vulnerability fix code pairs containing the vulnerable code and the patched code from the vulnerability fix code. Processor 210 performs character-level difference comparisons on the vulnerability fix code pairs to identify modified code blocks. Preferably, processor 210 performs character-level difference comparisons on each vulnerability fix submission in the underlying dataset.

[0072] S102: Processor 210 extracts lines of code from the modified code block to obtain vulnerable code fragments and safe code fragments.

[0073] Preferably, processor 210 extracts the deleted lines of code as vulnerable code segments from each modified code block. Preferably, processor 210 excludes comment lines from the extracted deleted lines of code.

[0074] Preferably, processor 210 discards modified code blocks that do not contain the deleted lines of code. Preferably, processor 210 extracts the added lines of code from each modified code block as safe code fragments.

[0075] S103: Processor 210 performs code retrieval based on vulnerable code fragments to obtain benign code fragments.

[0076] Processor 210 searches within the same code repository, outside of modified code blocks, based on lexical similarity and / or semantic similarity, to find lines of code similar to fragile code fragments, and selects the lines of code with the highest similarity as benign code fragments among non-fragile code fragments. Preferably, processor 210 uses fragile code fragments as query conditions for retrieval. Preferably, processor 210 employs BM25 retrieval based on the bag-of-words model, cosine similarity retrieval based on code embedding vectors, or similarity matching based on abstract syntax tree structures to achieve code retrieval.

[0077] The specific implementation method of the above-mentioned benign code snippet retrieval is as follows.

[0078] Processor 210 uses a pre-trained code embedding model (such as the OpenAI text-embedding-large embedding model) to map fragile code snippets and each candidate line of code into dense vector representations. and , where d e For the embedding dimension, the formula for calculating the cosine similarity between the vulnerable code snippet and each candidate line of code is: .

[0079] Processor 210 sorts candidate code lines in descending order of cosine similarity and selects the candidate code line with the highest similarity as a benign code segment.

[0080] like Figure 2 As shown, the process of retrieving benign code snippets and the role of the four comparison types are illustrated using the CWE-326 (weak encryption) vulnerability as an example.

[0081] like Figure 2 As shown, the code file services / file_crypto.py contains the code context before and after the vulnerability modification. Lines 112 (plaintext=json.dumps(metadata).encode("utf-8")), 113 (ENCRYPT_MODE=AES.MODE_ECB), 116, and 117 are unchanged. Line 114 is a deleted line (KEY_SIZE=128). Line 115 is a newly added line (KEY_SIZE=256).

[0082] exist Figure 2 In the example, the constant 128 has different semantics in different contexts. For example... Figure 2As shown, the vulnerable code snippet is "KEY_SIZE=128". The source of this vulnerable code snippet is the deleted line before modification (used as a baseline for benign comparison). In the vulnerable code snippet, 128 is used as the AES key length parameter, specifying an insecure key length and constituting a weak encryption vulnerability. The relevance score corresponding to the vulnerable code snippet is +1.

[0083] like Figure 2 As shown, the secure code snippet is "KEY_SIZE=256". The secure code snippet originates from the modified added line. In the secure code snippet, 256 replaces 128 as the secure AES key length, fixing the weak encryption issue. The relevance score corresponding to the secure code snippet is -1.

[0084] like Figure 2 As shown, the benign code snippet is "RANDOM_SEED=128". The benign code snippet was extracted from an unmodified code context retrieval. In the benign code snippet, 128 is used as a random seed and is irrelevant to cryptographic security. The relevance score corresponding to the benign code snippet is 0.

[0085] like Figure 2 As shown, the content of a neutral code snippet is, for example, `plaintext=json.dumps(metadata)`. The source of the neutral code snippet is the unmodified code context. The relevance score corresponding to the neutral code snippet is 0.

[0086] Vulnerable and secure code snippets are used to learn vulnerability characteristics and their corresponding remediation patterns. Vulnerable and benign code snippets are used to learn the differences in characteristics of the same tag under different security semantics. Neutral code snippets are used to provide a broader range of code patterns as a benchmark for comparison.

[0087] Processor 210 performs a character-level difference comparison operation on the code in services / file_crypto.py, extracting the vulnerable code fragment KEY_SIZE=128. Processor 210 then uses the extracted vulnerable code fragment KEY_SIZE=128 as input to perform a semantic search operation within the code repository. The code file services / review_sampler.py contains the context of the searched code.

[0088] The processor 210 performs the retrieval process for the aforementioned benign code fragment as described below.

[0089] Using the vulnerable code snippet as the query criterion, all lines of code other than the modified code block were retrieved and ranked within the same code repository. In candidate line 114, RANDOM_SEED=128 was selected as a similar line due to containing the same term 128 as the vulnerable code snippet, achieving the highest similarity score. Since the candidate line and its context fall within the neutral code snippet range, confirming that the candidate line is similar to the vulnerable code snippet but does not involve cryptographic security, RANDOM_SEED=128 was chosen as the benign code snippet.

[0090] The introduction of benign code snippets can effectively reduce false positives caused by the same term (e.g., 128) having different security semantics in different contexts. The above construction process yielded 1,068 valid code pairs, of which 689 were from the SVEN dataset and 379 were from the SafeCoder dataset. These code pairs cover 21 CWE categories.

[0091] S104: Processor 210 marks unmodified code contexts, other than fragile code segments, safe code segments, and benign code segments, as neutral code segments.

[0092] S105: Preferably, processor 210 filters out CWE categories with fewer than 5 vulnerable code segments. Processor 210 performs the filtering operation to ensure the statistical reliability of the analysis.

[0093] S200: Neuron Probing: Input code snippets from the comparative analysis dataset into the code language model, and obtain the activation values ​​of each neuron in the code language model on fragile and non-fragile code snippets respectively.

[0094] The goal of this step is to traverse all code snippets in the comparative analysis dataset and record the highest activation value of each neuron in the code language model and its corresponding comparison type label.

[0095] Preferably, the processor 210 initializes a highest activation value buffer for each neuron in the code language model. The highest activation value buffer is configured to store a preset number of activation values.

[0096] Processor 210 performs lexicalization on code snippets in the contrastive analysis dataset and assigns contrastive type labels to the generated lexical tokens. Processor 210 constructs code prefixes for the code snippets and inputs them into the code language model to perform forward propagation to obtain neuron activation values ​​corresponding to lexical positions.

[0097] Preferably, the processor 210 configures a highest activation value buffer for the target neuron in the code language model and updates the highest activation value buffer using a streaming update strategy. Specifically, the processor 210 determines whether the current activation value of the target neuron meets a preset replacement condition. The preset replacement condition includes at least that the current activation value is greater than or equal to the minimum value in the highest activation value buffer; if the preset replacement condition is met, the minimum value in the highest activation value buffer is updated using the current activation value, the corresponding code prefix, and the comparison type label.

[0098] Processor 210 receives the input code language model and performs neuron detection operations, such as... Figure 3 As shown.

[0099] S201: Initialize a buffer of the highest activation value of size k for each neuron.

[0100] Initialize a highest activation value buffer TABν of size k for each neuron ν in the code language model. The capacity of the highest activation value buffer is configured as k. In this embodiment, k is set to 15.

[0101] Step S202: Lexicalize the code snippet and label each lexical with a comparison type label.

[0102] Specifically, the processor 210 performs lexicalization on each code snippet s in the contrast analysis dataset to obtain a lexical sequence t, and labels each lexical with its corresponding contrast type label l.

[0103] Step S203: Construct a code prefix, input the code language model, perform forward propagation, and obtain the activation values ​​of all neurons.

[0104] Specifically, processor 210 constructs a code prefix p=t for each position j in the lexical sequence. 1..j The code prefix is ​​input into the code language model to perform forward propagation, and the activation values ​​'a' of all neurons are obtained.

[0105] The process of obtaining the activation values ​​of all neurons in step S203 is achieved by performing parallel matrix operations on the high-dimensional activation tensor using the graphics processing unit (GPU) 212. This parallel processing mechanism logically implies a traversal of the massive number of neurons within the model (such as the 327,680 neurons in the CodeGen-2B model). Therefore, the activation value 'a' output in step S203 is a vector containing the states of all neurons. The minimum value replacement operation in subsequent step S204 is a synchronous judgment performed on each element of the vector of all neuron states; the single-step serial loop traversal process is not explicitly shown in the flowchart.

[0106] To clarify the boundaries of the underlying algorithm operations and the internal loop control logic, the central processing unit 211 and the graphics processing unit 212 work together to form nested loop control loops. While the code language model is in the outer traversal state of continuous word-by-word forward reasoning, the graphics processing unit 212 establishes an inner comparison and update loop for L×D independent neurons. For each word position, the graphics processing unit 212 reads the activation tensor of the current layer neurons in parallel and compares and replaces it with the minimum value in the corresponding highest activation value buffer of each neuron, enabling the processor to obtain the maximum activation value of each neuron in the full forward propagation step.

[0107] Step S204: Determine if the activation value is greater than the minimum value of the buffer. If yes, update the buffer and replace the minimum value. Then (or regardless of whether it is updated), determine if all word positions have been traversed. If yes, proceed to step S205. If no, proceed to step S203.

[0108] Specifically, for each neuron ν, if the current activation value a ν If the value is greater than the minimum value in the buffer TABν, then perform an update operation: replace the minimum item in the buffer with the current code prefix p and activation value a. ν And contrast type label l j .

[0109] Step S205: Output the buffer set of all neurons.

[0110] After traversing all code snippets, return the buffer set {TAB} of all neurons. ν} ν The final output is the neuron detection result.

[0111] The advantage of the above streaming probing strategy is that it only needs to maintain a fixed-size buffer for each neuron, without needing to store all activation values ​​across all code segments, thus achieving high efficiency in memory usage. The underlying computational implementation of the streaming update strategy is explained in detail below.

[0112] The buffer for the highest activation value of each neuron, with a capacity of k, is implemented using a min-heap data structure. A min-heap is a complete binary tree structure where the root node stores the minimum activation value in the current buffer. It supports O(1) time complexity for querying the minimum value, and O(logk) time complexity for insertion and replacement operations. With k=15, the replacement operation takes a maximum of [time value missing]. The comparisons and swaps are performed. The buffer is deployed in the global video memory of the graphics processing unit (GPU) 212 during the forward propagation of the code language model, sharing the same physical storage device with the model parameters and intermediate activation tensors. For a model with L layers, each with a hidden dimension of D, and a total of L×D neurons, with each neuron's buffer storing k entries, each entry containing an activation value (float32, 4 bytes), a code prefix pointer (int64, 8 bytes), and a comparison type label (int8, 1 byte), the total video memory usage of the buffer is: .

[0113] Taking CodeGen-2B as an example, with L=32, D=10,240, and k=15, the total video memory usage of the buffer is approximately 13×15×32×10,240≈60.9MB, which is much smaller than the storage requirements of the model parameters themselves.

[0114] The streaming update operation is implemented through the forward hook mechanism of the deep learning framework. Specifically, a forward hook function is registered on each layer of the MLP module in the code language model. When the code language model performs forward propagation on the input code prefix, the registered hook function is automatically invoked after the computation of each layer of the MLP module is completed. The hook function directly reads the activation tensors of all neurons in the current layer from the GPU's 212 memory. For each neuron activation value The value of a is compared with the root node of the corresponding min-heap. If a is greater than the activation value of the root node, a heap replacement operation is performed, replacing the root node with the new entry and performing a sinking adjustment. The above comparison and update operations are all performed in the graphics processing unit (GPU) 212 to avoid data transfer overhead between the graphics processing unit (GPU) 212 and the central processing unit (CPU) 211.

[0115] Compared to conventional methods that require storing all activation values, streaming update strategies offer a significant advantage in terms of memory usage. With conventional full storage, storing all activation values ​​for a dataset containing N total words and a model with M = L × D neurons requires O(N × M) memory. For example, in CodeGen-2B, with M = 327,680 neurons and a dataset containing approximately 500,000 words, full storage would require 327,680 × 500,000 × 4 bytes ≈ 610.4 GB, far exceeding the GPU memory capacity of a single server. In contrast, streaming update strategies require only O(k × M) memory, approximately 60.9 MB when k = 15, achieving a memory reduction of over three orders of magnitude. Compared to integrated gradient methods, which require performing n forward and backward propagations for each sample (typically...), streaming update strategies offer a significant advantage. ), while streaming detection only requires one forward propagation for each sample, achieving higher computational efficiency. It achieves a speedup of several times, without the need to calculate and store gradient information.

[0116] S300: Neuron localization.

[0117] Processor 210 receives the buffer set of all neurons output from neuron detection step S200. The buffer set records the top k highest activation values ​​of each neuron on the contrastive analysis dataset and their corresponding contrast type labels. Based on the distribution differences of the activation values ​​of each neuron in the buffer set across fragile and non-fragile code segments, processor 210 identifies neurons with a response preference to fragile code segments and designates them as vulnerability-selective neurons. The specific execution flow is as follows... Figure 4 As shown.

[0118] The goal of this step is to identify neurons that are selectively associated with the generation of fragile code, based on the detection results.

[0119] Preferably, the processor 210 acquires a sorted list of target neurons that meet preset activation conditions, where the activation values ​​in the sorted list are ranked based on their numerical values. The processor 210 calculates a selection score for the target neuron based on the code segment type corresponding to the activation value in the sorted list. The selection score measures the concentration of activation values ​​on vulnerable code segments. Based on the selection score, the processor 210 filters out vulnerable selective neurons from the neurons of the code language model.

[0120] According to a preferred embodiment, the processor 210 determines the relevance weight of each rank based on the code segment type to which the activation value at each rank in the target neuron's ranking list belongs. The relevance weight corresponding to fragile code segments contributes positively to the selection score, while the relevance weight corresponding to non-fragile code segments contributes negatively or zero to the selection score; the selection score of the target neuron is calculated based on the positional loss weight and relevance weight associated with the rank.

[0121] like Figure 4 As shown, the specific process of neuron localization is as follows.

[0122] Step S301: The processor 210 sorts the top k highest activation values ​​of each neuron in descending order of activation value, forming a sorted list, such as... Figure 4 As shown.

[0123] Step S302: Assign correlation scores to the top k highest activation values ​​of neurons based on the comparison type label.

[0124] Specifically, each activation value in the sorted list is assigned a relevance score based on its comparison type label: activations corresponding to vulnerable code snippets are assigned a positive relevance (e.g., +1), activations corresponding to safe code snippets are assigned a negative relevance (e.g., -1), and activations corresponding to benign and neutral code snippets are assigned zero relevance.

[0125] Step S303: Based on the sorted list of activation values ​​assigned relevance, calculate the selectivity score for each neuron using the position-depreciated weighting method.

[0126] Specifically, based on the ranked list assigned relevance, a ranking quality score is calculated as the selectivity score of the neuron. Specifically, the ranking quality score function S is defined as: .

[0127] In the above formula, The position loss weight is the position of the i-th sorting position. The weights for the comparison type labels corresponding to the sorting positions are: +1 for fragile code segments, -1 for safe code segments, and 0 for benign and neutral code segments. The denominator is the maximum absolute score under ideal sorting, used to normalize S(ν) to the interval [-1, 1]. A higher S(ν) value indicates that the highest activation of neurons is more concentrated on fragile code segments.

[0128] Preferably, .

[0129] In the above formula, To prevent the minimum smoothing term from being abnormally divided by zero.

[0130] Step S304: For each CWE category, select the top t% of neurons in terms of selectivity score, and merge them across CWE categories to obtain a set of vulnerability-selective neurons.

[0131] Specifically, for each CWE category, all neurons are sorted in descending order of selectivity score, and the top 0.1% of neurons are selected as the vulnerability-selective neurons for that CWE category. The vulnerability-selective neurons for all CWE categories are then deduplicated and merged to obtain the vulnerability-selective neuron set for the code language model.

[0132] Figure 5 This is a diagram of a vulnerability detection architecture based on vulnerability-selective neuron features. The data processing flow of the vulnerability detection architecture is divided into a training phase and a detection phase.

[0133] During the training phase, the training set code is input into the code language model whose base layer parameters are frozen. The code language model contains a multi-layered structure. Figure 5 The layers are labeled as layer 1, layer 2, up to layer L. In a vulnerability detection architecture, such as... Figure 5 As shown, the base layer parameters of the code language model remain frozen. After processing the input data, the code language model extracts the activation values ​​of vulnerability-selective neurons. Based on the extracted activation values, the code language model trains a vulnerability detection classifier, enabling the classifier to perform classification judgments using the activation features of vulnerability-selective neurons, thereby achieving vulnerability status detection, reducing interference from code context ambiguity, and improving the accuracy and interpretability of vulnerability status prediction. In the detection phase, the code to be tested is input into the code language model, the corresponding activation values ​​are extracted and input into the trained vulnerability detection classifier. The vulnerability detection classifier performs vulnerability status prediction on the code to be tested and outputs the vulnerability status prediction result.

[0134] In this embodiment, vulnerability-selective neurons are identified using the following three code language models as examples (the method of this invention is also applicable to other code language models based on the Transformer architecture): CodeGen-2B (32 layers × 10,240 neurons = 327,680 total neurons), DeepSeek-Coder-1.3B (24 layers × 5,504 neurons = 132,096 total neurons), and Llama-3.2-1.2B (16 layers × 8,192 neurons = 131,072 total neurons). For each CWE category, the top 0.1% of neurons are selected as vulnerability-selective neurons, as shown in Table 1. In each CWE category, 327, 132, and 131 vulnerability-selective neurons are obtained in the three models, respectively. After deduplication across CWE categories, 6,755, 2,729, and 2,721 unique vulnerability-selective neurons are obtained, respectively, accounting for approximately 2.1% of the total number of neurons. According to the data in Table 1, the CodeGen-2B model, DeepSeek-Coder-1.3B model, and Llama-3.2-1.2B model showed 112, 43, and 30 overlapping neurons, respectively, across different CWE categories.

[0135] Table 1: Statistics on the number of Vulnerability-Selective Neurons (VSNs) in the three code language models

[0136] The initial 0.1% screening threshold is a hyperparameter that controls the balance between selectivity and coverage. Sensitivity analysis was conducted by varying the threshold from 0.01% to 1.0% (in 0.01% increments): at a threshold of 0.01%, only 13 to 32 highly selective neurons were selected per CWE category, posing a risk of missing key vulnerability-selective neurons; at a threshold of 1.0%, each CWE category contained approximately 1,300 to 3,300 neurons, but most of them lacked vulnerability selectivity.

[0137] S400: Vulnerability Detection.

[0138] The code to be tested is obtained, and the activation values ​​at the corresponding positions of the vulnerability selective neurons are extracted to form a feature vector. The feature vector is then input into the vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

[0139] This step utilizes the identified vulnerability-selective neurons for vulnerability detection.

[0140] Preferably, the processor 210 performs lexicalization on the code under test and extracts the scalar activation values ​​output by all vulnerability-selective neurons at each lexical position in the code under test to construct a lexical-level feature vector. The processor 210 performs feature aggregation on multiple lexical-level feature vectors belonging to the same code unit based on the code unit boundaries of the code under test, thereby obtaining a code unit-level feature vector. The processor 210 inputs the lexical-level feature vector or the code unit-level feature vector into the vulnerability detection classifier to output the vulnerability status of at least one code unit.

[0141] The vulnerability detection process based on vulnerability-selective neuron features is as follows: Step S401: Keep the base layer parameters of the code language model frozen and do not perform gradient updates.

[0142] Step S402: Input the code to be tested into the code language model, extract the activation values ​​of the vulnerability selective neurons at each word position, and construct the feature vector.

[0143] Step S403: Attach a trainable classification head to the feature vector of the vulnerability-selective neuron for vulnerability detection. The specific network structure and feature construction method of the classification head are as follows: The feature vector x at word position u in the code to be tested u It is composed of the activation values ​​of all identified vulnerable selective neurons at the lexical position: .

[0144] In the above formula, where ν1,ν2,...,ν |VSN| For the identified set of vulnerable selective neurons, D vsn =|VSN| represents the total number of vulnerability-selective neurons.

[0145] The classification head uses a single-layer linear classifier, whose mathematical expression is: .

[0146] in, For a trainable weight matrix, For bias vectors, For word position The probability distribution predicting the vulnerability status at a given location has two components corresponding to the vulnerable and non-vulnerable categories, respectively. The total number of parameters in the classification head is 2×D. vsn +2, for example, for the CodeGen-2B model (D vsn =6,867), the classification head contains only 13,736 trainable parameters, which is negligible compared to the billions of parameters in the model itself.

[0147] In one implementation, it is directly based on the word-level feature vector x. u Perform vulnerability status prediction on a word-by-word basis. In another implementation, word-level feature vectors can be extracted first, and then multiple word-level feature vectors can be aggregated according to the code line, statement, function, or file boundary to which the word belongs to obtain the corresponding code unit-level features.

[0148] The aggregation method uses mean pooling: it takes the arithmetic mean of the feature vectors of all T word positions belonging to the same code unit. The calculation formula for mean pooling in the aggregation method is as follows: .

[0149] Then the vulnerability detection unit 140 will aggregate the code unit-level features. Input the classification header to perform vulnerability classification prediction. In addition to mean pooling, feature aggregation can also be achieved by summation pooling, max pooling, or weighted aggregation based on attention mechanisms.

[0150] Preferably, before training the classification head, the processor 210 acquires training set code containing known vulnerability status labels. The processor 210 freezes the code language model with input parameters from the training set code and extracts the scalar activation values ​​of all vulnerability-selective neurons at the corresponding lexical positions in the training set code, following the same feature extraction logic as the code to be tested, to construct a set of baseline feature vectors for model training.

[0151] Step S404: Formulate vulnerability detection as a binary classification task, predict the vulnerability status (vulnerable or non-vulnerable) of each word, and use 1:1 vulnerable-benign word resampling and 9:1 training-test set split.

[0152] Step S405: Train the classification head using the AdamW optimizer (learning rate 2×10⁻⁵) and the cross-entropy loss function.

[0153] Table 2: Performance Comparison of Different Methods on Vulnerability Detection Datasets

[0154] As shown in Table 2, experimental results demonstrate that using the activation values ​​of vulnerability-selective neurons as features (VSN-FT) achieves the highest accuracy across all settings, reaching 79.3% to 84.4% accuracy on the BigVul dataset and 78.7% to 80.9% accuracy on the CrossVul dataset. The DeepSeek-Coder-1.3B model performs best on both datasets. Compared to using all hidden states as features (FT), VSN-FT achieves a maximum improvement of 15.6% on BigVul and 11.9% on CrossVul. Using random neurons as features (Rand.FT) only achieves 54.8% to 59.1% accuracy; the significant gap between this and VSN-FT confirms that the identified vulnerability-selective neurons specifically encode vulnerability-related concepts.

[0155] Example 2 This embodiment is a further illustration of the vulnerability-selective neuron characteristics and interpretability in Embodiment 1.

[0156] This example illustrates the cross-model consistency of vulnerable selective neurons.

[0157] This invention analyzes the similarity of vulnerability-selective neuron layer distributions across different models. Layer positions are normalized to the [0,1] interval, and the proportion of vulnerability-selective neurons in each layer is interpolated to form a 100-dimensional density vector. Cosine similarity is calculated under three settings: (1) The mean cosine similarity of the same CWE category across different models is 0.82; (2) The similarity between different CWE categories across models is 0.78; (3) The similarity of random neurons was 0.63. The cross-model similarity of the same CWE was significantly higher than that of the other two baselines (p<0.001), indicating that the vulnerability-specific neuron pattern reflects the underlying code semantics rather than model-specific artifacts.

[0158] This embodiment illustrates the interpretability of vulnerable selective neurons.

[0159] Taking CodeGen-2B as an example, representative vulnerability-selective neurons for each of the 21 CWE categories were manually analyzed. Two authors with over 5 years of software engineering experience independently reviewed each vulnerability-selective neuron, recording its associated vulnerability type, activation conditions, representative code snippets, and whether the activation pattern was recurring. The analysis revealed two distinct types of vulnerability-selective neurons: (1) Lexical vulnerability-selective neurons identify specific keywords or API calls related to vulnerabilities and are mainly located in the early layers of the model. For example, the vulnerability-selective neurons of CWE-611 (XML External Entities) are consistently focused on the early layers and respond to specific tags such as XML_PARSE_NOENT.

[0160] (2) Semantic vulnerability-selective neurons detect structural patterns that require contextual understanding and are concentrated in the middle and deep layers of the model. For example, the vulnerability-selective neurons in CWE-078 (operating system command injection) are located in the middle layer and respond to the format placeholder %s in shell commands.

[0161] This distinction corresponds to the differentiation in the CWE classification between inherently unsafe constructs (such as gets, yaml.load) and the misuse of benign constructs (such as string formatting in shell commands, integer multiplication in buffer allocation).

[0162] Example 3 This embodiment is a detailed description of the robustness verification in Embodiment 1.

[0163] To verify that the vulnerability-selective neurons learn generalizable vulnerability features rather than the memory of specific lexical patterns, four semantically preserving code transformations were applied to the vulnerable code fragments: (1) Blank transformation: randomly insert blank lines; (2) Comment transformation: Add comments; (3) Renaming transformation: Replace local variable names with synonyms; (4) Dead code transformation: inserting declarations of unused variables.

[0164] For each CWE category, 10 vulnerable code fragments were selected, and four transformations were applied, resulting in a total of 840 transformed fragments. For each pair of original and transformed codes, neurons were sorted according to activation magnitude, and the Spearman correlation coefficient ρ between the two sorts was calculated.

[0165] Figure 9 The distribution of Spearman correlation coefficients (Spearman ρ) of vulnerable selective neurons under four types of code transformations is shown. Figure 9 The horizontal axis represents the Spearman correlation coefficient. The Spearman correlation coefficient indicates the consistency of the neuron activation values ​​ranking before and after code transformation in fragile code segments. Figure 9The vertical axis represents the probability density of the Spearman correlation coefficient. The legend (Operator) corresponds to four semantically preserving code transformation operations, including: whitespace transformation, which randomly inserts blank lines; comment transformation, which adds comments; rename transformation, which replaces local variable names with synonyms; and dead code transformation, which inserts declarations of unused variables. Figure 9 The vertical solid line in the graph represents the average Spearman correlation coefficient, with a value of 0.965. The probability density distribution curves are concentrated in the region with values ​​close to 1.

[0166] like Figure 9 As shown, the experimental results indicate that the average Spearman correlation coefficient is (Standard deviation = 0.011), indicating that vulnerability-selective neurons maintain a consistent activation pattern under code transformation. All four transformation types exhibit high stability: blank transformation (ρ = 0.964), comment transformation (ρ = 0.964), rename transformation (ρ = 0.965), and dead code transformation (ρ = 0.965). The recall of vulnerability-selective neurons is 0.74, indicating that 74% of the vulnerability-selective neurons identified in the original code are still identified as vulnerability-selective neurons in the transformed version.

[0167] Figure 9 The data distribution results show that after applying various code transformation operations to the vulnerable code fragment, the vulnerability-selective neurons maintain a consistent activation pattern. The validation results confirm that the vulnerability-selective neurons learn generalizable vulnerability features, rather than memorizing specific lexical patterns, thus verifying the robustness of the code vulnerability detection method.

[0168] Example 4 This embodiment provides a code vulnerability detection device 100 or a code vulnerability detection apparatus based on internal features of a model.

[0169] like Figure 8 As shown, the code vulnerability detection device 100 includes a comparison dataset construction unit 110, a neuron detection unit 120, a neuron localization unit 130, and a vulnerability detection unit 140.

[0170] The comparison dataset construction unit 110 is used to acquire a comparison analysis dataset built based on vulnerability patching code. The comparison analysis dataset includes vulnerable code fragments and non-vulnerable code fragments for comparison. The neuron detection unit 120 is used to input the code fragments from the comparison analysis dataset into a code language model, and obtain the activation values ​​of each neuron in the code language model on both vulnerable and non-vulnerable code fragments. The neuron localization unit 130 is used to locate neurons with a response preference to vulnerable code fragments based on the distribution differences of the activation values ​​of each neuron on the vulnerable and non-vulnerable code fragments, and designates these as vulnerability-selective neurons. The vulnerability detection unit 140 is used to acquire the code to be tested, extract the activation values ​​at the corresponding positions of the vulnerability-selective neurons to form a feature vector, and input the feature vector into a vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

[0171] When extracting activation values, the vulnerability detection unit 140 can directly invoke the code language model configured in the neuron detection unit 120 through an internal call interface or cross-domain data bus. Based on the received index set, the vulnerability detection unit 140 controls the code language model to perform forward propagation of the code under test, thereby completing the extraction of feature vectors.

[0172] The data flow relationships between the above units are as follows: The comparison dataset construction unit 110 outputs a set of code snippets labeled with four comparison types in the comparison analysis dataset to the neuron detection unit 120. The neuron detection unit 120 outputs the activation values ​​of each neuron on the vulnerable and non-vulnerable code snippets (the k highest activation values ​​of each neuron) to the neuron localization unit 130. The neuron localization unit 130 outputs the index set of vulnerability-selective neurons to the vulnerability detection unit 140.

[0173] Each of the above units can be implemented by an independent computing module or by a software module on the same computing device. In a preferred embodiment, the device can be deployed on a workstation or server equipped with a GPU, implemented using the Python programming language, running a code language model using the Hugging Face Transformers library, and capturing internal activations using the baukit library. In other embodiments, other deep learning frameworks (such as Keras, TensorFlow, etc.), other activation capture modules, or other computing platforms can also be used.

[0174] The computing power platform is the infrastructure for deploying and supporting the operation of the code vulnerability detection device 100. The specific form of the computing power platform is a public cloud graphics processing unit 212 computing instance, an enterprise's local distributed computing cluster, a computing workstation configured with multiple graphics processing units 212, or an edge computing server with hardware acceleration modules.

[0175] Example 5 This embodiment describes an application scenario of a code vulnerability detection method based on internal features of a model in a real code development environment.

[0176] In the offline phase, using the comparative analysis dataset construction, neuron detection, and neuron localization steps in Example 1, vulnerability-selective neurons are pre-identified from the target code language model, and their layer positions and dimension indices are recorded. Simultaneously, using the vulnerability detection methods from S100 to S400 in Example 1, a vulnerability detection classifier based on the activation features of vulnerability-selective neurons is trained.

[0177] like Figure 6 As shown, in the online detection phase, the method of this invention can be integrated into an integrated development environment (IDE) to detect potential vulnerabilities in real time while developers are writing code. The specific interaction process is as follows: Figure 6 As shown.

[0178] The IDE submits code, which means that the code snippets written by the developer are sent to the backend vulnerability detection server through the API interface.

[0179] The backend vulnerability detection server receives the code snippet to be tested and enters the code vulnerability detection process. The backend vulnerability detection server performs lexicalization on the code snippet to be tested, and inputs the generated lexical sequence into the loaded code language model to perform forward propagation.

[0180] The backend vulnerability detection server retrieves and obtains a pre-stored list of vulnerability-selective neurons. Based on the retrieved list of vulnerability-selective neurons, the backend vulnerability detection server extracts the activation values ​​of the pre-identified vulnerability-selective neurons at the lexical positions corresponding to the code to be tested, and concatenates the activation values ​​to form a feature vector.

[0181] To ensure consistency between the online detection timing logic and the graphical boundaries of functional modules, a specific timing sequence is configured during real-time vulnerability prediction inference. The backend vulnerability detection server, after retrieving and loading a pre-generated list of vulnerability-selective neurons generated offline, obtains the index data of the target neurons. After obtaining the index data, the vulnerability detection unit 140 performs targeted extraction on the tensor output by the code language model based on the index data, obtaining the activation values ​​of neurons at corresponding positions. The vulnerability detection unit 140 internally includes a feature vector construction submodule, a feature aggregation processing submodule, and a vulnerability detection classifier. These components constitute a feature analysis structure, enabling the vulnerability detection unit 140 to independently output the vulnerability status of the code under test. The backend vulnerability detection server inputs the feature vectors into the trained vulnerability detection classifier, classifies and predicts the vulnerability status of each term or line of code, and generates vulnerability detection results. Furthermore, although in Figure 6In the interaction sequence, retrieving the vulnerability selective neuron index set appears as a concurrent or subsequent response. However, in actual logic execution, the system must first or concurrently acquire the vulnerability selective neuron index set, using it as an absolute prerequisite for executing subsequent targeted extraction of activation values ​​to ensure the causal loop of the detection logic. Vulnerability detection results include the vulnerability probability value at each code location and the corresponding CWE vulnerability category.

[0182] The backend vulnerability detection server sends the vulnerability detection results to the IDE. The IDE receives the vulnerability detection results and, based on these results, annotates and displays the code snippets identified as potentially vulnerable in the editor interface to show the corresponding vulnerability types.

[0183] The integrated development environment outputs alarm information and, upon receiving the resubmitted modified code, triggers a verification process to re-execute the code vulnerability detection.

[0184] Preferably, the integrated development environment (IDE) performs annotation on code snippets identified as potentially vulnerable within the editor interface. Annotation includes highlighting. Annotation also includes adding wavy or straight underlines to the corresponding code text area. The IDE renders warning icons in the sidebar or line number display area of ​​the editor interface. The IDE performs annotation by changing the foreground color of the code text, adjusting font weight parameters, or modifying the text background color. The IDE configures a hover tooltip component to perform annotation. When the IDE detects that the input device's locator is hovering over a specific code snippet area, it displays a hover tooltip showing the vulnerability category and detection data.

[0185] In online detection scenarios, input devices are used to generate hover locators to trigger the interactive display of code vulnerability information. Specific peripheral hardware components of input devices include computer mice, touchpads, trackballs, touchscreens, or digital styluses.

[0186] Example 6 Based on the same technical concept, this embodiment also provides a computer device.

[0187] like Figure 7 As shown, the computer device includes a memory and one or more processors, as well as a system bus 230 connecting the different components.

[0188] The memory 220 is used to store computer programs and related data executed by the processor 210. In hardware implementation, the memory 220 can be volatile memory, such as random access memory (RAM); it can also be non-volatile memory, such as read-only memory, flash memory, hard disk (HDD), or solid-state drive (SSD); or the memory 220 can be any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer.

[0189] Logically, the memory 220 mainly includes a program storage area 221 and a data storage area 222. The program storage area 221 is used to store the operating system, the vulnerability detection method program running the present invention, and the model inference engine. The data storage area 222 is used to store code language model parameters, comparative analysis datasets used for training and analysis, and key data such as a pre-identified and recorded list of vulnerability-selective neurons.

[0190] The model inference engine is located in the stored program area 221 and is used to perform forward inference of the code language model and low-level GPU memory scheduling. The framework of the code language model inference engine can be a large language model inference engine, a cross-platform deep learning runtime library, or an application programming interface environment based on a specific computing framework, covering runtime components such as vLLM, TensorRT-LLM, or ONNX Runtime.

[0191] The processor 210 comprises a central processing unit (CPU) 211 and a graphics processing unit (GPU) 212. The CPU 211 is used for program scheduling, flow control, and data preprocessing and post-processing of input and output data in the computer device 200. The GPU 212 is used to perform computationally intensive tasks, specifically including performing forward inference of a code language model to capture neuron activation values, and performing the training and inference process of a vulnerability detection classifier.

[0192] Specifically, the vulnerability detection classifier training and inference refers to the process of constructing and applying a vulnerability determination model using the identified vulnerability-selective neuron features. During the training phase, the graphics processing unit 212 acquires a set of training code feature vectors containing known vulnerability status labels, and iteratively updates the classifier's weight parameters by minimizing the cross-entropy loss function and using an optimization algorithm. During the inference phase, the vulnerability detection unit 140 inputs the feature vector corresponding to the code to be tested into the trained vulnerability detection classifier, outputting the probability distribution of whether the code to be tested belongs to the vulnerable or non-vulnerable category, thereby achieving automated determination.

[0193] In actual operation, computer device 200 receives external code input through a communication module connected to system bus 230. Processor 210 calls instructions and data from memory 220, utilizes the parallel computing capabilities of graphics processing unit 212 to extract features and classify the code under test, and finally outputs the detection report to the communication module via system bus 230, which then sends the output externally. The device structure of this invention, through coordinated optimization of hardware and software, ensures the accuracy and real-time response speed of vulnerability detection under large-scale code corpora.

[0194] The physical hardware components of the communication module can be an Ethernet network interface card, a wireless LAN communication module, a Fibre Channel adapter, or a cellular mobile network communication module. The software interface of the communication module can be a RESTful application programming interface endpoint, a remote procedure call interface, or a real-time communication interface.

[0195] The detection report is the vulnerability assessment result fed back by the code vulnerability detection device 100 to the external environment. Preferably, the data format of the detection report can be a machine-readable format that can be parsed by the integrated development environment, including JSON format data packets, XML documents, YAML files, or static analysis result exchange format files. The detection report can also be in a human-readable format, including PDF text, HTML web page reports, or spreadsheet files.

[0196] It should be noted that the specific embodiments described above are exemplary. Those skilled in the art can devise various solutions inspired by the disclosure of this invention, and these solutions all fall within the scope of this invention and its protection. Those skilled in the art should understand that this specification and its accompanying drawings are illustrative and not intended to limit the scope of the claims. The scope of protection of this invention is defined by the claims and their equivalents. This specification contains multiple inventive concepts; terms such as "preferredly," "according to a preferred embodiment," or "optionally" indicate that the corresponding paragraph discloses an independent concept. The applicant reserves the right to file divisional applications based on each inventive concept.

Claims

1. A code vulnerability detection device based on internal model features, characterized in that, The code vulnerability detection device (100) includes a processor (210), which is configured to: Obtain a comparative analysis dataset built based on vulnerability patching code, the comparative analysis dataset containing vulnerable code snippets and non-vulnerable code snippets for comparison; The code snippets in the comparative analysis dataset are input into the code language model, and the activation values ​​of each neuron in the code language model on the fragile code snippet and the non-fragile code snippet are obtained respectively. Based on the difference in the distribution of activation values ​​of each neuron on the vulnerable code segment and the non-vulnerable code segment, neurons that have a response preference to the vulnerable code segment are located and are regarded as vulnerability-selective neurons; The code to be tested is obtained, and the activation values ​​of the code to be tested at the corresponding positions of the vulnerability selective neurons are extracted to form a feature vector. The feature vector is then input into a vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

2. The device according to claim 1, characterized in that, The non-fragile code fragment includes at least one of secure code fragments, benign code fragments, and neutral code fragments; the step of the processor (210) constructing a comparative analysis dataset based on the vulnerability patch code includes: Obtain vulnerability patching code pairs containing the vulnerable code and the patch code from the vulnerability patching code; The vulnerability fix code is subjected to character-level difference comparison to identify the modified code block; Extract lines of code from the modified code block to obtain the vulnerable code fragment and the secure code fragment; Based on the vulnerable code fragment, code retrieval is performed to obtain the benign code fragment; Unmodified code contexts other than the aforementioned vulnerable, safe, and benign code snippets are marked as neutral code snippets.

3. The device according to claim 1 or 2, characterized in that, The steps of the processor (210) in constructing the comparative analysis dataset based on the vulnerability patch code include: Within the same code repository, a search is performed in the code outside the modified code block based on lexical similarity and / or semantic similarity to find lines of code similar to the fragile code fragment, and the line of code with the highest similarity is selected as the benign code fragment among the non-fragile code fragments.

4. The device according to any one of claims 1 to 3, characterized in that, The step of the processor (210) obtaining the activation values ​​of each neuron in the code language model on the fragile code segment and the non-fragile code segment includes: A highest activation value buffer is initialized for each neuron in the code language model, and the highest activation value buffer is configured to store a preset number of activation values; Lexicalization is performed on the code snippets in the comparative analysis dataset, and the generated lexical tags are labeled with the corresponding comparative type. Construct a code prefix for the code snippet and input it into the code language model to perform forward propagation, in order to obtain the neuron activation values ​​corresponding to the word positions.

5. The device according to any one of claims 1 to 4, characterized in that, The step of the processor (210) obtaining the activation values ​​of each neuron in the code language model on the fragile code segment and the non-fragile code segment further includes: Configure a highest activation value buffer for the target neuron in the code language model, and update the highest activation value buffer using a streaming update strategy; Specifically, for the target neuron, it is determined whether the current activation value meets a preset replacement condition; wherein, the preset replacement condition includes that the current activation value is greater than or equal to the minimum value in the highest activation value buffer; If the preset replacement condition is met, the minimum value in the highest activation value buffer is updated using the current activation value, the corresponding code prefix, and the comparison type label.

6. The device according to any one of claims 1 to 5, characterized in that, The step of the processor (210) locating neurons that have a response preference to the fragile code segment based on the difference in the distribution of activation values ​​of each neuron across the fragile and non-fragile code segments, and designating them as vulnerability-selective neurons, includes: Obtain a sorted list of target neurons that meet preset activation conditions, wherein the ranking of activation values ​​in the sorted list is determined based on their numerical values; The selectivity score of the target neuron is calculated based on the code fragment type corresponding to the activation value in the sorted list. The selectivity score is used to measure the concentration of activation values ​​on the vulnerable code fragment. The vulnerability-selective neurons are selected from the neurons of the code language model based on the selectivity score.

7. The device according to any one of claims 1 to 6, characterized in that, The step of the processor (210) calculating the selective score of the target neuron according to the code fragment type corresponding to the activation value in the sorted list includes: Obtain the sorted list of the target neurons, and determine the relevance weight of each rank according to the code segment type to which the activation value in each rank of the target neurons belongs; wherein, the relevance weight corresponding to the fragile code segment has a positive contribution to the selectivity score, and the relevance weight corresponding to the non-fragile code segment has a negative contribution or zero contribution to the selectivity score. The selectivity score of the target neuron is calculated based on the positional loss weight associated with the ranking and the relevance weight.

8. The device according to any one of claims 1 to 7, characterized in that, The processor (210) is configured to: extract the activation value to calculate the vulnerability status. Lexicalization is performed on the code to be tested, and the scalar activation values ​​output by all the vulnerability-selective neurons at each lexical position of the code to be tested are extracted to construct a lexical-level feature vector. Based on the code unit boundaries of the code to be tested, feature aggregation processing is performed on multiple word-level feature vectors belonging to the same code unit to obtain code unit-level feature vectors; The term-level feature vector or the code unit-level feature vector is input into the vulnerability detection classifier to output the vulnerability status of at least one code unit.

9. A code vulnerability detection method based on internal features of a model, characterized in that, The method includes: Obtain a comparative analysis dataset built based on vulnerability patching code, the comparative analysis dataset containing vulnerable code snippets and non-vulnerable code snippets for comparison; The code snippets in the comparative analysis dataset are input into the code language model, and the activation values ​​of each neuron in the code language model on the fragile code snippet and the non-fragile code snippet are obtained respectively. Based on the difference in the distribution of activation values ​​of each neuron on the vulnerable code segment and the non-vulnerable code segment, neurons that have a response preference to the vulnerable code segment are located and are regarded as vulnerability-selective neurons; The code to be tested is obtained, and the activation values ​​of the code to be tested at the corresponding positions of the vulnerability selective neurons are extracted to form a feature vector. The feature vector is then input into a vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

10. A code vulnerability detection device based on internal model features, characterized in that, The code vulnerability detection device (100) includes: The comparison dataset construction unit (110) is used to obtain a comparison analysis dataset built based on vulnerability patching code, the comparison analysis dataset containing vulnerable code fragments and non-vulnerable code fragments for comparison; The neuron detection unit (120) is used to input the code fragments in the comparative analysis dataset into the code language model and obtain the activation values ​​of each neuron in the code language model on the fragile code fragments and the non-fragile code fragments, respectively. The neuron localization unit (130) is used to locate neurons that have a response preference to the fragile code segment based on the distribution difference of the activation values ​​of each neuron on the fragile code segment and the non-fragile code segment, and to use them as vulnerability-selective neurons. The vulnerability detection unit (140) is used to acquire the code to be tested, extract the activation values ​​of the code to be tested at the corresponding positions of the vulnerability selective neurons to form a feature vector, and input the feature vector into the vulnerability detection classifier to obtain the vulnerability status of the code to be tested.

Images