A method and device for detecting source code vulnerabilities based on deep learning and anti-interference

Abstract

The present invention discloses an interference-resistant deep learning-based source code vulnerability detection method and device, including a neuron deviation abnormality degree calculation module, a neuron deviation abnormality degree control module, and a real-time verification and update parameter module; by adding a deep learning network structure defense layer to the deep learning neural network model, an interference-resistant deep learning-based source code vulnerability detection network model is constructed, the neuron inhibition probability is adaptively adjusted and the neuron activation probability is dynamically adjusted, the source code detection effect of the model under disturbance is enhanced, and the missed detection rate of the model detection is reduced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence information security technology, and in particular to an interference-resistant source code vulnerability detection method and device based on deep learning. Background Art

[0002] Source code detection refers to analyzing software source code to identify possible errors, vulnerabilities or irregularities in the software, thereby improving the quality and security of the software. As software systems continue to expand and become more complex, the quality of the source code is directly related to the reliability and stability of the software. Effective source code vulnerability detection can prevent potential security risks and reduce losses caused by software defects.

[0003] In order to improve the efficiency and accuracy of source code vulnerability detection, existing work has adopted deep learning technology. Source code vulnerability detection based on deep learning, such as convolutional neural networks and graph neural networks, can automatically extract features and learn the grammatical and semantic features of the code from a large amount of source code data, which improves the accuracy of vulnerability detection to a certain extent.

[0004] However, existing technologies are still insufficient in dealing with code perturbations. Code perturbation generation is to modify the code to generate adversarial samples that are functionally equivalent but have different syntax, structure, or style. Some attackers can make minor perturbations to the source code, such as modifying variable names, adding invalid code, adjusting the code format, etc. Although these perturbations do not change the function and semantics of the code, they will interfere with the feature extraction process of the deep learning model, making it difficult to effectively detect source code vulnerabilities, resulting in a decrease in detection accuracy and an increase in missed detection rates. Summary of the invention

[0005] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides an anti-interference source code vulnerability detection method and device based on deep learning. By effectively preprocessing the source code data, building a code perturbation strategy library and a source code vulnerability detection model, introducing a defense layer and adopting a dynamic defense strategy, the anti-interference ability and detection performance of the source code vulnerability detection model are improved.

[0006] The present invention constructs an anti-interference source code vulnerability detection network model based on deep learning. The model is based on a general deep learning neural network model such as TextCNN, etc., and a deep learning network structure defense layer is added. The constructed source code vulnerability detection network model includes: input layer, hidden layer, defense layer, fully connected layer and output layer; the input layer is responsible for receiving the pre-processed data of the source code; the hidden layer uses the convolution kernel of the deep network to extract the input features; the added deep learning network structure defense layer is the key innovation of the model, by dynamically adjusting the activation state of neurons, suppressing overactive neurons or activating low-response neurons to enhance the anti-interference ability of the model; the role of the fully connected layer is to further combine the features processed by the defense layer, so that the model can better summarize and understand all information, and help accurately judge whether there are vulnerabilities in the code and the specific circumstances of the vulnerabilities; the output layer maps the features passed through the fully connected layer to the final detection results, including the existence or non-existence of vulnerabilities. The method of the present invention not only has the ability to adaptively adjust the probability of neuron inhibition, but also can dynamically adjust the probability of neuron activation, thereby improving the expression ability of key neurons. Through this mechanism, the source code detection effect of the model under disturbance can be enhanced, and the missed detection rate of model detection can be reduced.

[0007] The technical solution of the present invention includes:

[0008] A method for detecting source code vulnerabilities based on deep learning and with interference resistance is proposed. By constructing a network model for detecting source code vulnerabilities based on deep learning and with interference resistance, a deep learning network structure defense layer is added to the deep learning neural network model, and the neuron inhibition probability and the neuron activation probability can be adjusted adaptively and dynamically, so as to enhance the source code detection effect of the model under disturbance. The method comprises the following steps:

[0009] Step 1) Build a source code dataset containing known vulnerabilities and a code perturbation strategy library, including:

[0010] 11) Build a source code dataset containing known vulnerabilities;

[0011] Split special characters in the source code. For example, separate symbols such as (,), {,}, *, / , +, -, =, ;, [, ], >, <, ", ->, >>, <<, , from other characters. Split into token sequences by spaces or other separators, and convert the source code into a sequence of keywords, identifiers, operators, constants, etc.

[0012] Determine whether a source code file contains a vulnerability based on its name or content. Traverse all source code files in the specified directory and read their contents. Preprocess each source code file, aggregate the tag sequences of all source code files, and construct a complete source code dataset.

[0013] Build a vocabulary and encode each source code token sequence according to the vocabulary to convert it into a numerical sequence. Use the embedding model to train the source code token sequence to generate an embedding matrix for subsequent deep learning models.

[0014] 12) Build a code perturbation strategy library;

[0015] The definition includes identifier obfuscation, grammatical structure transformation, no-operation code insertion, code block reordering, equivalent logic replacement, and annotation and format perturbation;

[0016] Use static analysis tools to parse the syntax and semantic structure of source code and identify key semantic units in the code (such as function definitions, variable declarations, control flow structures, etc.). Use deep learning models (such as CodeBERT and GNN) to understand the semantic meaning of code snippets and ensure that the perturbed code is equivalent to the original code in terms of logic and function.

[0017] Based on the results of static semantic analysis and deep semantic understanding, the most influential code units are selected for perturbation, such as replacing variable names, refactoring loop structures, or inserting no-operation code snippets. At the same time, semantic constraints are used to ensure that the perturbation does not destroy the original functionality of the code.

[0018] Local perturbation refers to modifying certain small fragments or specific units of the code without affecting the overall logic and structure of the code. For example, adjusting or replacing a single function, statement block, variable or operator. Global perturbation refers to modifying the large-scale hierarchy or global structure of the code, such as adjusting the function call order, reorganizing the module structure, changing the function implementation method, etc. Combining local perturbation and global perturbation strategies, perturbation generation is performed at different code levels (such as a single function, the entire module) to improve the diversity and complexity of adversarial samples.

[0019] Step 2) Add a deep learning network structure defense layer to perform dynamic defense based on the degree of neuron abnormal deviation and random optimization, including:

[0020] 21) Obtain the neuron activation value of the hidden layer data Z of the source code vulnerability detection network model, and calculate the neuron value (neuron activation value) of the defense layer based on the abnormal degree of neuron deviation of the hidden layer;

[0021] The calculation of the abnormal degree of neuronal deviation can be completed by the following formula:

[0022]

[0023] Wherein, Z represents the hidden layer data, that is, the activation value generated by the current neuron after the source code is input. It represents the response strength of the neuron to the input feature. In the present invention, Z is used to reflect the importance of the features extracted from the source code when it passes through the deep learning model to vulnerability detection. (1) Represents the neuron activation value after the deviation abnormality is adjusted. This value integrates the correction of neuron abnormal deviation, and is used to further optimize the model's feature expression ability and anti-interference ability, ensuring that the model can still work stably in the face of disturbance input. Z Represents the expected value of neuron activation, which is calculated based on training data and is used to measure the average response level of neurons under normal input conditions. Z It is the normal working benchmark of neurons and is used to determine whether there is abnormal activation. Z The variance of the neuron activation value reflects the stability of the neuron response. In the present invention, a larger variance means that the neuron is more sensitive to input changes, while a smaller variance indicates that the neuron has a more stable response to the input data. δ is a weight factor used to control the deviation adjustment, which adjusts the sensitivity of the adjustment amplitude. In the present invention, the value of δ determines the intensity of the correction when the neuron activation value deviates from the expected value.

[0024] 22) Use the abnormal degree value of neuron deviation to screen and adjust abnormal neurons;

[0025] To achieve the above adjustment, this method designs two strategies for adjusting abnormal neurons: dynamic adjustment based on the abnormal degree of neuron deviation and random adjustment. The selection and implementation details of each strategy are as follows:

[0026] Dynamic adjustment based on the degree of abnormal neuron deviation: This strategy directly adjusts the activation state of each neuron according to its degree of deviation, using a predefined adjustment strategy (i.e., a dynamic adjustment strategy based on the degree of abnormal neuron deviation) to inhibit overactive neurons and activate low-response neurons.

[0027] Random Tuning: This strategy generates a random mask matrix to decide whether to increase or decrease the bias of these neurons.

[0028] The specific method to determine whether to use dynamic adjustment or random adjustment is: first generate a random tensor R of the same size as the batch, where each element of R is a random value sampled from the uniform distribution Uniform(0,1); then compare R with the set threshold (which can be 0.5) to generate a binary mask M, and apply it to the input data batch size size, and calculate the decision parameter using the following formula: decison = sum(M)-size / 2; decison is the output after applying the mask. Finally, the decision parameter decision is used to determine which method to use to adjust the abnormal degree of neuron deviation; if the output value is greater than 0, dynamic adjustment is selected, and if it is less than or equal to 0, random adjustment is selected.

[0029] If decision>0 indicates to use a dynamic adjustment strategy based on the abnormal degree of neuron deviation, execute 23); if it indicates to use a random adjustment strategy, execute 24).

[0030] 23) Dynamic adjustment based on the degree of abnormality of neuronal deviation;

[0031] First, the corresponding position mask matrix e is generated according to the abnormal degree value of the neuron deviation mask , and then determine whether the degree of deviation at each position is abnormal; dynamically generate activation mask ME activate and suppression mask ME inhibit , which is used to control whether those overactive neurons or low-response neurons need to be adjusted. Both the activation mask and the inhibition mask obey the Bernoulli distribution. The specific operation of dynamically adjusting the abnormal degree of neuron deviation is based on the three sigma principle (3σ principle):

[0032] when When >3, the neuron activation value is considered abnormal and the neuron activation state needs to be reduced. The adjustment formula is:

[0033]

[0034] when When <-3, the neuron activation value is considered abnormal and the neuron activation state needs to be increased. The adjustment formula is:

[0035]

[0036] Where S α As a gain factor, it is used to control the increase in neuron activity; S β As a decay factor, it is used to control the reduction of the activation value; S α and S β The value range is 0~1. (2) is the adjusted neuron activation value.

[0037] 24) Random adjustment; according to the abnormality value Z of neuron deviation (1) Generate the corresponding position matrix r mask ; Generate random activation mask M using Bernoulli distribution activate and random suppression mask M inhibit , through the position matrix r mask Filter where activation and suppression masks are applied to increase uncertainty and diversity during model training.

[0038] Step 3) Train based on source code data and adjust the deviation coefficient and activation inhibition probability in real time;

[0039] 31) Calculate the current loss value; quantitatively evaluate the prediction performance of the model through the loss function to measure the degree of match between the model output and the true label.

[0040] 32) Adaptively adjust activation and inhibition probabilities; dynamically adjust the activation probability p of the model based on the relative change of the loss value activate With the inhibition probability p inhibit , through iteration, the model can adapt to the changes in data distribution during the learning process. The specific adjustment steps are as follows:

[0041] Calculate the relative change in loss value:

[0042]

[0043] Among them, t refers to the number of steps in the training process, that is, the number of times the model parameters are updated during training; Loss (t) is the loss value of the current step (step t); For the previous step ( step) of the loss value. It is the relative change in the loss value of the model in two consecutive parameter updates, which is used to help determine whether the activation and inhibition probabilities need to be adjusted.

[0044] The specific adjustment formula is as follows:

[0045]

[0046] Among them, P activate (t) is the activation probability of the neuron activation value in the defense layer of the constructed model at the tth iteration. inhibit (t) is the inhibition probability of the neuron activation value in the defense layer of the model at the tth iteration. θ is the set relative change threshold of loss (such as 0.01), and γ is the ratio of each adjustment of activation probability or inhibition probability (such as 0.01).

[0047] 33) Optimize the weight factor; use an adaptive adjustment algorithm based on the abnormal degree of neuron deviation to adaptively adjust the weight factor S according to the overall change of neuron deviation α and S β , in order to balance the convergence speed and stability of the model. Specific adjustment steps: In each iteration, by comparing the current and previous Z (1) value, and increase, decrease or maintain S accordingly according to its increase, decrease or flat situation α and S β .

[0048] The specific adjustment formula is as follows:

[0049]

[0050]

[0051] Where S α (t) is the gain factor at the tth iteration, which is used to control the increase of the neuron activity value; S β (t) is the attenuation factor at the tth iteration, which is used to control the reduction of the activation value.

[0052] 34) Integrate the training loop; integrate the above steps 31) to 33) into the training loop of the source code inspection model, use the constructed source code dataset to perform iterative training, and obtain a trained deep learning model.

[0053] Step 4) Use the perturbation code dataset for testing and application deployment;

[0054] 41) Use the perturbation code dataset to fully test the trained deep learning model.

[0055] The constructed perturbation code dataset is input into the model, and the source code is tested to obtain the prediction result after the test. The prediction result is calculated by the output layer of the model through the sigmoid activation function. Specifically, the output layer converts the feature vector Z that has passed through the fully connected layer into (fc) Converted into a probability value P, which indicates the probability that the data contains a vulnerability, with a value range of [0,1]. If the probability value is greater than the threshold (such as 0.5), the model outputs that the data contains a vulnerability; otherwise, the output is that there is no vulnerability.

[0056] The formula for the sigmoid activation function is as follows:

[0057]

[0058] The total feature vector calculation formula of the fully connected layer is:

[0059]

[0060] Among them, W (fc) is the weight matrix of the fully connected layer, b (fc) is the bias, Z (2) is the vector passing through the model defense layer, Z (fc) is the vector after linear transformation of the fully connected layer.

[0061] The prediction results of all samples are used to calculate the accuracy, recall, and F1 value indicators of source code vulnerability detection.

[0062] 42) Multi-level perturbation intensity grading test; divide the perturbation code data set into different intensity levels (such as mild, moderate and high intensity perturbations), and gradually input the model according to the gradient. Test the low-perturbation data first, then test the moderate-perturbation data, and finally test the high-intensity perturbation. After ensuring the basic performance of the model under mild perturbation conditions, gradually increase the perturbation intensity to refine the performance of the model under a progressive perturbation environment. In the present invention, mild perturbation: only a single operation is performed, the code is slightly changed, the impact is limited to the local area, and the semantics and logic remain almost unchanged. Such as identifier obfuscation (modifying variable names, function names or class names), comments and whitespace perturbations (adding, deleting or modifying comments, adjusting the format of the code such as indentation, line breaks), adding code that has no effect on the function (such as empty statements or empty loops), etc. Moderate perturbation: combining two operations to make larger changes to the code. High-intensity perturbation: combining three or more complex operations.

[0063] 43) Multi-dimensional performance indicators and differentiation thresholds;

[0064] Existing performance requirement indicators include accuracy, recall and F1 value. Accuracy is an indicator to measure the correctness of the overall prediction of the model, indicating the proportion of samples correctly predicted by the model to the total number of samples. Recall is a measure of the model's detection ability for positive samples (such as vulnerability samples), that is, code samples with actual vulnerabilities, indicating the proportion of vulnerability samples correctly detected by the model to all actual vulnerability samples. It reflects the model's coverage of positive samples and is an important indicator for evaluating whether the model misses detection. F1 value is the harmonic mean of precision and recall, which is used to comprehensively evaluate the balance between false positives and missed detections of the model. In addition to the existing traditional performance indicators, this test method sets differentiated performance requirements for different types of perturbations (such as identifier obfuscation and logic reconstruction). For example, for identifier obfuscation perturbations, the F1 value can be required to reach a specific level (such as ≥87%); while for grammatical structure transformation perturbations, more attention is paid to recall (such as recall ≥85%). Differentiated thresholds are set for different types of perturbations (mild, moderate and high-intensity perturbations) to reflect the real adaptability of the model in actual and changing code scenarios. The differentiation threshold refers to the minimum acceptable standard set in the model performance evaluation index, which is used to measure whether the model has achieved the expected performance under specific test conditions. In specific implementation, the differentiation threshold under mild disturbance is set as: recall rate ≥ 90%, recall rate under moderate disturbance ≥ 85%, and recall rate under high-intensity disturbance ≥ 75%.

[0065] When the present invention is implemented, a fully tested and optimized deep learning model is integrated into the actual application environment to ensure that the model can process source code and automatically detect potential vulnerabilities.

[0066] According to the above method, an embodiment of the present invention provides an anti-interference source code vulnerability detection device based on deep learning, including a module for building a source code data set and its perturbation library, a model defense module, a real-time verification and parameter update module, and a model deployment module. Among them:

[0067] Build a source code dataset and its perturbation library module; segment and tokenize the source code by special characters, traverse and preprocess all source code files, summarize the token sequence, build a vocabulary and encode it into a numerical sequence. Use static analysis and deep semantic understanding to identify key code units, and combine local and global strategies to select the most influential parts for perturbation to generate adversarial samples.

[0068] Model defense module: By obtaining the output data of the hidden layer, the deviation degree value of each neuron is calculated. Based on the calculation results, a random mask of the same size as the batch is generated, and the mask is binarized. Finally, according to the result of the mask, it is decided whether to adopt a dynamic adjustment strategy based on the abnormal degree of neuron deviation or a random adjustment strategy based on the abnormal degree of neuron deviation. If the neuron regulation mode is selected, a position mask matrix is ​​generated based on the abnormal degree of neuron deviation, and activation and inhibition masks are dynamically generated to control the activation state of the neuron. By judging whether the abnormal degree of neuron deviation exceeds the preset threshold, it is decided to increase or decrease the activation deviation degree value of the neuron; otherwise, the randomly generated mask is combined to further enhance the anti-interference ability of the model.

[0069] Real-time verification and parameter update module: By real-time monitoring of the loss value on the verification set, the activation probability and inhibition probability of the model are dynamically adjusted. According to the relative change of the loss, the optimization weight factor is adaptively adjusted.

[0070] Model deployment module; use the perturbation library to test the model, evaluate model indicators and ensure reliability; deploy the model to the actual environment, realize source code processing and automatic vulnerability detection.

[0071] Compared with the prior art, the present invention has the following beneficial effects:

[0072] The present invention provides an anti-interference method and device for source code vulnerability detection based on deep learning. By effectively preprocessing the source code data, building a code perturbation strategy library and a source code vulnerability detection model, introducing a defense layer and adopting a dynamic defense strategy, the anti-interference ability and detection performance of the source code vulnerability detection model are improved. The specific technical advantages include the following:

[0073] First, the present invention adds a neuron defense layer before the fully connected layer of the traditional deep learning model. This defense layer monitors the activation value of each neuron in real time and calculates the degree of abnormal deviation of the neuron, and screens out neurons with abnormal deviation based on the statistical 3σ principle. This mechanism can filter abnormal activations caused by code disturbances and improve the detection accuracy and robustness of the model when processing disturbed source code.

[0074] Second, the present invention proposes a dynamic adjustment method based on the abnormal degree of neuron deviation and random optimization. By calculating the activation value deviation of each neuron, the model can automatically identify and suppress overactive neurons, while activating low-response neurons. In addition, random masks are introduced to increase uncertainty and diversity in the model training process. This adjustment strategy can enhance the model's ability to resist input disturbances, reduce missed detection rates, and improve overall detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1It is a flowchart of an interference-resistant deep learning-based source code vulnerability detection method provided according to an embodiment of the present invention.

[0076] Figure 2 It is a flow chart of a dynamic defense method based on neuron abnormal deviation degree and random optimization provided according to an embodiment of the present invention.

[0077] Figure 3 A flowchart of dynamic adjustment based on the abnormal degree of neuron deviation is provided according to an embodiment of the present invention.

[0078] Figure 4 It is a structural diagram of an anti-interference deep learning-based source code vulnerability detection device provided according to an embodiment of the present invention.

[0079] Figure 5 A schematic diagram of the structure of an electronic device with anti-interference and deep learning source code vulnerability detection provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0080] The present invention will be further described below by way of embodiments in conjunction with the accompanying drawings, but the scope of the present invention is not limited in any way.

[0081] The traditional source code vulnerability detection model receives the source code preprocessed data, that is, the tensor converted from the sequence of keywords, identifiers, operators, constants, etc., and then processes it through the input layer and the intermediate hidden layer. Among them, the linear transformation of the hidden layer weight matrix W and the bias vector b obtains the output of the hidden layer: , that is, the data entering the fully connected layer of the traditional deep learning model is The method of the present invention introduces a neuron defense layer before the fully connected layer of the traditional deep learning model. The data first enters S1 of the defense layer to calculate the degree of abnormal deviation of the neuron. The calculation formula is: ,in is the parameter that controls the adjustment sensitivity, μ Z is the mean activation value of the neuron, σ Z is the variance of the neuron activation value. In S2 of the defense layer, the 3σ principle in statistics is combined to make a threshold judgment, and then the abnormal neurons are activated or inhibited to determine the output of the neurons in the defense layer. .in , S is the threshold comparison matrix. Therefore, compared with the traditional model, the data entering the fully connected layer through the defense layer is in For example: the value of a neuron in a fully connected layer of a traditional neural network is 0.012498. After the defense layer is adjusted, the value of the neuron is z 1 becomes , which becomes 0.017750.

[0082] The present invention aims to solve the problem that the deep learning model has insufficient source code vulnerability detection performance under disturbance conditions, and provides an interference-resistant deep learning source code vulnerability detection method and device. In the present invention, the source of the source code may include: CVE (Common Vulnerabilities and Exposures Database), NVD (National Vulnerability Database), GitHub (open source project). The overall process of the method of the present invention is as follows: Figure 1 As shown, it includes steps S1 to S4.

[0083] S1) Build a source code dataset containing known vulnerabilities and a code perturbation strategy library, including:

[0084] S11) Build a source code dataset containing known vulnerabilities;

[0085] Split special characters in the source code. For example, separate symbols such as (,), {,}, *, / , +, -, =, ;, [, ], >, <, ", ->, >>, <<, , from other characters. Split into token sequences by spaces or other separators, and convert the source code into a sequence of keywords, identifiers, operators, constants, etc.

[0086] Determine whether a source code file contains a vulnerability based on its name or content. Traverse all source code files in the specified directory and read their contents. Preprocess each source code file, aggregate the tag sequences of all source code files, and construct a complete source code dataset.

[0087] Build a vocabulary and encode each source code token sequence according to the vocabulary to convert it into a numerical sequence. Use the embedding model to train the source code token sequence to generate an embedding matrix for subsequent deep learning models.

[0088] S12) Build a code perturbation strategy library;

[0089] The code perturbation methods adopted by the present invention include identifier obfuscation, grammatical structure transformation, no-operation code insertion, code block reordering, equivalent logic replacement, and annotation and format perturbation;

[0090] Specifically, identifier obfuscation: modify the variable name, function name or class name in the code to make it unreadable or random, but do not affect the logic and function of the code. Syntax structure transformation: by modifying the syntax structure of the code (such as changing if-else to a ternary operator, or changing a for loop to a while loop, etc.), keep the logic and function of the code equivalent. No-operation code insertion: insert code snippets that have no effect on the function (such as an empty statement pass or debugging code) into the code to keep the logic and semantics of the code consistent. Code block reordering: rearrange the order of code blocks, keep the logic function of the code unchanged, but change the execution order of the code writing. Equivalent logic replacement: replace the code logic with functionally equivalent but different implementation methods, such as loop structure conversion or logical expression rewriting. Comment and format perturbation: modify the comment content of the code, add or delete comments, and adjust the code format (such as indentation, space, line breaks).

[0091] Static analysis tools are used to parse the syntax and semantic structure of source code and identify key semantic units in the code (such as function definitions, variable declarations, control flow structures, etc.). Through the trained deep learning model, the semantic meaning of the code snippets is understood to ensure that the disturbed code is equivalent to the original code in logic and function.

[0092] Based on the results of static semantic analysis and deep semantic understanding, the most influential code units are selected for perturbation, such as replacing variable names, refactoring loop structures, or inserting no-operation code snippets. At the same time, semantic constraints are used to ensure that the perturbation does not destroy the original functionality of the code.

[0093] Combining local perturbation and global perturbation strategies, perturbation generation is performed at different code levels (such as a single function, the entire module) to improve the diversity and complexity of adversarial samples.

[0094] S2) Add a deep learning network structure defense layer, and perform dynamic defense based on the degree of abnormal deviation of neurons and random optimization. Specifically, it includes the following steps: Figure 2 As shown:

[0095] S21) obtaining the neuron activation value of the hidden layer data Z of the source code vulnerability detection network model, where the number of neurons in the defense layer corresponds to the hidden layer; and calculating the neuron value (neuron activation value) of the defense layer based on the abnormal degree of neuron deviation in the hidden layer.

[0096] The calculation of the abnormal degree of neuronal deviation can be completed by the following formula:

[0097]

[0098] Wherein, Z represents the hidden layer data, that is, the activation value generated by the current neuron after the source code is input. It represents the response strength of the neuron to the input feature. In the present invention, Z is used to reflect the importance of the features extracted from the source code when it passes through the deep learning model to vulnerability detection. (1) Represents the neuron activation value after the deviation abnormality is adjusted. This value integrates the correction of neuron abnormal deviation, and is used to further optimize the model's feature expression ability and anti-interference ability, ensuring that the model can still work stably in the face of disturbance input. Z Represents the expected value of neuron activation, which is calculated based on training data and is used to measure the average response level of neurons under normal input conditions. Z It is the normal working benchmark of neurons and is used to determine whether there is abnormal activation. Z The variance of the neuron activation value reflects the stability of the neuron response. In the present invention, a larger variance means that the neuron is more sensitive to input changes, while a smaller variance indicates that the neuron has a more stable response to the input data. δ is a weight factor used to control the deviation adjustment, which adjusts the sensitivity of the adjustment amplitude. In the present invention, the value of δ determines the intensity of the correction when the neuron activation value deviates from the expected value.

[0099] S22) using the abnormal degree value of neuron deviation to screen and adjust abnormal neurons;

[0100] To achieve the above adjustment, this method designs two strategies for adjusting abnormal neurons: dynamic adjustment based on the degree of abnormal neuron deviation and random adjustment. The selection and implementation details of each strategy are as follows:

[0101] Dynamic adjustment based on the degree of neuron deviation anomaly: This strategy adjusts the activation state of each neuron directly according to the degree of deviation anomaly, using a pre-defined adjustment strategy to inhibit overactive neurons while activating low-response neurons.

[0102] Random Tuning: This strategy generates a random mask matrix to decide whether to increase or decrease the bias of these neurons.

[0103] The specific method is:

[0104] First, generate a random tensor R of the same size as the batch, where each element of R is a random value sampled from the uniform distribution Uniform(0,1); then, compare R with the set relative change threshold, generate a binary mask M, and apply it to the input data batch size size, and calculate the decision parameter by the following formula: decison = sum(M)-size / 2; decison is the output after applying the mask. Finally, decide which method to use for dynamic adjustment based on the decision parameter decision;

[0105] If decision>0 indicates to use a dynamic adjustment strategy based on the abnormal degree of neuron deviation, then execute S23); if it indicates to use a random adjustment strategy based on the abnormal degree of neuron deviation, then execute S24).

[0106] S23) Dynamic adjustment based on the degree of abnormality of neuronal deviation; e.g. Figure 3 As shown, first, according to the abnormal degree of neuron deviation Generate the corresponding position mask matrix e mask , and then determine whether the deviation of each position can reach the deviation abnormality threshold; dynamically generate activation mask ME activate and suppression mask ME inhibit , which is used to control whether neurons that have reached the threshold or neurons that have not reached the threshold need to be adjusted. Both the activation mask and the inhibition mask obey the Bernoulli distribution. The specific operation of dynamically adjusting the abnormal degree of neuron deviation is based on the 3σ principle:

[0107] when When >3, the neuron activation value is considered abnormal and the neuron activation state needs to be reduced. The adjustment formula is:

[0108]

[0109] when When <-3, the neuron activation value is considered abnormal and the neuron activation state needs to be increased. The adjustment formula is:

[0110]

[0111] Where S α and S β As a weight factor (called gain factor or attenuation factor) of the abnormal degree of neuron deviation, the value range is 0-1 and is used to control the increase and decrease of the activation value. (2) is the adjusted neuron.

[0112] S24) Random adjustment; Generate the corresponding position matrix r according to the abnormal degree of neuron deviation mask；Generate a random activation mask M using Bernoulli distribution activate and random suppression mask M inhibit , through the position matrix r mask Filter where to apply activation and suppression masks to increase noise immunity during model training.

[0113] S3) Training based on source code data and adjusting the deviation coefficient and activation inhibition probability in real time, specifically including the following steps:

[0114] S31) Calculate the current loss value; quantitatively evaluate the prediction performance of the model through the loss function to measure the matching degree between the model output and the true label; this embodiment selects the following formula as the loss function:

[0115]

[0116] Where: N is the number of samples, C is the number of categories, y i,c is the true label of sample i, which is 1 for the correct category and 0 otherwise, p i,c is the predicted probability that the i-th sample output by the model belongs to category c.

[0117] S32) Adaptively adjust the activation and inhibition probabilities; based on the relative changes in the loss value, dynamically adjust the activation probability p of the model activate With the inhibition probability p inhibit , through iteration, the model adapts to the changes in data distribution during the learning process. The specific adjustment steps are as follows: Calculate the relative change of loss value :

[0118]

[0119] The specific adjustment formula is as follows:

[0120]

[0121]

[0122] Among them, θ is the set relative change threshold (such as 0.01), and γ is the adjustment rate (such as 0.01).

[0123] S33) Optimize the weight factor; use an adaptive adjustment algorithm based on the abnormal degree of neuron deviation to adaptively adjust the weight factor S according to the overall change of neuron deviation α and S β , in order to balance the convergence speed and stability of the model. Specific adjustment steps: In each iteration, by comparing the current and previous Z (1) value, and increase, decrease or maintain S accordingly according to its increase, decrease or flat situationα and S β .

[0124] The specific optimization adjustment formula is as follows:

[0125]

[0126]

[0127] S34) Integrate the training loop; integrate the above steps (31 to 33) into the training loop of the source code inspection model, and use the constructed source code dataset to perform iterative training.

[0128] S4) Test and deploy using the perturbation code dataset

[0129] S41) Use the perturbation code dataset to conduct a comprehensive test; input the constructed perturbation code dataset into the trained deep learning model to comprehensively evaluate the detection performance of the model under different perturbations. The specific evaluation steps include:

[0130] S42) Multi-level perturbation intensity grading test: Divide the perturbation code data set into different intensity levels (such as mild, medium and high intensity perturbations) and gradually input them into the model according to the gradient. After ensuring the basic performance of the model under mild perturbation conditions, gradually increase the perturbation intensity to refine the performance of the model under progressive perturbation environments.

[0131] S43) Multi-dimensional performance indicators and differentiated thresholds; In addition to traditional accuracy, recall, and F1-score, this test method sets differentiated performance requirements for different types of perturbations (such as identifier confusion and logic reconstruction). For example, for identifier confusion perturbations, the F1 value can be required to reach a certain level (such as ≥87%); while for grammatical structure transformation perturbations, more emphasis is placed on recall (such as ≥85%). Differentiated thresholds are set for different types of perturbations to reflect the model's true adaptability in actual and changing code scenarios.

[0132] S44) Integrate the fully tested and optimized deep learning model into the actual application environment to ensure that the model can process the source code and automatically detect potential vulnerabilities.

[0133] Figure 4 This is a structural diagram of an anti-interference deep learning source code vulnerability detection device according to an embodiment of the present invention, including a source code data set and its perturbation library module 1, a model defense module 2, a real-time verification and parameter update module 3, and a model deployment module 4. Among them:

[0134] Build a source code dataset and its perturbation library module; segment and tokenize the source code by special characters, traverse and preprocess all source code files, summarize the token sequence, build a vocabulary and encode it into a numerical sequence. Use static analysis and deep semantic understanding to identify key code units, and combine local and global strategies to select the most influential parts for perturbation to generate adversarial samples.

[0135] Model defense module: By obtaining the output data of the hidden layer, the deviation degree value of each neuron is calculated. Based on the calculation results, a random mask of the same size as the batch is generated, and the mask is binarized. Finally, according to the result of the mask, it is decided whether to adopt a dynamic adjustment strategy based on the abnormal degree of neuron deviation or a random adjustment strategy based on the abnormal degree of neuron deviation. If the neuron regulation mode is selected, a position mask matrix is ​​generated based on the abnormal degree of neuron deviation, and activation and inhibition masks are dynamically generated to control the activation state of the neuron. By judging whether the abnormal degree of neuron deviation exceeds the preset threshold, it is decided to increase or decrease the activation deviation degree value of the neuron; otherwise, the randomly generated mask is combined to further enhance the anti-interference ability of the model.

[0136] Real-time verification and parameter update module: By real-time monitoring of the loss value on the verification set, the activation probability and inhibition probability of the model are dynamically adjusted. According to the relative change of the loss, the optimization weight factor is adaptively adjusted.

[0137] Model deployment module; use the perturbation library to test the model, evaluate model indicators and ensure reliability; deploy the model to the actual environment, realize source code processing and automatic vulnerability detection.

[0138] Figure 5 It is shown that an embodiment of the present invention provides an electronic device, including: a processor, a memory, a bus, and a computer program stored in the memory and executable on the processor; wherein the processor and the memory communicate with each other through the bus; when the processor executes the computer program, the method described above is implemented, and the method focuses on the regularization layer in the neural network. Specifically, it includes: calculating the deviation abnormality value of each neuron in the layer according to the hidden layer data; constructing a random mask related to the neuron deviation abnormality value and performing binarization processing, and deciding to adopt a dynamic adjustment or random adjustment strategy based on the neuron deviation abnormality according to the mask result; dynamically generating an activation mask and an inhibition mask to control the activation or inhibition of the neurons in the layer; real-time monitoring of the loss value on the validation set and adjusting the activation and inhibition probabilities and adjustment factors according to the loss changes.

[0139] The embodiment of the present invention provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method as described above is implemented, and the method is applied on a specific layer of a neural network. For example, it includes: calculating the deviation abnormality value of each neuron in the layer according to the hidden layer data; constructing a random mask related to the neuron deviation abnormality value and performing binarization processing, and deciding to adopt a dynamic adjustment or random adjustment strategy based on the neuron deviation abnormality according to the mask result; dynamically generating an activation mask and an inhibition mask to control the activation or inhibition of the neurons in the layer; real-time monitoring of the loss value on the validation set and adjusting the activation and inhibition probabilities and adjustment factors according to the loss changes.

[0140] It should be noted that the purpose of publishing the embodiments is to help further understand the present invention, but those skilled in the art can understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the contents disclosed in the embodiments, and the scope of protection claimed by the present invention shall be subject to the scope defined in the claims.

Claims

1. An anti-interference method for detecting source code vulnerabilities based on deep learning, characterized in that: By adding a deep learning network structure defense layer to the deep learning neural network model, an interference-resistant deep learning-based source code vulnerability detection network model is constructed, the neuron inhibition probability is adaptively adjusted and the neuron activation probability is dynamically adjusted to enhance the source code detection effect of the model under disturbance; the steps include: 1) Build a source code dataset containing known vulnerabilities and a code perturbation strategy library; 2) Add a deep learning network structure defense layer to the deep learning neural network model, build an interference-resistant deep learning-based source code vulnerability detection network model, and perform dynamic defense based on the degree of neuron abnormal deviation and random optimization; including: 21) Obtain the neuron activation value of the hidden layer data of the source code vulnerability detection network model, and calculate the neuron activation value of the defense layer based on the abnormal degree of neuron deviation in the hidden layer; 22) Design decision parameters, use the abnormal degree of neuron deviation to screen and adjust abnormal neurons; the adjustment methods of abnormal neurons include: dynamic adjustment based on the abnormal degree of neuron deviation and random adjustment; Among them, dynamic adjustments based on the degree of abnormality of neuronal deviation include: Generate the corresponding position mask matrix e according to the abnormal degree value of neuron deviation mask ; Determine whether the degree of deviation of each position is abnormal; Dynamically generate activation masks and inhibition masks to control whether active neurons or low-response neurons need to be adjusted; Dynamically adjust the abnormality value of neuron deviation based on the three sigma criterion; 3) Perform model training based on source code data and adjust the deviation coefficient and activation inhibition probability in real time; including: 31) Calculate the current loss value; quantitatively evaluate the prediction performance of the model through the loss function to measure the degree of match between the model output and the true label; 32) Adaptive adjustment of activation and inhibition probabilities: Based on the relative changes in the loss value, dynamically adjust the activation probability and inhibition probability of the model, and iteratively adapt the model to changes in data distribution during the learning process; 33) Optimize weight factors; use an adaptive adjustment algorithm based on the abnormal degree of neuron deviation to adaptively adjust the weights according to the changes in neuron deviation, thereby balancing the convergence speed and stability of the model; 34) Model training; Integrate the above steps 31) to 33) into the training loop of the source code inspection model, use the constructed source code dataset to perform iterative training, and obtain a trained deep learning model, that is, a trained anti-interference deep learning-based source code vulnerability detection network model; 4) Use the perturbed code dataset to test and verify the source code detection model; Through the above steps, interference-resistant deep learning-based source code vulnerability detection is achieved.

2. The anti-interference deep learning source code vulnerability detection method according to claim 1, characterized in that: In step 21), the calculation method of the abnormal degree of neuron deviation is expressed as: ， Among them, Z (1) represents the neuron activation value after the deviation abnormality is adjusted; Z represents the hidden layer data, that is, the activation value of the current neuron after the source code is input; μ Z represents the expected value of neuron activation; σ Z represents the variance of the neuron activation value; δ is the weight factor used to control the bias adjustment.

3. The anti-interference deep learning source code vulnerability detection method according to claim 1, characterized in that: In step 22), dynamically adjusting neurons or randomly adjusting neurons is determined according to the decision parameters; the calculation method of the decision parameter decision is expressed as: decison = sum(M)-size / 2; where decison is the output after applying the mask; M is the generated binary mask; size is the data batch size.

4. The anti-interference method for detecting source code vulnerabilities based on deep learning as claimed in claim 1, characterized in that: In step 22), both the activation mask and the inhibition mask obey the Bernoulli distribution; dynamically adjusting the abnormality value of the neuron deviation based on the three sigma criterion includes: when When >3, the neuron activation value is abnormal, and the neuron activation state is reduced by adjustment; the adjustment method is expressed as: ， when When <-3, the neuron activation value is abnormal. Adjustments are made to increase the neuron activation state. The adjustment method is expressed as: ， Among them, S α As a gain factor, it is used to control the increase in neuron activity; S β As a decay factor, it is used to control the reduction of activation value; Z (2) is the adjusted neuron activation value.

5. The anti-interference method for detecting source code vulnerabilities based on deep learning as claimed in claim 4, characterized in that: The specific random adjustment is: according to the abnormal degree value Z of neuron deviation (1) Generate the corresponding position matrix; use Bernoulli distribution to generate random activation masks and random inhibition masks, and use the position matrix to filter the positions where activation and inhibition masks are applied to increase uncertainty and diversity in the model training process.

6. The anti-interference method for detecting source code vulnerabilities based on deep learning as claimed in claim 5, characterized in that: In step 32), the loss function is expressed as: ， Where: N is the number of samples, C is the number of categories, y i,c is the true label of sample i, the correct category is 1, otherwise it is 0; p i,c is the predicted probability that the i-th sample output by the model belongs to category c; Calculate the relative change in loss value , expressed as: ， Among them, t is the number of training updates; Loss (t) is the loss value of the tth step; For the The loss value of the step; It is the relative change of the loss value of the model in two consecutive parameter updates, which is used to determine whether the activation and inhibition probabilities need to be adjusted.

7. The anti-interference method for detecting source code vulnerabilities based on deep learning as claimed in claim 6, characterized in that: In step 32), the specific adjustment method is expressed as: ， ， Among them, P activate (t) is the activation probability of the neuron activation value in the defense layer of the constructed model at the tth iteration; P inhibit (t) is the inhibition probability of the neuron activation value in the defense layer of the model at the tth iteration; θ is the set relative change threshold of the loss; γ is the ratio of the activation probability or inhibition probability adjusted each time; In step 33), the weight factor is optimized; it is expressed as: ， ， Among them, S α (t) is the gain factor at the tth iteration; S β (t) is the attenuation factor at the tth iteration.

8. The anti-interference method for detecting source code vulnerabilities based on deep learning as claimed in claim 1, characterized in that: In step 1): Building a source code dataset containing known vulnerabilities includes: segmenting special characters in the source code; splitting into token sequences using delimiters, converting the source code into a sequence consisting of keywords, identifiers, operators, and constants; judging whether a source code file contains vulnerabilities based on its name or content; building a complete source code dataset; building a vocabulary and encoding each source code token sequence according to the vocabulary and converting it into a numerical sequence; using an embedding model to train the source code token sequence and generate an embedding matrix; The perturbation methods used to build the code perturbation strategy library include identifier obfuscation, grammatical structure transformation, and no-operation code insertion perturbation.

9. An anti-interference deep learning source code vulnerability detection device implemented by the method of claim 1, characterized in that: It includes a neuron deviation abnormality degree calculation module, a neuron deviation abnormality degree control module, and a real-time verification and update parameter module; among which: The neuron deviation abnormality degree calculation module is used to calculate the deviation degree value of each neuron according to the hidden layer output data; The neuron deviation abnormality degree control module is used to generate activation or inhibition masks according to the deviation degree value and dynamically adjust the activation state of the neuron; The real-time verification and parameter update module is used to monitor the loss value of the verification set and dynamically adjust the activation probability and inhibition probability based on the change of the loss value.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the anti-interference source code vulnerability detection method based on deep learning described in claim 1 is implemented.

Images