Network security vulnerability analysis method and system based on pre-training language model

Abstract

The invention relates to a network security vulnerability analysis method and system based on a pre-training language model, belongs to the technical field of network security, and solves the problems of low vulnerability detection efficiency, high cost and difficulty in discovering unknown vulnerabilities in the prior art. The method comprises the following steps: preprocessing source code data to obtain a preprocessed code snippet; constructing a vulnerability detection model, wherein the vulnerability detection model comprises a pre-training language model and a semantic inference agent; inputting the preprocessed code snippets into the vulnerability detection model, and processing the preprocessed code snippets by the pre-training language model to obtain hidden state representation of each code snippet as code features, the semantic inference agent identifies the code features to judge whether vulnerabilities exist in the source code data or not. Automatic detection and deduction of network security vulnerabilities are realized, and vulnerability detection efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a network security vulnerability analysis method and system based on a pre-trained language model. Background Art

[0002] With the rapid development of information technology, network security threats emerge in an endless stream, and network security vulnerabilities have become the main means of hacker attacks. Therefore, the study of network security vulnerabilities in information systems is of great significance. It not only helps to understand the current threat situation of network security, but also provides a scientific basis for formulating effective security protection strategies.

[0003] Computer network systems have various vulnerabilities during the development process, which attackers can exploit to invade the system. Currently, the main security vulnerabilities are as follows:

[0004] Remote code execution vulnerabilities: These vulnerabilities allow attackers to remotely execute malicious code without authorization, thereby controlling the target system. Remote code execution vulnerabilities are common in various web applications, database systems, and operating systems.

[0005] SQL injection vulnerability: An attacker constructs a malicious SQL query statement and injects it into the input parameters of the application to obtain data from the database or perform malicious operations on the database. SQL injection vulnerability is one of the most common vulnerabilities in Web applications.

[0006] Cross-site scripting (XSS) vulnerabilities: Attackers exploit security vulnerabilities in web applications to embed malicious scripts into web pages browsed by users. When users browse these web pages, the malicious scripts will be executed, thereby stealing user information or performing other malicious operations.

[0007] Privilege escalation vulnerabilities: These vulnerabilities allow attackers to gain higher access rights than normal, thereby controlling the entire system or accessing sensitive data. Privilege escalation vulnerabilities usually exist in operating and database systems and various application software.

[0008] The exploitation of network security vulnerabilities presents the following situations: 1) The means of exploiting vulnerabilities are diversified. Hackers not only use known vulnerabilities to attack, but also use multiple vulnerabilities in combination, or combine other means such as social engineering to attack; 2) The vulnerability exploitation cycle is shortened. With the acceleration of vulnerability disclosure and patching, hackers' attacks are also faster. Once a new vulnerability is disclosed, hackers will quickly exploit it, causing significant losses; 3) The targets of vulnerability exploitation are widespread. Hackers' attacks are no longer limited to specific industries or systems, but are widely attacked against various information systems. Therefore, both enterprises and individuals need to improve network security awareness and strengthen security protection. Traditional vulnerability analysis methods mainly rely on manual auditing and rule matching, which have problems such as low efficiency, high cost, and difficulty in discovering unknown vulnerabilities. Summary of the invention

[0009] In view of the above analysis, the embodiments of the present invention aim to provide a network security vulnerability analysis method and system based on a pre-trained language model, so as to solve the problems of low detection efficiency, high cost and difficulty in discovering unknown vulnerabilities in existing vulnerability analysis methods.

[0010] On the one hand, an embodiment of the present invention provides a network security vulnerability analysis method based on a large-scale pre-trained language model, the method comprising:

[0011] Preprocess the source code data to obtain preprocessed code snippets;

[0012] Constructing a vulnerability detection model, wherein the vulnerability detection model includes a pre-trained language model and a semantic inference agent;

[0013] The preprocessed code snippets are input into the vulnerability detection model, the pre-trained language model processes the preprocessed code snippets to obtain the hidden state representation of each code snippet as a code feature, and the semantic inference agent identifies the code feature to determine whether there is a vulnerability in the source code data and outputs the vulnerability category.

[0014] Based on the further improvement of the above method, the semantic inference agent is a reinforcement learning semantic inference model based on policy gradient;

[0015] The semantic inference agent comprises:

[0016] Critic target network, estimates the Q value of the state-action pair at the next moment, which is used to calculate the TD target;

[0017] The critic training network outputs the Q value of the current state-action pair, which is used to calculate the loss value to update the parameters of the critic network;

[0018] Actor target network, which provides the strategy for the next state to calculate the target action;

[0019] The Actor training network generates actions under the current strategy, and calculates the policy gradient of the Actor network during parameter update according to the Q value of the current state-action pair output by the Critic training network.

[0020] Based on the further improvement of the above method, the calculation formula of the loss value is:

[0021]

[0022] a i =π θ (S i )+ε;

[0023] y i =r i +γQ ω′ (S i+1 ,π θ′ (S i+1 ));

[0024] In the formula, loss is the loss value, y i is the TD target at the current moment, Q ω (S i ,a i ) is the Q value of the current state-action pair, ε represents the exploration noise on the behavior strategy; π θ is the strategy of the current state; π θ′ is the strategy for the next state; S i is the state at time i, a i is the action at time i; r i is the immediate reward, γ is the discount factor, Q ω′ (S i+1 ,π θ′ (S i+1 ) is the Q value of the state-action pair at the next moment, i is the moment, i=1…N, and N is the total time.

[0025] Based on the further improvement of the above method, the calculation formula of the strategy of the Actor network during parameter update is:

[0026]

[0027] In the formula, is the policy gradient, i is the moment, i=1…N, N is the total time, is the gradient of the Q-value function with respect to action a, |s=s i ,a=π θ (s i) means in state s i and strategy π θ Evaluate the gradient at the action a selected next; is π θ (s) is the gradient of the parameter θ, |s i Indicates that in state s i Evaluate the gradient at.

[0028] Based on the further improvement of the above method, the pre-training tasks of the pre-trained language model include: masked language modeling tasks and replacement word detection tasks;

[0029] In the masked language modeling task, a data point of the NL-PL pair is taken as input, a preset proportion of words in the NL word sequence and PLtoken sequence of the data point are randomly selected for masking, and then the masked original word is predicted;

[0030] In the replacement word token detection task, the unimodal data is used to train the NL generator and the GL generator to generate possible replacement words for the masked words, and a discriminator is trained to determine whether a word is the original word.

[0031] Based on the further improvement of the above method, the method further includes:

[0032] If the pre-trained language model cannot understand the input code snippet, the structure of the code snippet is parsed to extract preset feature values, and the extracted preset feature values ​​are converted into a data format that can be processed by the vulnerability identification agent and then input into the semantic inference agent.

[0033] Based on the further improvement of the above method, the method further includes:

[0034] If the vulnerability detection model detects that the input code features contain missing information, the code features containing the missing information are sent to the missing information filling module for processing;

[0035] The missing information filling module processes the code features of the missing information and includes the following steps:

[0036] Use natural language processing to extract the semantic features of missing information;

[0037] Based on the pre-built vulnerability and malicious code corpus, the semantic features of the missing information are analyzed to determine its rough vulnerability classification;

[0038] The semantic features of the missing information and the rough vulnerability classification results are used as the input of the knowledge graph, and the network security vulnerability knowledge graph is used to infer the vulnerability intent of the missing information to form an intermediate result;

[0039] After converting the intermediate results into machine-understandable features, the vulnerability detection model is corrected using the feature specification.

[0040] Based on the further improvement of the above method, the preprocessing of the source code data includes:

[0041] Tokenize, truncate, and / or add special tokens to source code data.

[0042] Based on the further improvement of the above method, the method further includes:

[0043] After the vulnerability identification agent identifies the existence of a vulnerability in the source code data, it performs attack verification on the source code data to obtain the vulnerability type and vulnerability location.

[0044] On the other hand, the present invention also provides a network security vulnerability analysis system based on a large-scale pre-trained language model, the system comprising:

[0045] A preprocessing module preprocesses the source code data and obtains the preprocessed code fragments;

[0046] A vulnerability detection model, the vulnerability detection model includes a pre-trained language model and a semantic inference agent; wherein the pre-trained language model processes the pre-processed code snippets to obtain the hidden state representation of each code snippet as a code feature, and the semantic inference agent identifies the code feature to determine whether there is a vulnerability in the source code data and outputs the vulnerability category.

[0047] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0048] 1. In the present invention, by combining the powerful text processing capabilities of the pre-trained language model and the semantic reasoning capabilities of the semantic inference agent, the automatic detection and inference of network security vulnerabilities are realized, which is conducive to the identification of unknown vulnerabilities and combined vulnerabilities, improves the efficiency and accuracy of vulnerability detection, reduces the cost of manual review, and provides strong technical support for network security.

[0049] 2. In the present invention, when there is missing information in the code data and the pre-trained language model cannot understand the input code snippet, the structure of the code snippet is parsed to extract preset feature values, and the extracted preset feature values ​​are converted into a data format that can be processed by the vulnerability identification agent and then input into the semantic inference agent, thereby completely transmitting the information of the code snippet to the semantic inference agent, which is beneficial to improving the vulnerability detection capability of the model.

[0050] 3. In the present invention, when the vulnerability identification agent recognizes that there is missing information in the code data, the code data containing the missing information is sent to the missing information filling module for processing. The missing information filling module uses relevant methods of natural language processing to extract the semantic features of the missing information, combines the vulnerability and malicious code corpus and human computer prior knowledge to obtain the vulnerability intention and form an intermediate result, and processes the intermediate result into a feature that can be understood by the machine, which is used to regulate and modify the vulnerability detection model, thereby obtaining a more humane vulnerability detection model and improving the accuracy of vulnerability detection.

[0051] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0053] Figure 1 A flowchart of a network security vulnerability analysis method based on a pre-trained language model according to an embodiment of the present invention;

[0054] Figure 2 Schematic diagram of the structure of a vulnerability detection model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0056] An embodiment of the present invention discloses a network security vulnerability analysis method based on a pre-trained language model, such as Figure 1 The method comprises:

[0057] Preprocess the source code data to obtain preprocessed code snippets;

[0058] Constructing a vulnerability detection model, wherein the vulnerability detection model includes a pre-trained language model and a semantic inference agent;

[0059] The preprocessed code snippets are input into the vulnerability detection model, the pre-trained language model processes the preprocessed code snippets to obtain the hidden state representation of each code snippet as a code feature, and the semantic inference agent identifies the code feature to determine whether there is a vulnerability in the source code data and outputs the vulnerability category.

[0060] like Figure 2 As shown in , the vulnerability detection model includes a pre-trained language model and a semantic inference agent. Among them, pre-trained language models (PLMs) are a type of deep learning models pre-trained on large-scale text data, which can capture the complex structure and patterns of language. Semantic inference agents refer to agents that integrate semantic understanding and reasoning capabilities. They can process and interpret semantic information, make decisions and perform tasks.

[0061] Compared with the prior art, the embodiments of the present invention achieve automatic detection and inference of network security vulnerabilities by combining the powerful text processing capabilities of the pre-trained language model and the semantic reasoning capabilities of the semantic inference agent, and effectively identify unknown vulnerabilities and combined vulnerabilities, thereby improving the efficiency and accuracy of vulnerability detection, reducing manual review costs, and providing strong technical support for network security.

[0062] Specifically, the preprocessing of the source code data includes: performing operations such as word segmentation, truncation and / or adding special tags to the source code data.

[0063] (1) Participle.

[0064] When implemented, tokenize the code data using a tokenizer such as BertTokenizer. A tokenizer is a program or function that breaks text into smaller units such as words, subwords, or characters.

[0065] In the embodiment of the present invention, considering that the words in the code data, such as variable names and function names, may be very long and may contain non-standard characters such as "_", "-", etc., many unknown tags -[UNK] may be generated. Therefore, the code data is decomposed into smaller units, such as identifiers, keywords, operators, literals, etc., and these units can be further mapped to the BERT vocabulary.

[0066] (2) Add special tags

[0067] Among them, adding special tags in the code helps the model understand the structure of the code.

[0068] During implementation, a preset start marker, such as a classification marker [CLS], is added at the beginning of the input sequence, and a preset separator [SEP] is added between each segment and at the end of the input sequence.

[0069] Furthermore, additional special tags can be introduced to denote different parts of the code, such as [VAR] for variable names, [KW] for keywords, [LIT] for literals, etc.

[0070] (3) Truncation processing

[0071] When the source code data exceeds a preset length, the source code data is truncated (segmented).

[0072] For example, the CodeBERT model has a maximum input length limit, usually 512 tokens. If the code snippet is too long, it needs to be truncated or segmented. When segmenting, each code snippet should start with [CLS] and end with [SEP], and pass them to the model as independent sequences.

[0073] In a specific embodiment, the semantic inference agent is a reinforcement learning semantic inference model based on policy gradient. The semantic inference agent includes:

[0074] Critic target network, estimates the Q value of the state-action pair at the next moment, which is used to calculate the TD target;

[0075] The critic training network outputs the Q value of the current state-action pair, which is used to calculate the loss value to update the parameters of the critic network;

[0076] Actor target network, which provides the strategy for the next state to calculate the target action;

[0077] The Actor training network generates actions under the current strategy, and calculates the policy gradient of the Actor network during parameter update according to the Q value of the current state-action pair output by the Critic training network.

[0078] It should be noted that the policy gradient-based reinforcement learning semantic inference model is a technology that combines reinforcement learning (RL) and natural language processing (NLP). The policy gradient method is a reinforcement learning algorithm that directly optimizes the policy. It updates the policy parameters by calculating the gradient of the policy with respect to the expected return. Among them, the Q-value function is a core concept in reinforcement learning, which is used to evaluate the expected return of taking a specific action in a specific state. The Q-value function, also known as the state-action value function, represents the expected total return that the agent can obtain from the current state to the future state after taking action a and following a certain strategy in a given state s. That is, the Q-value is an indicator for evaluating the quality of a specific state and action pair.

[0079] Among them, the Critic target network and the Critic training network are both Critic networks, but they play different roles in the reinforcement learning algorithm and jointly promote the stability and efficiency of the learning process. Similarly, the Actor target network and the Actor training network are both Actor networks, and they play different roles in the reinforcement learning algorithm.

[0080] Specifically, the Critic target network is used to estimate the Q value Q of the state-action pair at the next moment ω′ (S i+1 ,π θ′ (S i+1 )). The Critic target network provides a stable Q-value estimate for calculating the TD target (TD Target).

[0081] Among them, the action value at the next moment is π estimated by the Actor target network θ′ (S i+1 ). Then the TD target in the current state is obtained, and its calculation formula is:

[0082] y i =r i +γQ ω′ (S i+1 ,π θ′ (S i+1 ));

[0083] In the formula, y i is the TD target at the current moment, Q ω′ (S i+1 ,π θ′ (S i+1 )) is the Q value of the state-action pair at the next moment, r iis the immediate reward, γ is the discount factor, i is the moment, i=1…N, and N is the total time.

[0084] The critic training network outputs the Q value of the current state-action pair for evaluating the current strategy. In the embodiment of the present invention, in order to increase the agent's exploration in the environment, a Gaussian noise function is added to the behavior strategy. The parameters of the critic network are updated by minimizing the loss value (mean square error loss). The loss function of the critic network when updating is:

[0085]

[0086] Among them, a i =π θ (S i )+ε.

[0087] In the formula, loss is the loss value, y i is the TD target at the current moment, Q ω (S i ,a i ) is the Q value of the current state-action pair, ε represents the exploration noise on the behavior strategy; π θ is the strategy of the current state; π θ′ is the strategy for the next state; S i is the state at time i, a i is the action at time i; r i is the immediate reward, γ is the discount factor, i is the moment, i=1…N, and N is the total time.

[0088] The Actor target network is used to provide the strategy for the next state, while the Actor training network provides the strategy for the current state. Combined with the Q value function of the Critic training network, the strategy gradient of the Actor during parameter update can be obtained. The calculation formula of the Actor network strategy during parameter update is:

[0089]

[0090] In the formula, is the policy gradient, i is the moment, i=1…N, is the gradient of the Q-value function with respect to action a, |s=s i ,a=π θ (s i ) means in state s i and strategy π θ Evaluate the gradient at the action a selected next; is π θ (s) is the gradient of the parameter θ, |s i Indicates that in state s iEvaluate the gradient at.

[0091] Among them, for the update of the target network parameters ω′ and θ′, the soft update mechanism (updating some parameters each time learning) ensures that the parameters can be updated slowly, thereby improving the stability of learning. The update formula is:

[0092] ω′←ξω+(1-ξ)ω′;

[0093] θ′←ξθ+(1-ξ)θ′;

[0094] In the formula, θ and ω are the parameters of the current network, and ξ represents the transfer ratio from the current network to the target network parameters at each update.

[0095] In a specific embodiment, the pre-training tasks of the pre-trained language model include: a masked language modeling task and a replacement word detection task;

[0096] In the masked language modeling task, a data point of the NL-PL pair is taken as input, a preset proportion of words in the NL word sequence and PLtoken sequence of the data point are randomly selected for masking, and then the masked original word is predicted;

[0097] In the replacement word token detection task, the unimodal data is used to train the NL generator and the GL generator to generate possible replacement words for the masked words, and a discriminator is trained to determine whether a word is the original word.

[0098] Among them, NL-PL pairs refer to the paired data of natural language (NL) and programming language (PL).

[0099] In the embodiment of the present invention, through the two pre-training tasks of masked language modeling task and replacement word detection task, the pre-trained language model can learn the complex relationship between natural language and programming language, and provide powerful semantic understanding ability for the subsequent vulnerability identification task of the semantic inference agent.

[0100] Specifically, the pre-trained language model is a CodeBERT model.

[0101] In the Masked Language Modeling (MLM) task, given a data point x = {ω, c} of an NL-PL pair as input, where ω is the NL word sequence and c is the PL token sequence, we first select a random set of positions for NL-PL to mask, namely m ω and m c , and then replace the selected position with [MASK] token.

[0102] 15% of the tokens in data point x are masked.

[0103]

[0104] ω masked =REPLACE(ω,m ω ,[MASK]);

[0105] c masked =REPLACE(c,m c ,[MASK]);

[0106] x = ω + c;

[0107] The goal of the masked language modeling task is to predict the masked original token. The formula is as follows:

[0108]

[0109] In the formula, is a discriminator that predicts a token from a large vocabulary.

[0110] In the Replaces Token Detection task RTD (Replaces Token Detection), there are two data generators: NL generator and GL generator, which are used to generate available replacements for a random masked position set.

[0111]

[0112]

[0113] x corrupt =ω corrupt +c corrupt .

[0114] The discriminator is trained to determine whether a word is the original word, which is a binary classification problem. The RTD target is applied to each position in the input. It differs from the Generative Adversarial Network (GAN) in that if the generator happens to produce the correct token, the label of the token is "real" instead of "fake". The loss function of RTD for the parameterized discriminator is as follows:

[0115]

[0116] Where δ(i) is an indicator function; is the discriminator, used to predict the probability that a word is the original word.

[0117] In the embodiment of the present invention, two effective bidirectional contextual n-gram language models are implemented, one for NL and one for PL, and they are trained using corresponding unimodal data points, respectively. The training data for the PL generator is unimodal code, and the training data for the NL generator comes from documents in the bimodal data.

[0118] In summary, the loss function of the pre-trained language model is: min(L MLM (θ)+L RTD (θ)).

[0119] In one embodiment, the method further comprises:

[0120] If the pre-trained language model cannot understand the input code snippet, the structure of the code snippet is parsed to extract preset feature values, and the extracted preset feature values ​​are converted into a data format that can be processed by the vulnerability identification agent and then input into the semantic inference agent.

[0121] In the present invention, when there is missing information in the code data and the pre-trained language model cannot understand the input code snippet, if the pre-trained language model cannot understand the input code snippet, the structure of the code snippet is parsed to extract preset feature values, and the extracted preset feature values ​​are converted into a data format that can be processed by the vulnerability identification agent and then input into the semantic inference agent, thereby completely transmitting the information of the code snippet to the semantic inference agent, which is beneficial to improving the vulnerability detection capability of the model.

[0122] It should be noted that when there are syntax errors or semantic errors in the code, the model will have difficulty parsing the code, resulting in missing information in the code.

[0123] Specifically, the extracted preset feature values ​​include meaningful features such as syntax, control flow, function calls, etc.

[0124] In one embodiment, the method further comprises:

[0125] If the vulnerability detection model detects that the input code features contain missing information, the code features containing the missing information are sent to the missing information filling module for processing;

[0126] The missing information filling module processes the code features of the missing information and includes the following steps:

[0127] Use natural language processing to extract the semantic features of missing information;

[0128] Based on the pre-built vulnerability and malicious code corpus, the semantic features of the missing information are analyzed to determine its rough vulnerability classification;

[0129] The semantic features of the missing information and the rough vulnerability classification results are used as the input of the knowledge graph, and the network security vulnerability knowledge graph is used to infer the vulnerability intent of the missing information to form an intermediate result;

[0130] After converting the intermediate results into machine-understandable features, the vulnerability detection model is corrected using the feature specification.

[0131] In an embodiment of the present invention, when the vulnerability identification agent recognizes that there is missing information in the code data, the code data containing the missing information is sent to the missing information filling module for processing. The missing information filling module uses the relevant method of natural language processing to extract the semantic features of the missing information, and combines the vulnerability and malicious code corpus and the network security vulnerability knowledge graph to obtain the vulnerability intention, thereby using human computer prior knowledge to obtain the vulnerability intention of the missing information, and the intermediate results are processed into feature specifications that can be understood by the machine, which are used to correct the vulnerability detection model, thereby obtaining a more humanized vulnerability detection model and improving the accuracy of vulnerability detection. It should be noted that when the code features contain missing information, the semantic inference agent will fill in the missing information, turn it into a known code defect, and output the vulnerability type.

[0132] Specifically, the natural language processing methods include cleaning, feature extraction, model recognition, model prediction and optimization, etc. Through these methods, the semantic features of missing information can be effectively stripped out, and the effect and accuracy of text processing can be improved.

[0133] Cleaning processing: Clean and preprocess the input text to remove irrelevant information.

[0134] Feature extraction: Convert text into feature vectors using methods such as bag-of-words, TF-IDF, or word embedding.

[0135] Pattern recognition: Use machine learning or deep learning models to train feature vectors and learn patterns of missing information.

[0136] Model prediction and optimization: Use the trained model to predict new text and optimize and adjust based on the results.

[0137] During implementation, a network security vulnerability knowledge graph is constructed by analyzing open source codes with known vulnerabilities, or the existing network security vulnerability knowledge graph is directly used. Among them, the intermediate results obtained by inferring vulnerability intent using the network security vulnerability knowledge graph refer to a series of structured information about the vulnerability and its exploitation intent obtained through knowledge graph deduction and analysis, including key attributes of vulnerability entities, vulnerability attack intent, attack mode, potential impact and other information. These intermediate results provide security analysts with a structured perspective to understand the context of the vulnerability and potential security risks.

[0138] Feature reduction is an important step in machine learning, which involves selecting the most relevant and informative feature subsets from the original dataset to improve the performance and efficiency of the model. Feature reduction methods include embedded methods, wrapper methods, filtering methods, etc.

[0139] In one embodiment, the method further comprises:

[0140] After the vulnerability identification agent identifies the existence of a vulnerability in the source code data, it performs attack verification on the source code data to verify the vulnerability type.

[0141] On the other hand, the present invention also provides a network security vulnerability analysis system based on a large-scale pre-trained language model. The system comprises:

[0142] A preprocessing module preprocesses the source code data and obtains the preprocessed code fragments;

[0143] A vulnerability detection model, the vulnerability detection model includes a pre-trained language model and a semantic inference agent; wherein the pre-trained language model processes the pre-processed code snippets to obtain the hidden state representation of each code snippet as a code feature, and the semantic inference agent identifies the code feature to determine whether there is a vulnerability in the source code data and outputs the vulnerability category.

[0144] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0145] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A network security vulnerability analysis method based on a pre-trained language model, characterized in that: The method comprises: Preprocess the source code data to obtain preprocessed code snippets; Constructing a vulnerability detection model, wherein the vulnerability detection model includes a pre-trained language model and a semantic inference agent; The preprocessed code snippets are input into the vulnerability detection model, the pre-trained language model processes the preprocessed code snippets to obtain the hidden state representation of each code snippet as a code feature, and the semantic inference agent identifies the code feature to determine whether there is a vulnerability in the source code data and outputs the vulnerability category.

2. The method according to claim 1, characterized in that The semantic inference agent is a reinforcement learning semantic inference model based on policy gradient; The semantic inference agent comprises: Critic target network, estimates the Q value of the state-action pair at the next moment, which is used to calculate the TD target; The critic training network outputs the Q value of the current state-action pair, which is used to calculate the loss value to update the parameters of the critic network; Actor target network, which provides the strategy for the next state to calculate the target action; The Actor training network generates actions under the current strategy, and calculates the policy gradient of the Actor network during parameter update according to the Q value of the current state-action pair output by the Critic training network.

3. The method according to claim 2, characterized in that The loss value is calculated as follows: a i =π θ (S i )+e; y i =r i +γQ ω′ (S i+1 ,π θ′ (S i+1 )); In the formula, loss is the loss value, y i is the TD target at the current moment, Q ω (S i ,a i ) is the Q value of the current state-action pair, ε represents the exploration noise on the behavior strategy; π θ is the strategy of the current state; π θ′ is the strategy for the next state; S i is the state at time i, a i is the action at time i; r i is the immediate reward, γ is the discount factor, Q ω′ (S i+1 ,π θ′ (S i+1 ) is the Q value of the state-action pair at the next moment, i is the moment, i=1…N, and N is the total time.

4. The method according to claim 2, characterized in that: The calculation formula of the Actor network's strategy during parameter update is: In the formula, is the policy gradient, i is the moment, i=1…N, N is the total time, is the gradient of the Q-value function with respect to action a, Indicates that in state s i and strategy π θ Evaluate the gradient at the action a selected next; is π θ (s) is the gradient with respect to the parameter θ, Indicates that in state s i Evaluate the gradient at.

5. The method according to any one of claims 1 to 4, characterized in that The pre-training tasks of the pre-trained language model include: a masked language modeling task and a replacement word detection task; In the masked language modeling task, a data point of the NL-PL pair is taken as input, a preset proportion of words in the NL word sequence and PLtoken sequence of the data point are randomly selected for masking, and then the masked original word is predicted; In the replacement word token detection task, the unimodal data is used to train the NL generator and the GL generator to generate possible replacement words for the masked words, and a discriminator is trained to determine whether a word is the original word.

6. The method according to any one of claims 1 to 4, characterized in that The method further comprises: If the pre-trained language model cannot understand the input code snippet, the structure of the code snippet is parsed to extract preset feature values, and the extracted preset feature values ​​are converted into a data format that can be processed by the vulnerability identification agent and then input into the semantic inference agent.

7. The method according to any one of claims 1 to 4, characterized in that The method further comprises: If the vulnerability detection model detects that the input code features contain missing information, the code features containing the missing information are sent to the missing information filling module for processing; The missing information filling module processes the code features of the missing information and includes the following steps: Use natural language processing to extract the semantic features of missing information; Based on the pre-built vulnerability and malicious code corpus, the semantic features of the missing information are analyzed to determine its rough vulnerability classification; The semantic features of the missing information and the rough vulnerability classification results are used as the input of the knowledge graph, and the network security vulnerability knowledge graph is used to infer the vulnerability intent of the missing information to form an intermediate result; After converting the intermediate results into machine-understandable features, the vulnerability detection model is corrected using the feature specification.

8. The method according to any one of claims 1 to 4, characterized in that The preprocessing of the source code data includes: Tokenize, truncate, and / or add special tokens to source code data.

9. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: After the vulnerability identification agent identifies the existence of a vulnerability in the source code data, it performs attack verification on the source code data to obtain the vulnerability type and vulnerability location.

10. A network security vulnerability analysis system based on a pre-trained language model, characterized in that: The system comprises: A preprocessing module preprocesses the source code data and obtains the preprocessed code fragments; A vulnerability detection model, the vulnerability detection model includes a pre-trained language model and a semantic inference agent; wherein the pre-trained language model processes the pre-processed code snippets to obtain the hidden state representation of each code snippet as a code feature, and the semantic inference agent identifies the code feature to determine whether there is a vulnerability in the source code data and outputs the vulnerability category.

Images