Automatic vulnerability repair model based on Bayesian inference, construction method thereof and automatic vulnerability repair method

Abstract

The invention discloses an automatic vulnerability repair method based on Bayesian inference, and belongs to the technical field of computers.The method comprises the steps that an original data set is constructed by obtaining vulnerability codes and historical repair record data of the vulnerability codes, and the original data set is preprocessed to obtain a vulnerability repair data set; carrying out training learning on a Bayesian-based automatic vulnerability repair model by utilizing the vulnerability repair data set; a real vulnerability code is repaired by utilizing an automatic vulnerability repairing model based on Bayesian; and feeding back the first n repair patches with the highest prediction result to the user by utilizing a beam search technology. According to the method, the accuracy of automatic bug repair can be improved, and the code bug repair efficiency of a user is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more specifically, to a vulnerability automatic repair model based on Bayesian inference and a construction method thereof and a vulnerability automatic repair method. Background Art

[0002] Software vulnerabilities refer to computational logic or code defects in software systems, which may pose a serious threat to the system. Relying on manual repair of vulnerabilities not only consumes a lot of time and energy, but may also introduce new vulnerabilities. With the expansion of software scale and the increase of code complexity, the number of network security vulnerabilities has increased significantly, gradually becoming one of the main threats to network security. The sharp increase in the number of vulnerabilities has posed a severe challenge to network security. In recent years, with the rapid development of machine learning and deep learning, learning-based automatic vulnerability repair technology has attracted widespread attention from researchers. This type of technology aims to assist developers in quickly locating and repairing vulnerabilities, thereby reducing the losses caused by vulnerabilities.

[0003] Existing automatic vulnerability repair technologies have been well applied. Most of these vulnerability repair technologies are built based on learning. They improve the learning ability of vulnerabilities by learning more vulnerability data, and then use this knowledge to repair the vulnerabilities. However, these vulnerability repair systems still face great challenges. Traditional vulnerability repair systems often try to learn more vulnerability data, but ignore the diverse characteristics of vulnerability codes, and have the problem of inaccurate vulnerability positioning. They are unable to locate and learn the diverse characteristics of vulnerabilities well, resulting in poor repair results. Summary of the invention

[0004] The first object of the present invention is to solve the above-mentioned problems.

[0005] The second object of the present invention is to provide a Bayesian inference-based vulnerability automatic repair model and a construction method thereof.

[0006] The third purpose of the present invention is to provide a method for automatic vulnerability repair based on Bayesian inference, which uses the constructed automatic vulnerability repair model based on Bayesian inference to solve the problems of inaccurate positioning and inaccurate repair in the existing field of automatic vulnerability repair and improve vulnerability efficiency.

[0007] In order to achieve these objectives of the present invention, the present invention provides an automatic vulnerability repair method based on Bayesian inference, comprising:

[0008] 1) Data acquisition and preprocessing step: obtain data containing vulnerability codes and their historical repair records, and preprocess the data to generate a vulnerability data set containing vulnerability codes and correct codes after repair; further perform word segmentation processing on the vulnerability data set.

[0009] Specifically, the data can come from public vulnerability libraries, software developers' repair records, etc. After obtaining the data, the present invention preprocesses the data, including removing irrelevant information, formatting data, etc., to generate a vulnerability data set containing vulnerability codes and correct codes after repair. Furthermore, the present invention performs word segmentation processing on the vulnerability data set, decomposing the code into smaller units, such as variable names, function names, etc., to facilitate subsequent processing and analysis.

[0010] 2) Model building and training steps: Building a vulnerability automatic repair model based on Bayesian inference, and using the vulnerability dataset to train the vulnerability automatic repair model to learn the mapping relationship between vulnerability code and repair patch.

[0011] After data preprocessing is completed, the present invention establishes an automatic vulnerability repair model based on Bayesian inference. Bayesian inference is a probability-based reasoning method that uses known prior information and observation data to update the understanding of unknown events. In the present invention, Bayesian inference is used to learn the mapping relationship between vulnerability code and repair patch.

[0012] Specifically, the present invention divides the vulnerability data set into a training set and a test set. The training set is used to train the vulnerability automatic repair model, while the test set is used to evaluate the performance of the model. During the training process, the present invention uses the Bayesian inference algorithm to learn the training set to obtain the mapping relationship between the vulnerability code and the repair patch.

[0013] 3) Vulnerability code repair step: Use the trained vulnerability automatic repair model to automatically repair the real vulnerability code.

[0014] After the model training is completed, the present invention uses the trained vulnerability automatic repair model to automatically repair the real vulnerability code. Specifically, the present invention inputs the vulnerability code to be repaired into the model, and the model generates a corresponding repair patch based on the learned mapping relationship. Then, the present invention applies the repair patch to the vulnerability code to repair the vulnerability.

[0015] 4) Repair patch feedback step: using a beam search method to obtain the top n repair patches with the highest prediction results from the repair patches generated by the automatic vulnerability repair model, and feeding back these repair patches to the user for screening.

[0016] In order to improve the accuracy of repair, the present invention also adopts a beam search method. Beam search is a heuristic search algorithm that can find the optimal solution or the approximate optimal solution in a given search space. In the present invention, beam search is used to obtain the top n repair patches with the highest prediction results from the repair patches generated by the automatic vulnerability repair model.

[0017] Specifically, the present invention sorts the repair patches generated by the model according to the probability values ​​of the prediction results, and selects the first n repair patches with the highest probability values ​​as candidate repair patches. Then, the present invention feeds back these candidate repair patches to the user for the user to screen and use. The user can select the most suitable repair patch for application according to actual needs.

[0018] Preferably, in step 1), the vulnerability code refers to the vulnerability code existing in the system that may cause a serious threat to the system, and the historical repair record refers to the correct code after the developer actually repairs the vulnerability after discovering it, also called a real repair patch.

[0019] Preferably, in step 1), the pretreatment comprises:

[0020] a) Cleaning block-level vulnerability code: Clean the acquired vulnerability code data and remove block-level code information irrelevant to vulnerability repair, such as comments and irrelevant functions, so as to retain the key vulnerability code segments;

[0021] b) Adding hint information: In the cleaned vulnerability code and the corresponding correct code after repair, add appropriate hint information according to the grammatical structure and semantic information of the code. This hint information helps the model better understand the structure and intention of the code, such as the type annotation of variable names, parameter description of function calls, etc.

[0022] c) Code text sequence conversion: The cleaned and prompted block-level program code is converted into a code text sequence, that is, each element in the code (such as variables, operators, keywords, etc.) is arranged into a continuous text sequence according to the order in which it appears in the code;

[0023] d) Constructing a binary vulnerability dataset: The converted sequence vulnerability code and the corresponding correct code after repair are combined into binary tuples to form a binary vulnerability dataset. Each binary tuple is represented as i ,T i >, where S i represents the sequence vulnerability code, T i Indicates the correct code after repair corresponding to the vulnerability code.

[0024] In one embodiment of the present invention, the preprocessing specifically includes: firstly deleting blank lines, spaces and comments of irrelevant codes, then extracting and processing each line of block-level codes and splicing them into a sequence, then adding prompt information to the sequence code, that is, adding the vulnerability type starting with CWE-ID at the beginning of the sequence text, and adding positioning marks at the positions where the vulnerability is generated and ended: <startloc>and <endloc>,The same process is performed on the correct code after the real repair of the historical repair record (real repair patch), and finally a representation of a two-tuple vulnerability dataset is formed.

[0025] Preferably, the vulnerability code and its historical repair records are obtained, the vulnerability code is regarded as training data, and the historical repair records are regarded as model expected data.

[0026] Preferably, in step 1) of data acquisition and preprocessing, the "performing word segmentation processing on it" specifically refers to:

[0027] The BPE (Byte-Pair Encoding) word segmentation algorithm is used to segment the sequence vulnerability code S obtained after preprocessing. i And the correct code after repair T i Perform word segmentation. The BPE word segmentation algorithm builds an effective vocabulary by iteratively merging the most frequently occurring character pairs, and divides the text into a series of subword units based on this vocabulary.

[0028] Through the BPE word segmentation algorithm, the sequence vulnerability code S i is converted into vulnerability code vectors i , the correct code after repair T i is converted into a true inpainting patch vector t i , that is, forming i ,t i >Such a vector pair. These two vectors not only retain the structure and semantic information of the original code, but also represent it in a more compact and efficient form, which is convenient for subsequent model learning and processing.

[0029] The specific calculation formula is as follows:

[0030] s i =BPE(S i ), t i =BPE(T i )

[0031] Where S i Indicates the sequence vulnerability code text; T i Indicates the correct code text after repair.

[0032] After the word segmentation is completed, the obtained i ,t i >The vector pairs are passed as input data to the Bayesian inference-based automatic vulnerability repair model for subsequent model training and vulnerability repair work.

[0033] ​​Preferably, in step 2), a vulnerability automatic repair model based on Bayesian inference is established through a deep learning algorithm, and the model mainly consists of the following three parts:

[0034] a) Encoder: used to receive the vulnerability code vector s after BPE word segmentation processing i As input, it is encoded into a series of hidden states or vector representations that can capture the structural and semantic characteristics of the vulnerable code;

[0035] b) Feature Mapper: Based on the hidden state or vector representation output by the encoder, further extract and transform features to generate an intermediate representation that can reflect the key information required for vulnerability repair. The feature mapper can use Bayesian inference or other statistical learning methods to optimize the feature extraction process, thereby improving the accuracy and generalization ability of the model for vulnerability repair;

[0036] c) Decoder: Receives the intermediate representation output by the feature mapper and decodes it into the correct code vector t after repair i The decoder should be able to generate a fixed, syntactically and semantically correct code sequence corresponding to the input vulnerable code;

[0037] In the model training phase, the parameters of the encoder, feature mapper, and decoder are continuously optimized iteratively, so that the model can gradually learn the mapping relationship from the vulnerable code to the correct code after repair. In the model application phase, the vulnerable code to be repaired is input into the model, and the correct code after repair is output through the encoding, feature mapping, and decoding processes.

[0038] The established automatic vulnerability repair model based on Bayesian inference can realize automatic detection and repair of vulnerable codes and improve code quality and security.

[0039] Preferably, the specific structure of the encoder (Encoder) of the present invention is as follows:

[0040] The encoder is mainly composed of 12 layers of Transformer blocks, each of which contains the following key components:

[0041] a) Normalization Layer: The main function of this layer is to normalize the input vulnerability code vector s i Normalization is performed to ensure the stability and efficiency of the data during model training. Through normalization, the vulnerability code vector s i The value range of is adjusted to an appropriate interval, which is helpful for model learning and optimization;

[0042] b) Self-Attention Layer: This layer is the core part of the Transformer block. It uses the self-attention mechanism to calculate the correlation scores between each word in the input sequence, thereby helping the model understand the complex relationship between words. The self-attention layer can capture the dependencies and context information between each element in the vulnerability code, which is crucial for subsequent feature extraction and vulnerability repair.

[0043] c) Feedforward Neural Network: This layer is located after the self-attention layer and is mainly used to perform in-depth processing and feature extraction on the representation of each word. Through a series of nonlinear transformations and parameter learning, the feedforward neural network can extract more complex and abstract vulnerability features, thereby helping the model better understand the connotation and characteristics of the vulnerability.

[0044] Through the stacking and mutual cooperation of 12 layers of Transformer blocks, the encoder is able to input the vulnerability code vector s i Perform efficient feature extraction and conversion to generate hidden feature data containing rich vulnerability information i '. The specific calculation process is as follows:

[0045] s i '=Encoder(s i )

[0046] Among them, s i Represents the input vulnerability code vector, s i ' represents the hidden feature data after the encoder feature extraction. These hidden feature data will be further processed and utilized by the subsequent feature mapper and decoder to achieve automatic repair of vulnerabilities.

[0047] Preferably, the feature mapper of the present invention is mainly used to map the vulnerability features extracted by the encoder to a probability distribution to facilitate subsequent sampling operations. The specific process is as follows:

[0048] a) Feature Mapping Conversion: First, a trainable embedding matrix P is used to transform the hidden feature data s output by the encoder i 'Indicates that the vector m is obtained by using the embedding matrix P i The embedding matrix P is a trainable parameter of the model and is continuously optimized during the training process to better map the vulnerability features. The calculation formula is:

[0049] m i = Ф(P,s i ')

[0050] Where Ф(·) represents the process of obtaining a vector through an embedding matrix, and the embedding matrix P is a trainable parameter;

[0051] b) Probability distribution construction: In order to quantify the uncertainty of model parameters and support Bayesian inference, the feature mapper further uses the AverageSelfAttention algorithm to transform the vector m i Calculate and get the mean μ and variance σ; the mean self-attention algorithm can comprehensively consider the vector m i , so as to accurately calculate the mean and variance; then, use the mean μ and variance σ to construct a probability distribution D of vulnerability features. The specific calculation formula is D = (μ, σ 2 ); This probability distribution D not only reflects the statistical characteristics of the vulnerability features, but also provides a basis for subsequent sampling operations; the calculation formula for the above process is:

[0052] μ,σ=AverageSelfAttention(m i )

[0053] D=(μ,σ 2 )

[0054] Where AverageSelfAttention(·) represents the calculation process of mean μ and variance σ, and D represents the constructed vulnerability feature probability distribution.

[0055] Through the above operation of the feature mapper, the hidden feature data s extracted by the encoder can be i 'It is effectively converted into the probability distribution D of vulnerability features, providing key information for subsequent Bayesian inference and automatic vulnerability repair.

[0056] Preferably, step 2) further includes: obtaining vulnerability feature information by performing expected sampling on the probability distribution D using the reparameterization technique, and then using Bayesian inference to estimate the posterior distribution of the vulnerability feature. Specifically, the following steps are included:

[0057] c) Reparameterization technique sampling: Based on the obtained vulnerability feature probability distribution D, the reparameterization technique is used to perform expected sampling to obtain specific vulnerability feature information. Specifically, sampling is performed using the following formula:

[0058] Z=Sample(D)

[0059] =μ+σ·∈ i

[0060] Where Z represents the sample obtained from the probability distribution D, which represents the specific vulnerability feature information; μ and σ represent the mean and variance of the vulnerability feature, respectively, which are calculated by the aforementioned feature mapper; ∈ i It is a noise term randomly sampled from a standard normal distribution (i.e., probability distribution D), which is used to introduce randomness and implement sampling operations.

[0061] d) Bayesian inference estimates the posterior distribution: Using the vulnerability feature information Z obtained by sampling, Bayesian inference is performed to estimate the posterior distribution of the vulnerability feature. This step is based on the Bayesian theorem, combining prior knowledge and observed data (i.e., the vulnerability feature information Z obtained by sampling) to calculate the posterior distribution, thereby quantifying the uncertainty of the vulnerability feature.

[0062] Through the operations of steps c) and d), the Bayesian inference method can be further used, combined with the vulnerability feature information obtained by sampling with the reparameterization technique, to estimate the posterior distribution of the vulnerability features, providing a more accurate and reliable basis for subsequent automatic vulnerability repair.

[0063] Preferably, in step 2), when using Bayesian inference to estimate the posterior distribution of vulnerability features, since the posterior distribution is usually not directly analytically solved, sampling is used to effectively approximate the true posterior distribution. Specifically, the core idea of ​​this method is to use the sample Z sampled from the probability distribution D to update the posterior distribution of the parameter, thereby fitting the true posterior part of the vulnerability feature.

[0064] In order to measure the effectiveness of Bayesian inference and optimize it, this method uses KL divergence (Kullback-Leibler Divergence) as an evaluation indicator; KL divergence is used to measure the difference between two probability distributions, and the specific calculation method is as follows:

[0065]

[0066] Among them, q(θ) represents the computable distribution that uses variational inference to approximate the posterior p(θ|D); p(θ|D) represents the true posterior distribution of vulnerability features; D represents the observed data, that is, the vulnerability feature information Z obtained by sampling; the smaller the value of KL divergence, the smaller the difference between the approximate distribution q(θ) and the true posterior distribution p(θ|D), and the higher the inference quality; conversely, the larger the value of KL divergence, the less the approximate distribution can describe the true posterior distribution well, and the poorer the inference quality. For example, if D KL =0, it means that the approximate distribution is completely correct.

[0067] During the optimization process, the parameters of the approximate distribution q(θ) are continuously updated iteratively to minimize the KL divergence, thereby achieving effective estimation of the posterior distribution of vulnerability features, providing a more accurate and reliable basis for subsequent automatic vulnerability repair.

[0068] Preferably, in step 2), the decoder structure is mainly composed of 12 layers of Transformer blocks stacked together, and the decoder is used to decode and repair the sampled vulnerability feature Z. Each Transformer block specifically includes the following components:

[0069] A normalization layer, which is used to normalize the input vulnerability feature vector Z to improve the training efficiency and stability of the model;

[0070] A masked self-attention layer, which introduces a masking mechanism to provide contextual information to the decoding process, ensuring that the generated repair sequence is coherent and consistent;

[0071] A self-attention layer that enables the decoder to dynamically focus on relevant information in the encoder output during the generation process, thereby more accurately understanding and fixing vulnerability features;

[0072] A feedforward neural network layer, which further enriches the feature expression ability of the model through nonlinear transformation and improves the repair effect of the decoder.

[0073] In the decoding process, the vulnerability feature Z is first encoded by the encoder to obtain the encoded feature vector y i 'Input into the first Transformer block of the decoder and processed by each Transformer block in turn; at the end of each Transformer block, the processed feature vector is expanded to the dimension of the encoding dictionary size through a linear expansion layer to obtain the decoded predicted repair patch vector The specific calculation method is as follows:

[0074] y i '=Encoder(Z i )

[0075]

[0076] Among them, t i ' represents the output of the last Transformer block in the decoder, Represents the repair patch vector finally predicted by the decoder, which can be used for subsequent vulnerability repair operations.

[0077] Preferably, in step 2), a loss function loss is constructed to simultaneously measure the gap between the predicted patch vector and the true patch vector, and the gap between the predicted probability distribution of the vulnerability feature and the true posterior part of the vulnerability feature in Bayesian inference; specifically, the calculation formula of the loss function loss is as follows:

[0078]

[0079] in, Represents the reconstruction loss (ReconstructionLoss) between the predicted patch vector and the true patch vector, which is used to evaluate the similarity or difference between the repair patch generated by the model and the true repair patch; Represents the KL divergence (Kullback-Leibler Divergence) between the predicted vulnerability feature probability distribution and the true posterior part of the vulnerability feature, which is used to measure the difference between the two probability distributions;

[0080] During the training process, the backpropagation algorithm is used to optimize the trainable parameters in the vulnerability automatic repair model to minimize the value of the loss function. By continuously iteratively updating the model parameters, the predicted patch vector is closer to the real patch vector, and the predicted probability distribution of vulnerability features is closer to the real posterior distribution of vulnerability features, thereby improving the performance and accuracy of the vulnerability automatic repair model.

[0081] Preferably, in step 3), the trained Bayesian-based automatic vulnerability repair model is used to repair the real vulnerability code, and the specific method of use is as follows:

[0082]

[0083] Among them, si represents the input code feature vector containing vulnerabilities, BRepiar represents the Bayesian-based automatic vulnerability repair model, Indicates the vulnerability code s that the model targets for the input i The resulting repair patch vector.

[0084] Through this usage method, a repair patch for the real vulnerability code can be automatically generated, thereby improving the efficiency and accuracy of vulnerability repair.

[0085] Preferably, in step 4), in order to provide the user with the best repair suggestion, the beam search technology is used to filter out the top n repair patches with the highest prediction results from the repair patch vectors predicted by the model, and feed them back to the user. The specific method is as follows:

[0086]

[0087] in, represents the repair patch vector predicted by the Bayesian vulnerability automatic repair model; the softmax function is used to convert each element of the repair patch vector into a probability value for the convenience of probability sorting; BeamSearch represents the beam search algorithm, which is used to search for the optimal n repair patches from the candidate patches with the highest probability while maintaining a certain computational complexity; Tokenizer represents the word segmentation decoder, which is used to decode the vectorized repair patch output by the beam search algorithm into a readable code text; n It represents the nth optimal repair patch selected by the beam search algorithm, and n represents the preset number of output patch batches, which can be set as needed, such as n=1, 2, 3, 4, 5, 6, etc.

[0088] Through this method, the user can select the most suitable patch from the recommended n repair patches to repair the vulnerability code, thereby improving the efficiency and accuracy of vulnerability repair. In one embodiment of the present invention, n=5.

[0089] The present invention provides a Bayesian inference-based vulnerability automatic repair model established in the above-mentioned Bayesian inference-based vulnerability automatic repair method and a construction method thereof.

[0090] The present invention provides an automatic vulnerability repair system, comprising a processor, wherein the processor executes the automatic vulnerability repair method based on Bayesian inference.

[0091] The present invention has at least the following beneficial effects:

[0092] 1. The present invention uses the Bayesian inference method to map the vulnerability features to a probability distribution, allowing the uncertainty of the model parameters to be quantified, thus changing the problem of insufficient learning diversity caused by the traditional fixed coding to generate a single vulnerability feature.

[0093] 2. The present invention also proposes a Bayesian inference method to sample vulnerability feature data so that it gradually fits the true posterior distribution of vulnerability features, and optimizes the KL divergence to make the constructed distribution close to the true distribution, thereby being able to handle complex, high-dimensional and non-standard posterior distributions of vulnerability features.

[0094] 3. The Bayesian inference-based automatic vulnerability repair technology in the present invention makes full use of the limited public vulnerability data set, greatly improves the learning ability of the model, and uses a deep learning method with strong feature representation ability to build a model, thereby improving the accuracy of vulnerability repair and improving the vulnerability repair efficiency in the vulnerability repair field.

[0095] Other advantages, objectives and features of the present invention will be embodied in part through the following description, and in part will be understood by those skilled in the art through study and practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0096] Figure 1 A flowchart of the automatic vulnerability repair method based on Bayesian inference according to the present invention;

[0097] Figure 2 A network architecture diagram of the automatic vulnerability repair method based on Bayesian inference described in the present invention;

[0098] Figure 3 This is a comparison chart of the prediction accuracy of the model method of the present invention and the existing model. DETAILED DESCRIPTION

[0099] The present invention is further described in detail below with reference to examples so that those skilled in the art can implement the invention with reference to the description.

[0100] It should be understood that the terms such as “having”, “including” and “comprising” used herein do not exclude the existence or addition of one or more other elements or combinations thereof.

[0101] Example 1

[0102] The present invention provides a Bayesian inference-based vulnerability automatic repair model and a construction method thereof, comprising:

[0103] A Bayesian inference-based vulnerability automatic repair model is established through deep learning algorithms. The model mainly consists of an encoder, a feature mapper, and a decoder. The establishment and training process is as follows: Figure 2 The specific instructions are as follows:

[0104] 2a) First, the vulnerability data is encoded using an encoder. The encoder is mainly composed of 12 layers of Transformer blocks, which are mainly used to encode the vulnerability code vector s i For feature extraction, each Transformer block consists of a normalization layer, a self-attention layer, and a feedforward neural network. The normalization layer is mainly used to convert the vulnerability code vector s i Standardization is used to facilitate model training. The self-attention layer is used to help the model understand the relationship between words. The feedforward neural network mainly processes the representation of each word and extracts more complex vulnerability features to help the model better understand the connotation of the vulnerability. The specific calculation is as follows:

[0105] s i '=Encoder(s i )

[0106] where s i ' represents the hidden feature data after the vulnerability code feature is extracted, s i Represent the vulnerability code feature vector, and then go to the next step 2b).

[0107] 2b) Use the feature mapper to map the vulnerability features of the code to a probability distribution to facilitate the subsequent sampling operation. Specifically, the hidden feature data s extracted by the encoder is used. i 'to calculate the mean and variance, and thus construct the probability distribution of vulnerability features. Specifically:

[0108] First, the vulnerability vector s is transformed into i 'Convert to m i , the calculation formula is:

[0109] m i = Ф(P,s i ')

[0110] Where Ф(·) represents the process of obtaining a vector through an embedding matrix, and the embedding matrix P is a trainable parameter;

[0111] Secondly, in order to allow Bayesian inference to quantify the uncertainty of model parameters, the average self-attention algorithm is used to calculate m i Calculate the mean and variance, and then use the mean and variance to construct a probability distribution D of the vulnerability feature. The calculation formula is:

[0112] μ,σ=AverageSelfAttention(m i )

[0113] D=(μ,σ 2 )

[0114] Where AverageSelfAttention(·) represents the calculation process of mean μ and variance σ, and D represents the constructed vulnerability feature probability distribution.

[0115] Again, we use the reparameterization technique to perform expected sampling on the probability distribution D to obtain vulnerability feature information, which is then used for Bayesian inference to estimate the posterior distribution of vulnerability features. The specific calculation method is as follows:

[0116] Z=Sample(D)

[0117] =μ+σ·∈ i

[0118] Where D represents the probability distribution of vulnerability features, Z represents the sampled samples, μ represents the mean of vulnerability features, σ represents the variance of vulnerability features, ∈ i is a noise term randomly sampled from a standard normal distribution, go to the next step 2c).

[0119] 2c) Use Bayesian inference to estimate the posterior distribution of vulnerability features. Since the posterior distribution cannot be directly solved analytically, sampling is used to effectively approximate the true posterior distribution. The core idea is to use Z to update the posterior distribution of the parameters and then fit the true posterior part of the vulnerability features. KL divergence can be used to measure the effectiveness of Bayesian inference and optimize it. The specific calculation method is as follows:

[0120]

[0121] where q(θ) represents a computable distribution that is used to approximate the posterior p(θ|D) using variational inference, if D KL = 0, it means that the approximate distribution is completely correct. If D KL If it is very large, it means that the posterior distribution cannot be well described, the inference quality is poor, and then go to the next step 2d).

[0122] 2d) The decoder is used to decode the sampled data. The decoder is mainly composed of 12 layers of Transformer blocks, which are mainly used to decode and repair the sampled vulnerability features Z. Each Transformer block consists of a normalization layer, a masked self-attention layer, a self-attention layer and a feedforward neural network. Among them, the normalization layer is mainly used to standardize the vulnerability feature vector decoding Z to facilitate model training. The masked self-attention layer is used to provide context information to make the generated sequence coherent. The self-attention layer is used to help the decoder dynamically focus on the relevant information in the encoder output during the generation process. The feedforward neural network enriches the feature expression ability of the model through nonlinear transformation. After that, it is linearly expanded to the dimension of the encoding dictionary size for the subsequent loss construction. The specific calculation method is as follows:

[0123] y i '=Encoder(Z i )

[0124]

[0125] in Represents the repair patch vector predicted by the final decoding, and go to the next step 2e).

[0126] 2e) Construct a loss function to optimize the model. The loss function loss is mainly used to obtain the gap between the predicted patch vector and the true patch vector, as well as the gap between the predicted probability distribution of vulnerability features in Bayesian inference and the true posterior part of the vulnerability features. The back propagation algorithm is then used to train the trainable parameters in the vulnerability automatic repair model to optimize the vulnerability automatic repair model. The calculation formula of the loss function loss is:

[0127]

[0128] in represents the reconstruction loss between the predicted patch vector and the true patch vector, Represents the KL divergence between the predicted vulnerability feature probability distribution and the true posterior part of the vulnerability feature.

[0129] During the training process, the backpropagation algorithm is used to optimize the trainable parameters in the vulnerability automatic repair model to minimize the value of the loss function. By continuously iteratively updating the model parameters, the predicted patch vector is closer to the real patch vector, and the predicted probability distribution of vulnerability features is closer to the real posterior distribution of vulnerability features, thereby improving the performance and accuracy of the vulnerability automatic repair model.

[0130] Example 2

[0131] like Figure 1 and Figure 2 As shown, the present invention provides a method for automatically repairing vulnerabilities based on Bayesian inference, comprising:

[0132] 1) Obtain the data of vulnerability codes and their historical repair records, and pre-process the data to obtain a vulnerability dataset containing vulnerability codes and correct codes after repair, and perform word segmentation on it. The specific instructions are as follows:

[0133] 1a) Download the vulnerability datasets CVEFixes and Big-Vul, obtain their vulnerability code data and their historical repair record data, treat the vulnerability code as training data, and the historical repair record as the model expected data, and then go to the next step 1b).

[0134] 1b) Cleaning block-level vulnerability codes and adding hint word information, converting block-level program codes into code text sequences and adding hint word information; vulnerability codes and correct codes are preprocessed to obtain a binary vulnerability dataset: (sequence vulnerability code S i , the correct code after repair T i ),Right now i, T i >, then go to the next step 1c). ​

[0135] 1c) Use the BPE word segmentation algorithm to analyze the sequence vulnerability code S i And the correct code after repair T i Perform word segmentation to obtain the vulnerability code vector s i With the real repair patch vector t i ,Right now i , t i > and pass it as input to the model. The calculation formula is as follows:

[0136] s i =BPE(S i ), t i =BPE(T i )

[0137] Where S i Indicates the sequence vulnerability code text, T i Indicates the correct code text after repair.

[0138] 2) Establish a Bayesian inference-based vulnerability automatic repair model through deep learning algorithms. The model mainly consists of an encoder, a feature mapper, and a decoder. The establishment and training process is as follows: Figure 2 The specific instructions are as follows:

[0139] 2a) First, the vulnerability data is encoded using an encoder. The encoder is mainly composed of 12 layers of Transformer blocks, which are mainly used to encode the vulnerability code vector s i For feature extraction, each Transformer block consists of a normalization layer, a self-attention layer, and a feedforward neural network. The normalization layer is mainly used to convert the vulnerability code vector s i Standardization is used to facilitate model training. The self-attention layer is used to help the model understand the relationship between words. The feedforward neural network mainly processes the representation of each word and extracts more complex vulnerability features to help the model better understand the connotation of the vulnerability. The specific calculation is as follows:

[0140] s i '=Encoder(s i )

[0141] where s i ' represents the hidden feature data after the vulnerability code feature is extracted, s i Represent the vulnerability code feature vector, and then go to the next step 2b).

[0142] 2b) Use the feature mapper to map the vulnerability features of the code to a probability distribution to facilitate the subsequent sampling operation. Specifically, the hidden feature data s extracted by the encoder is used. i ​'to calculate the mean and variance, and thus construct the probability distribution of vulnerability features. Specifically:

[0143] First, the vulnerability vector s is transformed into i 'Convert to m i , the calculation formula is:

[0144] m i = Ф(P,s i ')

[0145] Where Ф(·) represents the process of obtaining a vector through an embedding matrix, and the embedding matrix P is a trainable parameter;

[0146] Secondly, in order to allow Bayesian inference to quantify the uncertainty of model parameters, the average self-attention algorithm is used to calculate m i Calculate the mean and variance, and then use the mean and variance to construct a probability distribution D of the vulnerability feature. The calculation formula is:

[0147] μ,σ=AverageSelfAttention(m i )

[0149] D=(μ,σ 2 )

[0150] Where AverageSelfAttention(·) represents the calculation process of mean μ and variance σ, and D represents the constructed vulnerability feature probability distribution.

[0151] Again, we use the reparameterization technique to perform expected sampling on the probability distribution D to obtain vulnerability feature information, which is then used for Bayesian inference to estimate the posterior distribution of vulnerability features. The specific calculation method is as follows:

[0152] Z=Sample(D)

[0153] =μ+σ·∈ i

[0154] Where D represents the probability distribution of vulnerability features, Z represents the sampled samples, μ represents the mean of vulnerability features, σ represents the variance of vulnerability features, σ·∈ i is a noise term randomly sampled from a standard normal distribution, go to the next step 2c).

[0155] 2c) Use Bayesian inference to estimate the posterior distribution of vulnerability features. Since the posterior distribution cannot be directly solved analytically, sampling is used to effectively approximate the true posterior distribution. The core idea is to use Z to update the posterior distribution of the parameters and then fit the true posterior part of the vulnerability features. KL divergence can be used to measure the effectiveness of Bayesian inference and optimize it. The specific calculation method is as follows:

[0156]

[0157] where q(θ) represents a computable distribution that is used to approximate the posterior p(θ|D) using variational inference, if D KL = 0, it means that the approximate distribution is completely correct. If D KL If it is very large, it means that the posterior distribution cannot be well described, the inference quality is poor, and then go to the next step 2d).

[0158] 2d) The decoder is used to decode the sampled data. The decoder is mainly composed of 12 layers of Transformer blocks, which are mainly used to decode and repair the sampled vulnerability features Z. Each Transformer block consists of a normalization layer, a masked self-attention layer, a self-attention layer and a feedforward neural network. Among them, the normalization layer is mainly used to standardize the vulnerability feature vector decoding Z to facilitate model training. The masked self-attention layer is used to provide context information to make the generated sequence coherent. The self-attention layer is used to help the decoder dynamically focus on the relevant information in the encoder output during the generation process. The feedforward neural network enriches the feature expression ability of the model through nonlinear transformation. After that, it needs to be linearly expanded to the dimension of the encoding dictionary size for the subsequent loss construction. The specific calculation method is as follows:

[0159] y i '=Encoder(Z i )

[0160]

[0161] in Represents the repair patch vector predicted by the final decoding, and go to the next step 2e).

[0162] 2e) Construct a loss function to optimize the model. The loss function loss is mainly used to obtain the gap between the predicted patch vector and the true patch vector, as well as the gap between the predicted probability distribution of vulnerability features in Bayesian inference and the true posterior part of the vulnerability features. The back propagation algorithm is then used to train the trainable parameters in the vulnerability automatic repair model to optimize the vulnerability automatic repair model. The calculation formula of the loss function loss is:

[0163]

[0164] in represents the reconstruction loss between the predicted patch vector and the true patch vector, Represents the KL divergence between the predicted vulnerability feature probability distribution and the true posterior part of the vulnerability feature.

[0165] 3) The method of using the Bayesian-based vulnerability automatic repair model to repair the real vulnerability code is as follows:

[0166]

[0167] Among them, s i represents the input code feature vector containing vulnerabilities, BRepiar represents the Bayesian-based automatic vulnerability repair model, Represents the inpaint patch vector generated by the model.

[0168] 4) Using beam search technology, the top n repair patches with the highest prediction results are fed back to the user. The specific method is as follows:

[0169]

[0170] where t n represents the predicted repair patch, n represents the number of output patch batches, represents the repair patch vector predicted by the model, and Tokenizer represents a word segmentation decoder, which is used to decode the vectorized data into code text. In one embodiment, n=5.

[0171] Effect example

[0172] The VRpair model in the existing document 1, the CodeBert model in the existing document 2, and the method model BRepiar of the present invention were tested on the same data set (CVEFixes and Big-Vul), n=5, and their perfect repair accuracy was compared. The results are shown in Tables 1 and Figure 3 As shown:

[0173] Document 1: G.Bhandari, A.Naseer, L.Moonen, Cvefixes: automated collection ofvulnerabilities and their fixes from open-source software, in: Proceedings of the 17th International Conference on Predictive Models and Data Analytics inSoftware Engineering, 2021, pp.30–39.

[0174] Document 2: J.Fan, Y.Li, S.Wang, TNNguyen, Ac / c++code vulnerability dataset with code changes and cve summaries, in: Proceedings of the 17th International Conference on Mining Software Repositories, 2020, pp.508–512.

[0175] Table 1 Comparison of prediction accuracy between the model method of the present invention and the existing model

[0176] Model Accuracy VRpair model in Reference 1 0.2321 CodeBert model in Reference 2 0.3142 Method model of the present invention BRepair 0.5277

[0177] From Table 1 and Figure 3 It can be seen that the prediction accuracy of the model of the method of the present invention is higher, which is significantly better than the existing VRpair model and CodeBert model.

[0178] Although the embodiments of the present invention have been disclosed above, they are not limited to the applications listed in the specification and the embodiments. They can be applied to various fields suitable for the present invention. For those familiar with the art, additional modifications can be easily realized.< / endloc> < / startloc> ​

Claims

1. A method for constructing an automatic vulnerability repair model based on Bayesian inference, characterized in that: include: Construct encoder: Input vulnerability code vector s i Perform feature extraction and output hidden feature data s i '; Construct feature mapper: Use embedding matrix P to transform s i 'Convert to vector m i , and calculate the mean μ and variance σ through the mean self-attention algorithm to construct the probability distribution D of the vulnerability features; use the reparameterization technique to sample on the probability distribution D to obtain the vulnerability feature information Z; Construct a decoder: Input the sampled vulnerability feature information Z, perform decoding and repair, and output the repair patch vector Constructing loss function: including reconstruction loss and KL divergence The model is trained using the back-propagation algorithm to optimize the trainable parameters.

2. A vulnerability automatic repair model based on Bayesian inference constructed by the construction method described in claim 1.

3. A Bayesian inference-based automatic vulnerability repair method, characterized in that: The following steps are involved: 1) Data acquisition and preprocessing step: acquiring data containing vulnerability codes and their historical repair records, and preprocessing the data to generate a vulnerability data set containing vulnerability codes and correct codes after repair; further performing word segmentation processing on the vulnerability data set; 2) Model building and training steps: building a vulnerability automatic repair model based on Bayesian inference, and using the vulnerability dataset to train the vulnerability automatic repair model to learn the mapping relationship between vulnerability code and repair patch; 3) Vulnerability code repair step: Use the trained vulnerability automatic repair model to automatically repair the real vulnerability code; 4) Repair patch feedback step: using a beam search method to obtain the top n repair patches with the highest prediction results from the repair patches generated by the automatic vulnerability repair model, and feeding back these repair patches to the user for screening.

4. The Bayesian inference-based automatic vulnerability repair method according to claim 3, characterized in that: In the step 2), the Bayesian inference-based automatic vulnerability repair model according to claim 1 is established through a deep learning algorithm, and the model mainly consists of an encoder, a feature mapper and a decoder.

5. The automatic vulnerability repair method based on Bayesian inference according to claim 4 is characterized in that: The encoder is mainly composed of 12 layers of Transformer blocks stacked together, and the input vulnerability code vector s i Perform feature extraction and conversion to generate hidden feature data containing rich vulnerability information i '.

6. The Bayesian inference-based automatic vulnerability repair method according to claim 5, characterized in that: The feature mapper is set to: a) Feature Mapping Conversion: Use a trainable embedding matrix P to transform the hidden feature data s output by the encoder i 'Convert to vector m i , the calculation formula is: m i = Ф(P,s i '); where Ф(·) represents the process of obtaining a vector through an embedding matrix, and the embedding matrix P is a trainable parameter; b) Probability distribution construction: Use the mean self-attention algorithm to transform the vector m i Calculate and get the mean μ and variance σ, the calculation formula is μ, σ = AverageSelfAttention (mi); and use the mean μ and variance σ to construct a probability distribution D of vulnerability features, the calculation formula is D = (μ, σ 2 ); c) Reparameterization technique sampling: Based on the obtained vulnerability feature probability distribution D, the reparameterization technique is used to perform expected sampling to obtain specific vulnerability feature information. The calculation formula is Z = Sample (D) = μ + σ · ∈ i , where Z represents the sample obtained from the probability distribution D, representing the specific vulnerability feature information, μ and σ represent the mean and variance of the vulnerability feature respectively, ∈ i is a noise term randomly sampled from the probability distribution D.

7. The automatic vulnerability repair method based on Bayesian inference according to claim 6 is characterized in that: It also includes Bayesian inference to estimate the posterior distribution: using the vulnerability feature information Z obtained by sampling, Bayesian inference is performed to estimate the posterior distribution of the vulnerability feature, using KL divergence as an evaluation indicator; it is used to measure the difference between two probability distributions, and the calculation method is as follows: Among them, q(θ) represents the computable distribution that uses variational inference to approximate the posterior p(θ|D); p(θ|D) represents the true posterior distribution of the vulnerability feature; D represents the observed data, that is, the vulnerability feature information Z obtained by sampling; the smaller the value of KL divergence, the smaller the difference between the approximate distribution q(θ) and the true posterior distribution p(θ|D), and the higher the inference quality; conversely, the larger the value of KL divergence, the greater the value of KL divergence, which indicates that the approximate distribution cannot describe the true posterior distribution well and the inference quality is poor.

8. The Bayesian inference-based automatic vulnerability repair method according to claim 7, characterized in that: The decoder is mainly composed of 12 layers of Transformer blocks stacked together; the vulnerability feature Z is encoded by the encoder to obtain the encoded feature vector y i 'Input into the first Transformer block of the decoder and processed by each Transformer block in turn; at the end of each Transformer block, the processed feature vector is expanded to the dimension of the encoding dictionary size through a linear expansion layer to obtain the repair patch vector predicted by the decoder The calculation is as follows: y i ’=Encoder(Z i ) Among them, t i ' represents the output of the last Transformer block in the decoder, Represents the repair patch vector finally predicted by the decoder.

9. The Bayesian inference-based automatic vulnerability repair method according to claim 8, characterized in that: Construct the loss function loss, the calculation formula is as follows: in, Represents the reconstruction loss between the predicted patch vector and the true patch vector, which is used to evaluate the similarity or difference between the repair patch generated by the model and the true repair patch; It represents the KL divergence between the predicted probability distribution of vulnerability features and the true posterior part of the vulnerability features, which is used to measure the difference between the two probability distributions.

10. The Bayesian inference-based automatic vulnerability repair method according to claim 9, characterized in that: The back propagation algorithm is used to optimize the trainable parameters in the automatic vulnerability repair model to minimize the value of the loss function.

Images