Software vulnerability automatic repair method based on thinking chain and storage medium

Abstract

The invention discloses a thinking chain-based software vulnerability automatic repair method and a storage medium. The method comprises the following steps of S1, preprocessing training data; constructing input and output of the model, including a vulnerability program, a repair position and a repair program; s2, model training; taking a vulnerability program as the input of the model, and constructing a sequence of {a repair position + a repair program} as the expected output; step S3, vulnerability restoration; and S2, inputting a given bug program to be repaired into the trained model obtained in the step S2 after utilizing the input in the step S1, searching possible outputs by utilizing a cluster search technology, and arranging the outputs from high to low according to the possibility to obtain candidate repair programs generated by CotRepair. The storage medium is used for storing a computer program for executing the method. The method has the advantages of simple principle, wide application range, high automation degree, high repairing efficiency and precision and the like.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of automatic repair of software program defects, and in particular to a method and storage medium for automatic repair of software vulnerabilities based on thought chains. Background Art

[0002] As the number and complexity of software vulnerabilities increase, researchers have proposed many automated methods to help developers discover, detect, and locate vulnerabilities, but fixing vulnerabilities still requires researchers to spend a lot of effort. In recent years, some researchers have begun to pay attention to software vulnerability automatic repair technology, but the existing technology only generalizes software vulnerability repair as a general text generation problem, without locating the defect repair location, resulting in a large repair program generation space and low repair program quality.

[0003] Current vulnerability repair methods can be divided into repair methods for specific types of vulnerabilities and repair methods for general types of vulnerabilities, depending on whether they target specific types of vulnerabilities. Among them, the repair methods for specific types of vulnerabilities usually customize proprietary vulnerability location and patch generation algorithms based on the common root causes and repair mechanisms of specific types of vulnerabilities. The general type method does not specify the vulnerability type. The mainstream method learns the vulnerability repair process based on the historical data of the vulnerability and the learning algorithm to achieve the repair of any type of vulnerability. For the methods of repairing specific types of vulnerabilities, current research is mainly carried out on vulnerabilities with greater comprehensive influence, such as common and harmful vulnerabilities such as buffer overflow and integer overflow. For example, practitioners Sidiroglou et al. proposed the Code Phage technology, which assumes that the non-vulnerable program contains branch checks that are missing in the vulnerable program, and the missing branch checks will lead to buffer overflow vulnerabilities. Based on this assumption, the technology searches for programs that can correctly handle failed cases from healthy programs, extracts the missing branch checks in the vulnerable program, and then modifies the branch conditions and transplants them to the vulnerable program to achieve repair. However, this technology has problems such as overly ideal assumptions and it is difficult to find qualified non-vulnerable programs as donors.

[0004] In order to improve the quality of patch generation, Huang et al. use program synthesis to generate vulnerability patching programs. Specifically, for buffer overflow vulnerabilities and integer overflow vulnerabilities, they first define a set of security constraints, and then use symbolic execution to detect whether the expression meets the security constraints. For expressions that do not meet the constraints, a set of pre-defined patch templates are used to generate patching programs.

[0005] As for the repair method of common vulnerabilities, practitioners Chen et al. proposed a vulnerability repair method VReapir based on transfer learning. This work uses the Transformer model, first pre-trained on the C language defect repair dataset, and then fine-tuned the model on a dataset containing 3754 vulnerabilities. The evaluation results show that compared with direct training on the vulnerability dataset, the repair accuracy after pre-training is improved from 12.58% to 17.30%. Despite this, VReapir still has the problem of a single pre-training task, resulting in poor code semantic capture capabilities.

[0006] In order to alleviate the above problems, another practitioner Fu et al. proposed a vulnerability repair method based on the T5 pre-trained model. This method used the T5 model obtained by pre-training 4 tasks on 8.35 million code instances as the base, and fine-tuned it on 8482 vulnerability data to alleviate the problem of poor code semantic capture of VRepair. The evaluation results show that compared with the VReapir method, the repair accuracy of VulRepair has increased from 23% to 44%. This result reveals the possibility of using the rich information contained in the pre-trained model to improve the repair effect. Despite this, the current general type vulnerability repair method still only attempts to directly translate the defective code into a repair program; without precise positioning information, all elements in the entire vulnerability program may be defective elements, resulting in a large search space for the generation of the repair program, which in turn affects the generation quality of the repair program. Summary of the invention

[0007] The technical problem to be solved by the present invention is: in response to the technical problems existing in the prior art, the present invention proposes a software vulnerability automatic repair method and storage medium based on a thinking chain, which has a simple principle, a wide range of applications, a high degree of automation, and high repair efficiency and precision.

[0008] In order to solve the above technical problems, the present invention adopts the following technical solutions: A method for automatically repairing software vulnerabilities based on thought chains, comprising: Step S1: training data preprocessing; constructing the input and output of the model, including the vulnerability program, repair location and repair program; Step S2: Model training; using the vulnerable program as the input of the model, constructing a sequence of {repair location + repair program} as the expected output, and reducing the cross entropy between the expected output and the actual output of the model as the optimization goal; Step S3: Vulnerability repair; for a given vulnerability program to be repaired, the input of step S1 is sent to the trained model obtained in step S2, and the possible outputs are searched using beam search technology. The outputs are arranged from high to low in terms of probability, and the candidate repair program generated by CotRepair is obtained.

[0009] As a further improvement of the method of the present invention: the process of step S1 includes: Step S11: data preprocessing; Step S12: data representation; Step S13: Code segmentation: For the input vulnerable program and repair program, the BPE algorithm is used to segment the input vulnerable program.

[0010] As a further improvement of the method of the present invention: in the step S11, for a vulnerability repair submission record existing in the data set, the code before and after the modification, that is, the original vulnerable program and the repair program, is extracted by searching the historical repair records; then tree-sitte is used to remove all comments in the original vulnerable program and the repair program; tree-sitter is used to eliminate all functions that have not been changed, and the functions that have been changed are segmented by the BPE word segmentation algorithm to obtain the vulnerable program and the repair program; the difflib tool is used to compare the two sequences and generate repair position information.

[0011] As a further improvement of the method of the present invention: in step S12, a submission record generated during the vulnerability repair process is used as the basic component unit of the data set; the input of the model follows the previous history-driven vulnerability automatic repair method, using the marked vulnerability program as input; two special tokens are used in the vulnerability program <startloc>and <endloc>Mark the first line of defective code in the vulnerable program; for defects that appear in multiple lines, only the first line of defective code is marked.

[0012] As a further improvement of the method of the present invention: the step S2 comprises: Step S21: embedding layer; the embedding layer consists of two parts: code embedding layer and position encoding layer; the code embedding layer is used to map the text code to the vector space through code embedding; the position encoding layer is used to capture the embedding matrix Any two elements in and The relative position relationship between to The relative position relationship is encoded as two vectors , ; Step S22: Encoder; composed of 12 layers of encoder layers with the same structure, which are linearly superimposed to encode the input data into a dense representation, to extract the most important information in the input data, and convert it into a low-dimensional vector, while retaining some features of the original data as much as possible; Step S23: Decoder: After obtaining the hidden layer state extracted by the encoder Afterwards, the model decodes the feature using a decoder; the decoder consists of a linear stack of 12 identical decoding layers, with the first layer input being the hidden layer state and the vector matrix of the expected output after the code embedding layer ; Vector matrix First, the multi-head self-attention layer is masked, and then the hidden layer state , sent to the multi-head self-attention layer, and after the feedforward neural network, the output of the first layer is obtained ; Step S24: Model training: The model is trained using a cross entropy loss function.

[0013] As a further improvement of the method of the present invention: the encoder layer is composed of a multi-head self-attention calculation module with relative position encoding and a feedforward neural network; the multi-head self-attention calculation module is composed of multiple self-attention heads, which are used to capture different interactive information in multiple different projection spaces and fuse them through different self-attention heads; for a single attention head with relative position encoding, its receiving matrix As input, three linear transformation matrices are used Map the input vectors to three matrices belonging to three subspaces, respectively denoted as , and ,Each subspace reflects different hidden features.

[0014] As a further improvement of the method of the present invention: in the embedding layer, it is assumed that the vulnerability program input , after word segmentation, we get a token stream sequence After the word embedding matrix Find the The semantic vector corresponding to the token , get the vulnerable program The embedding matrix ; The specific process can be expressed as: .

[0015] As a further improvement of the method of the present invention: the specific process of the position coding layer is expressed as follows:

[0016]

[0017]

[0018] The Clip function is used to clip the relative position between two elements that are too far apart. , then the relative position between them is considered to be ;Relative position encoding matrix , is the learnable weight matrix, where , ; Step S24: Model training: Use the cross entropy loss function to train the model; the specific process of loss function calculation can be expressed as:

[0019] in, The token stream generated for the model elements, is the probability distribution of tokens in an ideal situation, The token probability distribution generated for the model.

[0020] As a further improvement of the method of the present invention: the step S3 comprises: Step S31: extract the original vulnerable program, remove all comments and use existing defect location tools to obtain row-level defect information, and use the input of step S1. This data is the input required for step S32; Step S32: Input the preprocessed vulnerable program into the model trained in step S2, and the model extracts and decodes the sequence to generate a probability distribution of tokens that constitute the repair program; in the process of generating the repair program, the beam search strategy is used to select the token with the highest probability. k The candidate repair programs are outputted, that is, the candidate repair programs predicted by the model are obtained.

[0021] The present invention further provides a storage medium, which can be read by a computer or a processor, and stores a computer program for executing any one of the above methods.

[0022] Compared with the prior art, the advantages of the present invention are: 1. The automatic software vulnerability repair method and storage medium based on the thinking chain of the present invention have simple principles, wide application range, high degree of automation, high repair efficiency and precision. The present invention uses the thinking chain technology to help the model generate a repair program more accurately. Based on the T5 pre-trained model, by taking the vulnerable program as input, relying on the thinking chain technology to assist in learning with the repair location information during the training process, the candidate repair program is generated to help developers repair the vulnerable program faster and more accurately.

[0023] 2. The automatic software vulnerability repair method and storage medium based on the thinking chain of the present invention can accurately find the repair position of the vulnerability. Compared with the existing vulnerability repair technology that only relies on the row-level defect position markings provided by the existing defect location tools or developers to repair the vulnerable program, the present invention proposes to first rely on the thinking chain technology based on the existing defect position markings, require the model to predict the repair positions with higher probability, and then preferentially search the possible repair space at these positions to generate a repair program. Under the constraint of such fine-grained repair position prediction results, the repair program search space of the present invention is more accurate.

[0024] 3. The automatic software vulnerability repair method and storage medium based on the thinking chain of the present invention can generate high-quality repair programs based on the vulnerability repair technology proposed by the present invention. Compared with the existing vulnerability repair technology, when performing vulnerability repair, it only attempts to directly translate the defective code into a repair program. In the absence of repair location information, all elements in the vulnerability program may be identified as defective elements by the model, resulting in a larger search space for patch generation, thereby affecting the generation quality of the patch. The present invention predicts possible defective elements in the vulnerability program, so that the model can learn to first locate the repair location and then generate the repair program, thereby refining the search space for the repair program generation to improve the generation quality of the repair program.

[0025] 4. The automatic software vulnerability repair method and storage medium based on the thinking chain of the present invention have higher repair efficiency based on the defect location technology proposed in the present invention; based on the existing vulnerability repair technology, the search space of the repair tool is large due to the lack of accurate repair location information, and repair attempts are made to non-repair locations in the vulnerability program, resulting in the generation of a large number of incorrect repair programs. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0027] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0028] In order to further explain the technical solution of the present invention in detail, the following concepts are first described: Automatic software vulnerability repair technology: helps developers discover, detect and locate vulnerabilities through automated means, and generates corresponding repair programs.

[0029] Thinking chain: Taking the intermediate steps of solving mathematical problems as part of the output enables the model to have reasoning ability to a certain extent, thereby improving the ability to solve complex problems.

[0030] CodeT5 model: The CodeT5 model is a pre-trained encoder-decoder model for processing the task of converting between source code and target code.

[0031] Transformer architecture: is a language processing model that consists of multiple encoder layers and decoder layers, each of which contains a self-attention module and a feedforward network module.

[0032] Embedding layer: The embedding layer consists of two parts: code embedding layer and position encoding layer. The purpose of code embedding is to map the text code to the vector space, and the position encoding layer is used to capture the relative position relationship between any two elements in the embedding matrix.

[0033] Encoder: It is used to encode the input data into a dense representation, extract the most important information in the input data, and convert it into a low-dimensional vector.

[0034] Decoder: It is used to decode the hidden layer states extracted by the encoder and generate the final repair program.

[0035] The present invention is based on the idea of ​​thought chain, simulates the process of human beings repairing vulnerabilities, decomposes the vulnerability repair process into repair position positioning and repair program generation, solves the vulnerability repair in steps, and refines the search space of the repair program to improve the quality of repair program generation.

[0036] like Figure 1 As shown, the method for automatically repairing software vulnerabilities based on the thought chain of the present invention includes the following steps: Step S1: training data preprocessing; constructing the input and output of the model, including the vulnerability program, repair location and repair program; Step S2: Model training; using the vulnerable program as the input of the model, constructing a sequence of {repair location + repair program} as the expected output, and reducing the cross entropy between the expected output and the actual output of the model as the optimization goal; Step S3: Vulnerability repair; for a given vulnerability program to be repaired, after using the data preprocessing technology described in step S1 to obtain an input that meets the requirements, it is sent to the trained model obtained in step S2, and the possible outputs are searched using the beam search technology. The outputs are arranged from high to low in terms of probability, and the candidate repair program generated by CotRepair is obtained.

[0037] In a specific application example, the step S1 may include, according to actual needs: Step S11: data preprocessing; For a vulnerability repair submission record in the dataset, the code before and after modification, that is, the original vulnerability program and the repair program, are extracted by searching the historical repair records.

[0038] Then, tree-sitte is used to remove all comments in the original vulnerable program and the fixed program. The purpose of this operation is to eliminate updates that are not related to the code to avoid introducing noise in the data.

[0039] After removing all comments, use tree-sitter to eliminate all functions that have not been changed, and use the BPE word segmentation algorithm to segment the functions that have been changed to obtain the vulnerable program and the fixed program.

[0040] Finally, the difflib tool was used to compare the two sequences and generate repair position information; Step S12: data representation; The present invention uses a submission record generated during the vulnerability repair process as the basic component unit of the data set. In terms of model input, the previous history-driven automatic vulnerability repair method is followed, and the marked vulnerability program is used as input.

[0041] Among them, two special tokens are used in the vulnerable program <startloc>and <endloc>Mark the first line of defective code in the vulnerable program. For defects that appear in multiple lines, only the first line of defective code is marked.

[0042] Step S13: code segmentation; For the input vulnerable program and the fixed program, the BPE (Byte Pair Encoding) algorithm is first used to segment the input vulnerable program. The BPE algorithm divides longer code tokens into smaller but more common sub-token sequences. These root words help build better word embedding models and language models, and have been widely used in language models.

[0043] In a specific application example, the step S2 may include, according to actual needs: Step S21: embedding layer; The embedding layer consists of two parts: the code embedding layer and the position encoding layer.

[0044] The purpose of code embedding is to map textual codes to vector space. Through code embedding, the code is no longer regarded as a sequence of independent symbols, but is mapped to a dense, real-valued vector space.

[0045] Specifically, assume that for the vulnerable program input , after word segmentation, we get a token stream sequence After the word embedding matrix Find the The semantic vector corresponding to the token , get the vulnerable program The embedding matrix .

[0046] The specific process can be expressed as:

[0047] For the position encoding layer, since the Transformer architecture does not contain structures such as recursion and convolution that can capture position information, it is necessary to build an additional structure to capture position information. The position encoding layer is used to capture the embedding matrix Any two elements in and The relative position relationship between to The relative position relationship is encoded as two vectors , .

[0048] The specific process can be expressed as:

[0049]

[0050]

[0051] The Clip function is used to clip the relative position between two elements that are too far apart. , then the relative position between them is considered to be ;Relative position encoding matrix , is the learnable weight matrix, where , .

[0052] Step S22: encoder; The encoder part consists of 12 linearly stacked encoder layers with the same structure, which is used to encode the input data into a dense representation, aiming to extract the most important information in the input data and convert it into a low-dimensional vector while retaining certain features of the original data as much as possible.

[0053] Specifically, the encoder layer includes a multi-head self-attention calculation module with relative position encoding and a feedforward neural network. The self-attention mechanism aims to enable the model to weight the information at different positions in the sequence when processing the input sequence, thereby improving the model's attention to and ability to extract relevant information. The multi-head self-attention mechanism consists of multiple self-attention heads, and its purpose is to capture and fuse different interactive information in multiple different projection spaces through different self-attention heads. For a single attention head with relative position encoding, its receiving matrix As input, three linear transformation matrices are used Map the input vectors to three matrices belonging to three subspaces, respectively denoted as , and ,Each subspace reflects different hidden features.

[0054] The specific calculation process can be expressed as:

[0055] in, , , , , The calculation process is as follows:

[0056]

[0057]

[0058] After all the self-attention heads are calculated, their results are concatenated and sent to the feedforward neural network. Therefore, the specific process of the multi-head self-attention mechanism can be expressed as:

[0059]

[0060] in, For the The weight matrix that maps the input in each attention head; Used to linearly map the merged vector to obtain the output, where , is the number of subspaces, and the remaining parameter values ​​are .

[0061] Finally, a layer of feed-forward neural network processes the input. Its purpose is to make the model more flexible by adding nonlinear transformations, so that it can better capture the relationship between the elements of each input sequence and make full use of the information in the sequence. The specific process can be expressed as:

[0062] in, is the weight matrix for transforming the input, The weight matrix of the feedforward neural network in different encoder layers has independent parameters. The encoder consists of 12 layers of encoder layers with the same structure, and the output of the previous layer is the input of the next layer. Finally, the output of the last encoder layer is extracted as the hidden layer state. .

[0063] Step S23: decoder; After obtaining the hidden layer state extracted by the encoder Afterwards, the model decodes the feature using a decoder.

[0064] As a preferred embodiment, the decoder consists of a linear superposition of 12 identical decoding layers. The input of the first layer is the hidden layer state and the vector matrix of the expected output after the code embedding layer ; Vector matrix First, the multi-head self-attention layer is masked, and then the hidden layer state , sent to the multi-head self-attention layer, and after the feedforward neural network, the output of the first layer is obtained ; The specific process can be expressed as:

[0065]

[0066] After this, each The input of the layer encoder is The output of the layer encoder; the specific process can be expressed as:

[0067]

[0068] The output of the last layer is connected to a fully connected layer, which projects the output into a vector with a dimension equal to the size of the vocabulary, and then transforms the vector into a probability distribution on the vocabulary through the Softmax function to generate the final token probability distribution matrix; the specific process can be expressed as:

[0069] in, , is the vocabulary size.

[0070] Step S24: model training; The model is trained using the cross entropy loss function.

[0071] The specific process of loss function calculation can be expressed as:

[0072] in, The token stream generated for the model elements, is the probability distribution of tokens in an ideal situation, The token probability distribution generated for the model.

[0073] In a specific application example, the process of step S3 may include, according to actual needs: Step S31: extracting the original vulnerable program; After removing all annotations and using existing defect location tools to obtain row-level defect information, the data preprocessing technology described in step S1 is used to obtain input that meets the requirements. This data is the input required for step S32. Step S32: input the preprocessed vulnerable program into the model trained in step S2, and the model extracts and decodes the sequence to generate a probability distribution of tokens constituting the repair program; In the process of generating the repair program, a beam search strategy is used to select the most probable k The candidate repair programs are outputted, that is, the candidate repair programs predicted by the model are obtained.

[0074] The present invention further provides a storage medium, which can be read by a computer or a processor and stores a computer program for executing the above method.

[0075] Those skilled in the art should understand that the above-mentioned embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0076] The above are only preferred embodiments of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should be regarded as the protection scope of the present invention.< / endloc> < / startloc> < / endloc> < / startloc>

Claims

1. A method for automatically repairing software vulnerabilities based on thought chain, It is characterized in that include: Step S1: training data preprocessing; Construct the input and output of the model, including the vulnerability program, repair location and repair procedure; Step S2: Model training; using the vulnerable program as the input of the model, constructing a sequence of {repair location + repair program} as the expected output, and reducing the cross entropy between the expected output and the actual output of the model as the optimization goal; Step S3: Vulnerability repair; for a given vulnerability program to be repaired, the input of step S1 is sent to the trained model obtained in step S2, and the possible outputs are searched using beam search technology. The outputs are arranged from high to low in terms of probability, and the candidate repair program generated by CotRepair is obtained.

2. According to claim 1, the method for automatically repairing software vulnerabilities based on thought chain, It is characterized in that The process of step S1 includes: Step S11: data preprocessing; Step S12: data representation; Step S13: Code segmentation: For the input vulnerable program and repair program, the BPE algorithm is used to segment the input vulnerable program.

3. According to claim 2, the method for automatically repairing software vulnerabilities based on thought chain, It is characterized in that In step S11, for a vulnerability repair submission record existing in the data set, the code before and after the modification, i.e., the original vulnerable program and the repair program, is extracted by searching the historical repair records; then, all comments in the original vulnerable program and the repair program are removed by using tree-sitte; tree-sitter is used to eliminate all functions that have not been changed, and the functions that have been changed are segmented by using the BPE word segmentation algorithm to obtain the vulnerable program and the repair program; the difflib tool is used to compare the two sequences and generate repair position information.

4. The method for automatically repairing software vulnerabilities based on thought chain according to claim 2, It is characterized in that In step S12, a submission record generated during the vulnerability repair process is used as the basic component unit of the data set; the input of the model follows the previous history-driven vulnerability automatic repair method, using the marked vulnerability program as input; two special tokens are used in the vulnerability program <startloc>and <endloc> Mark the first line of defective code in the vulnerable program; for defects that appear in multiple lines, only the first line of defective code is marked.< / endloc> < / startloc> 5. The method for automatically repairing software vulnerabilities based on thought chain according to any one of claims 1 to 4, It is characterized in that The step S2 comprises: Step S21: embedding layer; the embedding layer consists of two parts: code embedding layer and position encoding layer; the code embedding layer is used to map the text code to the vector space through code embedding; the position encoding layer is used to capture the embedding matrix Any two elements in and The relative position relationship between to The relative position relationship is encoded as two vectors , ; Step S22: Encoder; composed of 12 layers of encoder layers with the same structure, which are linearly superimposed to encode the input data into a dense representation, to extract the most important information in the input data, and convert it into a low-dimensional vector, while retaining some features of the original data as much as possible; Step S23: Decoder: After obtaining the hidden layer state extracted by the encoder Afterwards, the model decodes the feature using a decoder; the decoder consists of a linear stack of 12 identical decoding layers, with the first layer input being the hidden layer state and the vector matrix of the expected output after the code embedding layer ; Vector matrix First, the multi-head self-attention layer is masked, and then the hidden layer state , sent to the multi-head self-attention layer, and after the feedforward neural network, the output of the first layer is obtained ; Step S24: Model training: The model is trained using a cross entropy loss function.

6. According to claim 5, the method for automatically repairing software vulnerabilities based on thought chain, It is characterized in that The encoder layer is composed of a multi-head self-attention calculation module with relative position encoding and a feedforward neural network; the multi-head self-attention calculation module is composed of multiple self-attention heads, which are used to capture and fuse different interactive information in multiple different projection spaces through different self-attention heads; for a single attention head with relative position encoding, its receiving matrix As input, three linear transformation matrices are used Map the input vectors to three matrices belonging to three subspaces, respectively denoted as , and ,Each subspace reflects different hidden features.

7. The method for automatically repairing software vulnerabilities based on thought chain according to claim 5, It is characterized in that In the embedding layer, it is assumed that for the input vulnerability program , after word segmentation, we get a token stream sequence After the word embedding matrix Find the The semantic vector corresponding to the token , get the vulnerable program The embedding matrix ; The specific process can be expressed as: 。 8. According to claim 5, the method for automatically repairing software vulnerabilities based on thought chain, It is characterized in that The specific process of the position encoding layer is expressed as: The Clip function is used to clip the relative position between two elements that are too far apart. , then their relative positions are considered to be ;Relative position encoding matrix , is the learnable weight matrix, where , ; Step S24: Model training: Use the cross entropy loss function to train the model; the specific process of loss function calculation can be expressed as: in, The token stream generated for the model elements, is the probability distribution of tokens in an ideal situation, The token probability distribution generated for the model.

9. The method for automatically repairing software vulnerabilities based on thought chain according to any one of claims 1 to 4, It is characterized in that The step S3 comprises: Step S31: extract the original vulnerable program, remove all comments and use existing defect location tools to obtain row-level defect information, and use the input of step S1. This data is the input required for step S32; Step S32: Input the preprocessed vulnerable program into the model trained in step S2, and the model extracts and decodes the sequence to generate a probability distribution of tokens that constitute the repair program; in the process of generating the repair program, the beam search strategy is used to select the token with the highest probability. k The candidate repair programs are outputted, that is, the candidate repair programs predicted by the model are obtained.

10. A storage medium capable of being read by a computer or a processor, It is characterized in that The storage medium stores a computer program for executing any one of the methods in claims 1 to 9.

Images