Statement-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion

Abstract

The invention discloses a statement-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion, and the method comprises the steps: converting a high-level program language code into a bimodal fusion information assembly code in which a source code statement is aligned with an assembly code instruction sequence and a source code variable is aligned with an assembly code register; the method comprises the following steps: preprocessing a code through a bimodal mutual switching and bimodal purification method to obtain a purified source code slice and an assembly code slice; sending the purified bimodal slice code segment into a coding network for bimodal feature fusion and learning a long dependency relationship of statements; sending the high-level vector representation of the code slice fused by the bimodal information into a decoding network, and converting a semantic vector obtained by an encoder into a specific sequence; and sending an output result of the decoding network into a classifier to judge whether the to-be-detected code statement contains vulnerabilities or not. According to the method, a complex vulnerability structure containing multiple types of vulnerabilities or spanning multiple lines of statements can be detected, and the false alarm rate and the missing report rate are effectively reduced.

Description

Technical Field

[0001] The present invention relates to a software vulnerability fine-grained detection method, and in particular to a statement-level vulnerability detection method integrating cross-modal fine-grained features. Background Art

[0002] Software vulnerability detection refers to identifying potential security issues or design flaws in software systems by analyzing, scanning or testing them to prevent attackers from exploiting these vulnerabilities for malicious activities. Currently, the commonly used vulnerability detection methods based on deep learning include methods based on token sequences, methods based on graph neural networks, and methods based on bimodal code. However, these existing methods can generally only detect whether a function or slice code segment contains vulnerabilities, but cannot locate the specific statement location that triggers the vulnerability. For this reason, it is necessary to study statement-level fine-grained vulnerability detection methods to assist software maintenance personnel in understanding and quickly repairing vulnerabilities based on the located statement locations. Fine-grained vulnerability detection methods, that is, vulnerability location technology, aim to automatically locate specific code statements that trigger vulnerabilities. Existing vulnerability location methods based on deep learning can be divided into sequence-based methods, methods based on Transformer architecture, methods combining Transformer architecture with graphs, and methods based on fine-tuning of large-scale language models. However, these methods have the following shortcomings: First, they do not utilize multimodal code semantic information such as source code and assembly code. Second, some methods can only output the statement line that is most likely to be a vulnerability, that is, predict the line of code that is most likely to contain a vulnerability. For code segments containing multiple lines of vulnerability statements, it is impossible to locate the vulnerability structure that causes the vulnerability, making this method lack of practicality. Existing methods that use bimodal information of assembly code and source code only use alignment information at the code snippet level, and do not go into more detailed levels. They do not fully utilize fine-grained bimodal alignment information and perform statement-level vulnerability detection. At present, no method has been retrieved that combines fine-grained alignment information of source code and assembly code for fine-grained software vulnerability location. Summary of the invention

[0003] The present invention provides a statement-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion. The method utilizes bimodal slice purification technology to purify high-level programming language source code slices and assembly code slices, effectively shortening the length of source code slices and reducing the impact of vulnerability-irrelevant statements on the model. Then, a bimodal encoder-decoder network based on the Transformer architecture is used to capture vulnerability features within statements, understand vulnerability contexts, and learn the relationships between vulnerability statements distributed at different locations in the code, thereby combining the fine-grained alignment information of source code and assembly code to improve the effect of vulnerability location tasks.

[0004] The objective of the present invention is achieved through the following technical solutions:

[0005] A sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion includes the following steps:

[0006] Step 1: Convert the high-level language source code into assembly code with bimodal fusion information in which the source code statements are aligned with the assembly code instruction sequences and the registers are aligned with the variables;

[0007] Step 2: Use static analysis tools to parse the source code and generate its abstract syntax tree and program dependency graph;

[0008] Step 3: Generate slice code segments of unsanitized source code and assembly code according to the source code slice criteria;

[0009] Step 4: purify the source code slice obtained in step 3, that is, delete the source code statement lines without corresponding assembly instruction sequences in the source code slice, and obtain the purified source code slice code segment based on the source code slice criterion;

[0010] Step 5: Generate sliced ​​code segments of unpurified source code and assembly code based on the slicing criteria of the assembly code;

[0011] Step 6: Merge the unpurified assembly code slicing segments obtained in step 3 and step 5, and perform purification processing, that is, replace the operand information of the assembly code slicing segment containing the bimodal fusion information with the comment information in the fusion assembly information, and form the purified assembly code slices into an assembly code slice dataset;

[0012] Step 7: Merge the cleansed source code slices based on the source code slice criterion obtained in step 4 and the source code slices based on the assembly code slice criterion obtained in step 5, and mark the line containing the vulnerability statement in the bimodal slice code segment containing the vulnerability statement as 1, indicating that it is a "vulnerable statement", and mark other statements as 0, indicating that they are "non-vulnerable statements", to form a source code slice dataset;

[0013] Step 8: Input each bimodal sentence in the bimodal slicing code segment into the sentence encoding network composed of CodeBERT to obtain the vector representation of the fusion information of each sentence, and then input the vector representation of the obtained sentence fusion information into the program encoding network composed of Self-attention to learn the long-term dependency relationship of the sentence;

[0014] Step 9: The high-level vector representation of the code slices obtained by the bimodal information fusion in step 8 is sent to the decoding network composed of BiGRU to learn the context relationship of the sentence sequence and enhance the structural features between the bimodal sentence sequences;

[0015] Step 10: Send the hidden vector representation of the sentence output by the decoding network in step 9 to the classifier composed of FCN, and then get the prediction result vector after passing the SoftMax function, and finally get the classification result. Calculate the cross entropy loss based on the output result of the classifier and the actual label of the code sentence, and adjust the multimodal representation learning model parameters according to the loss back propagation until the loss value no longer decreases, and the training ends;

[0016] Step 11: Use the trained fine-grained vulnerability localization model consisting of a sentence encoding network, a program encoding network, a decoding network, and a classifier to detect vulnerabilities in the code to be tested.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] (1) The dual-modal slicing purification method is introduced for the first time. According to the characteristics of source code and assembly code, the dual-modal slicing purification method can effectively shorten the length of source code slices and reduce the impact of source code statements without corresponding assembly instructions, i.e., vulnerability-irrelevant statements, on the model. This allows the model to more accurately capture the vulnerability characteristics of the source code and replace information that is not conducive to the semantic understanding of the assembly code with information that is helpful for the semantic understanding of the assembly code, thereby making it easier for the model to understand the semantics of the assembly code and more accurately extract the vulnerability characteristics of the assembly code.

[0019] (2) This invention combines the fine-grained alignment information of source code and assembly code for the first time to perform vulnerability localization tasks. The bimodal slices are input into a statement-level vulnerability detection model that integrates cross-modal fine-grained features and is composed of a statement encoding network, a program encoding network, a decoding network, and a classifier. This model can not only capture the vulnerability features within the statement through the information between the two modalities, but also capture the relationship between the vulnerability statements distributed in different locations in the code by understanding the context of the vulnerability, further improving the effect of the vulnerability localization task.

[0020] (3) The present invention can detect complex vulnerability structures that include multiple types of vulnerabilities or span multiple lines of statements, and effectively reduce the false positive rate and the missed negative rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a flowchart of a sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion.

[0022] Figure 2 This is a sample diagram of a vulnerability code.

[0023] Figure 3 This is an example diagram of the bimodal slice generation and source code slice purification process based on the source code slice criterion.

[0024] Figure 4This is an example diagram of the assembly code slicing purification process.

[0025] Figure 5 It is a schematic diagram of the sentence encoding network input structure.

[0026] Figure 6 This is a structural diagram of a multimodal code fine-grained vulnerability detection model based on the encoder-decoder structure of the Transformer architecture. DETAILED DESCRIPTION

[0027] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0028] The present invention provides a statement-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion. First, a high-level program language code is converted into an assembly code with bimodal fusion information in which source code statements are aligned with assembly code instruction sequences and source code variables are aligned with assembly code registers through bimodal fine-grained alignment technology. The code is preprocessed through bimodal mutual cutting and bimodal purification methods to obtain purified source code slices and assembly code slices to form source code slice data sets and assembly code slice data sets. Secondly, the purified bimodal slice code segments are sent to the encoding network composed of a statement encoding network and a program encoding network to perform bimodal feature fusion and learn the long-term dependency relationship of the statements. Then, the obtained high-level vector representation of the code slice with bimodal information fusion is sent to the decoding network, and the semantic vector obtained by the encoder is converted into a specific sequence. Finally, the output result of the decoding network is sent to a classifier composed of a fully connected layer and SoftMax to determine whether the code statement to be tested contains a vulnerability. Figure 1 As shown, the specific steps of the method are as follows:

[0029] Step 1: Convert the high-level language source code into assembly code with bimodal fusion information in which the source code statements are aligned with the assembly code instruction sequence and the registers are aligned with the variables. The specific steps are as follows:

[0030] Step 1.1: Download the complete and compilable C / C++ source code from the online open source code platform, annotate the location information of the vulnerable statements based on the vulnerability database information, and form a complete source code dataset with vulnerability information;

[0031] Step 1.2: Use the GCC compiler with the compilation mode with parameters to generate assembly code with corresponding comments for registers / variables;

[0032] Step 1.3: Use the GCC compiler single-step debugging function to mark the source code statement line number corresponding to the assembly code;

[0033] Step 1.4: Merge the information generated in step 1.2 and step 1.3 to generate assembly code containing bimodal fusion information of alignment of statements in the source code with instruction sequences in the assembly code, and alignment of variables in the source code with registers in the assembly code.

[0034] Step 2: Use static analysis tools to parse the source code and generate its abstract syntax tree (AST) and program dependency graph (PDG).

[0035] Step 3: Generate the sliced ​​code segments of the unsanitized source code and assembly code according to the source code slice guidelines. The specific steps are as follows:

[0036] Step 3.1: Extract the vulnerability candidate key points of the source code on the AST of the source code, use these vulnerability candidate key points as the slicing criteria of the source code, traverse the program dependency graph obtained in step 2 forward and backward, obtain the bidirectional slicing statement set of the source code, and generate the unpurified source code slicing code segment based on the source code slicing criteria;

[0037] Step 3.2: Based on the bimodal fusion information obtained in step 1, i.e., the mapping relationship between source code and assembly code, find the complete assembly instruction sequence corresponding to each statement in the source code slicing code segment, and obtain the unpurified assembly code slicing code segment based on the source code slicing criterion.

[0038] Step 4: purify the source code slice obtained in step 3, that is, delete the source code statement lines without corresponding assembly instruction sequences in the source code slice, and obtain the purified source code slice code segment based on the source code slice criterion.

[0039] Step 5: Generate unsanitized source code and assembly code slice code segments based on the assembly code slice criteria. The specific steps are as follows:

[0040] Step 5.1: According to the assembly code containing the bimodal fusion information generated in step 1, extract the memory operation instructions related to the variables in the source code in the assembly code;

[0041] Step 5.2: According to the mapping relationship between the source code and the assembly code obtained in step 1, find the source code statement corresponding to the memory operation instruction obtained in step 5.1, and obtain the slice code segment of the source code based on the assembly code slicing criterion;

[0042] Step 5.3: According to the mapping relationship between the source code and the assembly code obtained in step 1, find the complete assembly instruction sequence corresponding to each statement in the source code slice code segment, and obtain the unpurified assembly code slice code segment based on the assembly code slicing criterion.

[0043] Step 6: Merge the unpurified assembly code slicing segments obtained in step 3 and step 5, and perform purification processing, that is, replace the operand information of the assembly code slicing segment containing the bimodal fusion information with the comment information in the fusion assembly information. The purified assembly code slices constitute the assembly code slice dataset. The specific steps are as follows:

[0044] Step 6.1: Merge the unsanitized assembly slice code segments obtained in step 3 and step 5;

[0045] Step 6.2: Perform purification operations on the assembly slice code segment: If all operands after the assembly operator have corresponding variable names in the comments of the assembly instruction on that line, then replace all operands in the assembly instruction with corresponding comments; if the number of comments is less than the number of operands, then retain the operands without corresponding variable comments and only replace the operands with variable comments; if the instruction has no variable comments, then do not replace it.

[0046] Step 7: Merge the purified source code slices based on the source code slicing criteria obtained in step 4 and the source code slices based on the assembly code slicing criteria obtained in step 5, and mark the line containing the vulnerability statement in the bimodal slicing code segment containing the vulnerability statement as 1, indicating that it is a "vulnerable statement", and mark other statements as 0, indicating that they are "non-vulnerable statements", to form a source code slicing dataset.

[0047] Step 8: Input each bimodal sentence in the bimodal code segment into the sentence encoding network composed of CodeBERT to obtain the vector representation of the fusion information of each sentence. The bimodal code segment contains the source code segment C = [c1, c2, …c i …c n ] and assembly code slice A = [a1, a2…a i …a n ], where c i Represents the i-th statement in the source code slice, a i is the assembly code corresponding to the source code, n is the number of code statements in the slice, and then the vector representation of the obtained statement fusion information is input into the program encoding network composed of self-attention to learn the long-term dependency relationship of the statement. The specific steps are as follows:

[0048] Step 8.1: Add assembly language instruction and register tokens (e.g., 'mox', 'rax') to the CodeBERT vocabulary.

[0049] Step 8.2: Use CodeBERT tokenizer to tokenize source code and assembly code.

[0050] Step 8.3: Place the token of the source code sentence obtained by word segmentation in step 8.2 between [CLS] and [SEP] of CodeBERT, and place the token of the flattened assembly instruction sequence between [SEP] and [EOS] of CodeBERT. The specific formula is as follows:

[0051] c i =[T1, T2…T s ]

[0052] a i =[TA 1.1 , T.A. 1.2 , T.A. 1.3 …TA N.M ]

[0053] FS i =Concat(CLS,c i ,SEP,a i , EOS)

[0054] Among them, c i is the ith source code statement, T is the token in the statement, s is the number of tokens in the statement, T s is the sth token in the sentence, a i For source code statement c i The corresponding assembly instruction sequence, since one source code corresponds to several assembly instructions, N is the Nth assembly instruction, TA is a i The assembly instruction token in the instruction, M is the number of tokens in the instruction, TA N.M The Mth token of the Nth assembly instruction, CLS indicates the start of a sequence, SEP indicates the delimiter of a sequence, EOS indicates the end of a sequence, and concat() indicates connecting multiple sequences together to form a single sequence;

[0055] Step 8.4: Input the token sequence obtained in step 8.3 into the CodeBERT layer and output the sentence vector representation after bimodal feature fusion. The specific formula is as follows:

[0056] HS i =CodeBERT(FS i )

[0057] Among them, HS i =[CE f , c′ i , C.E. SEP , a′ i , C.E. EOS ] is the hidden vector representation of the bimodal fusion information of the i-th sentence output by the CodeBERT sentence encoding network, CEf ,CE SEP ,CE EOS , c′ i , a′ i For CLS, SEP, EOS, c i 、a i Output after CoedBERT model, FS i is the bimodal code of the ith sentence arranged according to the CodeBERT features and the method of the present invention, and CodeBERT() is the CodeBERT model;

[0058] Step 8.5: The vector representation HS of the sentence fusion information of each sentence in the bimodal slice obtained in step 8.4 is i The input is fed into the program encoding network composed of self-attention, so that the model can learn the long dependency relationship of vulnerability features between sentences and obtain the feature-enhanced bimodal hidden vector representation. The specific formula is as follows:

[0059] Q=W Q ·P

[0060] K=W K ·P

[0061] V=W V ·P

[0062]

[0063] Where P = [HS1, HS2...HS i ...HS n ], is the vector representation of the bimodal slice code segment after the sentence encoding network, Q is the query sequence, (K, V) is the key-value pair sequence, W Q , W K , W V is the learnable weight, P' is the hidden vector representation after the program encoding network, Attention() is the self-attention method, K T is the transposed matrix of K, d K is the dimension of Q.

[0064] Step 9: The code slice high-level vector representation p'=[HS'1, HS'2, ...HS' obtained in step 8 by fusing the bimodal information i ...HS′ n ] is sent to the decoding network composed of BiGRU to learn the relationship between the sentence sequence and enhance the structural features between the bimodal sentence sequences. The specific formula is as follows:

[0065] VS i =BiGRU(HS' i)

[0066] Among them, HS' i is the hidden vector representation of the i-th sentence after the encoding network, VS i is the hidden vector representation of the sentence HS' i The high-level vector representation of the decoding network output is obtained through the decoding network, and BiGRU() is a BiGRU network.

[0067] Step 10: Send the hidden vector representation of the sentence output by the decoding network in step 9 to the classifier composed of FCN, and then get the prediction result vector after passing the SoftMax function, and finally get the classification result. Calculate the cross entropy loss based on the output result of the classifier and the actual label of the code sentence, and adjust the multimodal representation learning model parameters according to the loss back propagation until the loss value no longer decreases, and the training ends. The specific steps are as follows:

[0068] Step 10.1: Send the sentence obtained in step 9 to the fully connected layer through the high-level vector representation output by the decoding network, and then pass it through the SoftMax function to obtain the prediction result vector. The specific formula is as follows:

[0069] r i =FCN(VS i )

[0070] R=[r1,r2…,r i ..., r n ]

[0071] where r i is the classification result of the i-th source code statement, VS i It is the high-level vector representation of the sentence obtained in step 9 output by the decoding network, FCN() is a fully connected neural network, and R is the vector of the classification result of the entire slice code segment.

[0072] Step 10.2: Calculate the cross entropy loss based on the output of the classifier in step 10.1 and the actual label of the code statement. Specifically, use the label value marked in step 7. When the label value is 0, it indicates a normal line, then initialize it to the vector [1, 0]. When the label value is 1, it indicates a vulnerable line, then initialize it to the vector [0, 1]. Calculate the cross entropy loss with the prediction result vector given by the multimodal representation learning model. Finally, adjust the multimodal representation learning model parameters according to the loss backpropagation until the loss value no longer decreases, and the training ends.

[0073] Step 11: Use the trained fine-grained vulnerability localization model to detect vulnerabilities in the code to be tested.

[0074] Example:

[0075] Figure 2is a sample of vulnerable code. Figure 2 (b) Yes Figure 2 (a) shows the assembly code corresponding to the source code mapping line number, where the comment of each assembly instruction contains the information of the variable in the source code statement corresponding to the register in the assembly instruction. Figure 2 (b) shows only the assembly instruction sequence corresponding to the statements in the vulnerable function of the source code. This vulnerability example is a relatively complex vulnerability code caused by multiple vulnerable statements, including both integer overflow vulnerabilities and buffer overflow vulnerabilities. In this loop, Line 9 and Line 10 do not check whether i exceeds the boundary of buf. After the for loop in Line 9 ends, the value of i is n. If the value of n is larger than the actual allocated size of buf, a buffer overflow will occur, resulting in an out-of-bounds subscript when accessing buf[n] in Line 12. Modifying buf[i] in Line 12 to buf[i-1] can fix the vulnerability in this line. The purpose of the check in line 20 is to prevent integer overflow. In actual applications, the main way to fix bugs is to add if conditions, but the if statement added here is invalid. The reason is that when n is a negative number, the if condition in line 20 is still true, and the test function call is executed. Since the parameter of the test function is an unsigned integer, when a negative number is passed to the parameter, it will be interpreted as a large unsigned integer, which may result in insufficient space for dynamic memory allocation in line 6, causing allocation failure. Checking whether n is less than 0 in line 20 can fix this vulnerability.

[0076] From the perspective of assembly code, the assembly instruction callmalloc@PLT instruction corresponding to Line6 is used to request memory allocation. If the allocated size is calculated incorrectly, the allocated memory may be smaller than expected. According to the present invention using the fusion information method to add annotations after the instructions, moveax, DWORDPTR-20[rbp] is an instruction to write data i into the buffer. Without proper boundary checking, these instructions may cause buffer overflows. The assembly instructions jmp.L4 and cmp instructions corresponding to Line20 are used to control loops and conditional execution. If the loop conditions are not set properly, the buffer access may exceed its allocated boundaries. The vulnerability features shown in the source code and assembly code are different, but the existence of vulnerabilities can be known through analysis. With the help of these comments, it is easier to read and understand the semantics of the assembly code, so that the vulnerability features extracted from the assembly code can better complement the source code vulnerability features.

[0077] by Figure 2 Taking the code shown in the figure as an example, the generation process of bimodal code slices based on source code slices is demonstrated, as shown in Figure 3As shown. Figure 3 (a) The abstract syntax tree (AST) and program dependency graph (PDG) generated for the source code, and Figure 2 The parameter n in line 4 of the code shown is the program slice based on the source code generated by the slice criterion. Figure 3 (b) is the generated assembly code slice code segment based on the source code slice criterion, Figure 3 (c) is a source code slicing code segment based on the source code slicing criterion after the purification process by deleting the executable assembly instructions without corresponding ones. Taking the function parameter n in Line 4 as the vulnerability candidate key node, the relevant statements are obtained forward and backward respectively, and then the unpurified source code slice based on the source code is obtained. Then, according to the assembly code containing the bimodal fusion information, the complete assembly code instruction sequence corresponding to the statement is mapped to obtain the unpurified assembly code slice. Specifically, since line 4 and line 5 have no assembly instructions executed when the program is executed, line 4 and line 5 in the source code slice are deleted to obtain the source code slice, that is, the bimodal Line 6, 9, 10, 11, 12, 14 source code statements and corresponding assembly codes. The example after source code slicing purification is reduced from 8 lines to 6 lines, which helps to reduce the impact of vulnerability-irrelevant statements in the source code on the model.

[0078] by Figure 2 Take the LINE12 statement in as an example, the assembly code containing the bimodal fusion information and the purified assembly code slice code segment corresponding to this statement are as follows Figure 4 As shown. According to the assembly code slicing purification rules, all operands in the assembly instructions of lines 1 and 3 are replaced with corresponding comments. For the assembly instructions of lines 2 and 6, operands without corresponding variable annotations are retained, and only operands with variable annotations are replaced. The assembly instruction of line 7 has no variable annotations, so it is not replaced. In the first assembly instruction, the two operand information that was difficult to understand were replaced by the mov operation of variable i and temporary variable tmp102, and the original semantics that were difficult to understand were changed to semantics that were easy to understand, and the assembly code corresponding to a sentence of source code was reduced from 38 tokens to 26 tokens. This helps the model understand the semantics of the assembly code to capture vulnerability features, and reduces the impact of instruction information in the assembly code that is not related to the vulnerability on the model.

[0079] Each bimodal sentence in the bimodal code slice is fed into the sentence encoding network, such as Figure 5As shown in , the vector representation of the bimodal sentence fusion information is obtained. Then it is sent to the program encoding network to learn the long-term dependency of the sentence. The output of the encoding network is sent to the program decoding network to learn the relationship between the sentence sequence. Finally, the result of the vector representation of each sentence output by the decoding network is classified by the classifier, and finally a vector equal to the number of code segment statements is output. The value of the index position of the vector is the vulnerability prediction result of the corresponding position statement, as shown in Figure 6 shown.

Claims

1. A sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion, characterized by The method comprises the following steps: Step 1: Convert the high-level language source code into assembly code with bimodal fusion information in which the source code statements are aligned with the assembly code instruction sequences and the registers are aligned with the variables; Step 2: Use static analysis tools to parse the source code and generate its abstract syntax tree and program dependency graph; Step 3: Generate slice code segments of unsanitized source code and assembly code according to the source code slice criteria; Step 4: purify the source code slice obtained in step 3, that is, delete the source code statement lines without corresponding assembly instruction sequences in the source code slice, and obtain the purified source code slice code segment based on the source code slice criterion; Step 5: Generate sliced ​​code segments of unpurified source code and assembly code based on the slicing criteria of the assembly code; Step 6: Merge the unpurified assembly code slicing segments obtained in step 3 and step 5, and perform purification processing, that is, replace the operand information of the assembly code slicing segment containing the bimodal fusion information with the comment information in the fusion assembly information, and form the purified assembly code slices into an assembly code slice dataset; Step 7: Merge the cleaned source code slices based on the source code slice criterion obtained in step 4 and the source code slices based on the assembly code slice criterion obtained in step 5, and mark the line containing the vulnerability statement in the bimodal slice code segment containing the vulnerability statement as 1, indicating that it is a "vulnerable statement", and mark other statements as 0, indicating that they are "non-vulnerable statements", to form a source code slice dataset; Step 8: Input each bimodal sentence in the bimodal slicing code segment into the sentence encoding network composed of CodeBERT to obtain the vector representation of the fusion information of each sentence, and then input the vector representation of the obtained sentence fusion information into the program encoding network composed of Self-attention to learn the long-term dependency relationship of the sentence; Step 9: The high-level vector representation of the code slices obtained by the bimodal information fusion in step 8 is sent to the decoding network composed of BiGRU to learn the context relationship of the sentence sequence and enhance the structural features between the bimodal sentence sequences; Step 10: Send the hidden vector representation of the sentence output by the decoding network in step 9 to the classifier composed of FCN, and then get the prediction result vector after passing the SoftMax function, and finally get the classification result. Calculate the cross entropy loss based on the output result of the classifier and the actual label of the code sentence, and adjust the multimodal representation learning model parameters according to the loss back propagation until the loss value no longer decreases, and the training ends; Step 11: Use the trained fine-grained vulnerability localization model consisting of a sentence encoding network, a program encoding network, a decoding network, and a classifier to detect vulnerabilities in the code to be tested.

2. The sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion according to claim 1 is characterized in that The specific steps of step 1 are as follows: Step 1.1: Download the complete and compilable C / C++ source code from the online open source code platform, annotate the location information of the vulnerable statements based on the vulnerability database information, and form a complete source code dataset with vulnerability information; Step 1.2: Use the GCC compiler with the compilation mode with parameters to generate assembly code with corresponding comments for registers / variables; Step 1.3: Use the GCC compiler single-step debugging function to mark the source code statement line number corresponding to the assembly code; Step 1.4: Merge the information generated in step 1.2 and step 1.3 to generate assembly code containing bimodal fusion information of alignment of statements in the source code with instruction sequences in the assembly code, and alignment of variables in the source code with registers in the assembly code.

3. The sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion according to claim 1 is characterized in that The specific steps of step 3 are as follows: Step 3.1: Extract the vulnerability candidate key points of the source code on the AST of the source code, use these vulnerability candidate key points as the slicing criteria of the source code, traverse the program dependency graph obtained in step 2 forward and backward, obtain the bidirectional slicing statement set of the source code, and generate the unpurified source code slicing code segment based on the source code slicing criteria; Step 3.2: Based on the bimodal fusion information obtained in step 1, i.e., the mapping relationship between source code and assembly code, find the complete assembly instruction sequence corresponding to each statement in the source code slicing code segment, and obtain the unpurified assembly code slicing code segment based on the source code slicing criterion.

4. The sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion according to claim 1 is characterized in that The specific steps of step 5 are as follows: Step 5.1: According to the assembly code containing the bimodal fusion information generated in step 1, extract the memory operation instructions related to the variables in the source code in the assembly code; Step 5.2: According to the mapping relationship between the source code and the assembly code obtained in step 1, find the source code statement corresponding to the memory operation instruction obtained in step 5.1, and obtain the slice code segment of the source code based on the assembly code slicing criterion; Step 5.3: According to the mapping relationship between the source code and the assembly code obtained in step 1, find the complete assembly instruction sequence corresponding to each statement in the source code slice code segment, and obtain the unpurified assembly code slice code segment based on the assembly code slicing criterion.

5. The sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion according to claim 1 is characterized in that The specific steps of step 6 are as follows: Step 6.1: Merge the unsanitized assembly slice code segments obtained in step 3 and step 5; Step 6.2: Perform purification operations on the assembly slice code segment: If all operands after the assembly operator have corresponding variable names in the comments of the assembly instruction on that line, then replace all operands in the assembly instruction with corresponding comments; if the number of comments is less than the number of operands, then retain the operands without corresponding variable comments and only replace the operands with variable comments; if the instruction has no variable comments, then do not replace it.

6. The sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion according to claim 1 is characterized in that The specific steps of step 8 are as follows: Step 8.1: Add assembly language instruction and register tokens to the CodeBERT vocabulary; Step 8.2: Use CodeBERT tokenizer to tokenize source code and assembly code. Step 8.3: Place the token of the source code sentence obtained by word segmentation in step 8.2 between [CLS] and [SEP] of CodeBERT, and place the token of the flattened assembly instruction sequence between [SEP] and [EOS] of CodeBERT. The specific formula is as follows: c i =[T1,T2…T s ] a i =[TA 1.1 ,TA 1.2 ,TA 1.3 …TA N.M ] FS i =Concat(CLS,c i ,SEP,a i ,EOS) Among them, c i is the ith source code statement, T is the token in the statement, s is the number of tokens in the statement, T s is the sth token in the sentence, a i For source code statement c i The corresponding assembly instruction sequence, since one source code corresponds to several assembly instructions, N is the Nth assembly instruction, TA is a i The assembly instruction token in the instruction, M is the number of tokens in the instruction, TA N.M The Mth token of the Nth assembly instruction, CLS indicates the start of a sequence, SEP indicates the delimiter of a sequence, EOS indicates the end of a sequence, and concat() indicates connecting multiple sequences together to form a single sequence; Step 8.4: Input the token sequence obtained in step 8.3 into the CodeBERT layer and output the sentence vector representation after bimodal feature fusion. The specific formula is as follows: HS i =CodeBERT(FS i ) Among them, HS i =[CE f , c′ i , C.E. SEP , a′ i , C.E. EOS ] is the hidden vector representation of the bimodal fusion information of the i-th sentence output by the CodeBERT sentence encoding network, CE f ,CE SEP ,CE EOS , c′ i , a′ i For CLS, SEP, EOS, c i 、a i Output after CoedBERT model, FS i is the bimodal code of the ith sentence, CodeBERT() is the CodeBERT model; Step 8.5: The vector representation HS of the sentence fusion information of each sentence in the bimodal slice obtained in step 8.4 is i The input is fed into the program encoding network composed of self-attention, so that the model can learn the long dependency relationship of vulnerability features between sentences and obtain the feature-enhanced bimodal hidden vector representation. The specific formula is as follows: Q=W Q ·P K=W K ·P V=W V ·P Where P = [HS1, HS2...HS i ...HS n ], is the vector representation of the bimodal slice code segment after the sentence encoding network, Q is the query sequence, (K, V) is the key-value pair sequence, W Q , W K , W V is the learnable weight, P' is the hidden vector representation after the program encoding network, Attention() is the self-attention method, K T is the transposed matrix of K, d K is the dimension of Q.

7. The sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion according to claim 1 is characterized in that In step 9, the code slice high-level vector representation p'=[HS'1, HS'2, ...HS' obtained in step 8 is fused with the bimodal information i ...HS′ n ] is sent to the decoding network composed of BiGRU to learn the relationship between the sentence sequence and enhance the structural features between the bimodal sentence sequences. The specific formula is as follows: VS i =BiGRU(HS’ i ) Among them, HS' i is the hidden vector representation of the i-th sentence after the encoding network, VS i is the hidden vector representation of the sentence HS' i The high-level vector representation of the decoding network output is obtained through the decoding network, and BiGRU() is a BiGRU network.

8. The sentence-level vulnerability detection method based on cross-modal fine-grained feature alignment and fusion according to claim 1 is characterized in that The specific steps of step 10 are as follows: Step 10.1: Send the sentence obtained in step 9 to the fully connected layer through the high-level vector representation output by the decoding network, and then pass it through the SoftMax function to obtain the prediction result vector. The specific formula is as follows: r i =FCN(VS i ) R＝[r1，r2…，r i ...，r n ] where r i is the classification result of the i-th source code statement, VS i is the high-level vector representation of the sentence obtained in step 9 output by the decoding network, FCN() is a fully connected neural network, and R is the vector of the classification result of the entire slice code segment; Step 10.2: Calculate the cross entropy loss based on the output of the classifier in step 10.1 and the actual label of the code statement. Specifically, use the label value marked in step 7. When the label value is 0, it indicates a normal line, then initialize it to the vector [1, 0]. When the label value is 1, it indicates a vulnerable line, then initialize it to the vector [0, 1]. Calculate the cross entropy loss with the prediction result vector given by the multimodal representation learning model. Finally, adjust the multimodal representation learning model parameters according to the loss backpropagation until the loss value no longer decreases, and the training ends.

Images