Binary program vulnerability detection method based on fine-tuning pre-trained model

Abstract

The application discloses a binary program vulnerability detection method based on fine-tuning of a pre-trained model, which divides instructions in function-level assembly code into branch and control class instructions, data class instructions and other class instructions according to the role of assembly instructions in the semantic reservation of control flow and data flow, and under the constraint of a preset maximum Token length threshold, selects the above three types of instructions in turn according to a ladder priority, and then reorders and flattens them according to original line number indexes to obtain a simplified one-dimensional assembly Token sequence. The method further adopts a partial freezing fine-tuning strategy to adapt the pre-trained encoding model to a task, and combines a double-layer bidirectional gated recurrent unit network and a multi-head attention aggregation mechanism with a mask to model the semantics and aggregate function-level features of function-level assembly code samples, and finally outputs a vulnerability classification result by a classifier. The application can effectively improve the performance of function-level binary program vulnerability detection.

Description

Technical Field

[0001] This invention belongs to the field of software vulnerability detection and relates to a binary program vulnerability detection method. Specifically, it relates to a function-level binary program vulnerability detection method based on a fine-tuning pre-trained model and an assembly code simplification strategy that preserves vulnerability semantics. Background Technology

[0002] As software systems become increasingly large and complex, software security issues are becoming more prominent. Influenced by factors such as intellectual property protection, commercial distribution models, and copyright restrictions, software products are typically distributed in binary form, such as executable files, dynamic link libraries, or firmware images. Security analysts often find it difficult to obtain the complete source code. Therefore, vulnerability detection targeting binary programs has significant practical implications.

[0003] Existing vulnerability detection methods for binary programs mostly use disassembled assembly code as input. Traditional methods rely primarily on expert rules or pattern matching, which are costly in terms of feature design and maintenance, and have limited adaptability to new and variant vulnerabilities. With the development of deep learning technology, researchers have begun to use neural networks to automatically learn vulnerability-related features from assembly code, mainly including graph-based methods, sequence-based methods, and fusion representation methods.

[0004] However, applying the above methods to function-level binary program vulnerability detection still has the following shortcomings:

[0005] First, function-level assembly code sequences are typically long and contain numerous instructions that are weakly related to vulnerability triggering, such as register storage, local data transfer, alignment, and padding. Directly inputting complete function-level assembly code into a neural network model would result in high computational and storage overhead. Furthermore, controlling the input length by truncating from the beginning or by random truncation could easily disrupt the critical control flow and data dependencies between the input and the vulnerability trigger point, thereby weakening the model's ability to identify vulnerability patterns.

[0006] Second, existing research has introduced pre-trained coding models into binary program vulnerability detection tasks. However, most of these pre-trained models are trained on high-level programming language corpora, which limits their ability to represent assembly code-specific vocabulary and their contextual relationships, such as register names, jump mnemonics, and memory addressing expressions. Directly using pre-trained coding models to encode assembly code makes it difficult to fully utilize their semantic representation capabilities; furthermore, fine-tuning all parameters of the pre-trained coding model may increase training costs and the risk of overfitting when the sample size is limited.

[0007] Currently, no method has been found that combines assembly code simplification strategies that preserve vulnerability semantics with pre-trained coding models that undergo partial freeze fine-tuning for function-level binary program vulnerability detection. Summary of the Invention

[0008] To address the problems of existing technologies, this invention provides a binary program vulnerability detection method based on fine-tuning a pre-trained model. This method classifies function-level assembly code according to the role of assembly instructions in preserving control flow and data flow semantics, and simplifies it according to a tiered priority system. This prioritizes the preservation of critical control flow and data flow semantics relevant to vulnerabilities while meeting model input length constraints. Simultaneously, a partial freeze fine-tuning strategy is employed to adapt the pre-trained encoding model to the task, and a bidirectional gated loop unit and a masked attention aggregation mechanism are combined to model function-level semantics, thereby improving the effectiveness of function-level binary program vulnerability detection.

[0009] The objective of this invention is achieved through the following technical solution:

[0010] A binary program vulnerability detection method based on a fine-tuned pre-trained model categorizes assembly instructions into branch and control instructions, data instructions, and other instructions based on their role in preserving the semantics of control and data flows. Under a preset maximum token length threshold, these three types of instructions are selected sequentially according to a tiered priority. After being reordered by their original line number indices and flattened, a simplified one-dimensional assembly token sequence is obtained. A partial freeze fine-tuning strategy is employed to adapt the pre-trained encoding model to the task. Furthermore, a two-layer bidirectional gated recurrent unit network and a masked multi-head attention aggregation mechanism are combined to perform semantic modeling and function-level feature aggregation on the function-level assembly code samples. Finally, a classifier outputs the vulnerability classification results. The method includes the following steps:

[0011] Step 1: Obtain a sample of the function-level assembly code to be processed;

[0012] Step 2: Normalize the function-level assembly code sample, including redundant instruction removal and tokenization, function name generalization mapping, and variable and operand normalization, to obtain the normalized function-level assembly code sample;

[0013] Step 3: Based on the role of assembly instructions in the semantic preservation of control flow and data flow, the assembly instructions in the normalized function-level assembly code sample are divided into branch and control instructions, data instructions, and other instructions; under the preset maximum token length threshold constraint, the three types of instructions are selected in order of priority; the selected instructions are reordered and flattened according to their line number index in the original assembly code to obtain the simplified one-dimensional assembly token sequence;

[0014] Step 4: Encode the simplified one-dimensional assembly token sequence using a pre-trained encoding model with partial freeze fine-tuning: Freeze the parameters of the bottom few layers of the pre-trained encoding model, and only allow supervised fine-tuning of the parameters of the top few layers and subsequent modules; output the semantic feature matrix corresponding to the function-level assembly token sequence;

[0015] Step 5: Input the semantic feature matrix into the two-layer bidirectional gated recurrent unit network in sequence, and use effective token masks to mask the filling positions to obtain a sequence-level feature matrix that integrates contextual dependencies;

[0016] Step 6: Based on the multi-head attention aggregation mechanism with mask, attention is calculated using the learnable global query vector and the key matrix and value matrix generated by the sequence-level feature matrix. The influence of the padding position is shielded by the mask matrix, and the sequence features are weighted and aggregated to obtain the function-level global feature vector.

[0017] Step 7: Feed the function-level global feature vector into a classifier composed of fully connected layers and output the original classification score;

[0018] Step 8: During the training phase, the loss between the original classification score and the sample label is calculated using the binary cross-entropy loss function with positive sample weight adjustment. The unfrozen high-level parameters, bidirectional gated recurrent unit network parameters, and classifier parameters of the pre-trained coding model are updated and optimized through backpropagation until training terminates.

[0019] Step 9: During the detection phase, the original classification score is mapped to a vulnerability probability using the Sigmoid function. The vulnerability probability is then compared with a preset threshold to determine whether the function-level assembly code sample corresponds to a vulnerable function.

[0020] Compared with the prior art, the present invention has the following advantages:

[0021] 1. This invention proposes an assembly code simplification strategy for vulnerability semantic preservation. The strategy categorizes assembly instructions based on their role in preserving control flow and data flow semantics, and selects instructions according to a tiered priority hierarchy of branch and control instructions, data instructions, and other instructions, under a preset maximum token length threshold constraint. The instructions are then reordered by their original line number index and flattened before output. This strategy can control input length and reduce model computational overhead while prioritizing the preservation of control flow structures and data flow dependencies closely related to vulnerability triggering. Compared to common length control methods such as header truncation and random truncation, it is more effective in preserving vulnerability-related semantics.

[0022] 2. This invention employs a partial freeze-tuning strategy to adapt a pre-trained encoding model to a specific task. This strategy freezes the parameters of several bottom layers of the pre-trained encoding model, only involving supervised fine-tuning of the parameters of the top layers and subsequent modules. While preserving the general code representation capabilities of the pre-trained encoding model, this strategy concentrates parameter adjustments for assembly code vulnerability detection tasks on higher-level representation spaces. This reduces training costs and overfitting risks with limited sample size and enhances the model's adaptability to assembly code semantic patterns.

[0023] 3. This invention introduces a two-layer bidirectional gated recurrent unit network and a masked multi-head attention aggregation mechanism in the sequence-level feature modeling and function-level feature aggregation stages. The bidirectional gated recurrent unit network can further model the instruction sequence relationship and contextual dependencies on the pre-trained encoding output; the masked multi-head attention aggregation mechanism can differentiate the weighting of sequence features within the effective token range, thereby strengthening the focus on key instructions related to vulnerabilities and further improving the effectiveness of function-level vulnerability identification. Attached Figure Description

[0024] Figure 1 This is a schematic diagram of the overall process of a binary program vulnerability detection method based on fine-tuning a pre-trained model.

[0025] Figure 2 A diagram illustrating the process of simplifying assembly code to preserve vulnerability semantics.

[0026] Figure 3 This is a schematic diagram of the assembly code encoding process for a pre-trained encoding model based on partial freeze fine-tuning.

[0027] Figure 4 This is a schematic diagram of the function-level feature extraction and vulnerability identification process based on bidirectional gated loop units and masked attention aggregation. Detailed Implementation

[0028] The technical solution of the present invention will be further described below with reference to the accompanying drawings, but it is not limited thereto. Any modifications or equivalent substitutions to the technical solution of the present invention that do not depart from the spirit and scope of the technical solution of the present invention should be covered within the protection scope of the present invention.

[0029] This invention provides a binary program vulnerability detection method based on a fine-tuned pre-trained model. During the training phase, source code labeled with vulnerabilities is compiled to obtain corresponding function-level assembly code samples and their labels. During the detection phase, the binary program under test is disassembled to obtain function-level assembly code samples. These function-level assembly code samples are then normalized and, based on instruction type and within a preset maximum token length threshold, are simplified according to a tiered priority, resulting in a length-controlled assembly token sequence that prioritizes the retention of vulnerability-related semantics. This assembly token sequence is then input into a pre-trained encoding model employing a partial freeze fine-tuning strategy to obtain a semantic feature matrix containing contextual semantic information. This feature matrix is ​​then sequentially passed through a two-layer bidirectional gated recurrent unit network and a masked multi-head attention aggregation mechanism to obtain a function-level global feature vector. Finally, the function-level global feature vector is fed into a classifier, which outputs the vulnerability classification result. This method, while controlling the input length, prioritizes the retention of key control flow and data flow semantics related to the vulnerability, and enables the pre-trained encoding model to better adapt to assembly code vulnerability detection tasks while retaining general code representation capabilities, effectively improving the performance of function-level binary program vulnerability detection. Figure 1 As shown, the specific steps include the following:

[0030] Step 1: Obtain the function-level assembly code sample to be processed. The sample is obtained by compiling the source code marked with vulnerabilities and extracting the function-level assembly code, or by disassembling the binary program to be tested and extracting the function-level assembly code. The specific steps are as follows:

[0031] Step 1.1: During the training phase, for the source code dataset with vulnerability annotations, the source code is compiled using a compiler, and auxiliary options related to source code line mapping are enabled in the compilation command, including but not limited to compiling only to the assembly stage, generating debugging information, adding assembly comment information, and disabling most compilation optimizations; thereby generating assembly code files that correspond to the source code lines.

[0032] Step 1.2: Based on the correspondence between the source code and the assembly code, extract function-level assembly code samples from the assembly code file at the function granularity, and pass labels to the function-level assembly code samples according to the vulnerability annotation information at the source code level: if there are statements marked as vulnerable in the source code function, then mark the corresponding function-level assembly code sample as a vulnerable function; otherwise, mark it as a non-vulnerable function.

[0033] Step 1.3: In the detection phase, the binary program to be tested is disassembled using a disassembler, and function-level assembly code samples are extracted at the function level as the processing objects for subsequent steps.

[0034] Step 2: Normalize the function-level assembly code sample. The specific steps are as follows:

[0035] Step 2.1: Redundant Instruction Removal and Tokenization. The assembly code is cleaned using pattern matching to remove compiler-generated metadata instructions, comments, linker-specific symbols, and semantically meaningless whitespace, while retaining assembly instructions, operands, and control flow labels.

[0036] Step 2.2: Function Name Generalization Mapping. Maintain a list of function names associated with system reserved keywords; for all function call targets that are not reserved keywords, remove their link-related suffixes, and map different specific function names to function generalization symbols in the order of appearance, thereby reducing the interference of project-specific naming on subsequent model learning.

[0037] Step 2.3: Variable and Operand Normalization. Identify operands in the assembly code that are neither instruction mnemonics, register names, nor immediate values, and map them uniformly to variable generalization symbols in the order of their appearance. This allows the subsequent model to focus on the assembly instruction structure information rather than specific variable names.

[0038] Step 3: Perform vulnerability-semantic preservation-oriented simplification on the normalized function-level assembly code sample. The specific steps are as follows:

[0039] Step 3.1: Based on the role of assembly instructions in preserving the semantics of control flow and data flow, the assembly instructions in the normalized function-level assembly code sample are divided into the following three categories:

[0040] (1) Branch and control instructions: Instructions used to affect the program execution path or function call relationship, including but not limited to conditional jump instructions, unconditional jump instructions, call instructions, return instructions, and comparison instructions and test instructions closely related to branch determination;

[0041] (2) Data instructions: Instructions used for register assignment, memory data transfer and arithmetic and logical operations, including but not limited to data transfer instructions, addition instructions, subtraction instructions, multiplication instructions and bitwise operation instructions;

[0042] (3) Other types of instructions: Instructions other than branch and control instructions and data instructions, including but not limited to alignment and padding instructions.

[0043] Step 3.2: Establish a line number-to-token index mapping table. The mapping table is used to record the line number of each assembly instruction in the function-level assembly code sample and its expanded token index range; and set a maximum token length threshold.

[0044] Step 3.3: Select instructions sequentially according to a tiered priority. First, within the remaining token length capacity, select branch and control instructions in order of appearance; second, within the remaining capacity, select data instructions in order of appearance; finally, within the remaining capacity, select other types of instructions in order of appearance. In the above three stages, for each candidate instruction, if its token length is greater than the current remaining token length capacity, skip the instruction and try subsequent candidate instructions.

[0045] Step 3.4: Reorder all selected instructions according to their line index in the original assembly code, and flatten them into a one-dimensional assembly token sequence in token order, which will serve as the input for subsequent encoding modules.

[0046] like Figure 2 As shown, by simplifying the function-level assembly code sample in the above manner, the branch control structure and necessary data transfer information related to vulnerability triggering can be retained first without significantly increasing the input length, while a large number of redundant alignment and padding instructions can be eliminated.

[0047] Step 4: Use a pre-trained encoding model with partial freeze fine-tuning to refine the one-dimensional assembly token sequence of a certain function sample. Encode, such as Figure 3 As shown, the specific steps are as follows:

[0048] Step 4.1: Select a pre-trained encoding model for the code understanding task as the base encoder. The base encoder adopts a multi-layer Transformer encoder structure and is configured with a corresponding tokenizer. In one optional implementation, the pre-trained encoding model is the CodeBERT model, but the present invention is not limited to this specific model.

[0049] Step 4.2: Use the token segmenter corresponding to the basic encoder to perform word segmentation and index mapping on the one-dimensional assembled token sequence output in Step 3.4 to obtain a token index sequence that meets the input requirements of the pre-trained encoding model.

[0050] Step 4.3: Apply a partial freeze fine-tuning strategy to the pre-trained coding model: Set the encoder parameters of the bottom preset number of layers of the pre-trained coding model to a frozen state, and only participate in the subsequent supervised fine-tuning of the remaining high-level encoder parameters; the specific number of frozen layers can be set according to the number of layers of the pre-trained coding model and the size of the training samples.

[0051] Step 4.4: Input the Token index sequence output in Step 4.2 into the pre-trained encoding model configured in Step 4.3 to obtain the context-dependent semantic feature matrix of the function sample. ,in To preset the maximum token length threshold, each column vector This indicates the first [item] after fusing global context information. The deep semantic features of each assembled token.

[0052] Step 5: Input the semantic feature matrix into a two-layer bidirectional gated recurrent unit network, and use an effective token mask to mask the padding positions to obtain a sequence-level feature matrix that incorporates contextual dependencies. The specific steps are as follows:

[0053] Step 5.1: Construct a two-layer bidirectional gated recurrent unit network connected in sequence, and set the hidden layer dimension for each gated recurrent unit in each direction.

[0054] Step 5.2: Obtain the semantic feature matrix of the function sample. The first and second layers of the bidirectional gated recurrent unit network are input sequentially, and padding positions are dynamically ignored using an effective token mask; the output sequence-level feature matrix is ​​denoted as... Its dimensions are obtained by bidirectional splicing.

[0055] Step 6: Based on the masked multi-head attention aggregation mechanism, the sequence-level feature matrix is ​​weighted and converged to obtain the function-level global feature vector. The specific steps are as follows:

[0056] Step 6.1: Initialize a learnable global query vector Q, and transform it from the sequence-level feature matrix through a linear mapping. Generate the key matrix K and the value matrix V.

[0057] Step 6.2: Construct a mask matrix M, where the position corresponding to the real token is set to 0 and the position of the padding is set to negative infinity, so as to shield the influence of the padding position on the attention calculation.

[0058] Step 6.3: Perform a dot product operation between the query vector Q and the key matrix K, divide by a preset scaling factor, and then superimpose the result onto a mask matrix M. Next, apply the mask matrix M to the softmax function to obtain the attention weight α. Multiply the attention weight α by the value matrix V and perform layer normalization to obtain the function-level global feature vector. .

[0059] Step 7: Feed the function-level global feature vector into a classifier composed of fully connected layers and output the original classification score. The original classification score is used to characterize the credibility of the corresponding function-level assembly code sample belonging to the vulnerable function.

[0060] Step 8: During the training phase, optimize and update the parameters that are not frozen. The specific steps are as follows:

[0061] Step 8.1: Use the real labels of the function-level assembly code samples as the supervision signal, where the vulnerable function corresponds to label 1 and the non-vulnerable function corresponds to label 0; and set the positive sample weight adjustment factor according to the ratio of vulnerable samples to non-vulnerable samples in the training data to alleviate the class imbalance problem.

[0062] Step 8.2: Calculate the loss based on the original classification score and the true label output in Step 7 using the binary cross-entropy loss function with integrated Sigmoid mapping.

[0063] Step 8.3: Using the loss as the optimization objective, calculate the gradient using the backpropagation algorithm, and combine it with the gradient pruning strategy to iteratively update the unfrozen high-level parameters, bidirectional gated recurrent unit network parameters, and classifier parameters of the pre-trained encoded model until the loss value converges or reaches the preset number of training rounds.

[0064] Step 9: In the detection phase, the trained model is used to identify vulnerabilities in the assembly code samples at the function level to be tested, such as... Figure 4 As shown, the specific steps are as follows:

[0065] Step 9.1: Input the simplified assembly token sequence obtained after processing the binary program under test in steps 1.3, 2 and 3 into the pre-trained encoding model, bidirectional gated recurrent unit network, attention aggregation module and classifier to obtain the original classification score.

[0066] Step 9.2: Map the original classification score to a vulnerability probability using the Sigmoid function, and compare the vulnerability probability with a preset threshold; if the vulnerability probability is not less than the preset threshold, determine that the function-level assembly code sample is a vulnerable function; otherwise, determine that it is a non-vulnerable function.

[0067] Example:

[0068] In this embodiment, the pre-trained encoding model used is a twelve-layer Transformer encoder structure, in which the parameters of the bottom three encoder layers are frozen, and only the parameters of the top nine encoder layers and subsequent modules are fine-tuned; the preset maximum token length threshold is set to 400; the unidirectional hidden layer dimension of the bidirectional gated recurrent unit network is set to 512; the dropout probability is set to 0.2; the optimizer is AdamW, and the initial learning rate is set to 2×10. -5 The batch size is set to 32.

[0069] It should be noted that the specific parameters such as the number of model layers, token length threshold, hidden layer dimension, dropout probability, learning rate, and optimizer type in the above embodiments are all illustrative and not intended to limit the scope of protection of this invention. Without departing from the spirit of this invention, those skilled in the art can adjust the parameters or make equivalent substitutions for the pre-trained encoding model based on the specific task scale, target program characteristics, and computing resource conditions.

Claims

1. A binary program vulnerability detection method based on a fine-tuned pre-trained model, characterized in that... The method includes the following steps: Step 1: Obtain a sample of the function-level assembly code to be processed; Step 2: Normalize the function-level assembly code sample, including redundant instruction removal and tokenization, function name generalization mapping, and variable and operand normalization, to obtain the normalized function-level assembly code sample; Step 3: Based on the role of assembly instructions in the semantic preservation of control flow and data flow, the assembly instructions in the normalized function-level assembly code sample are divided into branch and control instructions, data instructions, and other types of instructions; Under the constraint of a preset maximum token length threshold, three types of instructions are selected in order of priority; the selected instructions are reordered and flattened according to their line number index in the original assembly code to obtain a simplified one-dimensional assembly token sequence. Step 4: Encode the simplified one-dimensional assembly token sequence using a pre-trained encoding model with partial freeze fine-tuning: Freeze the parameters of the bottom few layers of the pre-trained encoding model, and only allow supervised fine-tuning of the parameters of the top few layers and subsequent modules; output the semantic feature matrix corresponding to the function-level assembly token sequence; Step 5: Input the semantic feature matrix into the two-layer bidirectional gated recurrent unit network in sequence, and use effective token masks to mask the filling positions to obtain a sequence-level feature matrix that integrates contextual dependencies; Step 6: Based on the multi-head attention aggregation mechanism with mask, attention is calculated using the learnable global query vector and the key matrix and value matrix generated by the sequence-level feature matrix. The influence of the padding position is shielded by the mask matrix, and the sequence features are weighted and aggregated to obtain the function-level global feature vector. Step 7: Feed the function-level global feature vector into a classifier composed of fully connected layers and output the original classification score; Step 8: During the training phase, the loss between the original classification score and the sample label is calculated using the binary cross-entropy loss function with positive sample weight adjustment. The unfrozen high-level parameters, bidirectional gated recurrent unit network parameters, and classifier parameters of the pre-trained coding model are updated and optimized through backpropagation until training terminates. Step 9: During the detection phase, the original classification score is mapped to a vulnerability probability using the Sigmoid function. The vulnerability probability is then compared with a preset threshold to determine whether the function-level assembly code sample corresponds to a vulnerable function.

2. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 1, characterized in that... The specific steps of step 1 are as follows: Step 1.1: During the training phase, for the source code dataset with vulnerability annotations, the source code is compiled using a compiler, and auxiliary options related to source code line mapping are enabled in the compilation command to generate assembly code files that correspond to the source code lines. Step 1.2: Based on the correspondence between the source code and the assembly code, extract function-level assembly code samples from the assembly code file at the function granularity, and pass labels to the function-level assembly code samples according to the vulnerability annotation information at the source code level: if there are statements marked as vulnerable in the source code function, then mark the corresponding function-level assembly code sample as a vulnerable function; Otherwise, mark it as a non-vulnerable function; Step 1.3: In the detection phase, the binary program to be tested is disassembled using a disassembler, and function-level assembly code samples are extracted at the function level as the processing objects for subsequent steps.

3. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 2, characterized in that... The specific steps of step 2 are as follows: Step 2.1: Redundant instruction removal and tokenization: The assembly code is cleaned using pattern matching methods to remove metadata instructions, comments, linker-specific symbols, and semantically meaningless whitespace generated by the compiler, while retaining assembly instructions, operands, and control flow labels; Step 2.2: Function Name Generalization Mapping: Maintain a list of function names related to system reserved keywords; for all function call targets that are not reserved keywords, remove their link-related suffixes, and map different specific function names to function generalization symbols in the order of appearance, thereby reducing the interference of project-specific naming on subsequent model learning; Step 2.3: Variable and operand normalization: Identify operands in the assembly code that are neither instruction mnemonics, register names, nor immediate values, and map them uniformly as variable generalization symbols according to their order of appearance, so that the subsequent model focuses on the assembly instruction structure information rather than specific variable names.

4. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 3, characterized in that... The specific steps of step 3 are as follows: Step 3.1: Based on the role of assembly instructions in preserving the semantics of control flow and data flow, the assembly instructions in the normalized function-level assembly code sample are divided into the following three categories: (1) Branch and control instructions: Instructions used to affect the program execution path or function call relationship, including but not limited to conditional jump instructions, unconditional jump instructions, call instructions, return instructions, and comparison instructions and test instructions closely related to branch determination; (2) Data instructions: Instructions used for register assignment, memory data transfer and arithmetic and logical operations, including but not limited to data transfer instructions, addition instructions, subtraction instructions, multiplication instructions and bitwise operation instructions; (3) Other types of instructions: Instructions other than branch and control instructions and data instructions, including but not limited to alignment and padding instructions; Step 3.2: Establish a line number-to-token index mapping table to record the line number of each assembly instruction in the function-level assembly code sample and its expanded token index range; and set a maximum token length threshold; Step 3.3: Select instructions in a tiered priority manner: First, within the remaining token length capacity, select the branch and control type instructions in the order of appearance; second, within the remaining capacity, select the data type instructions in the order of appearance; finally, within the remaining capacity, select the other types of instructions in the order of appearance. In the above three stages, for each candidate instruction, if its token length is greater than the current remaining token length capacity, the instruction is skipped and subsequent candidate instructions are tried. Step 3.4: Reorder all selected instructions according to their line index in the original assembly code, and flatten them into a one-dimensional assembly token sequence in token order, which will serve as the input for subsequent encoding modules.

5. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 4, characterized in that... The specific steps of step 4 are as follows: Step 4.1: Select a pre-trained encoding model for the code comprehension task as the base encoder; Step 4.2: Use the token segmenter corresponding to the basic encoder to perform word segmentation and index mapping on the one-dimensional assembled token sequence output in Step 3.4 to obtain a token index sequence that meets the input requirements of the pre-trained encoding model; Step 4.3: Apply a partial freeze fine-tuning strategy to the pre-trained encoding model: Set the encoder parameters of the bottom preset number of layers of the pre-trained encoding model to a frozen state, and only participate in the subsequent supervised fine-tuning of the remaining high-level encoder parameters; Step 4.4: Input the Token index sequence output in Step 4.2 into the pre-trained encoding model configured in Step 4.3 to obtain the context-dependent semantic feature matrix of the function sample. ,in To preset the maximum token length threshold, each column vector This indicates the first [item] after fusing global context information. The deep semantic features of each assembled token.

6. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 5, characterized in that... The specific steps of step 5 are as follows: Step 5.1: Construct a two-layer bidirectional gated recurrent unit network connected in sequence, and set the hidden layer dimension for each gated recurrent unit in each direction; Step 5.2: Obtain the semantic feature matrix of the function sample. The first and second layers of the bidirectional gated recurrent unit network are input sequentially, and padding positions are dynamically ignored using an effective token mask; the output sequence-level feature matrix is ​​denoted as... Its dimensions are obtained by bidirectional splicing.

7. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 6, characterized in that... The specific steps of step 6 are as follows: Step 6.1: Initialize a learnable global query vector Q, and transform it from the sequence-level feature matrix through a linear mapping. Generate the key matrix K and the value matrix V; Step 6.2: Construct a mask matrix M, where the positions corresponding to the real tokens are set to 0 and the positions filled with negative infinity, in order to shield the influence of the filling positions on the attention calculation; Step 6.3: Perform a dot product operation between the query vector Q and the key matrix K, divide by a preset scaling factor, and then superimpose the result onto a mask matrix M. Next, apply the mask matrix M to the softmax function to obtain the attention weight α. Multiply the attention weight α by the value matrix V and perform layer normalization to obtain the function-level global feature vector. .

8. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 1, characterized in that... The specific steps of step 8 are as follows: Step 8.1: Use the real labels of function-level assembly code samples as supervision signals, where vulnerable functions correspond to label 1 and non-vulnerable functions correspond to label 0; and set the positive sample weight adjustment factor according to the ratio of vulnerable samples to non-vulnerable samples in the training data to alleviate the class imbalance problem. Step 8.2: Calculate the loss based on the original classification score and the true label output in Step 7 using the binary cross-entropy loss function with integrated Sigmoid mapping; Step 8.3: Using the loss as the optimization objective, calculate the gradient using the backpropagation algorithm, and combine it with the gradient pruning strategy to iteratively update the unfrozen high-level parameters, bidirectional gated recurrent unit network parameters, and classifier parameters of the pre-trained encoded model until the loss value converges or reaches the preset number of training rounds.

9. The binary program vulnerability detection method based on a fine-tuned pre-trained model according to claim 4, characterized in that... The specific steps of step 9 are as follows: Step 9.1: Input the simplified assembly token sequence obtained after processing the binary program under test in steps 1.3, 2 and 3 into the pre-trained encoding model, bidirectional gated recurrent unit network, attention aggregation module and classifier to obtain the original classification score; Step 9.2: Map the original classification score to a vulnerability probability using the Sigmoid function, and compare the vulnerability probability with a preset threshold; if the vulnerability probability is not less than the preset threshold, determine that the function-level assembly code sample is a vulnerable function; otherwise, determine that it is a non-vulnerable function.

Images