Firmware patch existence detection method based on block embedding

Abstract

The invention belongs to the field of Internet of Things firmware software supply chain security, and particularly relates to a firmware patch existence detection method based on block embedding. Comprising the steps that S1, a target binary function of firmware of the Internet of Things equipment is preprocessed, an instruction set and a control flow diagram are extracted, each node in the control flow diagram represents a basic block, and each basic block comprises a continuous assembly instruction sequence; s2, taking the basic blocks as analysis units, and positioning the basic blocks which are changed due to introduction of the patches; s3, extracting features from the instructions in the basic blocks, and constructing a basic block embedding model; and S4, in the basic block embedding model, respectively calculating similarity scores of the patch related block and the vulnerability and the patch function related block in the target function, and if the similarity score of the patch related block and the vulnerability and the patch function related block exceeds the score of the vulnerability function, predicting that the unknown target binary vulnerability function has a patch. According to the method, positioning of the patch blocks can be effectively accelerated, and representation of semantic difference introduced to the patches is enriched.

Description

Technical Field

[0001] The present invention belongs to the field of Internet of Things firmware software supply chain security, and in particular is a firmware patch existence detection method based on block embedding. Background Art

[0002] With the popularity of open source software, the code reuse of third-party libraries has surged. The firmware of IoT devices is prone to introducing vulnerabilities in third-party code. Due to the lack of simple upgrade procedures and automated patch management, the patch status of firmware in most IoT devices needs to be manually confirmed. Therefore, it is crucial to quickly and accurately detect patches in IoT devices. Security patches usually introduce subtle code changes to fix vulnerabilities. Up to 50% of patches only modify less than 7 lines of code. Traditional binary code matching technology has a coarse detection granularity and is prone to misreporting fixed functions as vulnerabilities, which cannot achieve statement-level detection accuracy. Most of the current patch existence detection methods are based on the execution semantics of functions, abstracting the behavioral semantics of binary programs into high-level representations, generating signatures or numerical feature vectors for existing patches and vulnerability functions, and then using similarity scores to determine the similarity between the target function and the two to detect patches. If the target function is similar to the patch function, it is determined to have been patched. According to the different feature quantization forms, the methods are divided into the following two categories: Signature-based methods. This type of method identifies fine-grained semantic binary changes introduced by binary patches and analyzes these changes to extract patch signatures, such as the origins and execution paths of representative operators and operands. For example, Fiber proposes to extract patch signatures by performing symbolic execution to obtain the precise expression of any register and memory location at any point in the program, but its efficiency is relatively low due to the large amount of symbolic execution; Osprey extracts patch signatures by analyzing the data flow slices of basic blocks; Spain identifies patch blocks from all changed blocks between the patch function and the vulnerable function.

[0003] Methods based on binary code embedding. This type of technology disassembles the binary code of the function into assembly code, studies the program structure graph derived from the binary code, such as the control flow graph (CFG) and the abstract syntax tree, and combines the language embedding model in the field of natural language processing to learn assembly code embedding.

[0004] Current binary patch presence detection methods have a number of limitations: Compiler optimizations will introduce a large number of binary function changes that are unrelated to the patch, resulting in large differences in the compiled binary code.

[0005] The mapping algorithm for locating patch blocks has low accuracy and cannot accurately locate the basic blocks that have actually changed. The matching process needs to be optimized.

[0006] The embedding method of patch code semantics is relatively simple and difficult to adapt to instruction changes introduced by different compilation options. Summary of the invention

[0007] In order to solve the above problems, the present invention provides a firmware patch existence detection method based on block embedding.

[0008] The present invention adopts the following technical solution: a firmware patch existence detection method based on block embedding, comprising: S1: Preprocess the target binary function of the firmware of the IoT device and extract it into an instruction set and a control flow graph. Each node in the control flow graph represents a basic block, which includes a continuous assembly instruction sequence. S2: Take the basic block as the analysis unit and locate the basic blocks that have changed due to the introduction of the patch; S3: Extract features from instructions in basic blocks and build a basic block embedding model; S4: In the basic block embedding model, the similarity scores of the patch-related blocks in the target function and the vulnerability and patch function-related blocks are calculated respectively. If the similarity score with the patch function exceeds the score of the vulnerability function, the unknown target binary vulnerability function is predicted to have a patch.

[0009] In some embodiments, step S1 includes: S11: Use a disassembly tool to disassemble the target function, vulnerability function, and patch function in binary form, and extract them into instruction sets and control flow graphs; S12: Normalize and simplify the assembly instructions in the basic blocks of the control flow graph.

[0010] In some embodiments, in step S12, normalization includes the following rules: 1) Register replacement: replace all register names with "REG"; 2) Memory reference replacement: replace all memory references with "MEMACC"; 3) Address replacement: replace direct addresses with "ADDR"; 4) Constant replacement: replace constants with "Constant".

[0011] In some embodiments, step S2 includes: S21: randomly selecting a basic block node with an equal instruction hash value in each of the two control flow graphs to be matched, starting from the initial node, extracting their respective direct neighbors, and finding the best matching pair in the neighbors based on the instruction hash value; S22: Start a loop and explore the matched neighbors in the direct neighbors, sort the similarities between adjacent basic blocks, and select the basic block pair with the highest similarity; S23: Repeat step S22 until all neighbors of matching pairs are explored and matched.

[0012] S24: After the loop ends, unmatched basic blocks are found due to unreachable codes or low similarity between neighbors, and the matching results are returned.

[0013] In some embodiments, step S3 includes: S31: Use the CBOW model to embed assembly instructions in basic blocks; S32: Build and train a Bi-LSTM based basic block embedding model.

[0014] In some embodiments, step S31 includes: S311: Input the instruction sequence in the patch block into the CBOW model, treat the instructions as words, and the input layer is given a surrounding word w(t-2), w(t-1), w(t+1), w(t+2) of a given instruction; S312: For each word, select a context window size m, the goal is to predict the context words within the range, so that the probability Maximization, its objective function is as follows: in It is the central word. yes The context set; S313: The model first slides the context window in the input word sequence to obtain the context as input and the current word as output; at the same time, each word is represented as an encoding of size V, and then the model queries the semantic embedding of the input word from the embedding layer, that is, a word embedding matrix; the embeddings are averaged and the predicted output is obtained using the hierarchical Softmax calculation function based on the Huffman tree, which is also a vector of size V; After model training, the word vector of the assembly instruction is represented by the row vector in the connection matrix between the input layer and the embedding layer to complete the instruction embedding.

[0015] In some embodiments, step S32 includes: S321: Build the model: S322: Constructed training data set: S323: Perform model training.

[0016] In some embodiments, in S321, the model adopts a Siamese structure, including two identical Bi-LSTMs, the Siamese network takes the instruction embedding sets of two basic blocks to be tested as input, and outputs the similarity of the two basic blocks; Bi-LSTM consists of a forward LSTM and a backward LSTM. The input of Bi-LSTM is the instruction embedding sequence generated in step S31. The instruction embedding sequence is fed to the forward LSTM in sequence and fed to the backward LSTM in reverse order. The last hidden state of these two LSTM and Stitched together as an embedding of basic blocks.

[0017] In some embodiments, step S322 includes: Collect open source software projects and input the same source code into a modified LLVM compiler to generate a large number of similar basic block pairs for different instruction-level architectures; Measure the textual similarity between two basic blocks compiled from different source codes, set the similarity threshold to 0.5, and if the similarity score of two blocks is less than 0.5, they are marked as dissimilar; Compiling different source codes in different architectures results in obtaining a large number of different basic block pairs.

[0018] In some embodiments, step S4 includes: The instruction embedding sets of the target function and the other two functions are input into the trained basic block embedding model respectively, and the similarity scores of each pair are calculated. The similarity scores are calculated using the cosine distance: Where x represents the embedding vector of the target function patch block, and y represents the embedding vector of the patch block in the patch function; The similarity score between the function to be tested and the patch block in the patch function is expressed as , the scores of the patch blocks corresponding to the tested function and the vulnerability function are expressed as ,like Greater than , then the unknown target binary vulnerability function is predicted to have a patch.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention proposes a basic block matching method, which can effectively speed up the positioning of patch blocks.

[0020] 2. The present invention uses Bi-LSTM for basic block embedding, enriching the representation of semantic differences introduced by patches. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 The figure is a schematic diagram of the firmware patch existence detection process based on block embedding of the method of the present invention. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0023] The present invention will be described in detail below through specific embodiments in conjunction with the accompanying drawings, but the present invention is not limited thereto.

[0024] The idea of ​​the present invention to solve the above problems is: first, fully preprocess the target binary function. Secondly, take the basic block as the analysis unit, and use the greedy block matching algorithm to eliminate the unmodified basic blocks, and roughly locate the basic blocks that have changed due to the introduction of the patch. Then, extract features from the instruction sequence in the patch block and the entire patch block as the feature representation of the patch function. Among them, for the sequence in the patch block, the CBOW model is used to generate instruction embeddings; for the entire patch block, the Bi-LSTM model is used to complete the basic block embedding. Finally, the patch block embeddings of the function to be tested and the patch and vulnerability functions are generated respectively, and the similarity scores between the target function and them are calculated. If the similarity with the patch function is greater, the unknown target binary vulnerability function is predicted to have a patch.

[0025] This embodiment provides an example of firmware patch existence detection using the method of the present invention.

[0026] A firmware patch existence detection method based on block embedding, comprising: S1: Preprocess the target binary functions of the firmware of the IoT device and extract them into instruction sets and control flow graphs. Each node in the control flow graph represents a basic block, which contains a continuous sequence of assembly instructions.

[0027] Step S1 includes: S11: Use a disassembly tool to disassemble the target function, vulnerability function, and patch function in binary form, and extract them into instruction sets and control flow graphs; S12: Normalize and simplify the assembly instructions in the basic blocks of the control flow graph. The normalization includes the following rules: 1) Register replacement: replace all register names with "REG"; 2) Memory reference replacement: replace all memory references with "MEMACC"; 3) Address replacement: replace direct addresses with "ADDR"; 4) Constant replacement: replace constants with "Constant".

[0028] Specifically, the public vulnerability CVE-2012-6702 is selected as the target of this method. The object that has changed in multiple versions of this vulnerability is the function generate_hash_secret_salt in the open source project Expat. Versions 2.1.1-2 and 2.1.1-3 are the patched version and the vulnerable version of this function respectively. The target function to be detected is a binary file of version 2.1.1-2.

[0029] First, IDA Pro is used to disassemble the target function and patch function in binary form, extract the assembly instruction set and control flow graph. The assembly instructions in the basic blocks of the control flow graph are normalized and simplified. Instruction normalization includes the following steps: iRegister Replacement Replace all register names with "REG" For example: "push ebp"->"push REG".

[0030] ii Memory Reference Replacement: Replace all memory references with "MEMACC" For example: "mov eax, [ebp-0x10]" -> "mov eax, MEMACC".

[0031] iii Address replacement: Replace the direct address with "ADDR" For example: “call 0x400567” -> “call ADDR”.

[0032] iv Constant Replacement: Replace the constant with " Constant " For example: "sub esp, 0x98" -> "sub esp, Constant".

[0033] S2: Take the basic block as the analysis unit and locate the basic blocks that have changed due to the introduction of the patch.

[0034] Step S2 includes: S21: Randomly select a basic block node with an equal instruction hash value in each of the two control flow graphs to be matched, start from the initial node, extract their respective direct neighbors, and find the best matching pair in the neighbors based on the instruction hash value.

[0035] S22: The algorithm starts to loop and explore the already matched neighbors in the direct neighbors, sorts the similarities between adjacent basic blocks, and selects the basic block pair with the highest similarity based on the predefined threshold of 0.6.

[0036] S23: Repeat S22 until all neighbors of matching pairs are explored and matched.

[0037] S24: After the loop ends, unmatched basic blocks are found due to unreachable codes or low similarity between neighbors, and the matching results are returned.

[0038] Specifically, a basic block node with an equal instruction hash value is randomly selected from each of the two versions of the control flow graph of generate_hash_secret_salt. Starting from the initial node, each direct neighbor is extracted, and the best matching pair is found in the neighbors based on the instruction hash value until all neighbors of the matching pair are explored and matched. After the loop ends, unmatched basic blocks are found due to unreachable code or low similarity between neighbors. The basic blocks that fail to match are the parts that differ between the patch function and the vulnerable function, and are defined as patch blocks. These patch blocks have changed due to the modification of the patch, including the deletion, addition, and modification of instructions within the block.

[0039] S3: Extract features from instructions in basic blocks and build a basic block embedding model.

[0040] Step S3 includes: S31: Use the CBOW model to embed assembly instructions in basic blocks.

[0041] Step S31 includes: The instruction sequence in the patch block is input into the CBOW model. The CBOW model consists of an input layer, an embedding layer, and a Softmax layer. The instructions are regarded as words, and the input layer is given a surrounding word w(t-2), w(t-1), w(t+1), w(t+2) of a given instruction.

[0042] For each word, a context window size m is selected, and the goal is to predict the context words within the range so that the probability Maximization, its objective function is shown as follows. It is the central word. yes The context set The model first slides the context window in the input word sequence to obtain the context as input and the current word as output. At the same time, each word is represented as an encoding of size V. Then, the model queries the semantic embedding of the input word from the embedding layer, which is a word embedding matrix. The embeddings are then averaged and the predicted output is obtained using a hierarchical Softmax calculation function based on the Huffman tree, which is also a vector of size V.

[0043] After model training, the word vector of the assembly instruction is represented by the row vector in the connection matrix between the input layer and the embedding layer to complete the instruction embedding.

[0044] S32: Build and train a Bi-LSTM based basic block embedding model.

[0045] Step S32 includes: 1) Model structure: The model adopts a Siamese structure and contains two identical Bi-LSTMs. The Siamese network takes the instruction embedding sets of the two basic blocks to be tested as input and outputs the similarity of the two basic blocks.

[0046] Bi-LSTM consists of a forward LSTM and a backward LSTM. The input of Bi-LSTM is the instruction embedding sequence generated in step S31. The instruction embedding sequence is fed to the forward LSTM in order and fed to the backward LSTM in reverse order. and Stitched together as an embedding of basic blocks.

[0047] 2) Model training data set construction: Generate Similar Basic Block Pairs: Collect open source software projects and input the same source code into the modified LLVM compiler to generate a large number of similar basic block pairs for different instruction-level architectures.

[0048] Generate different basic block pairs: Use n-gram to measure the text similarity between two basic blocks compiled from different source codes, set the similarity threshold to 0.5, and if the similarity score of two blocks is less than 0.5, they are marked as dissimilar. Compile different source codes in different architectures to obtain a large number of different basic block pairs.

[0049] 3) Model training Each LSTM unit receives an input in sequence at each time step, and the input basic block is represented as a sequence of instruction embeddings. The basic blocks are fed to the various parts of the Bi-LSTM separately, and all LSTM units update their hidden states at each time step.

[0050] The goal of training is to make the embeddings generated by the network meet the following requirements: the embeddings of similar basic blocks are as close as possible, while the embeddings of dissimilar blocks are as far apart as possible.

[0051] S4: In the basic block embedding model, the similarity scores of the patch-related blocks in the target function and the vulnerability and patch function-related blocks are calculated respectively. If the similarity score with the patch function exceeds the score of the vulnerability function, the unknown target binary vulnerability function is predicted to have a patch.

[0052] Step S4 includes: The instruction embedding sets of the target function and the other two functions are input into the trained basic block embedding model respectively, and the similarity scores of each pair are calculated. The similarity is calculated using the cosine distance: Where x represents the embedding vector of the target function patch block, and y represents the embedding vector of the patch block in the patch function. The similarity score between the test function and the patch block in the patch function is expressed as , the scores of the patch blocks corresponding to the tested function and the vulnerability function are expressed as ,like Greater than , then the unknown target binary vulnerability function is predicted to have a patch.

[0053] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A firmware patch existence detection method based on block embedding, characterized in that: include: S1: Preprocess the target binary function of the firmware of the IoT device and extract it into an instruction set and a control flow graph. Each node in the control flow graph represents a basic block, which includes a continuous assembly instruction sequence. S2: Take the basic block as the analysis unit and locate the basic blocks that have changed due to the introduction of the patch; S3: Extract features from instructions in basic blocks and build a basic block embedding model; S4: In the basic block embedding model, the similarity scores of the patch-related blocks in the target function and the vulnerability and patch function-related blocks are calculated respectively. If the similarity score with the patch function exceeds the score of the vulnerability function, the unknown target binary vulnerability function is predicted to have a patch.

2. The method for detecting the existence of firmware patches based on block embedding according to claim 1, characterized in that: The step S1 comprises: S11: Use a disassembly tool to disassemble the target function, vulnerability function, and patch function in binary form, and extract them into instruction sets and control flow graphs; S12: Normalize and simplify the assembly instructions in the basic blocks of the control flow graph.

3. The method for detecting the existence of firmware patches based on block embedding according to claim 2, characterized in that: In step S12, the normalization includes the following rules: 1) Register replacement: replace all register names with "REG"; 2) Memory reference replacement: replace all memory references with "MEMACC"; 3) Address replacement: replace direct addresses with "ADDR"; 4) Constant replacement: replace constants with "Constant".

4. The method for detecting the existence of firmware patches based on block embedding according to claim 1, characterized in that: The step S2 comprises: S21: randomly selecting a basic block node with an equal instruction hash value in each of the two control flow graphs to be matched, starting from the initial node, extracting their respective direct neighbors, and finding the best matching pair in the neighbors based on the instruction hash value; S22: Start a loop and explore the matched neighbors in the direct neighbors, sort the similarities between adjacent basic blocks, and select the basic block pair with the highest similarity; S23: Repeat step S22 until all neighbors of matching pairs are explored and matched. S24: After the loop ends, unmatched basic blocks are found due to unreachable codes or low similarity between neighbors, and the matching results are returned.

5. The method for detecting the existence of firmware patches based on block embedding according to claim 1, characterized in that: The step S3 comprises: S31: Use the CBOW model to embed assembly instructions in basic blocks; S32: Build and train a Bi-LSTM based basic block embedding model.

6. The method for detecting the existence of firmware patches based on block embedding according to claim 5, characterized in that: The step S31 comprises: S311: Input the instruction sequence in the patch block into the CBOW model, treat the instructions as words, and the input layer is given a surrounding word w(t-2), w(t-1), w(t+1), w(t+2) of a given instruction; S312: For each word, select a context window size m, the goal is to predict the context words within the range, so that the probability Maximization, its objective function is as follows: in It is the central word. yes The context set; S313: The model first slides the context window in the input word sequence to obtain the context as input and the current word as output; at the same time, each word is represented as an encoding of size V, and then the model queries the semantic embedding of the input word from the embedding layer, that is, a word embedding matrix; the embeddings are averaged and the predicted output is obtained using the hierarchical Softmax calculation function based on the Huffman tree, which is also a vector of size V; After model training, the word vector of the assembly instruction is represented by the row vector in the connection matrix between the input layer and the embedding layer to complete the instruction embedding.

7. The method for detecting the existence of firmware patches based on block embedding according to claim 5, characterized in that: The step S32 comprises: S321: Build the model: S322: Constructed training data set: S323: Perform model training.

8. The method for detecting the existence of firmware patches based on block embedding according to claim 7, characterized in that: In S321, the model adopts a Siamese structure, including two identical Bi-LSTMs, the Siamese network takes the instruction embedding sets of two basic blocks to be tested as input, and outputs the similarity of the two basic blocks; Bi-LSTM consists of a forward LSTM and a backward LSTM. The input of Bi-LSTM is the instruction embedding sequence generated in step S31. The instruction embedding sequence is fed to the forward LSTM in sequence and fed to the backward LSTM in reverse order. The last hidden state of these two LSTM and Stitched together as an embedding of basic blocks.

9. The firmware patch existence detection method based on multi-feature semantics according to claim 7, characterized in that: The step S322 includes: Collect open source software projects and input the same source code into a modified LLVM compiler to generate a large number of similar basic block pairs for different instruction-level architectures; Measure the textual similarity between two basic blocks compiled from different source codes, set the similarity threshold to 0.5, and if the similarity score of two blocks is less than 0.5, they are marked as dissimilar; Compiling different source codes in different architectures results in obtaining a large number of different basic block pairs.

10. The method for detecting the existence of firmware patches based on block embedding according to claim 5, characterized in that: The step S4 comprises: The instruction embedding sets of the target function and the other two functions are input into the trained basic block embedding model respectively, and the similarity scores of each pair are calculated. The similarity scores are calculated using the cosine distance: Where x represents the embedding vector of the target function patch block, and y represents the embedding vector of the patch block in the patch function; The similarity score between the function to be tested and the patch block in the patch function is expressed as , the scores of the patch blocks corresponding to the tested function and the vulnerability function are expressed as ,like Greater than , then the unknown target binary vulnerability function is predicted to have a patch.

Images