A binary code similarity detection method, device and storage medium

Abstract

This invention discloses a binary code similarity detection method, apparatus, and storage medium, comprising: obtaining the control flow graph of functions in a binary firmware file; extracting semantic information to obtain code block embedding vectors from the control flow graph; obtaining deep semantic features of the control flow graph from the code block embedding vectors, and determining the order-aware features of the code block embedding vectors; fusing the deep semantic features and the order-aware features to obtain a graph embedding vector; and calculating function similarity using the graph embedding vectors. Using this invention, the detection accuracy can be significantly improved. It can arbitrarily extend semantic feature extraction across architectures, thereby improving the accuracy of binary code similarity comparison.

Description

Technical Field

[0001] This invention relates to the field of information security, and in particular to a binary code similarity detection method, apparatus, and storage medium. Background Technology

[0002] Binary code analysis is a crucial research area in information security, with one goal being to detect similar binary functions without accessing the source code. Due to the heterogeneous nature of IoT devices, the same program can compile into different functions on different platforms. To find relationships between vulnerabilities, it's necessary to identify connections between the code itself to determine if vulnerabilities exist or if there is malicious code plagiarism.

[0003] Currently, binary file similarity comparison schemes mainly fall into two categories: traditional methods and deep learning-based methods, based on their features.

[0004] Traditional methods primarily involve analyzing assembly code to identify features such as the number of instructions, function calls, string constants, and numeric constants, and then using graph matching algorithms to calculate the similarity between two binary files. Examples of traditional methods include SIGMA and discovRE.

[0005] Deep learning-based methods primarily extract function features from assembly instructions or control flow graphs using deep neural networks, and finally calculate binary file similarity using cosine distance. Methods based on binary instructions include SAFE and Asm2vec; methods based on control flow graphs include GEMINI and SANN.

[0006] The shortcoming of the current technology is that the accuracy of binary file similarity comparison can still be improved. Summary of the Invention

[0007] This invention provides a binary code similarity detection method, apparatus, and storage medium to improve the accuracy of binary file similarity comparison.

[0008] This invention provides the following technical solutions:

[0009] A binary code similarity detection method, comprising:

[0010] Control flow diagram of the function that obtains the binary firmware file;

[0011] Extract semantic information to obtain the code block embedding vector of the control flow diagram;

[0012] The deep semantic features of the control flow graph are obtained by analyzing the code block embedding vectors, and the order-aware features of the code block embedding vectors are determined.

[0013] By fusing deep semantic features and sequence-aware features, a graph embedding vector is obtained;

[0014] Calculate functional similarity using graph embedding vectors.

[0015] In practice, the control flow diagram of the function that obtains the binary firmware file is obtained through IDA.

[0016] In practice, the code block embedding vectors of the control flow graph are obtained by extracting semantic information using BERT.

[0017] During implementation, it further includes:

[0018] Link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code, and node category information of different platforms is learned through node classification and node clustering.

[0019] In implementation, link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code. Node classification and clustering are used to learn node category information for different platforms, including:

[0020] Add graph-level tasks: link prediction (LP), node classification (BCf), and node clustering information (BCt). LP is used to determine the link relationship between two blocks as one of the following: strong correlation, weak correlation, or no correlation. BCf classifies blocks, and BCt clusters blocks to determine the platform, compiler, and optimization options to which the block belongs.

[0021] In implementation, the deep semantic features of the control flow graph are obtained by embedding the code blocks into vectors, which is achieved through a message-passing neural network; and / or,

[0022] The order-aware features of the code block embedding vector are determined through a pointer network.

[0023] In implementation, the order-aware features of the code block embedding vectors are determined, including:

[0024] Given the node characteristics of the input code that show connection relationships, output the order of node connections;

[0025] Using intermediate layer features as sequence features, we learn the sequential relationships between nodes.

[0026] In implementation, the order-aware features of the code block embedding vectors are determined, including:

[0027] A Pointer network is used, and LSTM is used to encode a branch in the same control flow diagram;

[0028] During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network.

[0029] At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output;

[0030] When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

[0031] In implementation, when calculating functional similarity using graph embedding vectors, the following further steps are included:

[0032] Using Siamese neural networks for training reduces graph embedding vector loss.

[0033] In implementation, functional similarity is calculated using graph embedding vectors, including:

[0034] The cosine distance is used to calculate function similarity. Multiple functions in the firmware are traversed, and the similarity scores of each function are weighted and used as the firmware similarity score. Then, the function similarity is calculated.

[0035] A binary code similarity detection device, comprising:

[0036] The processor is used to read programs from memory and execute the following procedures:

[0037] Control flow diagram of the function that obtains the binary firmware file;

[0038] Extract semantic information to obtain the code block embedding vector of the control flow diagram;

[0039] The deep semantic features of the control flow graph are obtained by analyzing the code block embedding vectors, and the order-aware features of the code block embedding vectors are determined.

[0040] By fusing deep semantic features and sequence-aware features, a graph embedding vector is obtained;

[0041] Calculate function similarity using graph embedding vectors;

[0042] A transceiver is used to receive and send data under the control of a processor.

[0043] In practice, the control flow diagram of the function that obtains the binary firmware file is obtained through IDA.

[0044] In practice, the code block embedding vectors of the control flow graph are obtained by extracting semantic information using BERT.

[0045] During implementation, it further includes:

[0046] Link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code, and node category information of different platforms is learned through node classification and node clustering.

[0047] In implementation, link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code. Node classification and clustering are used to learn node category information for different platforms, including:

[0048] Add graph-level tasks: link prediction (LP), node classification (BCf), and node clustering information (BCt). LP is used to determine the link relationship between two blocks as one of the following: strong correlation, weak correlation, or no correlation. BCf classifies blocks, and BCt clusters blocks to determine the platform, compiler, and optimization options to which the block belongs.

[0049] In implementation, the deep semantic features of the control flow graph are obtained by embedding the code blocks into vectors, which is achieved through a message-passing neural network; and / or,

[0050] The order-aware features of the code block embedding vector are determined through a pointer network.

[0051] In implementation, the order-aware features of the code block embedding vectors are determined, including:

[0052] Given the node characteristics of the input code that show connection relationships, output the order of node connections;

[0053] Using intermediate layer features as sequence features, we learn the sequential relationships between nodes.

[0054] In implementation, the order-aware features of the code block embedding vectors are determined, including:

[0055] A Pointer network is used, and LSTM is used to encode a branch in the same control flow diagram;

[0056] During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network.

[0057] At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output;

[0058] When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

[0059] In implementation, when calculating functional similarity using graph embedding vectors, the following further steps are included:

[0060] Using Siamese neural networks for training reduces graph embedding vector loss.

[0061] In implementation, functional similarity is calculated using graph embedding vectors, including:

[0062] The cosine distance is used to calculate function similarity. Multiple functions in the firmware are traversed, and the similarity scores of each function are weighted and used as the firmware similarity score. Then, the function similarity is calculated.

[0063] A binary code similarity detection device, comprising:

[0064] The flowchart module is used to obtain the control flow diagram of the functions that retrieve the binary firmware file;

[0065] The embedding vector module is used to extract semantic information to obtain the code block embedding vectors of the control flow graph;

[0066] The semantic and sequence module is used to obtain deep semantic features of the control flow graph by analyzing the code block embedding vectors, and to determine the sequence-aware features of the code block embedding vectors.

[0067] The fusion module is used to fuse deep semantic features and sequence-aware features to obtain graph embedding vectors.

[0068] The similarity module is used to calculate the similarity of functions using graph embedding vectors.

[0069] In practice, the flowchart module is further used to control the flow chart of the function that obtains the binary firmware file through IDA.

[0070] In practice, the embedding vector module is further used to obtain the code block embedding vectors of the control flow graph by extracting semantic information using BERT.

[0071] In implementation, the semantic and sequence module is further used to connect the link relationships between nodes in the prediction learning code through link prediction, node classification and node clustering information, and to learn the node category information of different platforms through node classification and node clustering.

[0072] In implementation, the semantic and sequence module is further used to connect the link relationships between nodes in the prediction learning code through link prediction, node classification, and node clustering information, and to learn node category information for different platforms through node classification and node clustering, including:

[0073] Add graph-level tasks: link prediction (LP), node classification (BCf), and node clustering information (BCt). LP is used to determine the link relationship between two blocks as one of the following: strong correlation, weak correlation, or no correlation. BCf classifies blocks, and BCt clusters blocks to determine the platform, compiler, and optimization options to which the block belongs.

[0074] In implementation, the semantic and sequence module is further used to obtain deep semantic features of the control flow graph through a message passing neural network; and / or to determine the sequence-aware features of the code block embedding vector through a pointer network.

[0075] In implementation, the semantic and sequence module is further used to determine the sequence-aware features of the code block embedding vector, including:

[0076] Given the node characteristics of the input code that show connection relationships, output the order of node connections;

[0077] Using intermediate layer features as sequence features, we learn the sequential relationships between nodes.

[0078] In implementation, the semantic and sequence module is further used to determine the sequence-aware features of the code block embedding vector, including:

[0079] A Pointer network is used, and LSTM is used to encode a branch in the same control flow diagram;

[0080] During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network.

[0081] At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output;

[0082] When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

[0083] In practice, the similarity module is further used to reduce the graph embedding vector loss when calculating the functional similarity through graph embedding vectors.

[0084] In implementation, the similarity module is further used when calculating functional similarity via graph embedding vectors, including:

[0085] The cosine distance is used to calculate function similarity. Multiple functions in the firmware are traversed, and the similarity scores of each function are weighted and used as the firmware similarity score. Then, the function similarity is calculated.

[0086] A computer-readable storage medium storing a computer program that performs the binary code similarity detection method described above.

[0087] The beneficial effects of this invention are as follows:

[0088] In the technical solution provided by the embodiments of the present invention, after obtaining the deep semantic features of the control flow graph of the code block embedding vector and determining the sequence-aware features of the code block embedding vector, the graph embedding vector is fused to obtain the graph embedding vector, and then a similarity comparison is performed. Since the binary file comparison based on the fusion of deep semantic information and sequence features contains richer information, the graph embedding vector can better improve the detection accuracy.

[0089] Furthermore, by extracting the graph embedding vector framework, the pointer network learns the order information between nodes, which can arbitrarily extend the semantic feature extraction across architectures and improve the accuracy of binary code similarity comparison. Attached Figure Description

[0090] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0091] Figure 1 This is a schematic diagram of the implementation process of the binary code similarity detection method in this embodiment of the invention. Figure 1 ;

[0092] Figure 2 This is a schematic diagram of the implementation process of the binary code similarity detection method in this embodiment of the invention. Figure 2 ;

[0093] Figure 3 This is a schematic diagram of the semantic information extraction model in an embodiment of the present invention;

[0094] Figure 4 This is a schematic diagram of the sequence perception model in an embodiment of the present invention;

[0095] Figure 5 This is a schematic diagram of the binary code similarity detection device in an embodiment of the present invention. Detailed Implementation

[0096] The inventor noticed the following during the invention process:

[0097] An analysis of deep learning methods based on control flow graphs reveals that they suffer from at least one of the following drawbacks:

[0098] 1. Although features based on deep learning can learn semantic information, they cannot capture the order information between blocks in the control flow diagram more accurately.

[0099] 2. Currently, neural networks discard code blocks whose input exceeds a certain threshold, resulting in information loss.

[0100] 3. Although deep learning-based methods can learn high-level semantic information from a large amount of cross-platform data, unsupervised training can only rely on the model itself and cannot fully learn cross-platform information.

[0101] 4. Existing technologies have low scalability and poor flexibility.

[0102] Based on this, the technical solution provided by the embodiments of the present invention aims to improve the accuracy of binary code similarity comparison and enhance cross-platform scalability. It proposes a cross-platform binary code similarity detection scheme that fuses deep semantic features and sequence-aware features. The goal is to solve at least one or a combination of the following problems:

[0103] 1. Pointer networks can accurately learn the node sequence information in the control flowchart.

[0104] 2. The pointer network can accept arbitrary input lengths and learn all code block information of the entire function, thus solving the problem of code block information loss caused by fixed-length inputs.

[0105] 3. By incorporating node link relationships, node classification, and node clustering information, the BERT network is trained under supervision, enabling cross-platform information to be learned in the code block vectors, thereby solving the cross-platform problem of binary code similarity detection.

[0106] 4. Automatically extracts function feature vectors, and can be extended to different platforms simply by adding training data and training labels, improving the flexibility of the solution.

[0107] Based on this, the technical solution provided in the embodiments of the present invention proposes a cross-platform binary code similarity detection scheme based on the fusion of deep semantic features and sequence-aware features. The specific implementation of the present invention will be described below with reference to the accompanying drawings.

[0108] Figure 1 Implementation flow diagram of binary code similarity detection method Figure 1 As shown in the figure, it can include:

[0109] Step 101: Control flow diagram of the function for obtaining the binary firmware file;

[0110] Step 102: Extract semantic information to obtain the code block embedding vector of the control flow diagram;

[0111] Step 103: Obtain the deep semantic features of the control flow graph by analyzing the code block embedding vectors, and determine the order-aware features of the code block embedding vectors.

[0112] Step 104: Fuse deep semantic features and sequence-aware features to obtain graph embedding vectors;

[0113] Step 105: Calculate the function similarity using graph embedding vectors.

[0114] Figure 2 Implementation flow diagram of binary code similarity detection method Figure 2 As shown in the figure, it can include:

[0115] Obtain the binary firmware file;

[0116] The binary firmware file is unpacked and its contained executable files are disassembled to extract the control flow diagrams of each function in all executable files.

[0117] The control flow graph is converted into a natural language processing task, and the pre-trained BERT network is used to obtain block encoding.

[0118] A message-passing neural network is constructed based on block coding, and a sequence-aware model is trained to obtain sequence-aware features;

[0119] Train a deep neural network based on the fusion of deep semantic features and sequence-aware features to obtain the embedding vector of the control flow graph;

[0120] Compare functions from different binary firmware files, and determine whether there are identical functions in the firmware and whether the firmware code is similar based on the function similarity results.

[0121] Specifically, it mainly consists of the following parts:

[0122] I. Control flow diagram of all functions for obtaining the binary firmware file;

[0123] II. Semantic Information Extraction Module: This module utilizes BERT to extract semantic information and obtain code block embedding vectors. It incorporates link prediction, node classification, and node clustering information. Link prediction learns the link relationships between nodes in the code, while node classification and clustering learn node category information for different platforms.

[0124] Third, deep semantic features are obtained by learning deep semantic features from code block embedding vectors through message passing neural networks. The module that implements this function is called the graph semantic information extraction module.

[0125] IV. Obtaining sequence-aware features by embedding code block vectors through a pointer network. The input consists of node features with connections in the code, and the output is the node connection order. The intermediate layer features are used as sequence features to learn the sequential relationships between nodes. The module that implements this function is called the sequence-aware feature extraction module.

[0126] 5. By fusing deep semantic features and sequence-aware features, graph embedding vectors are obtained;

[0127] VI. Use Siamese neural network training on the graph embedding vector extraction model to reduce graph embedding vector loss;

[0128] 7. Use cosine distance to calculate graph similarity.

[0129] The implementation of each step will be explained again below with examples.

[0130] I. Control flow diagram of all functions for obtaining binary firmware files.

[0131] In practice, the control flow diagram of the function that obtains the binary firmware file is obtained through IDA.

[0132] Specifically, function control flow diagrams can be obtained using tools such as IDA (Interactive Disassembler).

[0133] II. Semantic Information Extraction Module.

[0134] In practice, the code block embedding vectors of the control flow graph are obtained by extracting semantic information using BERT.

[0135] Figure 3 The figure shows a schematic diagram of the semantic information extraction model. In practice, BERT (Bidirectional Encoder Representations from Transformers) can be used to pre-train the control flow graph to obtain the semantic information of the code blocks.

[0136] BERT was originally used in the field of NLP (Natural Language Processing) for pre-training on words and sentences. The tasks in this embodiment are similar to NLP tasks; code blocks in the control flow graph can be considered sentences, and tokens within those blocks can be considered words. BERT is used to extract word embedding vectors from the control flow graph, mainly including five tasks: MaskNM (Masked Node Modeling), ANP (Adjacency Node Prediction), LP (Link Prediction), BCf (Block Classification), and BCT (Block Clustering).

[0137] MNM and ANP are two tasks similar to those in the original BERT paper. MNM is a token-level task that masks tokens in a block and makes predictions, similar to language models. ANP is a block-level task. Although the control flow graph does not have the language order found in NLP, it is a directed graph with a topological order of nodes. Therefore, all adjacent nodes in the control flow graph can be extracted and treated as adjacent "sentences." These adjacent block pairs serve as positive examples for ANP, and non-adjacent block pairs within the same graph are randomly selected as negative examples.

[0138] In practice, it may further include:

[0139] Link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code, and node category information of different platforms is learned through node classification and node clustering.

[0140] In specific implementation, it may include:

[0141] Add graph-level tasks: link prediction (LP), node classification (BCf), and node clustering information (BCt). LP is used to determine the link relationship between two blocks as one of the following: strong correlation, weak correlation, or no correlation. BCf classifies blocks, and BCt clusters blocks to determine the platform, compiler, and optimization options to which the block belongs.

[0142] Specifically, in order to obtain more graph-level information, three auxiliary graph-level tasks, LP, BCf, and BCT, were added to the implementation.

[0143] LP and ANP are similar in approach, the difference being the selection of positive and negative examples for the pair. The goal of the LP task is to enable the model to determine the link relationship between two blocks: strongly correlated, weakly correlated, or completely uncorrelated, so that the model can learn this information as much as possible, thus helping with graph-level tasks. Therefore, in LP tasks, block pairs with direct links within the same graph are strongly correlated, block pairs with indirect links within the same graph are weakly correlated, and block pairs from different graphs are completely uncorrelated.

[0144] BCf and BCT are graph-level block classification tasks. In the scenario described in the example, the obtained block information differs under different platforms, compilers, and optimization options. The goal of the implementation is to enable the model to include this information in the block embedding. BCf classifies blocks, while BCT clusters blocks to determine which platform, compiler, and optimization option a block belongs to.

[0145] III. Graph Semantic Information Extraction Module

[0146] In practice, the deep semantic features of the control flow graph are obtained by embedding the code block vectors, which is achieved through a message passing neural network.

[0147] Specifically, after BERT pre-training, the message-passing neural network MPNN is used to compute the graph semantic information of the control flow graph.

[0148] IV. Sequence-aware feature extraction module.

[0149] In practice, the order-aware features of the code block embedding vector are determined through a pointer network.

[0150] In specific implementation, the order-aware features of the code block embedding vector are determined, including:

[0151] Given the node characteristics of the input code that show connection relationships, output the order of node connections;

[0152] Using intermediate layer features as sequence features, we learn the sequential relationships between nodes.

[0153] In specific implementation, the order-aware features of the code block embedding vector are determined, including:

[0154] A Pointer network is used, and LSTM is used to encode a branch in the same control flow diagram;

[0155] During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network.

[0156] At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output;

[0157] When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

[0158] Figure 4 The sequence-aware model is illustrated in the figure. This module is used to extract information about the order of nodes. Since there is no output vocabulary in this task, each element in the output sequence comes from an element in the input. Therefore, a pointer network can be used in the implementation. LSTM (Long Short-Term Memory) is used to encode a branch in the same graph. During decoding, it points to an element in the input at each time step, serving as the output for the current time step.

[0159] The decoder considers three pieces of information at each time step: block semantic features from the BERT network, relational features, and node features from the previous time step. At each time step, the network filters out nodes that have already been predicted as outputs, and the decoder selects the node with the highest probability as the current output. The network input is of arbitrary variable length, capable of capturing the sequential information of all code blocks in the function. Through supervised learning, the connection information of blocks in the control flow graph can be learned more accurately, and ultimately, the hidden layer features can be used as sequence-aware features.

[0160] Fifth, by fusing deep semantic features and sequence-aware features, graph embedding vectors are obtained.

[0161] VI. Use Siamese neural network training on the graph embedding vector extraction model to reduce graph embedding vector loss.

[0162] VII. Calculate functional similarity using graph embedding vectors, including:

[0163] The cosine distance is used to calculate function similarity. Multiple functions in the firmware are traversed, and the similarity scores of each function are weighted and used as the firmware similarity score. Then, the function similarity is calculated.

[0164] Specifically, cosine distance can be used to calculate function similarity. By traversing multiple functions in the firmware and weighting the similarity scores of each function, the firmware similarity score can be obtained, thereby determining whether there are vulnerabilities, malicious code plagiarism, etc.

[0165] Based on the same inventive concept, this invention also provides a binary code similarity detection device and a computer-readable storage medium. Since the principle of these devices in solving the problem is similar to that of the binary code similarity detection method, the implementation of these devices can refer to the implementation of the method, and the repeated parts will not be described again.

[0166] When implementing the technical solutions provided in the embodiments of the present invention, they can be implemented in the following manner.

[0167] Figure 5 The figure shows a schematic diagram of a binary code similarity detection device. The device includes:

[0168] Processor 500 is used to read the program from memory 520 and execute the following procedures:

[0169] Control flow diagram of the function that obtains the binary firmware file;

[0170] Extract semantic information to obtain the code block embedding vector of the control flow diagram;

[0171] The deep semantic features of the control flow graph are obtained by analyzing the code block embedding vectors, and the order-aware features of the code block embedding vectors are determined.

[0172] By fusing deep semantic features and sequence-aware features, a graph embedding vector is obtained;

[0173] Calculate function similarity using graph embedding vectors;

[0174] Transceiver 510 is used to receive and send data under the control of processor 500.

[0175] In practice, the control flow diagram of the function that obtains the binary firmware file is obtained through IDA.

[0176] In practice, the code block embedding vectors of the control flow graph are obtained by extracting semantic information using BERT.

[0177] During implementation, it further includes:

[0178] Link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code, and node category information of different platforms is learned through node classification and node clustering.

[0179] In implementation, link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code. Node classification and clustering are used to learn node category information for different platforms, including:

[0180] Add graph-level tasks: link prediction (LP), node classification (BCf), and node clustering information (BCt). LP is used to determine the link relationship between two blocks as one of the following: strong correlation, weak correlation, or no correlation. BCf classifies blocks, and BCt clusters blocks to determine the platform, compiler, and optimization options to which the block belongs.

[0181] In implementation, the deep semantic features of the control flow graph are obtained by embedding the code blocks into vectors, which is achieved through a message-passing neural network; and / or,

[0182] The order-aware features of the code block embedding vector are determined through a pointer network.

[0183] In implementation, the order-aware features of the code block embedding vectors are determined, including:

[0184] Given the node characteristics of the input code that show connection relationships, output the order of node connections;

[0185] Using intermediate layer features as sequence features, we learn the sequential relationships between nodes.

[0186] In implementation, the order-aware features of the code block embedding vectors are determined, including:

[0187] A Pointer network is used, and LSTM is used to encode a branch in the same control flow diagram;

[0188] During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network.

[0189] At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output;

[0190] When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

[0191] In implementation, when calculating functional similarity using graph embedding vectors, the following further steps are included:

[0192] Using Siamese neural networks for training reduces graph embedding vector loss.

[0193] In implementation, functional similarity is calculated using graph embedding vectors, including:

[0194] The cosine distance is used to calculate function similarity. Multiple functions in the firmware are traversed, and the similarity scores of each function are weighted and used as the firmware similarity score. Then, the function similarity is calculated.

[0195] Among them, Figure 5 In this context, the bus architecture may include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 500) and memory (memory 520). The bus architecture may also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 510 may be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over a transmission medium. The processor 500 is responsible for managing the bus architecture and general processing, and the memory 520 may store data used by the processor 500 during operation.

[0196] This invention also provides a binary code similarity detection device, comprising:

[0197] The flowchart module is used to obtain the control flow diagram of the functions that retrieve the binary firmware file;

[0198] The embedding vector module is used to extract semantic information to obtain the code block embedding vectors of the control flow graph;

[0199] The semantic and sequence module is used to obtain deep semantic features of the control flow graph by analyzing the code block embedding vectors, and to determine the sequence-aware features of the code block embedding vectors.

[0200] The fusion module is used to fuse deep semantic features and sequence-aware features to obtain graph embedding vectors.

[0201] The similarity module is used to calculate the similarity of functions using graph embedding vectors.

[0202] In practice, the flowchart module is further used to control the flow chart of the function that obtains the binary firmware file through IDA.

[0203] In practice, the embedding vector module is further used to obtain the code block embedding vectors of the control flow graph by extracting semantic information using BERT.

[0204] In implementation, the semantic and sequence module is further used to connect the link relationships between nodes in the prediction learning code through link prediction, node classification and node clustering information, and to learn the node category information of different platforms through node classification and node clustering.

[0205] In implementation, the semantic and sequence module is further used to connect the link relationships between nodes in the prediction learning code through link prediction, node classification, and node clustering information, and to learn node category information for different platforms through node classification and node clustering, including:

[0206] Add graph-level tasks: link prediction (LP), node classification (BCf), and node clustering information (BCt). LP is used to determine the link relationship between two blocks as one of the following: strong correlation, weak correlation, or no correlation. BCf classifies blocks, and BCt clusters blocks to determine the platform, compiler, and optimization options to which the block belongs.

[0207] In implementation, the semantic and sequence module is further used to obtain deep semantic features of the control flow graph through a message passing neural network; and / or to determine the sequence-aware features of the code block embedding vector through a pointer network.

[0208] In implementation, the semantic and sequence module is further used to determine the sequence-aware features of the code block embedding vector, including:

[0209] Given the node characteristics of the input code that show connection relationships, output the order of node connections;

[0210] Using intermediate layer features as sequence features, we learn the sequential relationships between nodes.

[0211] In implementation, the semantic and sequence module is further used to determine the sequence-aware features of the code block embedding vector, including:

[0212] A Pointer network is used, and LSTM is used to encode a branch in the same control flow diagram;

[0213] During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network.

[0214] At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output;

[0215] When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

[0216] In practice, the similarity module is further used to reduce the graph embedding vector loss when calculating the functional similarity through graph embedding vectors.

[0217] In implementation, the similarity module is further used when calculating functional similarity via graph embedding vectors, including:

[0218] The cosine distance is used to calculate function similarity. Multiple functions in the firmware are traversed, and the similarity scores of each function are weighted and used as the firmware similarity score. Then, the function similarity is calculated.

[0219] For ease of description, the various parts of the device described above are divided into modules or units according to their functions. Of course, in implementing this invention, the functions of each module or unit can be implemented in one or more software or hardware components.

[0220] This invention also provides a computer-readable storage medium storing a computer program that performs the above-described binary code similarity detection method.

[0221] For details on implementation, please refer to the implementation of the binary code similarity detection method.

[0222] In summary, the technical solution provided by this invention proposes that the sequence perception module, through supervised training of a pointer network, can more accurately capture the sequence information between blocks in the control flow diagram.

[0223] Furthermore, semantic information extraction is proposed, and link prediction, node classification and node clustering are introduced. Supervised learning enables the feature vector to fully capture cross-platform information.

[0224] Furthermore, the binary file alignment method based on the fusion of deep semantic information and sequential features has an overall framework that allows graph embedding vectors to contain richer information, thereby improving detection accuracy.

[0225] Furthermore, the solution can be applied in areas such as vulnerability detection, code plagiarism, and software theft.

[0226] It is evident that by learning the sequential features between nodes through pointer networks, supervised learning enables the sequence-aware features to fully learn the sequential relationships between nodes.

[0227] By supervising the training of pointer networks, with input being a vector of code blocks of arbitrary length, without discarding any code blocks in functions, the network can learn the order information and high-level semantic information of all code blocks.

[0228] By introducing link prediction, node classification, and node clustering information, BERT is trained in a supervised manner, making the code block embedding vectors rich in various cross-platform and cross-compiler information, and arbitrarily extended with cross-architecture information.

[0229] By extracting graph embedding vector frameworks, pointer networks are introduced to learn the order information between nodes, which can arbitrarily extend semantic feature extraction across architectures and improve the accuracy of binary code similarity comparison.

[0230] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0231] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0232] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0233] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0234] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A binary code similarity detection method, characterized in that, include: Control flow diagram of the function that obtains the binary firmware file; Extract semantic information to obtain the code block embedding vector of the control flow diagram; The deep semantic features of the control flow graph are obtained by analyzing the code block embedding vectors, and the order-aware features of the code block embedding vectors are determined. By fusing deep semantic features and sequence-aware features, a graph embedding vector is obtained; Calculate function similarity using graph embedding vectors; The deep semantic features of the control flow graph are obtained by using the code block embedding vectors, which is achieved through a message passing neural network; and / or, the order-aware features of the code block embedding vectors are determined by a pointer network. Determine the order-aware features of the code block embedding vectors, including: Given the node characteristics of the input code that show connection relationships, output the order of node connections; Using intermediate layer features as sequential features, we learn the sequential relationships between nodes; Determining the order-aware features of code block embedding vectors also includes: A pointer network is used, and a Long Short-Term Memory (LSTM) is used to encode a branch in the same control flow graph; During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network. At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output; When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

2. The method as described in claim 1, characterized in that, The control flow diagram of the function that obtains the binary firmware file was obtained using the interactive disassembler IDA.

3. The method as described in claim 1, characterized in that, The code block embedding vectors of the control flow diagram are obtained by extracting semantic information, which is extracted by using the bidirectional encoder of the transformer to represent BERT.

4. The method as described in claim 3, characterized in that, Further includes: Link prediction, node classification, and node clustering information are used to connect the link relationships between nodes in the prediction learning code, and node category information of different platforms is learned through node classification and node clustering.

5. The method as described in claim 4, characterized in that, Link prediction, node classification, and node clustering information are used to establish the link relationships between nodes in the prediction learning code. Node classification and clustering are used to learn node category information for different platforms, including: Add graph-level tasks: Link Prediction (LP), Node Classification (BCf), and Node Clustering Information (BCt). LP is used to determine the link relationship between two blocks as one of the following: strong correlation, weak correlation, or no correlation. BCf classifies blocks, and BCt clusters blocks to determine the platform, compiler, and optimization options to which the block belongs.

6. The method as described in claim 1, characterized in that, When calculating functional similarity using graph embedding vectors, the following further applies: Using Siamese neural networks for training reduces graph embedding vector loss.

7. The method as described in claim 6, characterized in that, Calculating functional similarity using graph embedding vectors includes: The cosine distance is used to calculate function similarity. Multiple functions in the firmware are traversed, and the similarity scores of each function are weighted and used as the firmware similarity score. Then, the function similarity is calculated.

8. A binary code similarity detection device, characterized in that, include: The processor is used to read programs from memory and execute the following procedures: Control flow diagram of the function that obtains the binary firmware file; Extract semantic information to obtain the code block embedding vector of the control flow diagram; The deep semantic features of the control flow graph are obtained by analyzing the code block embedding vectors, and the order-aware features of the code block embedding vectors are determined. By fusing deep semantic features and sequence-aware features, a graph embedding vector is obtained; Calculate function similarity using graph embedding vectors; A transceiver is used to receive and send data under the control of a processor; The deep semantic features of the control flow graph are obtained by using the code block embedding vectors, which is achieved through a message passing neural network; and / or, the order-aware features of the code block embedding vectors are determined by a pointer network. Determine the order-aware features of the code block embedding vectors, including: Given the node characteristics of the input code that show connection relationships, output the order of node connections; Using intermediate layer features as sequential features, we learn the sequential relationships between nodes; Determining the order-aware features of code block embedding vectors also includes: A pointer network is used, and a Long Short-Term Memory (LSTM) is used to encode a branch in the same control flow graph; During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network. At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output; When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

9. A binary code similarity detection device, characterized in that, include: The flowchart module is used to obtain the control flow diagram of the functions that retrieve the binary firmware file; The embedding vector module is used to extract semantic information to obtain the code block embedding vectors of the control flow graph; The semantic and sequence module is used to obtain deep semantic features of the control flow graph by analyzing the code block embedding vectors, and to determine the sequence-aware features of the code block embedding vectors. The fusion module is used to fuse deep semantic features and sequence-aware features to obtain graph embedding vectors. The similarity module is used to calculate the similarity of functions using graph embedding vectors; The deep semantic features of the control flow graph are obtained by using the code block embedding vectors, which is achieved through a message passing neural network; and / or, the order-aware features of the code block embedding vectors are determined by a pointer network. Determine the order-aware features of the code block embedding vectors, including: Given the node characteristics of the input code that show connection relationships, output the order of node connections; Using intermediate layer features as sequential features, we learn the sequential relationships between nodes; Determining the order-aware features of code block embedding vectors also includes: A pointer network is used, and a Long Short-Term Memory (LSTM) is used to encode a branch in the same control flow graph; During decoding, it points to a certain element in the input at each time step as the output of the current time step. The information processed by the decoder at each time step includes: block semantic features, relation features and node features output from the previous time step from the BERT network. At each time step, nodes that have already been predicted as outputs are filtered out, and the decoder selects the node with the highest probability as the current output; When the input is of arbitrary variable length, capture the order information of all code blocks in the function.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that performs the method of any one of claims 1 to 7.

Images