A binary function similarity detection method and apparatus

Abstract

This invention provides a binary function similarity detection method and apparatus, comprising: preprocessing two binary functions to be detected to obtain intermediate language sequences and control flow diagrams for each binary function; extracting semantic features from the intermediate language sequences of each binary function based on a first preset neural network model to obtain semantic feature vector representations for each binary function; and extracting structural features from the control flow diagrams of each binary function based on a second preset neural network model to obtain structural feature vector representations for each binary function; and inputting the semantic feature vector representations and structural feature vector representations of each binary function into a preset multilayer perceptron classifier to obtain the similarity detection results of the two binary functions. The method of this invention effectively solves the shortcomings of existing technologies, such as the susceptibility of grammatical and structural features to interference, and improves the accuracy and robustness of binary function similarity detection.

Description

Technical Field

[0001] This invention relates to the field of computer information security technology, and in particular to a binary function similarity detection method and apparatus. Background Technology

[0002] Binary function similarity detection technology is used to find and locate the differences between two binary programs when the program is closed source. It can serve as a key technology for multiple security scenarios such as similarity vulnerability search, code plagiarism detection, and malicious code tracing, and has great research value.

[0003] Currently, existing binary function similarity detection methods suffer from technical defects such as high error rates in statistical features, susceptibility of structural features to disturbances, and lack of program execution flow variation in sequence features, resulting in low detection accuracy and poor robustness.

[0004] Therefore, how to better perform binary function similarity detection has become an urgent problem to be solved in the industry. Summary of the Invention

[0005] This invention provides a binary function similarity detection method and apparatus for better binary function similarity detection.

[0006] This invention provides a binary function similarity detection method, comprising:

[0007] The two binary functions to be detected are preprocessed separately to obtain the intermediate language sequence and control flow diagram of each binary function;

[0008] Based on the first preset neural network model, semantic features are extracted from the intermediate language sequences of each binary function to obtain semantic feature vector representations of each binary function. Based on the second preset neural network model, structural features are extracted from the control flow charts of each binary function to obtain structural feature vector representations of each binary function.

[0009] The semantic feature vector representation and structural feature vector representation of each binary function are input into a preset multilayer perceptron classifier to obtain the similarity detection results of the two binary functions.

[0010] According to an embodiment of the present invention, a binary function similarity detection method is provided, wherein semantic features are extracted from the intermediate language sequences of each binary function based on a first preset neural network model to obtain semantic feature vector representations of each binary function, including:

[0011] Based on the intermediate language sequence of each of the binary functions, determine the vector representation of each intermediate language statement in each of the intermediate language sequences;

[0012] Based on the first preset sub-neural network model and the vector representation of each intermediate language statement in each intermediate language sequence, the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence is determined;

[0013] Based on the second preset sub-neural network model and the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence, the semantic feature vector representation of each binary function is determined;

[0014] The first preset neural network model includes the first preset sub-neural network model and the second preset sub-neural network model.

[0015] According to an embodiment of the present invention, a binary function similarity detection method is provided, wherein the method extracts structural features from the control flow graphs of each binary function based on a second preset neural network model to obtain structural feature vector representations of each binary function, including:

[0016] Based on the control flow diagrams of each of the binary functions, determine the vector representation of each basic block in each of the control flow diagrams;

[0017] Based on the second preset neural network model and the vector representation of each basic block in each of the control flow diagrams, the structural feature vector representation of each of the binary functions is determined.

[0018] According to an embodiment of the present invention, a binary function similarity detection method is provided, wherein determining the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence based on a first preset sub-neural network model and the vector representation of each intermediate language statement in each intermediate language sequence includes:

[0019] The vector representation of each intermediate language statement in each intermediate language sequence is input into the first preset bidirectional long short-term memory neural network model, and the hidden state corresponding to the vector representation of each intermediate language statement in each intermediate language sequence is output.

[0020] The hidden state corresponding to the vector representation of each intermediate language sentence in each intermediate language sequence is input into a preset self-attention neural network model, and the second semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is output.

[0021] The first preset sub-neural network model includes the first preset bidirectional long short-term memory neural network model and the preset self-attention neural network model.

[0022] According to an embodiment of the present invention, a binary function similarity detection method is provided, wherein determining the semantic feature vector representation of each binary function based on a second preset sub-neural network model and the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence includes:

[0023] The second semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is input into the first preset cross-attention neural network model, and the third semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is output.

[0024] Based on the second preset bidirectional long short-term memory neural network model, the third semantic feature vector representation of each intermediate language statement in each intermediate language sequence is aggregated to obtain the semantic feature vector representation of each binary function.

[0025] The second preset sub-neural network model includes the first preset cross-attention neural network model and the second preset bidirectional long short-term memory neural network model.

[0026] According to an embodiment of the present invention, a binary function similarity detection method is provided, wherein determining the vector representation of each basic block in each control flow diagram based on the control flow diagram of each binary function includes:

[0027] Based on the control flow diagram of each of the binary functions, determine the vector representation of all intermediate language statements in each basic block of each of the control flow diagrams;

[0028] The vector representations of all intermediate language statements in each basic block of each control flow diagram are weighted and summed to obtain the vector representation of each basic block of each control flow diagram.

[0029] or,

[0030] Based on each basic block in the control flow graph of each binary function, determine the adjacency matrix and syntax feature matrix of each control flow graph;

[0031] The adjacency matrix and syntax feature matrix of each of the control flow graphs are input into a preset graph convolutional neural network model, and the vector representation of each basic block in each of the control flow graphs is output.

[0032] According to an embodiment of the present invention, a binary function similarity detection method is provided, wherein determining the structural feature vector representation of each binary function based on a second preset neural network model and the vector representation of each basic block in each control flow diagram includes:

[0033] The vector representation of each basic block in each of the control flow diagrams is input into the second preset cross-attention neural network model, and the structural feature vector representation of each basic block in each of the control flow diagrams is output.

[0034] Based on the third preset bidirectional long short-term memory neural network model, the structural feature vector representations of each basic block in each control flow diagram are aggregated to obtain the structural feature vector representations of each binary function.

[0035] The second preset neural network model includes the second preset cross-attention neural network model and the third preset bidirectional long short-term memory neural network model.

[0036] This invention also provides a binary function similarity detection device, comprising:

[0037] The preprocessing module is used to preprocess the two binary functions to be detected separately to obtain the intermediate language sequence and control flow diagram of each binary function;

[0038] The feature extraction module is used to extract semantic features from the intermediate language sequences of each binary function based on a first preset neural network model to obtain semantic feature vector representations of each binary function, and to extract structural features from the control flow diagrams of each binary function based on a second preset neural network model to obtain structural feature vector representations of each binary function.

[0039] The detection module is used to input the semantic feature vector representation and structural feature vector representation of each binary function into a preset multilayer perceptron classifier to obtain the similarity detection result of the two binary functions.

[0040] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of any of the binary function similarity detection methods described above.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the binary function similarity detection method described above.

[0042] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the binary function similarity detection methods described above.

[0043] This invention provides a binary function similarity detection method and apparatus. By preprocessing the two binary functions to be detected, intermediate language sequences and control flow diagrams for each binary function are obtained. Based on the vectorized representation of each basic block in the intermediate language sequences and control flow diagrams, a preset neural network model is used to extract semantic features from each intermediate language statement, obtaining the contextual relationships between each intermediate language statement. Structural features are also extracted from each basic block in the control flow diagram, obtaining the connection relationships between the basic blocks. Simultaneously, by constructing a cross-attention mechanism during the two types of feature extraction, the fine-grained differences between the two functions are further reflected in the embedding representations of each function, thereby obtaining semantic feature vector representations and structural feature vector representations of each binary function. These are used as input to a preset multilayer perceptron classifier, which then outputs the similarity detection results. This effectively solves the shortcomings of existing technologies where syntactic and structural features are easily interfered with and semantic features lack code execution flow information, improving the detection accuracy and robustness of binary function similarity detection. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0045] Figure 1 This is one of the flowcharts of the binary function similarity detection method provided in the embodiments of the present invention;

[0046] Figure 2 This is the second flowchart of the binary function similarity detection method provided in this embodiment of the invention;

[0047] Figure 3 This is a schematic diagram of the binary function similarity detection device provided in an embodiment of the present invention;

[0048] Figure 4 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0049] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0050] The following is combined with Figures 1-4 The present invention describes a binary function similarity detection method and apparatus.

[0051] Figure 1 This is one of the flowcharts illustrating the binary function similarity detection method provided in this embodiment of the invention, such as... Figure 1 As shown, it includes steps 101, 102 and 103.

[0052] In step 101, the two binary functions to be detected are preprocessed to obtain the intermediate language sequence and control flow diagram of each binary function.

[0053] It should be noted that binary (code) programs refer to executable programs obtained after preprocessing, compilation, assembly, and linking processes from programs written in high-level languages. Currently, binary program detection typically uses functional functions as the smallest granularity, and the degree and location of similarity between two binary programs are determined by comprehensively analyzing the detection results between functional functions.

[0054] The binary functions described in the embodiments of this invention refer to the functional functions in a binary program.

[0055] The intermediate language sequence and control flow diagram described in this embodiment of the invention are two commonly used intermediate representations of binary functions. The intermediate representation refers to the source program's appearance between the end of syntax analysis and the generation of the target code.

[0056] A control flow graph is an abstract representation of a process or program. It is an abstract data structure used in the compiler and maintained internally. It represents all the paths that a program will traverse during execution. Each node in the control flow graph represents a basic block.

[0057] Specifically, considering that binary code from the same source code can differ significantly in instruction sets, code offsets, and calling conventions due to compilation on different platforms and architectures, the semantic features of the binary function's intermediate language statement sequence are extracted to achieve the universality of the method of this invention.

[0058] In this embodiment, the two binary functions to be detected are preprocessed separately. After removing all library functions, the Angr tool is used to extract two intermediate representations of the binary functions. One is an intermediate language sequence of the binary function divided into basic blocks, which has a certain semantic correspondence with the program source code. The other is the control flow graph of the binary function, which reflects the overall execution logic of the function. Functions with similar graph structures often have similar functions.

[0059] Compared to other tools, Angr is based on the Python platform, is simple and easy to use, facilitates integration with subsequent code processed by this invention, and has relatively complete documentation; the intermediate language syntax can use VEX-IR statements, which is an architecture-independent and side-effect-free language that supports a large number of target machine languages. It abstracts the differences between several different architectures, including register names, memory access, memory segmentation, instruction side effects, etc., allowing for unified analysis on top of these differences.

[0060] Step 102: Based on the first preset neural network model, semantic features are extracted from the intermediate language sequences of each binary function to obtain the semantic feature vector representation of each binary function. Based on the second preset neural network model, structural features are extracted from the control flow diagram of each binary function to obtain the structural feature vector representation of each binary function.

[0061] The first preset neural network model described in the embodiments of the present invention refers to a preset neural network model that extracts and represents semantic features of the intermediate language sequences of each binary function.

[0062] The second preset neural network model described in this embodiment of the invention refers to a preset neural network model that extracts and represents the structural features of the control flow diagram of each binary function.

[0063] The semantic feature vector representation of the binary function described in the embodiments of the present invention refers to the feature vector representation that can characterize the contextual semantic relationship between each intermediate language statement in the intermediate language sequence of the binary function and the semantic interaction features between functions.

[0064] The structural feature vector representation of a binary function described in this embodiment of the invention refers to a feature vector representation that can characterize the relationship between the basic blocks in a binary function and the structural interaction features between functions.

[0065] In this embodiment, a cross-attention mechanism is constructed in both types of preset neural network models during feature extraction, thereby ensuring that the fine-grained differences between the two binary functions can be reflected in the embedded representation of each function.

[0066] In some embodiments, step 102, based on a first preset neural network model, extracts semantic features from the intermediate language sequences of each binary function to obtain semantic feature vector representations of each binary function, including:

[0067] Based on the intermediate language sequences of each binary function, determine the vector representation of each intermediate language statement in each intermediate language sequence;

[0068] Based on the first preset sub-neural network model and the vector representation of each intermediate language statement in each intermediate language sequence, the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence is determined;

[0069] Based on the second preset sub-neural network model and the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence, the semantic feature vector representation of each binary function is determined;

[0070] The first preset neural network model includes a first preset sub-neural network model and a second preset sub-neural network model.

[0071] In an embodiment of the present invention, after extracting the intermediate language sequence of the binary function using the Angr tool, the present invention proposes a proprietary method to obtain the vector representation of a single intermediate language sentence in the intermediate language sequence, which is similar to the word embedding model in Natural Language Processing (NLP). The vector distance between intermediate language sentences with similar contexts is also similar.

[0072] In this embodiment, based on the FastText structure and the word embedding model of random walk, the intermediate language sequence of each binary function is vectorized to determine the vector representation of each intermediate language statement in each intermediate language sequence.

[0073] Specifically, in this embodiment, since the connection relationship between basic blocks can reflect the context semantics of the actual execution of the function, the function control flow graph is first converted into a path sequence composed of different execution paths by randomly walking on the function basic blocks. Each execution path is set to cover 4 basic blocks, which can ensure that enough context execution flow information is obtained.

[0074] In this embodiment, since it is impossible to extract all paths, the walk stops when all basic blocks appear in at least two random walks. Finally, this set of path sequences is used as the dataset for training the unsupervised intermediate language sentence embedding model, where each intermediate language sentence is considered a "word" and each path is considered a "sentence" composed of multiple "words".

[0075] Furthermore, in order to train the model, this embodiment adopts the FastText structure. In order to overcome the problem of loss of internal morphological information of words in the traditional word2vec model structure, FastText uses character-level N-grams to represent a word. This will be better for generating word vectors for low-frequency words, because their N-grams can be shared with other words. In addition, word vectors can still be constructed for words outside the training vocabulary.

[0076] In this embodiment, we assume an intermediate language statement sequence l1, l2, ..., l t The initial vectors are represented as v1, v2, ..., v t The training objective is to maximize the probability function P(l), as shown in the following formula:

[0077]

[0078] Where w is the size of the sliding window, and C is the number of statements in the sliding window. Represents the sequence l corresponding to time t within the sliding window. t The initial vector representation.

[0079] Since each intermediate language statement is represented as an assignment expression, the above embedding model does not consider the order of the two sides of the expression when predicting statement embeddings. For example, given two statements "GET:I bit(offset=24)=tv" and "tv=GET:I bit(offset=48)", the embedding model will assume that the first one is closer to "tv=GET:I bit(offset=24)" in vector representation than the second one.

[0080] To address the aforementioned issues, this embodiment extracts the N-gram sub-vectors of each sentence obtained during the training of the intermediate language sentence embedding model to construct a new dictionary. For each intermediate language sentence, its embedding representation is formed by concatenating the two sub-vectors generated on both sides of the assignment expression. This effectively solves the problem and improves the model's performance.

[0081] Therefore, based on the above process, the vector representation of each intermediate language statement in each intermediate language sequence can be obtained.

[0082] The first preset neural network model described in this embodiment of the invention includes a first preset sub-neural network model and a second preset sub-neural network model. The first preset sub-neural network model is used to learn the long-distance contextual relationship between each intermediate language statement in the intermediate language sequence, and the second preset sub-neural network model is used to learn the relationship between each intermediate language statement.

[0083] The second semantic feature vector representation described in this embodiment of the invention refers to the semantic feature vector representation obtained after extracting the semantic features of the long-distance association relationship of the vector representation of each intermediate language statement in each intermediate language sequence.

[0084] Furthermore, the vector representation of each intermediate language statement in each intermediate language sequence is input into the first preset sub-neural network model, and the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence is output. Then, the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence is input into the second preset sub-neural network model for semantic feature extraction and representation, and the semantic feature vector representation of each binary function can be obtained.

[0085] In embodiments of the present invention, the intermediate language sequences of each binary function are vectorized using the FastText structure and a random walk word embedding model. Then, based on the first preset sub-neural network model and the second preset sub-neural network model, the contextual relationships between each intermediate language statement are learned layer by layer to strengthen the correlation between the high-dimensional vector representations of similar functions, which is beneficial to improving the accuracy of subsequent function semantic feature similarity detection.

[0086] It is understandable that using only the semantic features of binary functions for similarity analysis has certain limitations and may reduce the accuracy of similarity matching. Therefore, in this embodiment of the invention, the relationship between the basic blocks in the binary function control flow graph is also considered as a structural feature used in similarity analysis.

[0087] In some embodiments, step 102, based on the second preset neural network model, extracts structural features from the control flow charts of each binary function to obtain structural feature vector representations of each binary function, including:

[0088] Based on the control flow diagrams of each binary function, determine the vector representation of each basic block in each control flow diagram;

[0089] Based on the second preset neural network model and the vector representation of each basic block in each control flow diagram, the structural feature vector representation of each binary function is determined.

[0090] Specifically, in embodiments of the present invention, determining the vector representation of each basic block in each control flow diagram can be implemented in two specific ways.

[0091] The first implementation method is to determine the vector representation of all intermediate language statements in each basic block of each control flow diagram based on the control flow diagram of each binary function.

[0092] The vector representations of all intermediate language statements in each basic block of each control flow diagram are weighted and summed to obtain the vector representation of each basic block of each control flow diagram.

[0093] In this embodiment, after extracting the control flow diagrams of each binary function, based on the aforementioned implementation method for determining the vector representation of intermediate language statements, the vector representations of all intermediate language statements in each basic block of each control flow diagram can be obtained.

[0094] Furthermore, a weighted summation is performed on the vector representations of all intermediate language statements in each basic block of the control flow diagram, and the vector representation obtained by the weighted summation is used as the vector representation of that basic block.

[0095] The second implementation method is to determine the adjacency matrix and syntax feature matrix of each control flow graph based on each basic block in the control flow graph of each binary function.

[0096] The adjacency matrix and syntax feature matrix of each control flow graph are input into a pre-defined graph convolutional neural network model, which outputs a vector representation of each basic block in each control flow graph.

[0097] Specifically, traditional methods for extracting and representing structural features are generally based on graph embedding algorithms such as matrix factorization or random walks to extract low-dimensional dense vector representations of graph nodes (basic blocks). However, these algorithms are computationally complex on large-scale graphs, typically cannot cover all node connections, and only focus on node associations, failing to consider the features of the nodes themselves. Therefore, in this embodiment, a pre-defined Graph Convolution Network (GCN) model is used to fuse syntactic and structural features at the level of function basic blocks.

[0098] In this embodiment, it is assumed that the input binary function control flow graph has N basic block nodes. The initial features of the basic blocks are vectorized from common syntactic features of the basic blocks obtained based on statistics. The selected syntactic features are shown in Table 1. Let the features of these nodes form an N×20 matrix X, and the relationships between the nodes form an N×N adjacency matrix A. X and A are the inputs to the GCN model. GCN is also a neural network layer, and its propagation between layers is shown in the following formula:

[0099]

[0100] in, This represents the matrix obtained by adding the identity matrix I to matrix A. Let A represent the degree matrix, H represent the feature vector of each layer (for the input layer, H is X), W represent the trainable weighted matrix, and σ represent the non-linear activation function.

[0101] Table 1

[0102] Serial Number Feature Name Serial Number Feature Name 1 CPU instructions 11 Number of Xor instructions 2 Number of different temporary variables 12 And the number of instructions 3 Number of PUT commands 13 Number of Cmp commands 4 Number of GET commands 14 Number of Shl commands 5 Number of Add commands 15 Number of Shr commands 6 Number of Sub instructions 16 Number of Sar commands 7 Number of Mul commands 17 Number of ST instructions 8 Number of Div commands 18 Number of Clz commands 9 Number of LD instructions 19 Number of NOT commands 10 Or number of commands 20 Number of if statements

[0103] Therefore, by inputting the adjacency matrix and syntax feature matrix of each control flow diagram into the preset GCN model, the vector representation of each basic block in each control flow diagram can be obtained.

[0104] In this embodiment, the vector representation of each basic block contains information about the adjacent basic blocks of the function, reflecting the connection relationships between nodes. Thus, the structural features of the binary function are extracted by referencing the semantic features of the basic blocks.

[0105] In the embodiments of the present invention, by analyzing each basic block in the control flow diagram of each binary function, a vector representation of each basic block in the control flow diagram is obtained. This ensures that the vector representation of each basic block contains information about the adjacent basic blocks of the function, which helps to improve the accuracy of subsequent binary function structural feature similarity analysis.

[0106] In this embodiment, the second preset neural network model can be used to learn the interaction features between each basic block of one control flow diagram and each basic block of another control flow diagram.

[0107] Furthermore, after obtaining the vector representation of each basic block in each control flow diagram, the vector representation of each basic block in each control flow diagram is input into the second preset neural network model in this invention for structural feature extraction and representation, thereby obtaining the structural feature vector representation of each binary function.

[0108] The method of this invention obtains the semantics of each fused basic block in the control flow diagram, inputs it into a second preset neural network model for structural feature extraction and representation, so as to further learn the interaction features between the two functions, fully explore the interaction of local feature relationships between the two functions, and improve the accuracy of binary function similarity detection.

[0109] Step 103: Input the semantic feature vector representation and structural feature vector representation of each binary function into a preset multilayer perceptron classifier to obtain the similarity detection results of the two binary functions.

[0110] Specifically, after completing the above embodiments, vector representations with fixed lengths of semantic features and structural features of two binary functions to be detected can be obtained, that is, semantic feature vector representations and structural feature vector representations of each binary function are obtained.

[0111] In this embodiment, it is assumed that the two binary functions are fa and fb respectively, and the semantic feature vector representation and structural feature vector representation of function fa are respectively The semantic feature vector representation and structural feature vector representation of function fb are respectively

[0112] Input the semantic feature vector representation and structural feature vector representation of each binary function into the constructed preset multi-layer perceptron classifier to evaluate the similarity degree of the two binary functions.

[0113]

[0114] P(y丨fa,fb) = sigmoid(ReLU(WI + b));

[0115] Among them, P(y丨fa,fb) represents the similarity degree of the two binary functions, W represents the trainable weighted matrix, and b represents a constant.

[0116] As shown in the above formula, send I to the fully connected layer in the preset multi-layer perceptron classifier, the activation function used is sigmoid, and the obtained result is sent to the softmax layer to output the similarity score. If the two binary functions are more similar, the score is closer to 1, otherwise it is closer to 0.

[0117] The method of the embodiment of the present invention preprocesses the two binary functions to be detected respectively to obtain the intermediate language sequences and control flowcharts of each binary function. Based on vectorizing each basic block in the intermediate language sequences and control flowcharts, and based on a preset neural network model, extract the semantic features of each intermediate language statement, obtain the context association relationship between each intermediate language statement, and extract the structural features of each basic block in the control flowchart to obtain the connection relationship between each basic block in the control flowchart; at the same time, by constructing a cross-attention mechanism in the two types of feature extraction processes, further reflect the fine-grained differences between the two functions in the embedding representations of each function, so as to obtain the semantic feature vector representations and structural feature vector representations of each binary function as the input of the preset multi-layer perceptron classifier, and then through the preset multi-layer perceptron classifier, output the similarity detection result, which can effectively solve the existing technical defects that the syntax and structural features are easily interfered and the semantic features lack code execution flow information, and improve the detection accuracy and robustness of binary function similarity detection.

[0118] In some embodiments, based on a first preset sub-neural network model and the vector representation of each intermediate language statement in each intermediate language sequence, a second semantic feature vector representation of each intermediate language statement in each intermediate language sequence is determined, including:

[0119] Input the vector representation of each intermediate language statement in each intermediate language sequence into the first preset bidirectional long short-term memory neural network model, and output the hidden state corresponding to the vector representation of each intermediate language statement in each intermediate language sequence.

[0120] The hidden state corresponding to the vector representation of each intermediate language sentence in each intermediate language sequence is input into a preset self-attention neural network model, and the second semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is output.

[0121] The first preset sub-neural network model includes a first preset bidirectional long short-term memory neural network model and a preset self-attention neural network model.

[0122] Specifically, the first preset sub-neural network model described in the embodiments of the present invention includes a first preset bidirectional long short-term memory (LSTM) neural network model and a preset self-attention neural network model. The first preset LSTM neural network model and the preset self-attention neural network model are both pre-set neural network models used to extract the long-distance correlation between each intermediate language statement in the intermediate language sequence of each binary function.

[0123] Furthermore, the vector representation of each intermediate language statement in each intermediate language sequence is input into the first preset bidirectional LSTM neural network model, and the hidden state corresponding to the vector representation of each intermediate language statement in each intermediate language sequence is output.

[0124] In this embodiment, it is assumed that v i It is the initial intermediate language statement l i The vector representation of h i-1 This represents the hidden state output by the first preset bidirectional LSTM neural network model Bi-LSTM at time i. These hidden states generated by Bi-LSTM at each time step can be regarded as semantic feature vectors representing the corresponding intermediate language sentences containing contextual information, as shown below:

[0125]

[0126] To further address the long-term dependency issue between statements, this embodiment of the invention employs a self-attention mechanism at this layer, namely a pre-defined self-attention neural network model, which effectively enhances the extraction of information within the function. In this embodiment, the input to the pre-defined self-attention neural network model is a bidirectional LSTM hidden state representation h. i Attention matrix A is used to capture the degree of correlation between adjacent statements and the current statement;

[0127] The hidden state corresponding to the vector representation of each intermediate language sentence in each intermediate language sequence is input into a pre-defined self-attention neural network model to obtain the second semantic feature vector representation of each intermediate language sentence in each intermediate language sequence. The specific calculation formula is as follows:

[0128]

[0129] in, This indicates that two hidden states h have been captured. i and h i′ The relationship between them, σ represents the nonlinear activation function, W t and W x W represents the weight matrix corresponding to these two hidden states. a b represents the weight matrix corresponding to the nonlinear combination of these two hidden states. t and b a This represents the deviation vector.

[0130] Furthermore, the second semantic feature vector of each intermediate language statement after adding self-attention can be represented as:

[0131]

[0132] The method of this invention extracts semantic features from each intermediate language statement in the intermediate language sequence by employing a first preset bidirectional long short-term memory neural network model and a preset self-attention neural network model. This effectively obtains the long-distance association representation between each intermediate language statement in the intermediate language sequence of each binary function, which is beneficial to improving the accuracy and robustness of subsequent semantic feature similarity detection of two binary functions.

[0133] In some embodiments, the semantic feature vector representation of each binary function is determined based on the second preset sub-neural network model and the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence, including:

[0134] The second semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is input into the first preset cross-attention neural network model, and the third semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is output.

[0135] Based on the second preset bidirectional long short-term memory neural network model, the third semantic feature vector representation of each intermediate language statement in each intermediate language sequence is aggregated to obtain the semantic feature vector representation of each binary function.

[0136] The second preset sub-neural network model includes the first preset cross-attention neural network model and the second preset bidirectional long short-term memory neural network model.

[0137] Specifically, the third semantic feature vector representation described in the embodiments of the present invention refers to the feature vector representation of the semantic relationship between statements at the intermediate language statement level after obtaining the second semantic feature vector representation of the intermediate language statement.

[0138] The second preset sub-neural network model includes a first preset cross-attention neural network model and a second preset bidirectional LSTM neural network model. The first preset cross-attention neural network model and the second preset bidirectional LSTM neural network model are both preset neural network models. They are used to learn the correlation between each intermediate language statement in each binary function after obtaining the second semantic feature vector representation of each intermediate language statement, and obtain the semantic feature vector representation of multi-feature fusion.

[0139] In this embodiment, a cross-attention mechanism, namely a first preset cross-attention neural network model, is first added at the intermediate language statement level to learn the correlation between each statement of the two binary functions. Assume... For intermediate language statements in function fa i The vector representation of , For intermediate language statements in function fb j The vector representation of , The cross-attention weight matrix between the two intermediate language statements is shown in the following formula:

[0140]

[0141] Assuming function fa has n intermediate language statements and function fb has m intermediate language statements, based on the obtained cross-attention weight matrix, the intermediate language statements in function fa are represented as a weighted sum of the associated intermediate language statements in function fb, thus increasing local inference. The specific formula is as follows:

[0142]

[0143] Similarly, intermediate language statements in function fb are also represented using the corresponding method, as shown in the following formula:

[0144]

[0145] In this way, we obtain the weighted embedded representation of the intermediate language statement.

[0146] Next, in an embodiment of the present invention, to further enhance the interaction between each statement of the two functions, correlation analysis is performed to determine the differences between the two statements. Specifically, the original features and weighted features of the intermediate language statements in functions fa and fb are subtracted and multiplied bitwise, as shown in the following formula:

[0147]

[0148]

[0149] That is, we obtain the third semantic feature vector representation of each intermediate language statement in each intermediate language sequence.

[0150] In embodiments of this invention, researchers have found that, compared to attention mechanisms in other existing methods, this method focuses more on fine-grained semantic similarity between function pairs. Before mapping each function to its final vector representation, the method of this invention can capture the interaction features between two functions.

[0151] Furthermore, based on the second preset bidirectional LSTM neural network model, the third semantic feature vector representation of each intermediate language statement in each intermediate language sequence is aggregated to obtain the semantic feature vector representation of each binary function.

[0152] In this embodiment, a bidirectional LSTM network layer is added to aggregate all intermediate language statement context local inference information of the input function. That is, through the second preset bidirectional long short-term memory neural network model, the output of all hidden states in the network is used. They are considered as the final semantic feature vector representations at the intermediate language statement level, and are merged and stored in a sequence.

[0153] Since the vector lengths obtained for different functions are different, in order to facilitate the final similarity analysis, in this embodiment of the invention, the hidden state output sequence is subjected to max pooling and average pooling, as shown in the following formula:

[0154]

[0155]

[0156] Furthermore, the output of the last node in the second preset LSTM neural network model can be regarded as the semantic feature of the entire function. The pooled value of the above statement sequence vector and the semantic feature vector of the entire function are concatenated into a fixed-length vector, which serves as the semantic feature vector representation I of the binary function.semantic ,Right now

[0157]

[0158] Furthermore, the semantic feature vector representations of the two binary functions can be input into a pre-defined multilayer perceptron classifier for similarity detection.

[0159] The method of this invention uses a first preset cross-attention neural network model and a second preset bidirectional long short-term memory neural network model to further analyze the second semantic feature vector representation of each intermediate language statement. This effectively learns richer cross-level interaction features between the semantic features of the two functions, performs joint reasoning, and finally integrates the multi-level granular interaction features between the two functions. The similarity of binary functions is calculated in an end-to-end manner, improving the accuracy and robustness of binary function semantic feature similarity detection.

[0160] In some embodiments, based on a second preset neural network model and the vector representation of each basic block in each control flow diagram, the structural feature vector representation of each binary function is determined, including:

[0161] The vector representation of each basic block in each control flow diagram is input into the second preset cross-attention neural network model, and the structural feature vector representation of each basic block in each control flow diagram is output.

[0162] Based on the third preset bidirectional long short-term memory neural network model, the structural feature vector representations of each basic block in each control flow diagram are aggregated to obtain the structural feature vector representations of each binary function.

[0163] The second preset neural network model includes a second preset cross-attention neural network model and a third preset bidirectional long short-term memory neural network model.

[0164] Specifically, the second preset neural network model described in the embodiments of the present invention includes a second preset cross-attention neural network model and a third preset bidirectional LSTM neural network model, wherein the second preset cross-attention neural network model and the third preset bidirectional LSTM neural network model are both preset neural network models;

[0165] In this embodiment, the second preset cross-attention neural network model is used to effectively learn the interaction features between each node (basic block) of a control flow graph and the cross-graph nodes of another control flow graph. This not only considers the information aggregation of edges in a single graph, but also considers the cross-graph matching vector, thereby obtaining the structural feature vector representation of each basic block in the control flow graph. The third preset bidirectional LSTM neural network model is used to aggregate the structural feature vector representations of each basic block to obtain the final structural feature vector representation of the entire function control flow graph.

[0166] In this embodiment, it is assumed that V i a Basic block B in the control flow diagram Ga of function fa i The vector representation of , The basic block B in the control flow graph Gb of function fb j The vector representation of is then: in, This represents the cross-attention weight matrix between two basic functional blocks.

[0167] Assume Ga has n basic block nodes and Gb has m basic block nodes; and These are the vector representations of each basic block node in Ga and Gb after considering the cross-attention distribution, as shown in the following formulas:

[0168]

[0169]

[0170] Furthermore, a positional multiplication operation is performed on the original features and weighted features of the basic block nodes to reflect the differences between the two control flow graphs. The resulting interaction features of the basic block nodes are used as the structural feature vector representation of the basic block, as shown in the following formula:

[0171]

[0172]

[0173] Among them, W m This represents a trainable weighted matrix.

[0174] Based on the second preset cross-attention neural network model, the structural feature vector representation of each basic block in each control flow diagram can be obtained.

[0175] Furthermore, based on the third preset bidirectional LSTM neural network model, the structural feature vector representations of each basic block in each control flowchart are aggregated, that is, the last two hidden states in both directions are concatenated into a fixed-length vector, which serves as the structural feature vector representation I of the entire binary function control flowchart. structural ,but:

[0176]

[0177] Furthermore, the structural feature vector representations of the two binary functions can be input into a pre-defined multilayer perceptron classifier for similarity detection.

[0178] The method of this invention uses a second preset cross-attention neural network model and a third preset bidirectional long short-term memory neural network model to perform in-depth analysis on the vector representation of each basic block, further learn richer cross-level interaction features between the structural features corresponding to the two functions, and perform joint inference to improve the accuracy and robustness of binary function structural feature similarity detection.

[0179] Figure 2 This is a second schematic flowchart of the binary function similarity detection method provided in this embodiment of the invention, as shown below. Figure 2 As shown, the process includes function preprocessing, semantic feature analysis and extraction, structural feature analysis and extraction, function similarity detection, and generation of detection results.

[0180] Among them, function preprocessing is to preprocess the two binary functions to be detected by deleting library functions and using the Angr tool to extract the intermediate language sequence and function control flow graph of each binary function.

[0181] Semantic feature analysis and extraction involves obtaining semantic feature vector representations of two binary functions through sentence vectorization and sequence vectorization, and by adding a cross-function attention mechanism to the neural network.

[0182] Structural feature analysis and extraction, namely, through the vectorization representation of (basic block) nodes and graph vectorization representation, and by adding a cross-function cross-attention mechanism in the neural network, obtains the structural feature vector representation of two binary functions;

[0183] Function similarity detection involves inputting the semantic feature vector representations and structural feature vector representations of the two binary functions obtained above into a pre-built multilayer perceptron classifier for similarity detection, and obtaining a similarity score between the two binary functions.

[0184] The detection results are generated by inferring the similarity between two binary functions based on the proportion of similar function pairs in the two binary functions.

[0185] The method of this invention determines the similarity between programs by analyzing the similarity between the functions of two binary functions. It extracts semantic and structural features from the intermediate language representation and control flow diagram of the functions, and adds a cross-function attention mechanism to the neural network to perform joint reasoning on the function pairs, forming a multi-feature fusion feature matrix. This better reveals the differences between the two binary functions. By inputting the extracted multi-feature fusion feature matrix into a constructed multilayer perceptron classifier, the output value is used as the basis for determining the similarity of functions, which can effectively improve the accuracy and robustness of binary function similarity detection.

[0186] The binary function similarity detection device provided by the present invention will be described below. The binary function similarity detection device described below and the binary function similarity detection method described above can be referred to in correspondence.

[0187] Figure 3 This is a schematic diagram of the binary function similarity detection device provided in an embodiment of the present invention, as shown below. Figure 3 As shown, it includes:

[0188] The preprocessing module 301 is used to preprocess the two binary functions to be detected separately to obtain the intermediate language sequence and control flow diagram of each binary function;

[0189] The feature extraction module 302 is used to extract semantic features from the intermediate language sequences of each binary function based on the first preset neural network model, to obtain the semantic feature vector representation of each binary function, and to extract structural features from the control flow diagram of each binary function based on the second preset neural network model, to obtain the structural feature vector representation of each binary function.

[0190] The detection module 303 is used to input the semantic feature vector representation and structural feature vector representation of each binary function into a preset multilayer perceptron classifier to obtain the similarity detection results of the two binary functions.

[0191] The apparatus described in this embodiment can be used to execute the above method embodiments, and its principle and technical effects are similar, so they will not be repeated here.

[0192] The binary function similarity detection device provided in this invention preprocesses the two binary functions to be detected to obtain intermediate language sequences and control flow diagrams for each binary function. Based on the vectorized representation of each basic block in the intermediate language sequences and control flow diagrams, a preset neural network model is used to extract semantic features of each intermediate language statement to obtain the contextual relationships between each intermediate language statement. Structural features are also extracted from each basic block in the control flow diagram to obtain the connection relationships between the basic blocks. Simultaneously, by constructing a cross-attention mechanism during the two types of feature extraction, the fine-grained differences between the two functions are further reflected in the embedding representations of each function, thereby obtaining the semantic feature vector representation and structural feature vector representation of each binary function. These are used as input to a preset multilayer perceptron classifier, which then outputs the similarity detection result. This effectively solves the existing technical defects of susceptibility of grammatical and structural features to interference and lack of code execution flow information in semantic features, improving the detection accuracy and robustness of binary function similarity detection.

[0193] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communication interface 420, and the memory 430 communicate with each other through the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute the binary function similarity detection method provided by the above methods. The method includes: preprocessing the two binary functions to be detected to obtain intermediate language sequences and control flow diagrams of each binary function; extracting semantic features from the intermediate language sequences of each binary function based on a first preset neural network model to obtain semantic feature vector representations of each binary function; extracting structural features from the control flow diagrams of each binary function based on a second preset neural network model to obtain structural feature vector representations of each binary function; and inputting the semantic feature vector representations and structural feature vector representations of each binary function into a preset multilayer perceptron classifier to obtain similarity detection results of the two binary functions.

[0194] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0195] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the binary function similarity detection method provided by the above methods. The method includes: preprocessing two binary functions to be detected to obtain intermediate language sequences and control flow diagrams of each binary function; extracting semantic features from the intermediate language sequences of each binary function based on a first preset neural network model to obtain semantic feature vector representations of each binary function; and extracting structural features from the control flow diagrams of each binary function based on a second preset neural network model to obtain structural feature vector representations of each binary function; and inputting the semantic feature vector representations and structural feature vector representations of each binary function into a preset multilayer perceptron classifier to obtain similarity detection results of the two binary functions.

[0196] On another front, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the binary function similarity detection method provided by the methods described above. This method includes: preprocessing two binary functions to be detected to obtain intermediate language sequences and control flow diagrams for each binary function; extracting semantic features from the intermediate language sequences of each binary function based on a first preset neural network model to obtain semantic feature vector representations for each binary function; and extracting structural features from the control flow diagrams of each binary function based on a second preset neural network model to obtain structural feature vector representations for each binary function; and inputting the semantic feature vector representation and structural feature vector representation of each binary function into a preset multilayer perceptron classifier to obtain similarity detection results for the two binary functions. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0197] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0198] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A binary function similarity detection method, characterized in that, include: The two binary functions to be detected are preprocessed separately to obtain the intermediate language sequence and control flow diagram of each binary function; Based on the first preset neural network model, semantic features are extracted from the intermediate language sequences of each binary function to obtain the semantic feature vector representation of each binary function, including: Based on the intermediate language sequences of each of the binary functions, the vector representation of each intermediate language statement in each intermediate language sequence is determined, including: randomly walking on the basic blocks of each of the binary functions to convert the function flowcharts of each of the binary functions into path sequences composed of different execution paths; each execution path is composed of multiple intermediate language statements; using the set of path sequences to train an embedding model of the FastText structure to obtain the vector representation of each intermediate language statement output by the embedding model; the vector representation is formed by connecting two sub-vectors generated on both sides of the equal sign of the assignment expression represented by the corresponding intermediate language statement; Based on the second preset neural network model, structural features are extracted from the control flow diagrams of each binary function to obtain structural feature vector representations of each binary function; the structural feature vector representations are used to characterize the correlation between each basic block in a single binary function and the structural interaction features between functions. The semantic feature vector representation and structural feature vector representation of each binary function are input into a preset multilayer perceptron classifier to obtain the similarity detection results of the two binary functions.

2. The binary function similarity detection method according to claim 1, characterized in that, The step of extracting semantic features from the intermediate language sequences of each binary function based on the first preset neural network model to obtain the semantic feature vector representation of each binary function further includes: Based on the first preset sub-neural network model and the vector representation of each intermediate language statement in each intermediate language sequence, the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence is determined; Based on the second preset sub-neural network model and the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence, the semantic feature vector representation of each binary function is determined; The first preset neural network model includes the first preset sub-neural network model and the second preset sub-neural network model.

3. The binary function similarity detection method according to claim 1, characterized in that, The step of extracting structural features from the control flow graphs of each binary function based on the second preset neural network model to obtain structural feature vector representations of each binary function includes: Based on the control flow diagrams of each of the binary functions, determine the vector representation of each basic block in each of the control flow diagrams; Based on the second preset neural network model and the vector representation of each basic block in each of the control flow diagrams, the structural feature vector representation of each of the binary functions is determined.

4. The binary function similarity detection method according to claim 2, characterized in that, The determination of the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence based on the first preset sub-neural network model and the vector representation of each intermediate language statement in each intermediate language sequence includes: The vector representation of each intermediate language statement in each intermediate language sequence is input into the first preset bidirectional long short-term memory neural network model, and the hidden state corresponding to the vector representation of each intermediate language statement in each intermediate language sequence is output. The hidden state corresponding to the vector representation of each intermediate language sentence in each intermediate language sequence is input into a preset self-attention neural network model, and the second semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is output. The first preset sub-neural network model includes the first preset bidirectional long short-term memory neural network model and the preset self-attention neural network model.

5. The binary function similarity detection method according to claim 2, characterized in that, The determination of the semantic feature vector representation of each binary function based on the second preset sub-neural network model and the second semantic feature vector representation of each intermediate language statement in each intermediate language sequence includes: The second semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is input into the first preset cross-attention neural network model, and the third semantic feature vector representation of each intermediate language sentence in each intermediate language sequence is output. Based on the second preset bidirectional long short-term memory neural network model, the third semantic feature vector representation of each intermediate language statement in each intermediate language sequence is aggregated to obtain the semantic feature vector representation of each binary function. The second preset sub-neural network model includes the first preset cross-attention neural network model and the second preset bidirectional long short-term memory neural network model.

6. The binary function similarity detection method according to claim 3, characterized in that, The control flow diagram based on each of the binary functions determines the vector representation of each basic block in each of the control flow diagrams, including: Based on the control flow diagram of each of the binary functions, determine the vector representation of all intermediate language statements in each basic block of each of the control flow diagrams; The vector representations of all intermediate language statements in each basic block of each control flow diagram are weighted and summed to obtain the vector representation of each basic block of each control flow diagram. or, Based on each basic block in the control flow graph of each binary function, determine the adjacency matrix and syntax feature matrix of each control flow graph; The adjacency matrix and syntax feature matrix of each of the control flow graphs are input into a preset graph convolutional neural network model, and the vector representation of each basic block in each of the control flow graphs is output.

7. The binary function similarity detection method according to claim 3, characterized in that, The determination of the structural feature vector representation of each binary function based on the second preset neural network model and the vector representation of each basic block in each of the control flow diagrams includes: The vector representation of each basic block in each of the control flow diagrams is input into the second preset cross-attention neural network model, and the structural feature vector representation of each basic block in each of the control flow diagrams is output. Based on the third preset bidirectional long short-term memory neural network model, the structural feature vector representations of each basic block in each control flow diagram are aggregated to obtain the structural feature vector representations of each binary function. The second preset neural network model includes the second preset cross-attention neural network model and the third preset bidirectional long short-term memory neural network model.

8. A binary function similarity detection device, characterized in that, The method for performing binary function similarity detection as described in claim 1 includes: The preprocessing module is used to preprocess the two binary functions to be detected separately to obtain the intermediate language sequence and control flow diagram of each binary function; The feature extraction module is used to extract semantic features from the intermediate language sequences of each binary function based on a first preset neural network model to obtain semantic feature vector representations of each binary function, and to extract structural features from the control flow diagrams of each binary function based on a second preset neural network model to obtain structural feature vector representations of each binary function. The detection module is used to input the semantic feature vector representation and structural feature vector representation of each binary function into a preset multilayer perceptron classifier to obtain the similarity detection result of the two binary functions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the binary function similarity detection method as described in any one of claims 1 to 6.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the binary function similarity detection method as described in any one of claims 1 to 6.

Images