A Binary Function Similarity Detection Method and System Based on Composite Feature Model

Abstract

This invention discloses a binary function similarity detection method and system based on a composite feature model. The method includes: Step 1: Constructing a training sample database and training a cross-architecture binary function similarity detection model based on the constructed training sample database; the binary function similarity detection model is a Siamese network structure containing two identical graph neural networks; the graph neural network is an improved Structure2vec; the improvement of Structure2vec is that when iteratively calculating the embeddings of each basic block node, only the predecessor node of the current node is used as the calculation parameter instead of the embeddings of all adjacent nodes; Step 2: Performing target binary function similarity detection based on the constructed binary function similarity detection model. This invention can solve the problem of insufficient behavioral feature description affecting detection accuracy, effectively improving the accuracy and precision of binary function similarity detection.

Description

Technical Field

[0001] This invention relates to the field of information security technology, and in particular to a binary function similarity detection method and system based on a composite feature model. Background Technology

[0002] With the widespread application of computer technology across various fields, the scale of general-purpose computer software and embedded device software is growing rapidly. To achieve rapid software development and application, developers typically shorten development cycles and reduce development difficulty through code reuse and the use of third-party libraries. However, this development approach also exacerbates some problems: first, code reuse leads to the rapid proliferation of functional modules with the same or similar vulnerabilities or malicious logic across different brands, types, and architectures of software; second, it facilitates software code theft and misappropriation. The core solution to these problems lies in code similarity analysis and detection—that is, how to accurately analyze and identify whether there is code identical or similar to a target code object (such as malicious code samples, vulnerable code samples, and copyrighted code samples) in the software to be tested.

[0003] Based on the different objects of analysis, existing code similarity detection methods can be divided into two categories: one for source code and the other for binary code. In reality, the source code of many software (fixed) systems is unavailable, and due to the diversity of software (fixed) architectures, code similarity detection for a single architecture cannot achieve good results. Therefore, cross-platform binary code similarity detection methods have become a current research hotspot. This patent is used to achieve accurate analysis and detection of cross-platform binary function similarity.

[0004] Most recent research on cross-platform binary function similarity adopts the following approach: analyzing the control-flow graph (CFG) of the binary function, and using the set of instruction statistical characteristics in each basic block of the control-flow graph and the control flow... Figure 1 The original features of the binary function are used as the initial features; then, based on graph neural network models such as Structure2vec, the original features are embedded to transform them into function feature vectors; finally, an intelligent detection model is constructed to determine whether two binary functions are similar based on the distance between the feature vectors. Although existing methods have achieved high accuracy in detecting the similarity of cross-architecture binary functions, the detection accuracy needs to be further improved due to certain limitations in the function feature description methods. Summary of the Invention

[0005] This invention addresses the issue that existing cross-platform binary function similarity detection methods have insufficient detection accuracy. It proposes a binary function similarity detection method and system based on a composite feature model, which can solve the problem of insufficient behavioral feature description affecting detection accuracy and effectively improve the accuracy and precision of binary function similarity detection.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention discloses a binary function similarity detection method based on a composite feature model, comprising:

[0008] Step 1: Construct a training sample database and train a cross-architecture binary function similarity detection model based on the constructed training sample database; the binary function similarity detection model is a Siamese network structure containing two identical graph neural networks; the graph neural network is an improved Structure2vec; the improvement of Structure2vec is that when iteratively calculating the embedding of each basic block node, only the predecessor node of the current node is used as the calculation parameter instead of the embedding of all neighboring nodes;

[0009] Step 2: Perform target binary function similarity detection based on the constructed binary function similarity detection model.

[0010] Further, step 1 includes:

[0011] Step 1.1: Build a training sample database based on open-source software or firmware;

[0012] Step 1.2: Extract the original features of the corresponding function from all samples in the training sample database;

[0013] Step 1.3: Train the binary function similarity detection model based on the original features of the corresponding functions in all samples of the extracted training sample database.

[0014] Further, step 1.1 includes:

[0015] Step 1.1.1: For open source software or firmware, cross-compile to generate binary programs for different architectures. During compilation, different optimization levels can be set, and multiple binary programs with different compilation optimization levels can be generated for each architecture.

[0016] Step 1.1.2: Randomly select two function samples with the same function name generated by the same software or firmware under different architectures or different optimization levels as similar function pairs and mark them as 1; randomly select two function bodies with different function names as dissimilar function pairs and mark them as -1.

[0017] Step 1.1.3: All labeled function pairs together constitute the training sample database.

[0018] Further, step 1.2 includes:

[0019] Step 1.2.1: For the function body in each sample database, obtain its assembly program by reverse disassembly, and analyze the function's control flow graph (CFG) based on this. The structural feature G of the function is the control flow graph.

[0020] Step 1.2.2: Analyze each basic block in the control flow graph of each function. Based on the instruction set descriptions and characteristics under different architectures, analyze the statistical characteristics of each basic block. The statistical characteristics of each basic block are represented by a vector S =<Sum,Str,Con,Tra,Log,Mat,Bit,Bra,Sub,Out> The function is represented as follows: Sum represents the total number of instructions, Str represents the number of string constants, Con represents the number of numeric constants, Tra represents the number of jump instructions, Log represents the number of logical instructions, Mat represents the number of arithmetic instructions, Bit represents the number of bit manipulation instructions, Bra represents the number of branch instructions, Sub represents the number of subroutines, and Out represents the number of out-degrees. The statistical characteristics of a function are the set of statistical characteristic vectors of all basic blocks in the function.

[0021] Step 1.2.3: Analyze each basic block in the control flow graph of each function. Based on the instruction set and function call characteristics under different architectures, analyze the function behavior characteristics in each basic block, including: determining whether there are calls to Glibc API functions in the basic block. If so, analyze and extract the name of the called function; if not, mark it as Null. Focus only on the 18 types of Glibc API functions, namely memory object copy, object comparison, memory read, memory write, memory management, data type conversion, file management, file write, file read, output operation, computation operation, program management, system management, network connection management, network data acquisition, network data transmission, execution control, and pipe communication. The function behavior characteristics are represented by an 18-dimensional vector B, where each dimension corresponds to a type of API function, and the corresponding value is the cumulative number of Glibc API functions of that type called in all basic blocks of the function.

[0022] Further, step 1.3 includes:

[0023] Step 1.3.1: Divide the function pair samples in the training sample database into two disjoint sample sets in a 4:1 ratio, and use them for model training and testing respectively;

[0024] Step 1.3.2: Select a pair of functions from the training set and feed the features of the two functions in the sample into two graph neural networks in the Siamese network. Specifically, the structural features G and statistical features S are first processed by a graph neural network, namely the improved Structure2vec, to obtain an embedding Sg containing structural relationships. Then, the structural features G and Sg are concatenated to generate a new feature vector, which is then fed into a fully connected layer (FC) for processing. The two functions obtain a final embedding E1 and E2, respectively. Similarity calculation is performed on E1 and E2.

[0025] Step 1.3.3: Calculate the L2 Loss based on the BatchSize setting, and modify the relevant parameters in the Structure2vec and FC networks based on this value through backpropagation;

[0026] Step 1.3.4: Repeat steps 1.3.2 and 1.3.3 until all samples in the training set have been trained. At this point, a detection model is obtained, and the AUC and L2 Loss of the model on the training set are recorded.

[0027] Step 1.3.5: Repeat steps 1.3.2 to 1.3.4 until the set training loop limit is reached, and save all detection models and their corresponding AUC and L2 Loss;

[0028] Step 1.3.6: Select one of the detection models from Step 1.3.5 to configure a Siamese network;

[0029] Step 1.3.7: Select a function pair sample from the test set, and feed the features of the two functions in the sample into the Siamese network respectively. The operation is the same as in step 1.3.2, and calculate the L2 Loss based on the detection similarity results.

[0030] Step 1.3.8: Repeat step 1.3.7 until every sample in the test set has been tested;

[0031] Step 1.3.9: Calculate the AUC and L2 Loss on the test set;

[0032] Step 1.3.10: Repeat steps 1.3.6 to 1.3.9 until each model in step 1.3.5 has been tested, and save the AUC and L2 Loss for each round of testing;

[0033] Step 1.3.11: Based on the test results of Step 1.3.10, optimize the number of training loops, BatchSize, and learning rate. Repeat Steps 1.3.2 to 1.3.11 until the AUC and L2 Loss on the test set meet the requirements, and record the optimal detection model obtained at this time.

[0034] Another aspect of this invention proposes a binary function similarity detection system based on a composite feature model, comprising:

[0035] The module for building and training is used to build a training sample database and train a cross-architecture binary function similarity detection model based on the built training sample database. The binary function similarity detection model is a Siamese network structure containing two identical graph neural networks. The graph neural network is an improved Structure2vec. The improvement of Structure2vec is that when iteratively calculating the embedding of each basic block node, only the predecessor node of the current node is used as the calculation parameter instead of the embedding of all neighboring nodes.

[0036] The function similarity detection module is used to perform target binary function similarity detection based on the constructed binary function similarity detection model.

[0037] Furthermore, the construction and training module includes:

[0038] The construction submodule is used to build a training sample database based on open source software or firmware;

[0039] The original feature extraction submodule is used to extract the original features of the corresponding function from all samples in the training sample database;

[0040] The training submodule is used to train a binary function similarity detection model based on the original features of the corresponding functions in all samples of the extracted training sample database.

[0041] Furthermore, the construction submodule includes:

[0042] The compilation unit is used to cross-compile open-source software or firmware to generate binary programs for different architectures. By setting different optimization levels during compilation, multiple binary programs with different compilation optimization levels can be generated for each architecture.

[0043] The function pair generation unit is used to randomly select two function samples with the same name under different architectures or different optimization levels generated by the same software or firmware compilation as similar function pair samples, marked as 1; and randomly select two function bodies with different function names as dissimilar function pairs, marked as -1.

[0044] The training sample database generation unit is used to generate the training sample database from all labeled function pairs.

[0045] Furthermore, the original feature extraction submodule includes:

[0046] The structural feature extraction unit is used to obtain the assembly program of the function body in each sample database by reverse disassembling, and then analyze the control flow graph (CFG) of the function. The structural feature G of the function is the control flow graph.

[0047] The statistical feature extraction unit analyzes each basic block in the control flow graph of each function. Based on the instruction set specifications and characteristics under different architectures, it analyzes the statistical features of each basic block. The statistical features of each basic block are represented by a vector S =<Sum,Str,Con,Tra,Log,Mat,Bit,Bra,Sub,Out> The function is represented as follows: Sum represents the total number of instructions, Str represents the number of string constants, Con represents the number of numeric constants, Tra represents the number of jump instructions, Log represents the number of logical instructions, Mat represents the number of arithmetic instructions, Bit represents the number of bit manipulation instructions, Bra represents the number of branch instructions, Sub represents the number of subroutines, and Out represents the number of out-degrees. The statistical characteristics of a function are the set of statistical characteristic vectors of all basic blocks in the function.

[0048] The behavioral feature extraction unit is used to analyze each basic block in the control flow graph of each function. Based on the instruction set and function call characteristics under different architectures, it analyzes the function behavioral features in each basic block, including: determining whether there are calls to Glibc API functions in the basic block; if so, analyzing and extracting the name of the called function; if not, marking it as null. It only focuses on 18 types of Glibc API functions, namely memory object copy, object comparison, memory read, memory write, memory management, data type conversion, file management, file write, file read, output operation, computation operation, program management, system management, network connection management, network data acquisition, network data transmission, execution control, and pipe communication. The behavioral features of the function are represented by an 18-dimensional vector B, where each dimension corresponds to a type of API function, and the corresponding value is the cumulative number of Glibc API functions of that type called in all basic blocks of the function.

[0049] Furthermore, the training submodule includes:

[0050] The sample splitting unit is used to divide the function pair samples in the training sample database into two disjoint sample sets in a 4:1 ratio, which are used for model training and testing respectively.

[0051] The function embedding generation unit selects a pair of function samples from the training set and feeds the features of the two functions in the sample into two graph neural networks in the Siamese network. Specifically, the structural features G and statistical features S are first processed by a graph neural network, namely the improved Structure2vec, to obtain an embedding Sg containing structural relationships. Then, the structural features G and Sg are concatenated to generate a new feature vector, which is then fed into a fully connected layer (FC) for processing. The two functions obtain a final embedding E1 and E2, respectively. Similarity calculation is then performed on E1 and E2.

[0052] The parameter modification unit is used to calculate the L2 Loss based on the BatchSize setting, and to modify the relevant parameters in the Structure2vec and FC networks based on this value through backpropagation.

[0053] The first loop unit is used to repeatedly execute the function embedding generation unit and the parameter modification unit until all samples in the training set have been trained. At this point, a detection model is obtained, and the AUC and L2Loss of the model on the training set are recorded.

[0054] The second loop unit is used to repeatedly execute the function embedding generation unit to the first loop unit until the set training loop count limit is reached, and to save all detection models and their corresponding AUC and L2 Loss.

[0055] The network configuration unit is used to select one of the detection models from the second loop unit to configure the twin network;

[0056] The first computational unit is used to select a function pair sample from the test set, and feed the features of the two functions in the sample into the Siamese network respectively. The operation is the same as the function embedding generation unit, and the L2 Loss is calculated based on the detection similarity results.

[0057] The third loop unit is used to repeatedly execute the first calculation unit until every sample in the test set has been tested.

[0058] The second computation unit is used to calculate the AUC and L2 Loss on the test set;

[0059] The fourth loop unit is used to repeatedly execute the network configuration unit to the second computation unit until each model in the second loop unit has been tested, and saves the AUC and L2 Loss of each round of testing;

[0060] The fifth loop unit is used to optimize the number of training loops, batch size, and learning rate based on the test results of the fourth loop unit. It repeatedly executes the function embedding generator unit to the fifth loop unit until the AUC and L2Loss on the test set meet the requirements, and records the optimal detection model obtained at this time.

[0061] Compared with the prior art, the present invention has the following beneficial effects:

[0062] 1. Compared with existing research, this invention has a stronger ability to characterize the function and behavior of binary functions, and can better detect and distinguish different binary functions with similar instruction content and CFG structural features but different functional behaviors, thus effectively improving the ability to detect the similarity of binary functions.

[0063] 2. By optimizing the embedding generation process in the Structure2vec network, predecessor nodes are used instead of neighbor nodes to participate in the node embedding update iteration. Furthermore, the function behavior feature vector is concatenated with the embeddings obtained through statistical and structural features, maximizing the role of behavior features in determining the similarity between two functions. This effectively enhances the stability and accuracy of the binary function similarity detection model. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating a binary function similarity detection method based on a composite feature model according to an embodiment of the present invention.

[0065] Figure 2 This is a schematic diagram of the architecture of a binary function similarity detection system based on a composite feature model according to an embodiment of the present invention. Detailed Implementation

[0066] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:

[0067] like Figure 1 As shown, a binary function similarity detection method based on a composite feature model includes:

[0068] Step 1: Build and train a cross-architecture binary function similarity detection model.

[0069] S101: Building a training sample database based on open-source software.

[0070] (1) For commonly used open source software, cross-compile to generate binary programs for various architectures such as ARM, MIPS, and PowerPC. During compilation, different optimization levels (O0 to O3) are set, and four binary programs with four different compilation optimization levels can be generated for each architecture.

[0071] (2) Randomly select two function samples with the same function name generated by the same software (solid) under different architectures or different optimization levels as similar function pairs and mark them as 1; randomly select two function bodies with different function names as dissimilar function pairs and mark them as -1.

[0072] (3) All labeled function pairs together constitute the training sample database.

[0073] S102: Extract the original features of the corresponding function from all samples in the training sample database.

[0074] (1) For each function body in the sample database, the assembly program is obtained by reverse disassembly, and the control flow graph (CFG) of the function is analyzed on this basis. The structural feature G of the function is the control flow graph (CFG), and G = CFG.

[0075] (2) Analyze each basic block in the control flow graph of each function. Based on the instruction set descriptions and characteristics under different architectures, analyze the statistical characteristics of each basic block, namely: total number of instructions (Sum), number of string constants (Str), number of numeric constants (Con), number of branch instructions (Tra), number of logical instructions (Log), number of arithmetic instructions (Mat), number of bit manipulation instructions (Bit), number of branch instructions (Bra), number of subroutine calls (Sub), and number of out-degrees (Out). The statistical characteristics of each basic block are represented by the vector S =<Sum,Str,Con,Tra,Log,Mat,Bit,Bra,Sub,Out> This means that the statistical characteristics of a function are the set of statistical characteristic vectors of all the basic blocks in the function.

[0076] (3) Analyze each basic block in the control flow graph of each function. Based on the instruction set and function call characteristics under different architectures, analyze the function behavior characteristics in each basic block, that is: whether there are calls to Glibc API functions in the basic block. If so, analyze and extract the name of the called function; if not, mark it as Null. Through the functional analysis of Glibc API and the analysis of typical vulnerability types, focus only on 18 types of Glibc API functions, namely memory object copy, object comparison, memory read, memory write, memory management, data type conversion, file management, file write, file read, output operation, calculation operation, program management, system management, network connection management, network data acquisition, network data transmission, execution control, and pipe communication. The behavioral characteristics of the function are represented by an 18-dimensional vector B, where each dimension corresponds to a type of API function, and the corresponding value is the cumulative number of Glibc API functions of that type called in all basic blocks of the function.

[0077] Through the above process, the original features of the function are extracted from all samples in the training sample database. The original features of the function are structural features G, statistical features S, and behavioral features B.

[0078] S103: Training of binary function similarity detection model.

[0079] (1) Divide the function pair samples in the training sample database into two disjoint sample sets in a ratio of 4:1, and use them for model training and testing respectively.

[0080] (2) Select a pair of function samples from the training set and feed the features of the two functions in the sample into two graph neural networks in the Siamese network. The function structure features G and statistical features S are first processed by a graph neural network, namely the improved Structure2vec, to obtain an embedding Sg containing structural relationships. The improvement of Structure2vec is that when iteratively calculating the embedding of each basic block node, only the predecessor node of the current node is used instead of the embedding of all neighboring nodes as the calculation parameter. Then, the structure features G and Sg are connected to generate a new feature vector and fed into the fully connected layer FC for processing. The two functions obtain a final embedding E1 and E2 respectively. The similarity between E1 and E2 is calculated, and the cosine distance is used as the metric to measure the similarity between the two function embeddings.

[0081] Specifically, in this embodiment, the embedding of the functional structure features G and statistical features S is implemented based on Structure2vec. Structure2vec is a structured data embedding algorithm that operates with a mechanism similar to the reasoning process of a graphical model. Each step performs a nonlinear function mapping, which facilitates the use of stochastic gradient descent to learn parameters and can handle large-scale datasets.

[0082] The principle of Structure2vec is to use graph g f For each node v∈V, a b-dimensional feature embedding is computed. Then, graph g is obtained through aggregation node embedding. f The embedding vector. Its mean-field inference embedding algorithm first calculates the embedding vector at each node. Initializing to 0, and updating the embeddings for all v∈V in each iteration, can be parameterized as a neural network:

[0083]

[0084] in, Let N(v) be any nonlinear function mapping, where N(v) is the set of neighboring nodes of node v in graph g. Let be the embedding of node v in the t-th iteration. According to the update formula, the embedding update process is based on the graph's topology; that is, when updating a node embedding, the attributes of its neighbors and their embeddings from the previous iteration are used. Finally, if the update terminates after T iterations, the embedding of each node... This will include the graph topology, the node characteristics involved, and the nonlinear propagation function. Determined T-hop neighborhood information.

[0085] In-depth analysis reveals that, since the control flow graph is a directed graph, and considering the sequential execution of instructions, the semantics of a statement involve a directional movement from the entry node to the exit node along the execution path. Clearly, the execution of a node in the control flow graph is influenced by its predecessor node, while the execution path of its successor nodes has no effect on that node. Therefore, we improved the original Stucture2vec model by replacing the neighboring node N(v) with the predecessor node P(v) of the current basic block node, and iteratively updating the node embedding. We parameterize the improved embedding update process as follows:

[0086]

[0087] in W1 is a b×c matrix, P i Here, represents the coefficients of the i-th fully connected layer, n is the number of fully connected layers, and GeLU is the Gaussian error linear unit activation function.

[0088] Finally, the embeddings of all nodes obtained after T iterations are aggregated and concatenated with the function-level features to generate the final graph embedding through ρ:

[0089]

[0090] Among them, z f The function-level features representing function f are represented by ρ, which we use here as a fully connected network. The input to ρ is the connection between the b-dimensional aggregated node embeddings and the d-dimensional function-level features, and the output is the final p-dimensional function embedding. The number of hidden layers and neurons are adjusted based on the model training performance and computational cost.

[0091] (3) Calculate L2 Loss based on the BatchSize setting, and modify the relevant parameters in the Structure2vec and FC networks based on this value through backpropagation.

[0092] (4) Repeat S103(2)(3) until all samples in the training set have been trained. At this point, a detection model is obtained and the AUC and L2 Loss of the model on the training set are recorded.

[0093] (5) Repeat S103(2)(3)(4) until the set training loop limit is reached (e.g., 100 times), and save all detection models and their corresponding AUC and L2 Loss.

[0094] (6) Select one of the detection models in S103(5) to configure the twin network.

[0095] (7) Select a function pair sample from the test set and feed the features of the two functions in the sample into the Siamese network, the operation is the same as S103(2), and calculate L2 Loss based on the detection similarity results.

[0096] (8) Repeat S103(7) until every sample in the test set has been tested.

[0097] (9) Calculate the AUC and L2 Loss on the test set.

[0098] (10) Repeat S103(6)-(9) until each model in S103(5) has been tested, and save the AUC and L2 Loss for each round of testing.

[0099] (11) Based on the test results of S103(10), optimize the training loop number, BatchSize, learning rate and other hyperparameters in a targeted manner, repeat S103(2)-(11) until the AUC and L2 Loss on the test set meet the requirements, and record the optimal detection model obtained at this time.

[0100] The optimal detection model obtained during the training process serves as the basis for step 2.

[0101] Step 2: Binary function similarity detection based on the detection model, that is, detecting whether there are functions in a certain software (solid) that are the same as or similar to the function of interest.

[0102] S201: Extract function bodies from the objects of interest (e.g., malicious code, vulnerable code, or copyright-protected code) and construct a database of target functions.

[0103] S202: Extract the original features of all functions in the target function database. The specific process is the same as S102.

[0104] S203: Objective function embedding generation.

[0105] (1) Configure a twin network using the optimal detection model obtained by training S103.

[0106] (2) Input the original features of each function in the target function database into the Siamese network and obtain the corresponding embedding of the function. Store the embeddings of all functions in the target function embedding database.

[0107] S204: Function embedding generation in the software (firmware) to be tested.

[0108] (1) Based on the different situations of the software (solid) to be tested, decompression, structural analysis, disassembly and other processes are adopted to extract all the function objects contained therein, and the original features are extracted using the original feature extraction module. The specific process is the same as S102.

[0109] (2) For each function in the software (firmware) to be detected, extract the function embedding by referring to the operation in S203 and store it in the function embedding database to be detected.

[0110] S205: For each target function embedding in the target function embedding database, calculate the similarity between the embedding and each function embedding in the function embedding database to be detected, and output the similarity ranking.

[0111] Based on the obtained function similarity scores and rankings, the potential function objects in the software (or fixed software) to be tested that are most similar to the target function can be quickly located. This has important reference value for further analysis to determine whether there are vulnerabilities, malicious code, or copyright infringement issues in the software (or fixed software) to be tested.

[0112] Based on the method disclosed in Example 1, Example 2 describes the application of the present invention in vulnerability detection for the CVE-2014-9471 vulnerability. The CVE-2014-9471 vulnerability exists in the parse_datetime function of Coreutils, and affected versions include 8.13, etc.

[0113] Example 2:

[0114] (1) Based on the CVE-2014-9471 vulnerability information, obtain the Coreutils 8.13 software, and extract the vulnerable parse_datetime function from its PowerPC architecture binary version with compilation optimization level O3, and use this function as the target vulnerable function.

[0115] (2) Extract the original features of the parse_datetime function. For details, refer to S102.

[0116] (3) Convert the original features of the parse_datetime function into embeddings. For details, refer to S203.

[0117] (4) To illustrate the detection process and effect, based on the Coreutils 8.13 source code, a total of 8 Coreutils 8.13 binary programs were generated under the ARM and MIPS architectures, with each architecture corresponding to four optimization levels from O0 to O3. Then, these 8 binary programs were used as the programs to be detected to check whether there were any functions similar to the target vulnerability function parse_datetime under different architectures.

[0118] (5) For each binary program, refer to the processing procedures of S204 and S205 to obtain the similarity ranking of all functions in the binary program to be detected with the target function parse_datetime.

[0119] The actual testing yielded the results shown in the table below. The numbers in the table represent the similarity ranking of the correct vulnerable function `parse_datetime` among all functions in eight Coreutils programs with different architectures and optimization levels. A higher ranking indicates greater accuracy in detecting cross-architecture binary function similarity. The test results demonstrate that this invention achieves a high level of performance in cross-architecture binary function similarity detection, effectively improving the efficiency of tasks such as vulnerability searching.

[0120]

[0121] Example 3:

[0122] Based on the above embodiments, such as Figure 2 As shown, this invention also proposes a binary function similarity detection system based on a composite feature model, comprising:

[0123] The module for building and training is used to build a training sample database and train a cross-architecture binary function similarity detection model based on the built training sample database. The binary function similarity detection model is a Siamese network structure containing two identical graph neural networks. The graph neural network is an improved Structure2vec. The improvement of Structure2vec is that when iteratively calculating the embedding of each basic block node, only the predecessor node of the current node is used as the calculation parameter instead of the embedding of all neighboring nodes.

[0124] The function similarity detection module is used to perform target binary function similarity detection based on the constructed binary function similarity detection model.

[0125] Furthermore, the construction and training module includes:

[0126] The construction submodule is used to build a training sample database based on open source software or firmware;

[0127] The original feature extraction submodule is used to extract the original features of the corresponding function from all samples in the training sample database;

[0128] The training submodule is used to train a binary function similarity detection model based on the original features of the corresponding functions in all samples of the extracted training sample database.

[0129] Furthermore, the construction submodule includes:

[0130] The compilation unit is used to cross-compile open-source software or firmware to generate binary programs for different architectures. By setting different optimization levels during compilation, multiple binary programs with different compilation optimization levels can be generated for each architecture.

[0131] The function pair generation unit is used to randomly select two function samples with the same name under different architectures or different optimization levels generated by the same software or firmware compilation as similar function pair samples, marked as 1; and randomly select two function bodies with different function names as dissimilar function pairs, marked as -1.

[0132] The training sample database generation unit is used to generate the training sample database from all labeled function pairs.

[0133] Furthermore, the original feature extraction submodule includes:

[0134] The structural feature extraction unit is used to obtain the assembly program of the function body in each sample database by reverse disassembling, and then analyze the control flow graph (CFG) of the function. The structural feature G of the function is the control flow graph.

[0135] The statistical feature extraction unit analyzes each basic block in the control flow graph of each function. Based on the instruction set specifications and characteristics under different architectures, it analyzes the statistical features of each basic block. The statistical features of each basic block are represented by a vector S =<Sum,Str,Con,Tra,Log,Mat,Bit,Bra,Sub,Out> The function is represented as follows: Sum represents the total number of instructions, Str represents the number of string constants, Con represents the number of numeric constants, Tra represents the number of jump instructions, Log represents the number of logical instructions, Mat represents the number of arithmetic instructions, Bit represents the number of bit manipulation instructions, Bra represents the number of branch instructions, Sub represents the number of subroutines, and Out represents the number of out-degrees. The statistical characteristics of a function are the set of statistical characteristic vectors of all basic blocks in the function.

[0136] The behavioral feature extraction unit is used to analyze each basic block in the control flow graph of each function. Based on the instruction set and function call characteristics under different architectures, it analyzes the function behavioral features in each basic block, including: determining whether there are calls to Glibc API functions in the basic block; if so, analyzing and extracting the name of the called function; if not, marking it as null. It only focuses on 18 types of Glibc API functions, namely memory object copy, object comparison, memory read, memory write, memory management, data type conversion, file management, file write, file read, output operation, computation operation, program management, system management, network connection management, network data acquisition, network data transmission, execution control, and pipe communication. The behavioral features of the function are represented by an 18-dimensional vector B, where each dimension corresponds to a type of API function, and the corresponding value is the cumulative number of Glibc API functions of that type called in all basic blocks of the function.

[0137] Furthermore, the training submodule includes:

[0138] The sample splitting unit is used to divide the function pair samples in the training sample database into two disjoint sample sets in a 4:1 ratio, which are used for model training and testing respectively.

[0139] The function embedding generation unit selects a pair of function samples from the training set and feeds the features of the two functions in the sample into two graph neural networks in the Siamese network. Specifically, the structural features G and statistical features S are first processed by a graph neural network, namely the improved Structure2vec, to obtain an embedding Sg containing structural relationships. Then, the structural features G and Sg are concatenated to generate a new feature vector, which is then fed into a fully connected layer (FC) for processing. The two functions obtain a final embedding E1 and E2, respectively. Similarity calculation is then performed on E1 and E2.

[0140] The parameter modification unit is used to calculate the L2 Loss based on the BatchSize setting, and to modify the relevant parameters in the Structure2vec and FC networks based on this value through backpropagation.

[0141] The first loop unit is used to repeatedly execute the function embedding generation unit and the parameter modification unit until all samples in the training set have been trained. At this point, a detection model is obtained, and the AUC and L2Loss of the model on the training set are recorded.

[0142] The second loop unit is used to repeatedly execute the function embedding generation unit to the first loop unit until the set training loop count limit is reached, and to save all detection models and their corresponding AUC and L2 Loss.

[0143] The network configuration unit is used to select one of the detection models from the second loop unit to configure the twin network;

[0144] The first computational unit is used to select a function pair sample from the test set, and feed the features of the two functions in the sample into the Siamese network respectively. The operation is the same as the function embedding generation unit, and the L2 Loss is calculated based on the detection similarity results.

[0145] The third loop unit is used to repeatedly execute the first calculation unit until every sample in the test set has been tested.

[0146] The second computation unit is used to calculate the AUC and L2 Loss on the test set;

[0147] The fourth loop unit is used to repeatedly execute the network configuration unit to the second computation unit until each model in the second loop unit has been tested, and saves the AUC and L2 Loss of each round of testing;

[0148] The fifth loop unit is used to optimize the number of training loops, batch size, and learning rate based on the test results of the fourth loop unit. It repeatedly executes the function embedding generator unit to the fifth loop unit until the AUC and L2Loss on the test set meet the requirements, and records the optimal detection model obtained at this time.

[0149] Specifically, the function similarity detection module includes:

[0150] The target function database construction submodule is used to extract function bodies from the objects of interest (such as malicious code, vulnerable code, or copyright-protected code) and construct the target function database.

[0151] The objective function original feature extraction submodule is used to extract the original features of all functions in the objective function database. The specific process is the same as that of the original feature extraction submodule.

[0152] The objective function embedding generation submodule is used for objective function embedding generation:

[0153] (1) Configure a Siamese network using the optimal detection model obtained from the training submodule;

[0154] (2) Input the original features of each function in the target function database into the Siamese network and obtain the corresponding embedding of the function. Store the embeddings of all functions in the target function embedding database.

[0155] The function embedding generation submodule is used to generate function embeddings in the software (or firmware) to be detected.

[0156] (1) Based on the different situations of the software (solid) to be detected, decompression, structural analysis, disassembly and other processing are adopted to extract all the function objects contained therein, and the original features are extracted using the original feature extraction module. The specific process is the same as the original feature extraction submodule.

[0157] (2) For each function in the software (solid) to be detected, extract the embedding of the function by referring to the operation of the target function embedding generation submodule, and store it in the function embedding database to be detected.

[0158] The function similarity calculation submodule is used to calculate the similarity between each target function embedding in the target function embedding database and each function embedding in the function embedding database to be detected, and output the similarity ranking.

[0159] In summary, compared to existing research, this invention possesses a stronger ability to characterize the functional and behavioral features of binary functions. It can better detect and distinguish different binary functions with similar instruction content and CFG structural features but different functional behaviors, effectively improving the detection capability for binary function similarity. This invention optimizes the embedding generation process in the Structure2vec network, using predecessor nodes instead of neighbor nodes to participate in the node embedding update iteration. Furthermore, it connects the function behavioral feature vector with the embeddings obtained through statistical and structural features, maximizing the role of behavioral features in determining whether two functions are similar. This effectively enhances the stability and accuracy of the binary function similarity detection model.

[0160] The above description is only a preferred embodiment of the present invention. It should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A binary function similarity detection method based on a composite feature model, characterized in that, include: Step 1: Construct a training sample database and train a cross-architecture binary function similarity detection model based on the constructed training sample database; Step 2: Perform target binary function similarity detection based on the constructed binary function similarity detection model; Step 1 includes: Step 1.1: Build a training sample database based on open-source software or firmware; Step 1.2: Extract the original features of the corresponding functions from all samples in the training sample database. The original features include the structural features, statistical features and behavioral features of the functions. The behavioral features are obtained by analyzing the calls to Glibc API functions in the functions and are represented in vector form. Step 1.3: Train the binary function similarity detection model based on the original features of the corresponding functions in all samples of the extracted training sample database; The binary function similarity detection model is a Siamese network structure containing two identical graph neural networks; the graph neural network is an improved Structure2vec; the improvement of Structure2vec is that when iteratively calculating the embedding of each basic block node, only the predecessor node of the current node is used as the calculation parameter instead of the embedding of all adjacent nodes. The training process of the model includes: inputting the structural and statistical features of the function into the graph neural network to obtain an embedding representation containing structural relationships, and fusing the embedding representation with the behavioral feature vector to generate the final function embedding vector.

2. The binary function similarity detection method based on a composite feature model according to claim 1, characterized in that, Step 1.1 includes: Step 1.1.1: For open source software or firmware, cross-compile to generate binary programs for different architectures. During compilation, different optimization levels can be set, and multiple binary programs with different compilation optimization levels can be generated for each architecture. Step 1.1.2: Randomly select two function samples with the same function name generated by the same software or firmware under different architectures or different optimization levels as similar function pairs and mark them as 1; randomly select two function bodies with different function names as dissimilar function pairs and mark them as -1. Step 1.1.3: All labeled function pairs together constitute the training sample database.

3. The binary function similarity detection method based on a composite feature model according to claim 1, characterized in that, Step 1.2 includes: Step 1.2.1: For the function body in each sample database, obtain its assembly program by reverse disassembly, and analyze the function's control flow graph (CFG) based on this. The structural feature G of the function is the control flow graph. Step 1.2.2: Analyze each basic block in the control flow graph of each function. Based on the instruction set descriptions and characteristics under different architectures, analyze the statistical characteristics of each basic block. The statistical characteristics of each basic block are represented by a vector S =<Sum, Str, Con,Tra, Log, Mat, Bit, Bra, Sub, Out> The function is represented as follows: Sum represents the total number of instructions, Str represents the number of string constants, Con represents the number of numeric constants, Tra represents the number of jump instructions, Log represents the number of logical instructions, Mat represents the number of arithmetic instructions, Bit represents the number of bit manipulation instructions, Bra represents the number of branch instructions, Sub represents the number of subroutines, and Out represents the number of out-degrees. The statistical characteristics of a function are the set of statistical characteristic vectors of all basic blocks in the function. Step 1.2.3: Analyze each basic block in the control flow graph of each function. Based on the instruction set and function call characteristics under different architectures, analyze the function behavior characteristics in each basic block, including: determining whether there are calls to Glibc API functions in the basic block. If so, analyze and extract the name of the called function; if not, mark it as Null. Focus only on the 18 types of Glibc API functions, namely memory object copy, object comparison, memory read, memory write, memory management, data type conversion, file management, file write, file read, output operation, computation operation, program management, system management, network connection management, network data acquisition, network data transmission, execution control, and pipe communication. The function behavior characteristics are represented by an 18-dimensional vector B, where each dimension corresponds to a type of API function, and the corresponding value is the cumulative number of Glibc API functions of that type called in all basic blocks of the function.

4. The binary function similarity detection method based on a composite feature model according to claim 1, characterized in that, Step 1.3 includes: Step 1.3.1: Divide the function pair samples in the training sample database into two disjoint sample sets in a 4:1 ratio, and use them for model training and testing respectively; Step 1.3.2: Select a pair of functions from the training set and feed the features of the two functions in the sample into two graph neural networks in the Siamese network. Specifically, the structural features G and statistical features S are first processed by a graph neural network, namely the improved Structure2vec, to obtain an embedding Sg containing structural relationships. Then, the structural features G and Sg are concatenated to generate a new feature vector, which is then fed into a fully connected layer (FC) for processing. The two functions obtain a final embedding E1 and E2, respectively. Similarity calculation is performed on E1 and E2. Step 1.3.3: Calculate the L2 Loss based on the BatchSize setting, and modify the relevant parameters in the Structure2vec and FC networks based on this value through backpropagation; Step 1.3.4: Repeat steps 1.3.2 and 1.3.3 until all samples in the training set have been trained. At this point, a detection model is obtained, and the AUC and L2 Loss of the model on the training set are recorded. Step 1.3.5: Repeat steps 1.3.2 to 1.3.4 until the set training loop limit is reached, and save all detection models and their corresponding AUC and L2 Loss; Step 1.3.6: Select one of the detection models from Step 1.3.5 to configure a Siamese network; Step 1.3.7: Select a function pair sample from the test set, and feed the features of the two functions in the sample into the Siamese network respectively. The operation is the same as in step 1.3.2, and calculate the L2 Loss based on the detection similarity results. Step 1.3.8: Repeat step 1.3.7 until every sample in the test set has been tested; Step 1.3.9: Calculate the AUC and L2 Loss on the test set; Step 1.3.10: Repeat steps 1.3.6 to 1.3.9 until each model in step 1.3.5 has been tested, and save the AUC and L2 Loss for each round of testing; Step 1.3.11: Based on the test results of Step 1.3.10, optimize the number of training loops, BatchSize, and learning rate. Repeat Steps 1.3.2 to 1.3.11 until the AUC and L2 Loss on the test set meet the requirements, and record the optimal detection model obtained at this time.

5. A binary function similarity detection system based on a composite feature model, characterized in that, include: The construction and training module is used to build a training sample database and train a cross-architecture binary function similarity detection model based on the built training sample database. The function similarity detection module is used to perform target binary function similarity detection based on the constructed binary function similarity detection model; The construction and training module includes: The construction submodule is used to build a training sample database based on open source software or firmware; The original feature extraction submodule is used to extract the original features of the corresponding functions in all samples in the training sample database. The original features include the structural features, statistical features and behavioral features of the functions. The behavioral features are obtained by analyzing the calls to Glibc API functions in the functions and are represented in vector form. The training submodule is used to train a binary function similarity detection model based on the original features of the corresponding functions in all samples of the extracted training sample database. The binary function similarity detection model is a Siamese network structure containing two identical graph neural networks; the graph neural network is an improved Structure2vec; the improvement of Structure2vec is that when iteratively calculating the embedding of each basic block node, only the predecessor node of the current node is used as the calculation parameter instead of the embedding of all adjacent nodes. The training process of the model includes: inputting the structural and statistical features of the function into the graph neural network to obtain an embedding representation containing structural relationships, and fusing the embedding representation with the behavioral feature vector to generate the final function embedding vector.

6. The binary function similarity detection system based on a composite feature model according to claim 5, characterized in that, The construction submodule includes: The compilation unit is used to cross-compile open-source software or firmware to generate binary programs for different architectures. By setting different optimization levels during compilation, multiple binary programs with different compilation optimization levels can be generated for each architecture. The function pair generation unit is used to randomly select two function samples with the same name under different architectures or different optimization levels generated by the same software or firmware compilation as similar function pair samples, marked as 1; and randomly select two function bodies with different function names as dissimilar function pairs, marked as -1. The training sample database generation unit is used to generate the training sample database from all labeled function pairs.

7. The binary function similarity detection system based on a composite feature model according to claim 5, characterized in that, The original feature extraction submodule includes: The structural feature extraction unit is used to obtain the assembly program of the function body in each sample database by reverse disassembling, and then analyze the control flow graph (CFG) of the function. The structural feature G of the function is the control flow graph. The statistical feature extraction unit analyzes each basic block in the control flow graph of each function. Based on the instruction set specifications and characteristics under different architectures, it analyzes the statistical features of each basic block. The statistical features of each basic block are represented by a vector S =<Sum,Str, Con, Tra, Log, Mat, Bit, Bra, Sub, Out> The function is represented as follows: Sum represents the total number of instructions, Str represents the number of string constants, Con represents the number of numeric constants, Tra represents the number of jump instructions, Log represents the number of logical instructions, Mat represents the number of arithmetic instructions, Bit represents the number of bit manipulation instructions, Bra represents the number of branch instructions, Sub represents the number of subroutines, and Out represents the number of out-degrees. The statistical characteristics of a function are the set of statistical characteristic vectors of all basic blocks in the function. The behavioral feature extraction unit is used to analyze each basic block in the control flow graph of each function. Based on the instruction set and function call characteristics under different architectures, it analyzes the function behavioral features in each basic block, including: determining whether there are calls to Glibc API functions in the basic block; if so, analyzing and extracting the name of the called function; if not, marking it as null. It only focuses on 18 types of Glibc API functions, namely memory object copy, object comparison, memory read, memory write, memory management, data type conversion, file management, file write, file read, output operation, computation operation, program management, system management, network connection management, network data acquisition, network data transmission, execution control, and pipe communication. The behavioral features of the function are represented by an 18-dimensional vector B, where each dimension corresponds to a type of API function, and the corresponding value is the cumulative number of Glibc API functions of that type called in all basic blocks of the function.

8. The binary function similarity detection system based on a composite feature model according to claim 5, characterized in that, The training submodule includes: The sample splitting unit is used to divide the function pair samples in the training sample database into two disjoint sample sets in a 4:1 ratio, which are used for model training and testing respectively. The function embedding generation unit selects a pair of function samples from the training set and feeds the features of the two functions in the sample into two graph neural networks in the Siamese network. Specifically, the structural features G and statistical features S are first processed by a graph neural network, namely the improved Structure2vec, to obtain an embedding Sg containing structural relationships. Then, the structural features G and Sg are concatenated to generate a new feature vector, which is then fed into a fully connected layer (FC) for processing. The two functions obtain a final embedding E1 and E2, respectively. Similarity calculation is then performed on E1 and E2. The parameter modification unit is used to calculate the L2 Loss based on the BatchSize setting, and to modify the relevant parameters in the Structure2vec and FC networks based on this value through backpropagation. The first loop unit is used to repeatedly execute the function embedding generation unit and the parameter modification unit until all samples in the training set have been trained. At this point, a detection model is obtained, and the AUC and L2 Loss of the model on the training set are recorded. The second loop unit is used to repeatedly execute the function embedding generation unit to the first loop unit until the set training loop count limit is reached, and to save all detection models and their corresponding AUC and L2 Loss. The network configuration unit is used to select one of the detection models from the second loop unit to configure the twin network; The first computational unit is used to select a function pair sample from the test set, and feed the features of the two functions in the sample into the Siamese network respectively. The operation is the same as the function embedding generation unit, and the L2 Loss is calculated based on the detection similarity results. The third loop unit is used to repeatedly execute the first calculation unit until every sample in the test set has been tested. The second computation unit is used to calculate the AUC and L2 Loss on the test set; The fourth loop unit is used to repeatedly execute the network configuration unit to the second computation unit until each model in the second loop unit has been tested, and saves the AUC and L2 Loss of each round of testing; The fifth loop unit is used to optimize the number of training loops, batch size, and learning rate based on the test results of the fourth loop unit. It repeatedly executes the function embedding generator unit to the fifth loop unit until the AUC and L2 Loss on the test set meet the requirements, and records the optimal detection model obtained at this time.

Images