AST-based enhanced binary code similarity detection method and system

Abstract

The invention relates to an AST-based enhanced binary code similarity detection method and system, and the method comprises the steps: carrying out the traversal of an abstract syntax tree through a depth-first algorithm, screening out nodes which play a key role in a program control flow and a logic structure from the abstract syntax tree, constructing AST for the key nodes obtained through screening, and obtaining AST semantic features; extracting control flow characteristics of the binary code based on a jump perception mechanism, and extracting behavior characteristics of the binary code according to a function execution microtrace; carrying out feature fusion on the AST semantic features, the control flow features and the behavior features by adopting a channel attention mechanism; and detecting the similarity of the binary codes by adopting the fused features. According to the invention, a new hybrid architecture model SemASTer is provided, and the Tree-LSTM and the Transform are integrated, so that the understanding of deep semantics in the AST is enhanced; and a channel attention mechanism is introduced, so that a grammatical feature-based method and a behavior feature-based method can be efficiently combined.

Description

Technical Field

[0001] The present invention relates to the field of computers, and particularly to a method and system for enhancing binary code similarity detection based on AST. Background Art

[0002] Binary Code Similarity Detection (BSCD) is a technology that determines the similarity degree between different codes by analyzing features such as the instruction sequence and control flow graph of binary files. This technology has wide applications in multiple fields, including vulnerability mining, reverse engineering, malicious code family classification, etc. With the popularization of Internet of Things (IoT) devices and the increasing diversification of firmware, the number of binary programs is constantly rising, and at the same time, binary analysis tasks are becoming increasingly complex and diverse. Therefore, it is particularly important to develop a binary code similarity detection solution with stronger scalability and higher accuracy.

[0003] Binary similarity detection can be divided into three categories according to the types of extracted features: syntax feature-based methods, behavior feature-based methods, and syntax and behavior feature-based methods. 1) Syntax feature-based methods mainly convert binary code into structures such as control flow graphs and data flow graphs, and then compare and analyze these structures to extract syntax features for similarity detection, which can comprehensively analyze program code. However, since the program does not need to be run, there are cases where the true behavior of the code cannot be understood, resulting in the loss of some important behavior information. 2) Behavior feature-based methods obtain the execution information of the program by running the binary code in a specific environment, and then extract behavior features for similarity detection, which has strong anti-obfuscation ability. However, behavior features depend on specific inputs and environments, and unexecuted paths or uncovered behaviors may be ignored. 3) Syntax and behavior feature-based methods attempt to combine the advantages of the two methods, obtaining a global view through syntax features while also being able to obtain local semantic features through dynamic behaviors to improve the accuracy and robustness of binary code similarity detection.

[0004] However, binary code is usually optimized and obfuscated during the compilation process, making it difficult to extract feature information; in addition, there are differences in the representation methods of syntactic features and behavioral features, making it difficult to effectively fuse these two types of information. Specifically, there are the following two main challenges: First, binary code is usually optimized and obfuscated during the compilation process, making it difficult to extract high-level semantic information from machine code. Existing syntax feature-based solutions cannot fully obtain the semantic information of binary code. Building a model on the control flow graph (CFG), however, over-reliance on the CFG will affect the learning of semantics. Asteria uses Tree-LSTM to encode the AST to extract semantic information. However, the capabilities of Tree-LSTM are often limited to the tree structure itself and it is difficult to effectively capture long-distance dependencies between nodes. Therefore, how to extract comprehensive syntactic features to fully obtain semantic information is a challenge. Second, due to the differences in the representation methods of syntactic features and behavioral features, it is difficult to directly fuse the two types of information. Existing methods based on syntactic and behavioral features cannot effectively combine the information of both. A certain number of candidate functions are selected through syntactic features, and then dynamic analysis is used to extract behavioral features for refined analysis. However, the fusion method of this approach is relatively coarse-grained and usually only performs simple splicing or merging at the last stage, resulting in some feature information being ignored or having unbalanced weights. Therefore, how to effectively combine the information of both in the overall process remains a challenge. Summary of the Invention

[0005] In view of the above problems, in the first aspect of the present invention, a method for enhancing binary code similarity detection based on AST is provided, and the method includes the following steps:

[0006] Use a depth-first algorithm to traverse the abstract syntax tree, screen out the nodes that play a key role in the program control flow and logical structure, and build an AST for the selected key nodes to obtain AST semantic features; extract the control flow features of the binary code based on the jump-aware mechanism, and extract the behavioral features of the binary code according to the function execution microtrace.

[0007] Use a channel attention mechanism to fuse the AST semantic features, control flow features, and behavioral features.

[0008] Use the fused features to detect the similarity of binary code.

[0009] Preferably, the obtaining of the AST semantic features is specifically:

[0010] Input each node in the AST into the Tree-LSTM model to obtain node embedding vectors.

[0011] Add the root node of the AST to the beginning of the sequence, and perform position encoding for each node according to the node positions recorded during depth-first search.

[0012] After fusing the node embedding vectors and position encodings, obtain the AST semantic features through the Transformer model.

[0013] Preferably, the control flow features of the binary code are extracted based on the jump awareness mechanism, specifically as follows:

[0014] Decompose the mnemonics and operands of the binary code into independent tokens, uniformly replace string variables and constant values with preset special tokens, and retain the names and labels of external function calls.

[0015] Assign unique tokens to the source and target of each jump instruction.

[0016] Obtain the control flow features using the processed binary code.

[0017] Preferably, the behavior features of the binary code are extracted according to the function execution microtrace, specifically as follows:

[0018] Through micro-execution, simulate function execution under unconstrained input, extract the dynamic behavior patterns at the instruction level, and generate embedding vectors that can accurately map the semantics of function execution.

[0019] Preferably, the channel attention mechanism is adopted to fuse the AST semantic features, control flow features, and behavior features, specifically as follows:

[0020] Use weight matrices W1, W2, and W3 to transform the AST semantic feature T s , control flow feature T c , and behavior feature T e respectively:

[0021] E1 = T s W1

[0022] E2 = T c W2

[0023] E3 = T e W3

[0024] Calculate the weights for each channel using the formula:

[0025]

[0026] where, W att is the weight matrix of the attention layer, b att is the bias vector, E i represents the feature representation of the i-th channel, αi is the attention weight of the i-th channel.

[0027] Multiply the features of each channel by their corresponding attention weights, and aggregate the weighted features to form a feature fusion result:

[0028]

[0029] where α i E i represents the product of the features of the i-th channel and its attention weight, and T final is the sum of all channel weighted features, that is, the feature fusion result.

[0030] In the second aspect of the present invention, a system for detecting binary code similarity based on AST enhancement is provided. The system includes the following modules:

[0031] A feature extraction module that traverses the abstract syntax tree using a depth-first algorithm, filters out the nodes that play a key role in the program control flow and logical structure, constructs an AST for the filtered key nodes to obtain AST semantic features; extracts the control flow features of the binary code based on a jump-aware mechanism, and extracts the behavior features of the binary code according to the function execution microtrace.

[0032] A feature fusion module that performs feature fusion on AST semantic features, control flow features, and behavior features using a channel attention mechanism.

[0033] A similarity detection module that detects the similarity of binary code using the fused features.

[0034] Preferably, the obtaining of the AST semantic features is specifically as follows:

[0035] Input each node in the AST into a Tree-LSTM model to obtain node embedding vectors;

[0036] Add the root node of the AST to the beginning of the sequence, and perform position encoding for each node according to the node positions recorded during depth-first search.

[0037] After fusing the node embedding vectors and position encodings, obtain AST semantic features through a Transformer model.

[0038] Preferably, the extraction of the control flow features of the binary code based on the jump-aware mechanism is specifically as follows:

[0039] Decompose the mnemonics and operands of the binary code into independent tokens, uniformly replace string variables and constant values with preset special tokens, and at the same time retain the names and labels of external function calls.

[0040] A unique label is assigned to the source and target of each jump instruction.

[0041] The control flow features are obtained using the processed binary code.

[0042] Preferably, the behavior features of the binary code are extracted according to the function execution microtrace, specifically:

[0043] By means of micro-execution, the function execution is simulated under unconstrained input, the dynamic behavior patterns at the instruction level are extracted, and embedding vectors that can accurately map the function execution semantics are generated.

[0044] Preferably, the feature fusion of the AST semantic features, control flow features, and behavior features is performed using the channel attention mechanism, specifically:

[0045] Weight matrices W1, W2, and W3 are used to transform the AST semantic feature T s , control flow feature T c , and behavior feature T e respectively:

[0046] E1 = T s W1

[0047] E2 = T c W2

[0048] E3 = T e W3

[0049] The weights of each channel are calculated using the formula:

[0050]

[0051] where W att is the weight matrix of the attention layer, b att is the bias vector, E i represents the feature representation of the i-th channel, and α i is the attention weight of the i-th channel;

[0052] The features of each channel are multiplied by their corresponding attention weights, and the weighted features are aggregated to form the feature fusion result:

[0053]

[0054] where α i E i represents the product of the features of the i-th channel and its attention weight, and T final is the sum of all channel weighted features, i.e., the feature fusion result.

[0055] In a third aspect of the present invention, there is provided a computer-readable storage medium, and a computer program stored on the readable storage medium, when executed by a processor, implements the method described in the first aspect.

[0056] The present invention designs a new hybrid architecture model, SemASTer, which integrates Tree-LSTM and Transformer to enhance the understanding of deep semantics in the AST. Moreover, a channel attention mechanism is introduced, which can efficiently combine the method based on syntactic features and the method based on behavioral features. At the same time, it can dynamically adjust the contribution of the outputs of different models to ensure that the most relevant features are highlighted in different situations, further improving the detection effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a flowchart of the first embodiment;

[0058] Figure 2 It is the overall flowchart of SeMASTer;

[0059] Figure 3 It is the decompiled code and the corresponding AST under the x86 architecture;

[0060] Figure 4 It is the overall process of the SemFCA model. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] In the embodiments of the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner for easy understanding.

[0062] It can be understood that the "embodiments" mentioned throughout the specification mean that specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, the various embodiments mentioned throughout the specification do not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in one or more embodiments in any suitable manner. It can be understood that in various embodiments of the present application, the magnitude of the sequence numbers of the various processes does not mean the order of execution, and the execution order of the various processes should be determined by their functions and internal logics, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0063] In the present invention, unless otherwise specified, the same or similar parts among various embodiments may be referred to each other. In each embodiment of the present invention, as well as in each implementation manner / implementation method / realization method in each embodiment, if there is no special specification and logical conflict, the terms and / or descriptions among different embodiments, as well as among the various implementation manners / implementation methods / realization methods in each embodiment, are consistent and can be mutually referred to. The technical features in different embodiments, as well as in the various implementation manners / implementation methods / realization methods in each embodiment, can be combined to form new embodiments, implementation manners, implementation methods, or realization methods according to their inherent logical relationships. The implementation manners described below in this application do not constitute a limitation on the protection scope of this application.

[0064] Specific Embodiment 1, as Figure 1 shown, provides a method for enhancing binary code similarity detection based on AST. The method includes the following steps:

[0065] S1, Traverse the abstract syntax tree using the depth-first algorithm, and screen out the nodes that play a key role in the program control flow and logical structure. For the screened key nodes, construct an AST to obtain the AST semantic features; extract the control flow features of the binary code based on the jump awareness mechanism, and extract the behavior features of the binary code according to the function execution microtrace.

[0066] Traverse the abstract syntax tree (AST) using the depth-first algorithm, and screen out the nodes that play a key role in the program control flow and logical structure, such as if, return, for, and while. During the screening process, not only identify these key nodes, but also record their depth information in the AST and the relative positional relationship between the nodes, and generate position encoding based on this. For the screened key nodes, use the Tree-LSTM model to generate embedding vectors for them. Then, add the embedding vector of each key node to its corresponding position encoding element by element to obtain the final input vector representation. To effectively capture the long-distance dependence relationship between key nodes, use the Transformer model to process these input vectors. The multi-layer attention mechanism of the Transformer can deeply understand the complex semantic information contained in the AST structure, and improve the ability to parse the program semantics.

[0067] To deeply understand the semantic information contained in the AST, the present invention designs a hybrid architecture SemASTer, as Figure 2As shown, the advantages of the Tree-LSTM and Transformer models are integrated. In one embodiment, the SemASTer architecture includes an AST key node traversal module that obtains key nodes by traversing the AST using DFS; a fusion module that includes two branches. The first branch obtains an embedding vector through Tree-LSTM, and the second branch is a position encoding unit that obtains a position encoding containing depth information and relative position information between nodes, and then fuses the first branch and the second branch; and a Transformer module. While using the Tree-LSTM model to process tree-structured data, it can also use the Transformer to handle long-range dependencies between AST nodes.

[0068] For binary code, the extraction of the AST usually needs to be achieved through reverse engineering techniques. Specifically, the reverse engineering tool IDA Pro is used to convert the binary code into an intermediate representation form, which facilitates the extraction of AST nodes. After extracting the AST, the AST is traversed by depth-first search (DFS) to identify and mark those nodes that play key roles in the program control flow and logic, such as if, return, for, and while nodes. These key nodes are closely related to the logical structure and function implementation of the program and are of great significance for binary code similarity detection. During the traversal process, the positions of the key nodes in the tree, that is, the depths of the nodes, are recorded as the depth encodings of the nodes; at the same time, the positional relationships between the key nodes and their parent nodes and child nodes are recorded as the relative position encodings of the nodes. The depth encoding and the relative position encoding are combined to form the final position representation of each node. After obtaining the position encoding, an embedding vector is generated for the key nodes, and the Tree-LSTM model performs well in processing tree structures.

[0069] To enable AST nodes to be processed by deep learning models, these nodes are encoded into a format that the model can process. The AST is converted into the Left-Child Right-Sibling (LCRS) format. In the LCRS format, each node is represented as a triple (node value, left child index, right sibling index). Furthermore, the hierarchical structure of the AST is converted into a linear sequence while preserving the parent-child and sibling relationships between nodes. This conversion is crucial for the Tree-LSTM model because it allows the model to consider its context information, including parent nodes, child nodes, and sibling nodes, when processing nodes. Figure 3 Shows the process of AST construction.

[0070] After constructing the AST, the AST semantic features are obtained, specifically:

[0071] Input each node in the AST into the Tree-LSTM model to obtain the node embedding vector.

[0072] Add the root node of the AST to the beginning of the sequence, and perform position encoding for each node according to the node positions recorded during depth-first search.

[0073] After fusing the node embedding vector and the position encoding, obtain the AST semantic features through the Transformer model.

[0074] Input the preprocessed AST into the Tree-LSTM model. Tree-LSTM is a variant of the recurrent neural network (RNN) that is particularly suitable for processing tree-structured data. It captures the hierarchical relationships between nodes by recursively updating the representation of each node while considering the information of its child nodes and parent nodes.

[0075] Let \(T = \langle V, E\rangle\) be an AST, where \(V\) is the set of nodes and \(E\) is the set of edges. Each node \(v\in V\) will be mapped to a representation \(h\) in a \(d -\)dimensional vector space. v The Tree-LSTM model recursively updates the representation of each node through the following formula:

[0076]

[0077] where the LSTM cell contains components such as input gate, forget gate, output gate, and cell state, \(h\) v (t) represents the hidden state of node \(v\) at time step \(t\), \(v\) left and \(v\) right represent the left child node and right child node of node \(v\) respectively, and \(v\) parent represents the parent node of node \(v\).

[0078] During the training process, the Tree-LSTM model will generate a fixed-size vector representation for each node. The vector representation not only captures the information of the node itself but also integrates the information of its child nodes and parent nodes, thus providing rich context information. Among these nodes, the root node, as the starting point for traversing the entire code structure, directly or indirectly contains all child nodes and is crucial for semantic analysis. In addition, some nodes (such as if, return, for, and while) play important roles in the program control flow and logical structure. Therefore, select the hidden states of these nodes to represent the AST.

[0079] To effectively capture the complex dependencies between selected root nodes and key nodes, in one embodiment, a Transformer model based on the RoBERTa architecture is adopted. The self-attention mechanism in the Transformer model can capture the dependencies between various positions in the sequence and is particularly good at handling long-range dependencies. The semantic information of binary code is distributed among multiple nodes in the AST, and the dependencies between these nodes are intricate and difficult for traditional models to effectively model. However, Transformer can globally model these dependencies, enabling the model to more accurately understand the code semantics.

[0080] RoBERTa (Robustly Optimized BERT Pretraining Approach) is an improved version of BERT. By using a larger dataset and longer training time, it enhances the model's ability to learn code structure information. In the binary code analysis task, the AST has a complex hierarchical structure. If the training time is insufficient, it is difficult for the model to learn sufficiently accurate semantic representations. Therefore, the optimization of RoBERTa on large-scale data can enhance the model's understanding of AST semantics. In addition, RoBERTa adopts a dynamic masking strategy, that is, different masks are applied to the input in different training epochs, thereby improving the model's generalization ability. In AST processing, the representations of nodes are highly structured, and the same code structure may undergo minor but important changes under different compilation optimization levels or different compilers. The dynamic masking strategy enables the model to learn more robust AST semantic representations in different contexts, thus enhancing its adaptability across optimizations and compilers.

[0081] To better represent the hierarchical position relationships of different nodes in the AST, the position encoding in RoBERTa is modified. Since the root node contains the overall syntactic structure information of the AST, the root node vector is added to the beginning of the sequence, allowing the model to refer to the global information of the AST when processing subsequent nodes. Subsequently, according to the node positions recorded during depth-first search, position encoding is performed for each node, enabling the model to more accurately understand the hierarchical relationships and relative positions between nodes, thereby further enhancing the effect of semantic feature extraction.

[0082] After completing the integration of the sequence, the next step is to input this sequence into the Transformer model. Specifically, for a given binary function f = [x0,…,x k ,…,x n , x0 represents the root node, and x k represents the k-th selected node. The nodes will be converted into embedding vectors {h(x0),…,h(xk ), …, h(x n ). Then, the embedded vector is summed with the position embedding.

[0083] v(x k ) = h(x k ) + p(x k )

[0084] where p(x k ) is the position embedding of the k-th node, and then the mixed embedded vector {v(x0), …, v(x k ), …, v(x n )} is obtained. The multi-head attention mechanism is applied to the mixed embedded vector. The embedding of the m-th layer is represented as v m = {v m (x0), …, v m (x k ), …, v m (x n )}. First, the m-th embedding is projected onto Q m , K m and V m respectively. Then, the attention matrix is obtained by using the scaled dot-product attention

[0085]

[0086]

[0087] where is the transformation matrix of the m-th layer, d emb is the dimension of the embedded vector, and is the attention weight matrix. Let d represent the updated embedding.

[0088]

[0089] Assume the model has N attention heads, and the updated embedding is obtained as follows v m+1 , is the output transformation matrix of the m-th layer, and FFN m is the feed-forward network of the m-th layer.

[0090]

[0091] The final output of the SemASTer model is the last layer of the model, and the AST feature is named T s .

[0092] In one embodiment, extracting the control flow features of the binary code based on the jump awareness mechanism is specifically as follows:

[0093] Decompose the mnemonics and operands of the binary code into independent tokens, uniformly replace string variables and constant values with preset special tokens, and at the same time retain the names and labels of external function calls.

[0094] Assign unique tokens to the source and target of each jump instruction.

[0095] Obtain control flow features using the processed binary code.

[0096] To fully exploit the control flow information in the binary code, the present invention introduces a jump-aware mechanism. Under this mechanism, the source and target of each jump instruction are assigned unique tokens, and parameter sharing is implemented inside the model, enabling the model to accurately capture the direct associations between jump instructions, even if these instructions are far apart in the code sequence. Specifically, convert the source token of the jump instruction (such as the jump target address) into a token in a specific format (e.g., JUMP_XX), where XX corresponds to the sequence number of the jump target in the code. This conversion strategy endows the model with strong adaptability, enabling it to still keenly perceive the subtle changes in the jump relationships in the face of code changes caused by compiler optimizations.

[0097] Before formally applying the jump-aware mechanism, perform preprocessing and tokenization operations on the binary code using the IDAPro tool. This step includes decomposing mnemonics and operands into independent tokens, uniformly replacing string variables and constant values with preset special tokens, and at the same time retaining the names and labels of external function calls to ensure that the model can accurately identify the key information in the code.

[0098] Finally, obtain binary code embedding vectors containing rich control flow features using the pre-trained model provided by jTrans. These embedding vectors not only accurately reflect the control flow structure of the code but also lay the foundation for subsequent binary code analysis tasks, enabling the model to gain a deeper insight into the internal logic and potential associations of the code.

[0099] In one embodiment, extracting the behavioral features of the binary code according to the function execution microtrace is specifically as follows:

[0100] Through micro-execution, simulate the function execution under unconstrained input, extract the dynamic behavior patterns at the instruction level, and generate embedding vectors that can accurately map the semantics of the function execution.

[0101] To more accurately extract the behavioral features of binary code, the present invention introduces the function execution micro-trace (Micro-Trace Execution) method to capture the execution semantics of binary functions at the dynamic behavior level. The core of this method is to simulate function execution through micro-execution technology under the condition of under-constrained input, so as to extract the dynamic behavior patterns at the instruction level and generate embedding vectors that can accurately map the function execution semantics.

[0102] Specifically, the Unicorn CPU emulator is used to perform micro-execution on the binary function. First, the function to be analyzed is loaded into the memory-mapped simulation environment, and all general-purpose registers and stack spaces are initialized. Subsequently, the function is executed in the simulation environment, and the dynamic execution information of each instruction (such as register value changes, memory access patterns) is recorded and composed into function micro-traces. These micro-traces include information such as instruction sequences and the register states and memory operation values before and after their execution, effectively capturing the dynamic execution characteristics of the function while avoiding the large input generation cost required for real dynamic execution.

[0103] After obtaining the micro-trace data, it is input into the pre-trained model in the Trex framework for in-depth processing. When the micro-trace data flows through the model, the model will fully mobilize its internal neural network structure to comprehensively analyze and integrate key information such as the instruction sequence, register change trajectory, and memory operation pattern in the micro-traces. After layers of abstraction and transformation, the model finally outputs high-quality function behavior feature embedding vectors.

[0104] S2. Feature fusion of AST semantic features, control flow features, and behavior features is performed using the channel attention mechanism;

[0105] In binary code similarity detection, the main methods can be divided into methods based on syntactic features, methods based on behavior features, and methods based on syntactic and behavior features. The present invention comprehensively utilizes the hierarchical structure information, control flow information, and behavior feature information of the AST, and introduces the channel attention mechanism, which dynamically quantifies the importance weights of each feature channel to achieve focused fusion of key features. In one embodiment, the feature fusion of AST semantic features, control flow features, and behavior features using the channel attention mechanism is specifically as follows:

[0106] The weight matrices W1, W2, and W3 are used to transform the AST semantic feature T s , control flow feature T c , and behavior feature T e respectively, that is, dimension adjustment:

[0107] E1 = T s W1

[0108] E2 = T c W2

[0109] E3 = T e W3

[0110] Among them,

[0111] d s 、d c 、d e are the dimensions of the features.

[0112] The weights of each channel are calculated using the formula:

[0113]

[0114] Among them, W att is the weight matrix of the attention layer, b att is the bias vector, E i represents the feature representation of the i-th channel, and α i is the attention weight of the i-th channel.

[0115] Multiply the features of each channel by their corresponding attention weights, and aggregate the weighted features to form a feature fusion result:

[0116]

[0117] Among them, α i E i represents the product of the features of the i-th channel and its attention weight, and T final is the sum of the weighted features of all channels, that is, the feature fusion result.

[0118] S3, use the fused features to detect the similarity of binary codes.

[0119] In one embodiment, after obtaining the fused feature representation, a similarity metric method is used to calculate the similarity between two binary codes. The similarity metric method includes, but is not limited to, cosine similarity, Euclidean distance, or training a neural network to be able to distinguish between similar and dissimilar binary code pairs.

[0120] In another embodiment, it further includes a fine-tuning process, and the model needs to be fine-tuned to be applicable to the similarity detection task. The goal of the SemFCA model is to detect whether a given pair of functions is similar. The model is trained to maximize the similarity between similar binary function pairs and minimize the similarity between irrelevant function pairs. After obtaining the sum of the weighted features of the function passing through all channels, cosine similarity is used to calculate the function similarity.

[0121] Among them, as Figure 4 shown, the SemFCA model at least includes a function embedding network, and the function embedding network includes a multi-channel feature extraction module. The multi-channel feature extraction module includes an AST information channel for extracting an Abstract Syntax Tree (AST) from binary code; a control flow information channel for extracting a Control Flow Graph (CFG) and converting the CFG into a feature vector; and a behavior information channel for extracting behavioral features of the code, such as function calls, system calls, etc., and converting the behavioral features into a feature vector. The function embedding network further includes a channel attention mechanism module, and the channel attention mechanism module includes a dimension adjustment unit, a channel weight calculation unit, and a feature fusion unit. The channel attention mechanism module is connected behind the multi-channel feature extraction module.

[0122] Specifically, as Figure 4 shown, for any query function q, a function f + similar to it is selected as a positive sample (i.e., compiled from the same function). At the same time, a function f - unrelated to q is randomly selected as a negative sample. The embedding of the function is denoted as E q , and the triplet loss function is used. α is a hyperparameter representing the distance difference between the positive and negative samples, and its value ranges from 0 to 0.5. By minimizing the loss function, the cosine distance of the embedding representation is calculated to measure the similarity between two functions.

[0123] To comprehensively evaluate the effectiveness of the binary code similarity detection method, a dataset including ClamAV, Curl, Nmap, OpenSSL, Unrar, Z3, and Zlib is used for effectiveness judgment. The library files of each open-source project are compiled using four main versions of two major compiler families, GCC and Clang. Each library file is also compiled for three different architectures, x86-64, ARM, and MIPS, including two modes, 32-bit and 64-bit, and five different optimization levels (O0, O1, O2, O3, and Os).

[0124] Given a query function, it is necessary to find related homologous functions in function pools with different numbers. Mean Reciprocal Rank (MRR) and Recall@1 are selected as evaluation metrics. MRR measures the ranking of the positive sample of the query sample in the returned result list. Recall@k measures the proportion of the model that can correctly identify the positive sample among the top k results returned. In the present invention, Recall@1 is adopted.

[0125]

[0126] To comprehensively evaluate the accuracy of the model, the experiment was divided into 4 tasks for evaluation: 1) Cross-Architecture (XA): Function pairs have different architectures and bit widths, but use the same compiler, compiler version, and optimization. 2) Cross-Compiler (XC): Function pairs have different compilers, compiler versions, and optimizations, but use the same architecture and bit width. 3) Cross-Compilation Option (XO): Function pairs have different optimizations, but use the same compiler, compiler version, and architecture. 4) Cross-Compilation Option and Cross-Compiler (XM): Function pairs come from any combination of architecture, bit width, compiler, compiler version, and optimization.

[0127] The following methods were selected as the baseline methods:

[0128] 1) Asteria-pro: This method is an improvement of the Asteria method, which introduces domain knowledge by designing a pre-filtering module and a reordering module. The pre-filtering module uses lightweight syntax features to filter out non-homologous functions, thereby reducing computational overhead. Subsequently, the reordering module enhances the sorting of potentially vulnerable functions based on function call relationships.

[0129] 2) jTrans: This method uses an innovative jump-aware representation, enhancing its ability to understand function control flow and improving binary code representation learning. jTrans outperforms existing state-of-the-art methods in binary code similarity detection tasks, especially when dealing with large-scale function pools. This method only supports the x86 architecture.

[0130] 3) Trex: This method performs similarity detection by learning the execution semantics of binary functions. It extends existing micro-execution techniques and uses a hierarchical Transformer network structure to efficiently process this information.

[0131] 4) GMN: This method is a method based on graph neural networks (GNNs), aiming to solve the similarity learning problem of graph-structured objects. The advantage of GMN is that it can consider both structural and semantic similarities during graph comparison.

[0132] 5) SAFE: This method is based on self-attention neural networks. By modeling the semantic information of instruction sequences, it can directly generate embedding vectors for disassembled binary functions without manually extracting features or constructing control flow graphs.

[0133] The source codes of the above baseline methods are all publicly available, and the experiments are conducted according to the given original parameters.

[0134] In the experiments of XO, XA, XC, and XM, 1,000 functions were selected as query functions, and 1,000 functions were randomly selected from the corresponding compilation settings to construct a function pool. Since jTrans is a binary code similarity detection method for x86 settings, XO and XC function retrieval tasks were additionally set on x86.

[0135] Table 1 Results of the comparative experiments in the XA, XO, XC, XM, X86-XO, and X86-XC tasks (query pool is 1000). The measurement criteria are MRR / Recall@1

[0136] XA XO XC XM X86-XO X86-XC SemFCA 0.747 / 0.681 0.634 / 0.583 0.621 / 0.564 0.543 / 0.466 0.671 / 0.628 0.642 / 0.593 Asteria-Pro 0.654 / 0.598 0.442 / 0.385 0.423 / 0.346 0.401 / 0.352 0.426 / 0.363 0.407 / 0.348 Trex 0.213 / 0.145 0.413 / 0.364 0.391 / 0.338 0.195 / 0.154 0.425 / 0.373 0.402 / 0.346 SAFE 0.086 / 0.067 0.176 / 0.143 0.162 / 0.121 0.053 / 0.041 0.132 / 0.095 0.156 / 0.112 GMN 0.561 / 0.497 0.408 / 0.332 0.374 / 0.293 0.322 / 0.273 0.372 / 0.297 0.357 / 0.265 jTrans -- -- -- -- 0.589 / 0.546 0.563 / 0.509

[0137] As shown in Table 1, BSAE-FCA achieved the best performance in all tasks, significantly outperforming the existing baseline methods. Specifically, in the XA task, compared with the current state-of-the-art Asteria-Pro, Recall@1 increased by 13.4%. This is because SemFCA, while fully extracting the semantic information rich in AST, uses function execution microtraces to obtain cross-architecture behavior characteristics, thereby enhancing the model's cross-architecture adaptability. In the X86-XO and X86-XC tasks, compared with the advanced method jTrans, Recall@1 increased by 15.0% and 16.5% respectively, indicating that SemFCA can better adapt to code variations brought about by different compilers and optimization options. Compared with the early work SAFE, BSAE-FCA increased by approximately two to three times in Recall@1, which can be attributed to the ability to utilize multi-dimensional information. Compared with Trex, SemFCA increased by approximately three times in the XA task. This is because Trex only focuses on behavior characteristics, and there are differences between cross-architecture execution behaviors, resulting in a problem of low code coverage, while SemFCA is not restricted by this and has a more stable match. Compared with GMN, SemFCA increased by 37.0%, 75.6%, 92.5%, and 70.7% in the XA, XO, XC, and XM tasks respectively, indicating that using only the control flow graph as a feature representation is inappropriate, and the proposed comprehensive utilization of AST information features, control flow features, and behavior features can better represent binary code.

[0138] The comprehensive experimental results show that SemFCA exhibits excellent performance in cross-architecture, cross-optimization, and cross-compiler binary code similarity detection tasks, demonstrating strong generalization ability and robustness.

[0139] To evaluate the impact of different information channels and channel attention mechanisms on the model performance, the present invention designed and implemented ablation experiments. The model was mainly tested on the XO, XA, XC, and XM tasks. The specific experimental settings are as follows:

[0140] 1. Remove the AST information channel: To evaluate the role of AST syntax information in the model, this experiment removed the AST information channel and only retained the control flow information channel and the behavior information channel.

[0141] 2. Remove the control flow information channel: To analyze the role of the control flow information channel in the model, this experiment removed the control flow information channel while keeping the other channels unchanged.

[0142] 3. Remove the behavior information channel: This experiment removed the behavior information channel to evaluate the impact of behavior features (such as system calls, memory access patterns, etc.) on the model performance.

[0143] 4. Remove the control flow information channel and the behavior information channel: In this experiment, both the control flow information channel and the behavior information channel were removed, and only the AST syntax information channel was retained.

[0144] 5. Remove the channel attention mechanism: In this experiment, the channel attention mechanism was removed to evaluate its role in feature fusion and information weight assignment. In this case, the output embedding vectors of the three information channels were directly dimensionally aligned and added together to obtain the final fused feature representation. That is, for the output embedding vectors E1, E2, and E3 of the three models, the fused feature embedding was obtained as follows: E final = E1 + E2 + E3. When removing a certain channel, the parameters of the other channels were fixed, and only the classification head was trained to ensure fair comparison. The test results of different ablation experiments when the query pool was 1000 were summarized in Table 2.

[0145] Table 2 Results of ablation studies in XA, XO, XC, and XM tasks (query pool = 1000). The measurement criteria are MRR / Recall@1

[0146] XA XO XC XM SemFCA 0.747 / 0.681 0.634 / 0.583 0.621 / 0.564 0.543 / 0.476 Remove AST information channel 0.337 / 0.265 0.486 / 0.439 0.471 / 0.419 0.384 / 0.293 Remove control flow channel 0.712 / 0.653 0.606 / 0.557 0.598 / 0.531 0.517 / 0.441 Remove behavior channel 0.729 / 0.664 0.618 / 0.571 0.607 / 0.543 0.528 / 0.452 Remove control flow and behavior channels 0.683 / 0.618 0.565 / 0.507 0.542 / 0.497 0.461 / 0.405 Remove channel attention mechanism 0.715 / 0.649 0.601 / 0.553 0.592 / 0.525 0.513 / 0.432

[0147] As shown in Table 2, the complete SemFCA model achieved the best performance in all tasks, indicating that each component of the model played a key role in improving the accuracy of binary code similarity detection.

[0148] The ablation experiments showed that the AST structural information had a decisive impact on the model performance. After removing the AST channel, Recall@1 in the XA and XM tasks decreased by 61.1% and 38.4% respectively, verifying the role of AST as a cross-architecture semantic stabilizer - its syntax tree representation can effectively overcome instruction set differences (such as instruction-level heterogeneity between x86 and ARM). In the XO and XC tasks, Recall@1 also decreased by more than 24%, proving that AST can resist semantic perturbations brought by compiler optimizations and code mutations.

[0149] The influence of control flow and behavioral features on the presentation task is task-specific. When removing the control flow channel alone, Recall@1 for the XO and XC tasks decreases by 4.5% and 5.9% respectively, indicating that although it can maintain the basic execution logic (such as function call relationships), its adaptability to optimization strategies (such as loop unrolling) is limited. The removal of the behavioral channel results in a smaller decrease (2.1% for XO / 3.7% for XC), suggesting that it is mainly used to compensate for the register allocation differences caused by the compiler. When both are removed together, Recall@1 for the XO and XC tasks decreases by 13% and 11.9% respectively, revealing their complementarity in handling optimization-induced control flow deformations (such as basic block merging) and memory access pattern changes.

[0150] The channel attention mechanism is crucial for complex tasks. After removing this module, Recall@1 in XM decreases by 9.2% because it loses the ability to dynamically adjust weights: in the code obfuscation scenario, the weight of the AST channel needs to be increased from 42% of the benchmark to 67% to cope with semantic distortions, while the weight of the control flow needs to be decreased from 31% to 18% to suppress misleading execution paths. Quantitative analysis shows that the attention mechanism reduces the feature fusion error of cross-architecture tasks by 38% (from 0.27 to 0.17), significantly improving the compatibility of heterogeneous features.

[0151] After removing the AST information channel, the model performance drops most significantly, with Recall@1 for the XA task and the XM task decreasing by 61.1% and 38.4% respectively. This result indicates that the AST structural information is crucial for cross-architecture matching because there may be significant differences in low-level instructions between different instruction set architectures, and the AST provides a relatively stable code syntax representation that can maintain cross-architecture semantic consistency. In addition, in the XO and XC tasks, Recall@1 also drops by more than 24%, indicating that the AST can also provide semantic stability in the case of compilation optimization and mutation. If the model only relies on control flow or behavioral information and lacks the high-level structural representation of the AST, it is difficult to stably match the code after different optimizations or processed by different compilers.

[0152] After removing the control flow channel, Recall@1 decreases by 4.5% and 5.9% for the XO and XC tasks respectively, indicating that the control flow information has a certain impact on the overall performance. The control flow is mainly used to model the execution logic of the code. Although the instruction level may change due to optimization and compiler mutation, the control flow structure of the function is usually relatively stable. Therefore, after removing the control flow information, the performance of the XO and XC tasks does not drop too severely, but it still weakens the model's ability to adapt to optimization and compiler mutation.

[0153] After removing the behavior channel, Recall@1 decreased by 2.1% and 3.7% on the XO and XC tasks respectively, and its impact was slightly less than that of the control flow channel, indicating that the contribution of behavior information was relatively low. However, in the XO and XC tasks, the differences in optimization strategies and compilers would affect the register allocation and memory management methods, and behavior characteristics could help the model adapt to these changes, so it still had a certain role.

[0154] When both the control flow and behavior channels were removed, Recall@1 decreased by 13% and 11.9% on the XO and XC tasks respectively, indicating that the control flow and behavior information had a certain complementary effect. In the XO and XC tasks, the differences in optimization strategies and compilers would lead to changes in the control flow structure and behavior patterns. After removing these two channels, the model's adaptability to optimization and compiler mutations decreased significantly.

[0155] After removing the channel attention mechanism, Recall@1 decreased by 9.2% on the XM task, indicating that the channel attention mechanism made a great contribution to the overall model. The channel attention mechanism could dynamically adjust the information weights of different channels to ensure that the model could adapt to changes in different tasks. After removing this mechanism, the model could not automatically allocate information weights according to the needs of different tasks, resulting in a decrease in the feature fusion effect of each channel, which in turn affected the model's performance in complex tasks (such as XM).

[0156] From the comprehensive experimental results, it can be seen that the AST channel had the greatest impact on performance. The control flow channel and the behavior channel both contributed to improving the model performance, and the two could play a good synergistic role. The channel attention mechanism was helpful for the model to comprehensively utilize information from different channels and improve the robustness of the model in complex task scenarios.

[0157] Specific Embodiment 2 provides a system for detecting binary code similarity based on AST enhancement, and the system includes the following modules:

[0158] The feature extraction module traverses the abstract syntax tree using the depth-first algorithm, screens out the nodes that play a key role in the program control flow and logical structure, constructs an AST for the selected key nodes to obtain AST semantic features; extracts the control flow features of the binary code based on the jump awareness mechanism, and extracts the behavior features of the binary code according to the function execution microtrace.

[0159] The feature fusion module uses the channel attention mechanism to fuse the AST semantic features, control flow features, and behavior features.

[0160] The similarity detection module uses the fused features to detect the similarity of binary codes.

[0161] Preferably, the obtaining of the AST semantic features is specifically:

[0162] Input each node in the AST into the Tree-LSTM model to obtain the node embedding vector;

[0163] Add the root node of the AST to the beginning of the sequence, and perform position encoding for each node according to the node positions recorded during depth-first search.

[0164] After fusing the node embedding vector and the position encoding, obtain the AST semantic features through the Transformer model.

[0165] Preferably, extracting the control flow features of the binary code based on the jump-aware mechanism is specifically as follows:

[0166] Decompose the mnemonics and operands of the binary code into independent tokens, uniformly replace string variables and constant values with preset special tokens, and at the same time retain the names and labels of external function calls.

[0167] Assign unique tokens to the source and target of each jump instruction.

[0168] Obtain the control flow features using the processed binary code.

[0169] Preferably, extracting the behavioral features of the binary code according to the function execution microtrace is specifically as follows:

[0170] Through the micro-execution method, simulate the function execution under unconstrained input, extract the dynamic behavior patterns at the instruction level, and generate embedding vectors that can accurately map the function execution semantics.

[0171] Preferably, using the channel attention mechanism to perform feature fusion on the AST semantic features, control flow features, and behavioral features is specifically as follows:

[0172] Use weight matrices W1, W2, and W3 to transform the AST semantic feature T s , control flow feature T c , and behavioral feature T e respectively:

[0173] E1 = T s W1

[0174] E2 = T c W2

[0175] E3 = T e W3

[0176] Calculate the weight of each channel using the formula:

[0177]

[0178] where, Watt is the weight matrix of the attention layer, b att is the bias vector, E i represents the feature representation of the i-th channel, α i is the attention weight of the i-th channel.

[0179] Multiply the features of each channel by their corresponding attention weights, and aggregate the weighted features to form a feature fusion result:

[0180]

[0181] where α i E i represents the product of the features of the i-th channel and its attention weight, T final is the sum of the weighted features of all channels, that is, the feature fusion result.

[0182] Specific Example 3 provides a computer-readable storage medium. The computer program stored on the readable storage medium, when executed by a processor, implements the method described in Specific Example 1.

[0183] The above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).

[0184] The steps of the methods or algorithms described in the embodiments of the present application may be directly embedded in hardware, software units executed by a processor, or a combination of both. The software units may be stored in a RAM memory, a flash memory, a ROM memory, an EPROM memory, an EEPROM memory, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium may be connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Optionally, the storage medium may also be integrated into the processor. The processor and the storage medium may be provided in an ASIC.

[0185] These computer program instructions may also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0186] Although the present application has been described in conjunction with specific features and their embodiments, it is obvious that various modifications and combinations can be made without departing from the spirit and scope of the present application. Accordingly, this specification and the drawings are merely exemplary illustrations of the present application defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of the present application. Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these changes and modifications.

Claims

1. A method for enhancing binary code similarity detection based on AST, characterized in that The method includes the following steps: Traverse the abstract syntax tree using the depth - first algorithm, filter out the nodes that play a key role in the program control flow and logical structure, construct an AST for the filtered key nodes to obtain AST semantic features; extract the control flow features of the binary code based on the jump - awareness mechanism, and extract the behavior features of the binary code according to the function execution micro - trace; Use the channel attention mechanism to fuse the AST semantic features, control flow features, and behavior features; Use the fused features to detect the similarity of binary codes.

2. The method according to claim 1, characterized in that The obtaining of the AST semantic features is specifically as follows: Input each node in the AST into the Tree - LSTM model to obtain node embedding vectors; Add the root node of the AST to the beginning of the sequence, and perform position encoding for each node according to the node positions recorded during the depth - first search; After fusing the node embedding vectors and the position encoding, obtain the AST semantic features through the Transformer model.

3. The method according to claim 1, characterized in that, The extraction of the control flow features of the binary code based on the jump - awareness mechanism is specifically as follows: Decompose the mnemonics and operands of the binary code into independent tokens, uniformly replace string variables and constant values with preset special tokens, and at the same time retain the names and labels of external function calls; Assign unique tokens to the source and target of each jump instruction; Obtain the control flow features using the processed binary code.

4. The method according to claim 1, characterized in that, The extraction of the behavior features of the binary code according to the function execution micro - trace is specifically as follows: Through the micro - execution method, simulate the function execution under unconstrained input, extract the dynamic behavior patterns at the instruction level, and generate embedding vectors that can accurately map the function execution semantics.

5. The method according to claim 1, characterized in that The use of the channel attention mechanism to fuse the AST semantic features, control flow features, and behavior features is specifically as follows: The AST semantic features T, the control flow features T, and the behavior features T are respectively transformed by using the weight matrices W1, W2, and W3 s , the control flow features T c , the behavior features T e as follows: E1 = T s W1 E2 = T c W2 E3 = T e W3 Use the formula to calculate the weight of each channel: Among them, W att is the weight matrix of the attention layer, b att is the bias vector, E i represents the feature representation of the i-th channel, and α i is the attention weight of the i-th channel; Multiply the features of each channel by their corresponding attention weights, and aggregate the weighted features to form the feature fusion result: Among them, α i E i represents the product of the feature of the i-th channel and its attention weight, and T final is the sum of all channel-weighted features, that is, the feature fusion result.

6. An AST-enhanced binary code similarity detection system, characterized in that, The system includes the following modules: A feature extraction module that traverses the abstract syntax tree using the depth - first algorithm, filters out the nodes that play a key role in the program control flow and logical structure, constructs an AST for the filtered key nodes to obtain AST semantic features; Extract the control flow features of the binary code based on the jump - awareness mechanism, and extract the behavior features of the binary code according to the function execution micro - trace; A feature fusion module that uses the channel attention mechanism to fuse the AST semantic features, control flow features, and behavior features; A similarity detection module that uses the fused features to detect the similarity of binary codes.

7. The system according to claim 6, wherein The obtaining of the AST semantic features is specifically as follows: Input each node in the AST into the Tree - LSTM model to obtain node embedding vectors; Add the root node of the AST to the beginning of the sequence, and perform position encoding for each node according to the node positions recorded during the depth - first search; After fusing the node embedding vectors and the position encoding, obtain the AST semantic features through the Transformer model.

8. The system according to claim 6, wherein The extraction of the control flow features of the binary code based on the jump - awareness mechanism is specifically as follows: Decompose the mnemonic and operands of the binary code into independent tokens, uniformly replace string variables and constant values with preset special tokens, and retain the names and labels of external function calls at the same time; Assign unique tokens to the sources and destinations of each jump instruction; Obtain the control flow features using the processed binary code.

9. The system according to claim 6, wherein The behavior features of the binary code are extracted according to the function execution microtraces, specifically: Through micro-execution, simulate the function execution under unconstrained input, extract the dynamic behavior patterns at the instruction level, and generate embedding vectors that can accurately map the function execution semantics.

10. The system according to claim 6, wherein The channel attention mechanism is used to perform feature fusion on the AST semantic features, control flow features, and behavior features, specifically: Use weight matrices W1, W2, and W3 to transform the AST semantic features T s , control flow features T c , and behavior features T e respectively: E1 = T s W1 E2 = T c W2 E3 = T e W3 Use the formula to calculate the weight of each channel: Among them, W att is the weight matrix of the attention layer, b att is the bias vector, E i represents the feature representation of the i-th channel, and α i is the attention weight of the i-th channel; Multiply the features of each channel by their corresponding attention weights, and aggregate the weighted features to form the feature fusion result: where α i E i represents the product of the feature of the i-th channel and its attention weight, and T final is the sum of all channel-weighted features, i.e., the feature fusion result.

Images