Training / application method and device for binary code similarity detection model

Abstract

The present application provides a training / application method and device for a binary code similarity detection model. The training method includes: establishing a training sample set based on an open source binary code data set, wherein the training sample set includes multiple training samples based on an intermediate language; pre-training a basic model based on the Transformer model architecture based on the training sample set to obtain an intermediate language pre-training model; training a comparison model based on the intermediate language pre-training model and the training sample set; training an embedding model based on the intermediate language pre-training model and the training sample set, wherein the embedding model includes a query encoder and a reference encoder; constructing a similarity detection model capable of detecting the similarity of binary codes based on the embedding model and the comparison model. The similarity detection model trained by the present application can simultaneously improve the accuracy and efficiency of binary code similarity detection.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and in particular to a training / application method and device for a binary code similarity detection model. Background Art

[0002] Binary Code Similarity Detection (BCSD) is a technique used to identify similarities between binary code snippets, such as functions. BCSD can be applied to a variety of tasks, such as vulnerability detection, malware classification, and code plagiarism detection.

[0003] Existing BCSD techniques include graph matching, tree-based methods, and feature-based techniques. Deep learning techniques have become a popular method for BCSD due to its accuracy and ability to automatically learn complex features. In deep learning techniques, binary code similarity detection can be divided into two main methods: embedding-based methods and comparison-based methods. Embedding-based methods focus on extracting the features of functions and representing them in a lower-dimensional space, namely, embedding. In order to determine the similarity, the embeddings of these functions can be compared using distance metrics. Comparison-based methods can directly measure the similarity of functions using raw data or feature analysis. However, both of these deep learning-based binary code similarity detection methods have some problems: embedding-based methods can provide efficient reasoning and are therefore suitable for expansion to large-scale BCSD application scenarios, but have low accuracy; although comparison-based methods have high accuracy in similarity detection, they have low detection efficiency and are not suitable for large-scale BCSD application scenarios.

[0004] Therefore, there is a need for a binary code similarity detection scheme that can simultaneously improve the accuracy and efficiency of binary code similarity detection. Summary of the invention

[0005] The present application provides a training / application method and device for a binary code similarity detection model, which is used to solve the technical problem that the existing binary code similarity detection method cannot simultaneously guarantee the accuracy and efficiency of similarity detection.

[0006] In a first aspect, the present application provides a training method for a binary code similarity detection model, comprising:

[0007] Establishing a training sample set according to an open source binary code data set, wherein the training sample set includes a plurality of training samples based on an intermediate language;

[0008] Pre-training a basic model based on the Transformer model architecture according to the training sample set to obtain an intermediate language pre-training model;

[0009] According to the intermediate language pre-training model and the training sample set, a comparison model is trained;

[0010] According to the intermediate language pre-training model and the training sample set, an embedding model is trained, wherein the embedding model includes a query encoder and a reference encoder;

[0011] Based on the embedding model and the comparison model, a similarity detection model capable of detecting similarity of binary codes is constructed.

[0012] In a possible implementation, establishing a training sample set based on an open source binary code data set specifically includes:

[0013] Obtain an open source binary code dataset and use a function extraction tool to extract the code functions corresponding to each binary code in the dataset;

[0014] For each code function, standardize the code function to obtain a corresponding intermediate language IL;

[0015] Performing word segmentation processing on the intermediate language IL by using a word segmentation algorithm to obtain a training sample corresponding to the code function;

[0016] According to the training samples corresponding to each code function, a training sample set is established;

[0017] The open source binary code dataset includes one or more of a BinaryCorp dataset, a Cisco dataset, and a Trex dataset, and the function extraction tool includes a BinaryNinja tool.

[0018] In a possible implementation, the training to obtain an embedding model according to the intermediate language pre-training model and the training sample set specifically includes:

[0019] Performing parameter initialization processing on the intermediate language pre-training model to determine initial parameters of the query encoder and initial parameters of the reference encoder;

[0020] For each training sample in the training sample set, formatting the training sample to obtain a query function and a reference function corresponding to the training sample respectively;

[0021] Input each query function into the query encoder in batches to obtain the corresponding query embedding, and input each reference function into the reference encoder in batches to obtain the corresponding reference embedding and a reusable reference embedding cache;

[0022] Calculate the similarity between the query embedding in each batch and the reference embedding and the reference embedding cache, and use the contrastive learning loss function to iteratively update the parameters of the query encoder and the reference encoder to obtain the final query encoder and reference encoder;

[0023] According to the final query encoder and the reference encoder, an embedding model is constructed.

[0024] In a possible implementation, the use of the contrastive learning loss function to iteratively update the parameters of the query encoder and the parameters of the reference encoder to obtain the final query encoder and reference encoder specifically includes:

[0025] The contrastive learning loss for this batch of training is calculated using the following formula based on the similarity between the query embeddings in this batch and the reference embeddings and the reference embedding cache:

[0026]

[0027] in, is a pair of positive samples of query embedding and reference embedding / reference embedding cache, is a pair of negative samples of query embedding and reference embedding / reference embedding cache, is the similarity function between two embeddings, N is the total number of pairs of query embedding and reference embedding, L To contrast learning loss;

[0028] performing a gradient update on initial parameters of the query encoder according to the contrastive learning loss;

[0029] Perform momentum update on the initial parameters of the reference encoder according to the updated parameters of the query encoder;

[0030] Update the reusable reference embedding cache according to the next batch of reference embeddings and the embedding cache size;

[0031] Repeat the above steps, and iteratively update the parameters of the query encoder and the reference encoder by back-propagation according to the updated reusable reference embedding cache and the contrastive learning loss of each batch training to obtain the final query encoder and reference encoder.

[0032] In a possible implementation manner, performing momentum updating on initial parameters of the reference encoder according to updated parameters of the query encoder specifically includes:

[0033] The momentum update of the initial parameters of the reference encoder is performed using the following formula:

[0034]

[0035] in, represents the updated parameters of the reference encoder, represents the initial parameters of the reference encoder, Indicates querying the updated parameters of the encoder. m represents the momentum coefficient.

[0036] In a second aspect, the present application provides an application method of a binary code similarity detection model, comprising:

[0037] Extracting a code function of the binary code to be detected, and performing standardization and word segmentation processing on the code function to obtain a detection function;

[0038] Inputting the detection function into the embedding model of the similarity detection model to construct a corresponding detection embedding vector;

[0039] Determining a plurality of potential similarity functions corresponding to the binary code to be detected according to the detection embedding vector and the embedding model;

[0040] Inputting the detection function and each potential similarity function into a comparison model of a similarity detection model to determine the similarity between the detection function and each potential similarity function;

[0041] Determine a target function among potential similarity functions that has the highest similarity to the detection function;

[0042] According to the binary code corresponding to the target function, a similar code to the binary code to be detected is determined.

[0043] In a possible implementation, determining a plurality of potential similarity functions corresponding to the binary code to be detected according to the detection embedding vector and the embedding model specifically includes:

[0044] Determining a plurality of potential similar functions corresponding to the binary code to be detected according to the detection embedding vector and the vector index of each function in the embedding model;

[0045] The vector index of each function is constructed using a vector search engine based on an approximate nearest neighbor algorithm.

[0046] In a third aspect, the present application provides an electronic device, including:

[0047] A sample generation module, used to establish a training sample set according to an open source binary code data set, wherein the training sample set includes a plurality of training samples based on an intermediate language;

[0048] The model training module is used to pre-train the basic model based on the Transformer model architecture according to the training sample set to obtain an intermediate language pre-training model; train a comparison model according to the intermediate language pre-training model and the training sample set; train an embedding model according to the intermediate language pre-training model and the training sample set, and the embedding model includes a query encoder and a reference encoder; and construct a similarity detection model capable of detecting the similarity of binary codes according to the embedding model and the comparison model.

[0049] In a fourth aspect, the present application provides another electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0050] The memory stores computer-executable instructions;

[0051] The processor executes the computer-executable instructions stored in the memory to implement the above method.

[0052] In a fifth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to implement the above-mentioned method when executed by a processor.

[0053] In a sixth aspect, the present application provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0054] The training / application method and device of the binary code similarity detection model provided in the present application can establish a training sample set based on an open source binary code data set, and the training sample set includes multiple training samples based on an intermediate language; pre-train the basic model based on the Transformer model architecture according to the training sample set to obtain an intermediate language pre-training model; train a comparison model according to the intermediate language pre-training model and the training sample set; train an embedding model according to the intermediate language pre-training model and the training sample set, and the embedding model includes a query encoder and a reference encoder; construct a similarity detection model that can detect the similarity of binary codes according to the embedding model and the comparison model. The similarity detection model of the present application combines the embedding model and the comparison model, which can achieve the accuracy and efficiency of similarity detection in large-scale detection tasks. Specifically, the embedding model in the similarity detection model can detect potential similar codes with high similarity from large-scale detection tasks to ensure the efficiency of large-scale detection, and the comparison model can further detect the potential similar codes detected by the embedding model to obtain the most similar binary code to ensure the accuracy of detection. Furthermore, the embedding model and comparison model in the similarity detection model are trained based on the same intermediate language pre-training model, which improves the synergy and performance of the embedding model and the comparison model. The intermediate language pre-training model is trained based on the Transformer model architecture, which further improves the performance of the embedding model and the comparison model, and further improves the accuracy of detection. In addition, the embedding model is split into a query encoder and a reference encoder, which can effectively manage memory and computing resources, further improving the efficiency of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0056] Figure 1 It is a schematic diagram of the comparative learning process in the prior art;

[0057] Figure 2 It is a schematic diagram of the similarity detection process based on embedding in the prior art;

[0058] Figure 3 It is a schematic diagram of a similarity detection process based on comparison in the prior art;

[0059] Figure 4 A flowchart of a method for training a binary code similarity detection model according to an embodiment of the present application;

[0060] Figure 5 A schematic diagram of an embedding model training process according to an embodiment of the present application;

[0061] Figure 6 A flowchart of an application method of a binary code similarity detection model according to an embodiment of the present application;

[0062] Figure 7 A schematic diagram of the structure of an electronic device according to an embodiment of the present application;

[0063] Figure 8 A schematic diagram of the structure of an electronic device according to another embodiment of the present application;

[0064] Fig. 9 This is a schematic structural diagram of an electronic device according to another embodiment of the present application.

[0065] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0066] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0067] It should be noted that the training / application method and device of the binary code similarity detection model of the present application can be used in the field of machine learning, and can also be used in any field other than the field of machine learning, such as the computer field, the data processing field, etc. The application field of the training / application method and device of the binary code similarity detection model of the present application is not limited.

[0068] The training / application method and device of the binary code similarity detection model of the present application can be applied to any scenario related to binary code similarity detection, such as vulnerability detection, malware classification, and code plagiarism detection, etc. The application scenario of the training / application method and device of the binary code similarity detection model of the present application is not limited.

[0069] First, the terms involved in this application are explained:

[0070] Contrastive Learning,The goal of contrastive learning is to increase the similarity between semantically similar data points, which are called embeddings, while increasing the dissimilarity between semantically unrelated data points in the latent representation space. This is achieved by using pairwise comparisons in an unsupervised or self-supervised manner, using a contrastive loss function to measure the distance between instances, such as Figure 1 As shown, Figure 1 Schematic diagram of the contrastive learning process in the prior art. For example, Trex uses a pairwise loss function to minimize the distance to the true value. Some previous works use ternary loss to reduce the distance between positive sample pairs and increase the distance between negative sample pairs. Vulhawk applies cross entropy loss to reduce the distance to the true value and uses a many-to-many approach to maintain the distance between negative sample pairs.

[0071] Existing BCSD techniques include graph matching, tree-based methods, and feature-based techniques. Recently, deep learning techniques have become a popular approach for BCSD due to their accuracy and ability to automatically learn complex features. In deep learning techniques, binary code similarity detection can be divided into two main approaches: embedding-based methods and comparison-based methods.

[0072] The recent development of deep neural networks (DNNs) has inspired researchers to explore embedding-based BCSD. Embedding-based BCSD methods mainly focus on extracting features of functions and representing them in a lower dimensional space (i.e., embedding). Existing studies use DNNs as feature extractors to transform binary functions into an embedding space. To determine similarity, the embeddings of these functions can be compared using a distance metric. One advantage of embedding-based methods is the use of a fixed representation that can be pre-computed.

[0073] Comparison-based methods can directly measure the similarity of functions using raw data or feature analysis. These methods vary: FOSSIL evaluates the syntax, semantics, and behavior of FOSS functions by integrating Bayesian networks. In contrast, aDiff applies CNN to directly process the raw bytes of functions, requiring a large amount of training data. BinDNN combines CNN, LSTM, and deep neural networks to determine the equivalence of functions between different compilers and architectures, while another work breaks down the code into fragments and uses feedforward neural networks for fast and accurate analysis. GMN introduces a cross-graph attention mechanism for graph matching in DNN models to evaluate similarity scores between graph elements.

[0074] Embedding-based methods have become mainstream in BCSD research in recent years. Compared with comparison-based methods, they can provide efficient reasoning and are therefore suitable for scaling to large-scale BCSD applications, but with lower accuracy.

[0075] Figure 2 It is a schematic diagram of the similarity detection process based on embedding in the prior art, such as Figure 2 As shown, given the original input of the binary code x and y , which are fed into the embedding-based model f , and use a distance metric (such as cosine) to calculate similarity, such as cos( f ( x ), f ( y ))You can get x and y similarity. Figure 3 Schematic diagram of the similarity detection process based on comparison in the prior art, such as Figure 3 As shown, given the original input of the binary code x and y , which is fed into the comparison-based model g Calculate similarity, such as g( x , y ) to get x and y The embedding model focuses on extracting and encoding the features of binary functions, while the comparison model focuses more on evaluating the direct similarity between functions. Obviously, g( x , y ) is greater than cos( f ( x ), f ( y )) has strong expressive power. In other words, a well-trained comparison model can outperform a well-trained embedding model and better capture the features between two binary codes.

[0076] However, both deep learning-based binary code similarity detection methods have some problems: the embedding-based method can provide efficient reasoning and is therefore suitable for expansion to large-scale BCSD application scenarios, but the accuracy is low; although the comparison-based method has high similarity detection accuracy, it has low detection efficiency and is not suitable for large-scale BCSD application scenarios.

[0077] Based on this technical problem, the inventive concept of the present application is: how to provide a similarity detection model that can simultaneously improve the accuracy and efficiency of binary code similarity detection.

[0078] Specifically, a training sample set can be established based on an open source binary code data set, and the training sample set includes multiple training samples based on an intermediate language; the basic model based on the Transformer model architecture is pre-trained according to the training sample set to obtain an intermediate language pre-training model; a comparison model is trained according to the intermediate language pre-training model and the training sample set; an embedding model is trained according to the intermediate language pre-training model and the training sample set, and the embedding model includes a query encoder and a reference encoder; according to the embedding model and the comparison model, a similarity detection model capable of detecting the similarity of binary codes is constructed. The similarity detection model of the present application combines an embedding model and a comparison model, and can achieve the accuracy and efficiency of similarity detection in large-scale detection tasks. Specifically, the embedding model in the similarity detection model can detect potential similar codes with high similarity from large-scale detection tasks to ensure the efficiency of large-scale detection, and the comparison model can perform further detection based on the potential similar codes detected by the embedding model to obtain the most similar binary code to ensure the accuracy of detection. Furthermore, the embedding model and comparison model in the similarity detection model are trained based on the same intermediate language pre-training model, which improves the synergy and performance of the embedding model and the comparison model. The intermediate language pre-training model is trained based on the Transformer model architecture, which further improves the performance of the embedding model and the comparison model, and further improves the accuracy of detection. In addition, the embedding model is split into a query encoder and a reference encoder, which can effectively manage memory and computing resources, further improving the efficiency of detection.

[0079] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0080] Embodiment 1

[0081] Figure 4 This is a flow chart of a method for training a binary code similarity detection model according to an embodiment of the present application. This embodiment uses an electronic device as an execution subject to illustrate the method for training the binary code similarity detection model. Figure 4 As shown, the training method of the binary code similarity detection model may include the following steps:

[0082] S101: A training sample set is established according to an open source binary code data set, wherein the training sample set includes a plurality of training samples based on an intermediate language.

[0083] In this embodiment, the electronic device can be a terminal device such as a computer or a mobile phone, or a server device such as a physical server or a cloud server. The specific type of the electronic device can be flexibly set by those skilled in the art, and no limitation is imposed here, as long as the electronic device can be trained and applied with a similarity detection model.

[0084] In a possible implementation, establishing a training sample set according to an open source binary code data set in step S101 may include:

[0085] S11: Obtain an open source binary code dataset, and use a function extraction tool to extract the code function corresponding to each binary code in the dataset.

[0086] S12: For each code function, perform standardization processing on the code function to obtain a corresponding intermediate language IL.

[0087] S13: Perform word segmentation processing on the intermediate language IL using a word segmentation algorithm to obtain training samples corresponding to the code function.

[0088] S14: Establish a training sample set according to the training samples corresponding to each code function.

[0089] Among them, the open source binary code datasets include one or more of the BinaryCorp dataset, the Cisco dataset, and the Trex dataset, and the function extraction tool includes the BinaryNinja tool.

[0090] In this embodiment, the binary code data set includes but is not limited to the BinaryCorp data set, the Cisco data set and the Trex data set, and may also include any other binary code data set, without any limitation here. Similarly, the function extraction tool also includes but is not limited to the BinaryNinja tool, and the specific type of the function extraction tool can be flexibly set by those skilled in the art, without any limitation here, as long as the code function corresponding to the binary code can be extracted.

[0091] In this embodiment, the specific method of standardizing the code function into the intermediate language IL can refer to the existing function standardization processing method, which will not be described in detail here.

[0092] In this implementation, the word segmentation algorithm may be the WordPiece word segmentation algorithm or other word segmentation algorithms, and no limitation is made here.

[0093] In this embodiment, the binary code data set may include data sets of different versions / types of binary codes. Pre-training with training sample sets obtained by processing data sets from multiple sources can ensure that the model can handle different types of binary codes. Furthermore, a function extraction tool is used to extract the code function corresponding to the binary code, and the code function is standardized to obtain the intermediate language IL, so that the training sample set can adapt to different instruction set architectures (ISAs) and coding styles, ensuring consistency across architectures and improving model performance. Furthermore, using a word segmentation algorithm to segment the intermediate language IL can achieve efficient encoding of binary codes, which not only solves the common out-of-vocabulary (OOV) problem in traditional models, but also retains the key features of binary codes, such as strings and numbers, further improving model performance.

[0094] S102: Pre-train a basic model based on the Transformer model architecture according to the training sample set to obtain an intermediate language pre-training model.

[0095] In this embodiment, the basic model based on the Transformer model architecture can refer to the models that have performed well in binary code analysis in the past, such as the jTrans model. Those skilled in the art can flexibly set the basic model without any limitation.

[0096] In this embodiment, the process of pre-training the basic model according to the training sample set to obtain the intermediate language pre-training model can refer to the pre-training process of the existing Transformer model, which will not be described in detail here.

[0097] S103: According to the intermediate language pre-training model and the training sample set, a comparison model is trained.

[0098] In this embodiment, the training method of the comparison model can refer to the existing training process of the comparison model, which will not be described in detail here. The intermediate language pre-training model is trained based on the Transformer model architecture. Therefore, the comparison model trained based on the intermediate language pre-training model significantly improves the model performance.

[0099] In this embodiment, inspired by image similarity detection, the training method of the comparison model enables the model to perform direct, labeled-to-label comparison, thereby improving the accuracy of the model in the similarity detection task. The input of the comparison model is designed to accept a pair of functions at the same time, and its output represents the similarity between the pair of functions.

[0100] S104: According to the intermediate language pre-training model and the training sample set, an embedding model is trained, where the embedding model includes a query encoder and a reference encoder.

[0101] In this embodiment, the embedding model and the comparison model are trained based on the same intermediate language pre-training model, which improves the synergy and performance of the embedding model and the comparison model and further improves the accuracy of detection.

[0102] In a possible implementation, training the embedding model according to the intermediate language pre-training model and the training sample set in the above step S104 may include:

[0103] S21: Perform parameter initialization processing on the intermediate language pre-training model to determine initial parameters of the query encoder and initial parameters of the reference encoder.

[0104] S22: For each training sample in the training sample set, format the training sample to obtain a query function and a reference function corresponding to the training sample.

[0105] S23: Input each query function into the query encoder in batches to obtain the corresponding query embedding, and input each reference function into the reference encoder in batches to obtain the corresponding reference embedding and a reusable reference embedding cache.

[0106] S24: Calculate the similarity between the query embedding in each batch and the reference embedding and the reference embedding cache, and iteratively update the parameters of the query encoder and the parameters of the reference encoder using the contrastive learning loss function to obtain the final query encoder and reference encoder.

[0107] S25: Based on the final query encoder and the reference encoder, an embedding model is constructed.

[0108] In this implementation, the query function and the reference function are compiled and formatted from the same training sample (source code), so the query function and the reference function are semantically equivalent.

[0109] In this embodiment, there is a reusable reference embedded cache in the reference encoder, and the cache size can be flexibly set by those skilled in the art according to the actual structure of the reference encoder, and no limitation is made here.

[0110] In this embodiment, the embedding model can be split into a query encoder and a reference encoder, so that the model can effectively manage memory and computing resources without sacrificing performance. By separating the query encoder and the reference encoder, the model can reuse the encoded embeddings, thereby introducing a large number of negative samples while improving training efficiency. This mechanism significantly reduces the demand for large-scale GPU resources. Specifically, a reusable reference embedding cache can be set in the reference encoder, and the reference embedding cache can store previously encoded reference embeddings. These reference embeddings can be reused as reference embedding caches during iterative training. By reusing previously encoded embeddings, the computing requirements during the training process are greatly reduced, and the negative samples are greatly enriched, so that the final constructed embedding model can be suitable for large-scale detection tasks, improving code detection efficiency.

[0111] In a possible implementation, the step S24 of iteratively updating the parameters of the query encoder and the parameters of the reference encoder using the contrastive learning loss function to obtain the final query encoder and reference encoder may include:

[0112] S31: Using the following formula (1), the contrastive learning loss of this batch training is calculated based on the similarity between the query embedding in this batch and the reference embedding and the reference embedding cache:

[0113]

[0114] in, is a pair of positive samples of query embedding and reference embedding / reference embedding cache, is a pair of negative samples of query embedding and reference embedding / reference embedding cache, is the similarity function between two embeddings, N is the total number of pairs of query embedding and reference embedding, L To contrast learning loss;

[0115] S32: Perform gradient update on the initial parameters of the query encoder according to the contrastive learning loss.

[0116] S33: Perform momentum update on the initial parameters of the reference encoder according to the updated parameters of the query encoder.

[0117] S34: Update the reusable reference embedding cache according to the next batch of reference embeddings and the embedding cache size.

[0118] S35: Repeat the above steps S31-S34, and perform iterative back-propagation update on the parameters of the query encoder and the parameters of the reference encoder according to the updated reusable reference embedding cache and the contrastive learning loss of each batch training to obtain the final query encoder and reference encoder.

[0119] In this implementation, the reference embeddings stored in the reusable reference embedding cache can be dynamically updated as training progresses, that is, old reference embeddings are continuously removed and new reference embeddings are added.

[0120] Exemplarily, the size of a reference embedding cache is 64, that is, the reference embedding cache can store 64 reference embeddings. Assuming that the number of reference embeddings input in each batch is 8, there are 8 reference embeddings and 0 reference embedding cache in the first training. In the second training, there are 8 new reference embeddings input. The 8 reference embeddings in the first training are stored in the reference embedding cache and participate in the second training as the reference embedding cache. By analogy, there are 8 new reference embeddings input in the ninth training. The reference embedding cache stores the previous 64 reference embeddings and participates in the ninth training as the reference embedding cache. In the tenth training, there are 8 new reference embeddings input. The earliest 8 reference embeddings (in the first training) are removed from the reference embedding cache and the 8 new reference embeddings input in the ninth training are stored. A total of 64 reference embeddings are maintained and participate in the tenth training as the reference embedding cache. That is, the reference embedding cache can be dynamically updated as the training progresses.

[0121] For example, Figure 5 FIG. 1 is a schematic diagram of an embedding model training process according to an embodiment of the present application. Figure 5 As shown in Figure 1, the embedding model can be first divided into a query encoder and a reference encoder. The training sample is formatted as a pair of Q i , R i , where Q i and R i Represents the query function and reference function Q respectively i is input into the query encoder, R i is input to the reference encoder. After being encoded by the encoder, the query function and the reference function are encoded into embeddings, denoted as Q 1:n and R 1:n (n is the number of embeddings). The reusable reference embedding cache in the reference encoder is R' 1:k , the size of the embedding cache is denoted as k. When calculating the contrastive learning loss for iterative training, Q 1:n and Concate(R 1:n , R' 1:L ). Among them, only the pair Q 1:n * R1:n The pairing with is a positive sample, and all other pairs are negative samples. After updating the embedding model, the embedding cache will also be embedded with the newly encoded reference R 1:n to update.

[0122] In this embodiment, the InfoNCE contrastive learning loss function shown in formula (1) is used to iteratively train the model, which can enhance the model's ability to distinguish between positive and negative samples and maximize the mutual information between positive sample pairs and negative samples, which not only improves the accuracy of the model in the BCSD task, but also increases its adaptability to new scenarios and enhances the model's discrimination ability in different scenarios. Furthermore, the reference embeddings stored in the reusable reference embedding cache can be dynamically updated as the training progresses, further enriching the negative samples and improving the model performance.

[0123] In a possible implementation, the above step S33 performs momentum updating on the initial parameters of the reference encoder according to the updated parameters of the query encoder, which may include:

[0124] The momentum of the initial parameters of the reference encoder is updated using the following formula (2):

[0125]

[0126] in, represents the updated parameters of the reference encoder, represents the initial parameters of the reference encoder, Indicates querying the updated parameters of the encoder. m represents the momentum coefficient.

[0127] In this embodiment, the momentum coefficient m It can be set relatively large, such as 0.99, to improve the efficiency and accuracy of the parameter update of the reference encoder. The specific momentum coefficient can be flexibly set by those skilled in the art, and no limitation is made here.

[0128] In this embodiment, during each training process, the query encoder can be updated with gradients through contrastive learning loss, but the reference encoder is frozen during the encoding process and can be updated using a momentum-based method after the query encoder is updated. Through such a setting, more negative samples can be included by reusing reference embeddings without incurring excessive computational costs, greatly enriching negative samples without increasing a huge computational burden, greatly reducing the amount of computation, and making the model more suitable for large-scale detection tasks.

[0129] S105: Based on the embedding model and the comparison model, a similarity detection model capable of detecting similarity of binary codes is constructed.

[0130] In this embodiment, after the embedding model and the comparison model are obtained according to the above steps S101-S104, a similarity detection model CEBin capable of detecting the similarity of binary codes can be constructed. CEBin can achieve the goal of maintaining high efficiency without sacrificing detection accuracy in large-scale tasks through its unique two-stage training and reasoning design.

[0131] In this embodiment, the similarity detection model combines the embedding model and the comparison model, which can achieve the accuracy and efficiency of similarity detection in large-scale detection tasks. Specifically, the embedding model in the similarity detection model can detect potential similar codes with high similarity from large-scale detection tasks to ensure the efficiency of large-scale detection, and the comparison model can perform further detection based on the potential similar codes detected by the embedding model to obtain the most similar binary code to ensure the accuracy of detection. Furthermore, the embedding model and the comparison model in the similarity detection model are trained based on the same intermediate language pre-training model, which improves the synergy and performance of the embedding model and the comparison model, and the intermediate language pre-training model is trained based on the Transformer model architecture, which further improves the performance of the embedding model and the comparison model, and further improves the accuracy of detection. In addition, the embedding model is split into a query encoder and a reference encoder, which can effectively manage memory and computing resources, further improving the efficiency of detection.

[0132] Embodiment 2

[0133] Figure 6 This is a flow chart of an application method of a binary code similarity detection model according to an embodiment of the present application. This embodiment uses an electronic device as an execution subject to illustrate the application method of the binary code similarity detection model. Figure 6 As shown, the application method of the binary code similarity detection model may include the following steps:

[0134] S201: extracting a code function of a binary code to be detected, and performing standardization and word segmentation processing on the code function to obtain a detection function.

[0135] In this embodiment, the code function of the binary code to be detected is extracted, and the code function is standardized and segmented to obtain the specific implementation of the detection function. Please refer to steps S101 and S11-S13 in Example 1, which will not be repeated here.

[0136] S202: Input the detection function into the embedding model of the similarity detection model to construct a corresponding detection embedding vector.

[0137] In this embodiment, the detection function can be input into the query encoder in the embedding model to obtain the corresponding detection embedding vector.

[0138] S203: Determine multiple potential similarity functions corresponding to the binary code to be detected according to the detection embedding vector and the embedding model.

[0139] In a possible implementation, the above step S203 determines multiple potential similarity functions corresponding to the binary code to be detected according to the detection embedding vector and the embedding model, which may include:

[0140] Determine multiple potential similar functions corresponding to the binary code to be detected according to the detection embedding vector and the vector index of each function in the embedding model;

[0141] The vector index of each function is constructed using a vector search engine based on an approximate nearest neighbor algorithm.

[0142] In this embodiment, a reference encoder in the embedding model may be first used to construct a function embedding vector for each function in the function pool, and then a vector search engine based on an approximate nearest neighbor algorithm may be used to construct a vector index corresponding to each function embedding vector.

[0143] In this embodiment, after the detection embedding vector is obtained, multiple potential similar functions corresponding to the binary code to be detected can be simply and accurately determined according to the detection embedding vector and the vector index of each function in the embedding model. Furthermore, a vector search engine based on the approximate nearest neighbor (ANN) algorithm is used to construct a vector index for each function in the function pool, so that the model can efficiently perform similarity search and comparison, which is suitable for large-scale BCSD scenarios and improves the retrieval efficiency of large-scale tasks.

[0144] S204: Input the detection function and each potential similarity function into a comparison model of the similarity detection model to determine the similarity between the detection function and each potential similarity function.

[0145] S205: Determine the target function that has the highest similarity with the detection function among the potential similarity functions.

[0146] S206: Determine a similar code to the binary code to be detected according to the binary code corresponding to the target function.

[0147] In this embodiment, the similarity detection model CEBin combines embedded and comparative methods, and uses advanced machine learning technology to improve the accuracy and efficiency of detection. CEBin is divided into two parts: an embedded model and a comparative model, each of which is optimized for different goals. The embedded model focuses on extracting and encoding the features of binary functions, while the comparative model focuses more on evaluating the direct similarity between functions. The embedded model in the similarity detection model can quickly detect potential similar codes with high similarity to the binary code to be detected from a large number of detection tasks, ensuring the efficiency of large-scale detection, while the comparative model can perform further detection based on the potential similar codes detected by the embedded model, and quickly and accurately detect the most similar binary code from a small number of similar codes, ensuring the accuracy of detection. The CEBin model is suitable for software ecosystems of different types and sizes, especially in the field of software security, such as malware classification and code plagiarism detection, which improves the efficiency and accuracy of large-scale code detection tasks.

[0148] The following is an explanation of the training and application method of the binary code similarity detection model of the present application with a specific embodiment.

[0149] Embodiment 3

[0150] In a specific embodiment, a software company wants to perform plagiarism detection on a newly developed code A. The specific detection process is as follows:

[0151] In the first step, the staff inputs the open source binary code dataset into the computer and enters the model building instructions. The computer establishes a training sample set based on the open source binary code dataset, and pre-trains the basic model based on the Transformer model architecture based on the training sample set to obtain the intermediate language pre-training model.

[0152] In the second step, the computer trains a comparison model based on the intermediate language pre-training model and the training sample set, and trains an embedding model based on the intermediate language pre-training model and the training sample set.

[0153] In the third step, the computer constructs a similarity detection model that can detect the similarity of binary codes based on the embedding model and the comparison model.

[0154] In the fourth step, the staff inputs code A into the computer's similarity detection model and starts the detection task. The computer extracts the code function of code A, and performs standardization and word segmentation processing on the code function to obtain the detection function.

[0155] In the fifth step, the computer inputs the detection function into the embedding model of the similarity detection model, and the query encoder in the embedding model constructs the detection embedding vector corresponding to the detection function.

[0156] In the sixth step, the reference encoder in the computer embedding model constructs a function embedding vector for each function in the function pool, and uses a vector search engine based on the approximate nearest neighbor algorithm to construct a vector index corresponding to each function embedding vector.

[0157] In the seventh step, the computer embedding model determines 50 potential similar functions corresponding to code A based on the detection embedding vector and the vector index of each function.

[0158] In the eighth step, the computer inputs the detection function and each potential similarity function into a comparison model of the similarity detection model to determine the similarity between the detection function and each potential similarity function.

[0159] In the ninth step, the computer determines the target function with the highest similarity to the detection function among the potential similarity functions, and determines the similar code of code A based on the binary code corresponding to the target function. Finally, based on the similar code of code A, it is determined whether code A has a plagiarism risk.

[0160] Figure 7 Schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 7 As shown, the electronic device includes: a sample generation module 71, which is used to establish a training sample set based on an open source binary code data set, and the training sample set includes multiple training samples based on an intermediate language; a model training module 72, which is used to pre-train a basic model based on the Transformer model architecture according to the training sample set to obtain an intermediate language pre-training model; a comparison model is trained based on the intermediate language pre-training model and the training sample set; an embedding model is trained based on the intermediate language pre-training model and the training sample set, and the embedding model includes a query encoder and a reference encoder; a similarity detection model capable of detecting the similarity of binary codes is constructed based on the embedding model and the comparison model. In one embodiment, the description of the specific implementation function of the electronic device can refer to steps S101-S105 in Example 1, which will not be repeated here.

[0161] Figure 8 This is a schematic diagram of the structure of an electronic device according to another embodiment of the present application. Figure 8As shown, the electronic device includes: an embedding detection module 81, which is used to extract the code function of the binary code to be detected, and perform standardization and word segmentation processing on the code function to obtain a detection function; input the detection function into the embedding model of the similarity detection model to construct a corresponding detection embedding vector; determine multiple potential similar functions corresponding to the binary code to be detected according to the detection embedding vector and the embedding model; a comparison detection module 82, which is used to input the detection function and each potential similar function into the comparison model of the similarity detection model to determine the similarity between the detection function and each potential similar function; determine the target function with the highest similarity to the detection function among the potential similar functions; and determine the similar code of the binary code to be detected according to the binary code corresponding to the target function. In one embodiment, the description of the specific implementation function of the electronic device can refer to steps S201-S206 in Example 2, which will not be repeated here.

[0162] Fig. 9 This is a schematic diagram of the structure of an electronic device according to another embodiment of the present application. Fig. 9 As shown, the electronic device includes: a processor 101, and a memory 102 communicatively connected to the processor 101; the memory 102 stores computer execution instructions; the processor 101 executes the computer execution instructions stored in the memory 102 to implement the steps of the training / application method of the similarity detection model in the above-mentioned method embodiments.

[0163] In the above electronic device, the memory 102 and the processor 101 are electrically connected directly or indirectly to realize data transmission or interaction. For example, these elements can be electrically connected to each other through one or more communication buses or signal lines, such as through a bus connection. The memory 102 stores computer-executable instructions for implementing the data access control method, including at least one software function module that can be stored in the memory 102 in the form of software or firmware. The processor 101 executes various functional applications and data processing by running the software programs and modules stored in the memory 102.

[0164] The memory 102 may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable read-only memory (EEPROM), etc. The memory 102 is used to store programs, and the processor 101 executes the programs after receiving the execution instruction. Furthermore, the software programs and modules in the above-mentioned memory 102 may also include an operating system, which may include various software components and / or drivers for managing system tasks (such as memory management, storage device control, power management, etc.), and may communicate with various hardware or software components to provide an operating environment for other software components.

[0165] The processor 101 may be an integrated circuit chip having the ability to process signals. The processor 101 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The methods, steps, and logic diagrams disclosed in the embodiments of the present application may be implemented or executed. The general-purpose processor may be a microprocessor or the processor may be any conventional processor, etc.

[0166] An embodiment of the present application further provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the steps of each method embodiment of the present application.

[0167] An embodiment of the present application further provides a computer program product, including a computer program, which implements the steps of each method embodiment of the present application when executed by a processor.

[0168] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0169] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0170] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0171] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0172] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

[0174] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A training method for a binary code similarity detection model, characterized in that: include: Establishing a training sample set according to an open source binary code data set, wherein the training sample set includes a plurality of training samples based on an intermediate language; Pre-training a basic model based on the Transformer model architecture according to the training sample set to obtain an intermediate language pre-training model; According to the intermediate language pre-training model and the training sample set, a comparison model is trained; According to the intermediate language pre-training model and the training sample set, an embedding model is trained, wherein the embedding model includes a query encoder and a reference encoder; Constructing a similarity detection model capable of detecting similarity of binary codes according to the embedding model and the comparison model; The training to obtain an embedding model according to the intermediate language pre-training model and the training sample set specifically includes: Performing parameter initialization processing on the intermediate language pre-training model to determine initial parameters of the query encoder and initial parameters of the reference encoder; For each training sample in the training sample set, formatting the training sample to obtain a query function and a reference function corresponding to the training sample respectively; Input each query function into the query encoder in batches to obtain the corresponding query embedding, and input each reference function into the reference encoder in batches to obtain the corresponding reference embedding and a reusable reference embedding cache; Calculate the similarity between the query embedding in each batch and the reference embedding and the reference embedding cache, and use the contrastive learning loss function to iteratively update the parameters of the query encoder and the reference encoder to obtain the final query encoder and reference encoder; According to the final query encoder and the reference encoder, an embedding model is constructed.

2. The method according to claim 1, characterized in that: The training sample set is established based on the open source binary code data set, specifically including: Obtain an open source binary code dataset and use a function extraction tool to extract the code functions corresponding to each binary code in the dataset; For each code function, standardize the code function to obtain a corresponding intermediate language IL; Performing word segmentation processing on the intermediate language IL by using a word segmentation algorithm to obtain a training sample corresponding to the code function; According to the training samples corresponding to each code function, a training sample set is established; The open source binary code dataset includes one or more of a BinaryCorp dataset, a Cisco dataset, and a Trex dataset, and the function extraction tool includes a BinaryNinja tool.

3. The method according to claim 1, characterized in that The contrastive learning loss function is used to iteratively update the parameters of the query encoder and the parameters of the reference encoder to obtain the final query encoder and reference encoder, specifically including: The contrastive learning loss for this batch of training is calculated using the following formula based on the similarity between the query embeddings in this batch and the reference embeddings and the reference embedding cache: Among them, Q i , R i is a pair of positive samples of query embedding and reference embedding / reference embedding cache, Q i , R j is a pair of negative examples of query embedding and reference embedding / reference embedding cache, f(x) is the similarity function between the two embeddings, N is the total number of pairs of query embedding and reference embedding, and L is the contrastive learning loss; performing a gradient update on initial parameters of the query encoder according to the contrastive learning loss; Perform momentum update on the initial parameters of the reference encoder according to the updated parameters of the query encoder; Update the reusable reference embedding cache according to the next batch of reference embeddings and the embedding cache size; Repeat the above steps, and iteratively update the parameters of the query encoder and the reference encoder by back-propagation according to the updated reusable reference embedding cache and the contrastive learning loss of each batch training to obtain the final query encoder and reference encoder.

4. The method according to claim 3, characterized in that: The step of performing momentum updating on the initial parameters of the reference encoder according to the updated parameters of the query encoder specifically includes: The momentum update of the initial parameters of the reference encoder is performed using the following formula: i r -mθ' r +(1-m)θ q Among them, θ r represents the updated parameters of the reference encoder, θ′ r represents the initial parameters of the reference encoder, θ q represents the updated parameters of the query encoder, and m represents the momentum coefficient.

5. An application method of the binary code similarity detection model according to any one of claims 1 to 4, characterized in that: include: Extracting a code function of the binary code to be detected, and performing standardization and word segmentation processing on the code function to obtain a detection function; Inputting the detection function into the embedding model of the similarity detection model to construct a corresponding detection embedding vector; Determining a plurality of potential similarity functions corresponding to the binary code to be detected according to the detection embedding vector and the embedding model; Inputting the detection function and each potential similarity function into a comparison model of a similarity detection model to determine the similarity between the detection function and each potential similarity function; Determine a target function among potential similarity functions that has the highest similarity to the detection function; According to the binary code corresponding to the target function, a similar code to the binary code to be detected is determined.

6. The method according to claim 5, characterized in that The step of determining, according to the detection embedding vector and the embedding model, a plurality of potential similarity functions corresponding to the binary code to be detected specifically includes: Determining a plurality of potential similar functions corresponding to the binary code to be detected according to the detection embedding vector and the vector index of each function in the embedding model; The vector index of each function is constructed using a vector search engine based on an approximate nearest neighbor algorithm.

7. An electronic device comprising: A sample generation module, used to establish a training sample set according to an open source binary code data set, wherein the training sample set includes a plurality of training samples based on an intermediate language; A model training module is used to pre-train a basic model based on the Transformer model architecture according to the training sample set to obtain an intermediate language pre-training model; train a comparison model according to the intermediate language pre-training model and the training sample set; train an embedding model according to the intermediate language pre-training model and the training sample set, wherein the embedding model includes a query encoder and a reference encoder; and construct a similarity detection model capable of detecting similarity of binary codes according to the embedding model and the comparison model; The model training module is specifically used for: Performing parameter initialization processing on the intermediate language pre-training model to determine initial parameters of the query encoder and initial parameters of the reference encoder; For each training sample in the training sample set, formatting the training sample to obtain a query function and a reference function corresponding to the training sample respectively; Input each query function into the query encoder in batches to obtain the corresponding query embedding, and input each reference function into the reference encoder in batches to obtain the corresponding reference embedding and a reusable reference embedding cache; Calculate the similarity between the query embedding in each batch and the reference embedding and the reference embedding cache, and use the contrastive learning loss function to iteratively update the parameters of the query encoder and the reference encoder to obtain the final query encoder and reference encoder; According to the final query encoder and the reference encoder, an embedding model is constructed.

8. An electronic device, characterized in that: comprising a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 6 when executed by a processor.

Images