A Clone Code Detection Method and System for Mining Features from Assembly Language

Abstract

The present invention discloses a clone code detection method and system for mining features from assembly language. The method includes: obtaining a target code and a comparison code; generating a target assembly code set and a comparison assembly code set based on multiple target compilation platforms; using the Asm2vec model to generate a corresponding target semantic embedding feature set and a comparison semantic embedding feature set; respectively using the BERT model for the target code and the comparison code to generate a target text embedding feature and a comparison text embedding feature of the corresponding high-level source code; concatenating the target semantic embedding feature set and the target text embedding feature to form a target code feature representation; concatenating the comparison semantic embedding feature set and the comparison text embedding feature to form a comparison code feature representation; inputting the target code feature representation and the comparison code feature representation into a classification model of a DNN deep neural network to output a detection result. It can effectively solve the semantic cloning problem caused by the diversity and flexibility of source code.

Description

Technical Field

[0001] The present invention relates to the technical field of code clone detection, and particularly to a method and system for detecting cloned code by mining features from assembly language. Background Art

[0002] Code Clone refers to code segments that are the same or have a similarity reaching a set threshold in a code library, which is a common phenomenon in software development. By means of "copy-paste-modify", developers can rewrite code segments that are functionally similar to the source code in a short time, thereby shortening the development cycle and improving development efficiency. With the promotion of the open-source code community, in order to promote the learning and communication among developers in the community, the way of code clone reuse is being encouraged by the community. Although appropriate code clone reuse can bring convenience to development, code clone behaviors that involve a large amount of code modification by imitating existing code on the Internet will also have many negative impacts. In the field of education, the phenomenon of students plagiarizing code assignments is not uncommon, which to a certain extent affects educational fairness. In enterprises, it will cause sensitive issues such as intellectual property ownership. In the field of software development, such behaviors will not only bring great difficulties to the maintenance of software quality, but may also introduce hard-to-detect vulnerabilities, etc. Therefore, detecting code clones in software systems can enable developers to more intuitively face the problem of code clones and facilitate more fully optimizing and modifying the cloned code in the system.

[0003] Code clone detection technology is a very crucial basic task in software engineering and is widely applied to numerous downstream tasks, such as refactoring, code reuse, and code search. Generally, code clones are divided into four different types of similarity degrees, namely, complete clone, parameterized clone, gap clone, and semantic clone. For the first three types of clones, existing methods can detect them well. However, for semantic clones, different code segments have similar semantics. Due to the flexibility of source code, it is difficult to align different code representations in the semantic space. Therefore, code semantic clone detection has become a quite challenging problem. Summary of the Invention

[0004] The purpose of the present invention is to provide a method and system for detecting cloned code by mining features from assembly language. Based on the features extracted from high-level language and assembly language, the problem of detecting code semantic clones can be effectively solved.

[0005] To achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0006] In the first aspect, the present invention provides a method for detecting cloned code by mining features from assembly language, including the following steps:

[0007] S1. Obtain the target code and the comparison code; generate their respective corresponding target assembly code sets and comparison assembly code sets based on multiple target compilation platforms; use the Asm2vec model for the target assembly code set and the comparison assembly code set respectively to generate the corresponding target semantic embedding feature set and comparison semantic embedding feature set;

[0008] S2. For the target code and the comparison code, use the BERT model to generate the target text embedding feature and comparison text embedding feature of the corresponding high-level source code respectively;

[0009] S3. Concatenate the target semantic embedding feature set and the target text embedding feature to form a target code feature representation; concatenate the comparison semantic embedding feature set and the comparison text embedding feature to form a comparison code feature representation;

[0010] S4. Input the target code feature representation and the comparison code feature representation into a classification model based on a DNN deep neural network to output a detection result.

[0011] Further, in the Asm2Vec model in step S1, each function of the assembly code is regarded as a set of sequences, and the assembly code is converted into a sequence by means of edge sampling and random walk to obtain the vector representation of all functions in the assembly code.

[0012] Further, step S1 includes:

[0013] S11. Obtain the target code and the comparison code, and remove irrelevant information;

[0014] S12. Select multiple target compilation platforms, use the GCC cross-compiler tool as the assembly code generator, and use the parameter -S to dump the assembly code to generate their respective corresponding target assembly code sets and comparison assembly code sets;

[0015] S13. Use the Asm2vec model for each target assembly code set and comparison assembly code set respectively to obtain the vector representation of all functions in the target assembly code set, and obtain the vector representation of all functions in the comparison assembly code set;

[0016] S14. Perform an average pooling operation on the vector representation of all functions in the target assembly code set under each target compilation platform to obtain an average value; use the average values corresponding to multiple target compilation platforms respectively as the target semantic embedding feature set;

[0017] Perform an average pooling operation on the vector representation of all functions in the comparison assembly code set under each target compilation platform to obtain an average value; use the average values corresponding to multiple target compilation platforms respectively as the comparison semantic embedding feature set.

[0018] Further, the BERT model in the step S2 has a linear layer and a text classification block; the linear layer is used to adjust the output dimension of the BERT model and balance the ratio of the target text embedding features and the contrast text embedding features; the text classification block is used to generate task-level feature embeddings.

[0019] Further, in the step S4, a classification model based on a DNN (Deep Neural Network) is used to handle the clone detection task as a binary classification task; the input of the model is the feature representations of two source codes, and the output is the clone detection result; the clone detection result includes: two source codes under the same task are regarded as a clone pair, while two source codes under different tasks are regarded as a non-clone pair.

[0020] In a second aspect, the present invention further provides a clone code detection system for mining features from assembly language, including:

[0021] A semantic perception module, configured to obtain a target code and a comparison code; generate respective corresponding target assembly code sets and comparison assembly code sets based on multiple target compilation platforms; use the Asm2vec model for the target assembly code set and the comparison assembly code set respectively to generate corresponding target semantic embedding feature sets and comparison semantic embedding feature sets;

[0022] A text perception module, configured to respectively use a BERT model for the target code and the comparison code to generate corresponding target text embedding features and comparison text embedding features of the high-level source code;

[0023] A concatenation module, configured to concatenate the target semantic embedding feature set and the target text embedding features to form a target code feature representation; concatenate the comparison semantic embedding feature set and the comparison text embedding features to form a comparison code feature representation;

[0024] A classification module, configured to input the target code feature representation and the comparison code feature representation into a classification model based on a DNN (Deep Neural Network) and output a detection result.

[0025] Further, in the Asm2Vec model of the semantic perception module, each function of the assembly code is regarded as a set of sequences, and the assembly code is converted into a sequence by means of edge sampling and random walk to obtain the vector representation of all functions in the assembly code.

[0026] Further, the semantic perception module includes:

[0027] An acquisition unit, configured to obtain a target code and a comparison code and remove irrelevant information;

[0028] Generate an assembly code unit, which is used to select multiple target compilation platforms, use the GCC cross-compiler tool as the assembly code generator, and use the parameter -S to dump the assembly code to generate their respective corresponding target assembly code sets and comparison assembly code sets;

[0029] The function vector representation unit is used to use the Asm2vec model for each of the target assembly code set and the comparison assembly code set to obtain the vector representations of all functions in the target assembly code set and the vector representations of all functions in the comparison assembly code set;

[0030] The semantic embedding establishment unit is used to perform average pooling operations on the vector representations of all functions in the target assembly code set under each target compilation platform to obtain an average value; use the average values corresponding to each of the multiple target compilation platforms as the target semantic embedding feature set;

[0031] It is also used to perform average pooling operations on the vector representations of all functions in the comparison assembly code set under each target compilation platform to obtain an average value; use the average values corresponding to each of the multiple target compilation platforms as the comparison semantic embedding feature set.

[0032] Furthermore, the BERT model in the text perception module has a linear layer and a text classification block; the linear layer is used to adjust the output dimension of the BERT model to balance the ratio of the target text embedding features and the comparison text embedding features; the text classification block is used to generate task-level feature embeddings.

[0033] Furthermore, in the classification module, a classification model based on the DNN deep neural network is used to handle the clone detection task as a binary classification task; the input of the model is the feature representations of two source codes, and the clone detection result is output; the clone detection result includes: two source codes under the same task are regarded as a clone pair, while two source codes under different tasks are regarded as a non-clone pair.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] A clone code detection method for mining features from assembly language provided by an embodiment of the present invention extracts corresponding semantic embedding features and text embedding features from the high-level language domain and the assembly language domain respectively for two pieces of program source code to mine code behavior features; and simplifies the code representation process, saving time; finally, connects the code representations of the source code and the assembly code, inputs a classification model based on the DNN deep neural network, outputs the detection result, and completes the code clone detection, which can effectively solve the semantic clone problem caused by the diversity and flexibility of the source code. Description of the Drawings

[0036] Figure 1a Schematic diagram of source code A and the source code after compilation when it runs on the x86 architecture;

[0037] Figure 1b Schematic diagram of source code B and the source code after compilation when it runs on the x86 architecture;

[0038] Figure 2a Schematic diagram of the differences between CISC and RISC in two different pieces of code with the same semantics;

[0039] Figure 2b Schematic diagram of the differences between CISC and RISC in another two different pieces of code with the same semantics;

[0040] Figure 3 Flowchart of the clone code detection method for mining features from assembly language provided by an embodiment of the present invention;

[0041] Figure 4 Principle diagram of the clone code detection for mining features from assembly language provided by an embodiment of the present invention;

[0042] Figure 5 Structure diagram of adding a linear layer to the BERT model provided by an embodiment of the present invention;

[0043] Figure 6 For Figure 5 Structure diagram of adding a text classification block on the basis of;

[0044] Figure 7 Structure diagram of the classification model based on the DNN deep neural network provided by an embodiment of the present invention;

[0045] Figure 8 Block diagram of the clone code detection system for mining features from assembly language provided by an embodiment of the present invention. Detailed implementation manners

[0046] To make the technical means, creative features, achieved purposes and functions of the present invention easy to understand, the present invention will be further described below in conjunction with specific implementation manners.

[0047] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "upper", "lower", "inner", "outer", "front end", "rear end", "both ends", "one end", "the other end", etc. is the orientation or positional relationship based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0048] In the description of the present invention, it should be noted that, unless otherwise clearly specified and defined, the terms "installation", "equipped with", "connection", etc. shall be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0049] In previous related technology research, people used various static analysis techniques to extract information from high-level languages and understand the code, including Abstract Syntax Trees (ASTs), Control Flow Graphs (CFGs), Data Flow, and call relationships. However, due to the diversity of the expression methods and structural characteristics of high-level languages, relying solely on the information extracted from the high-level language domain can easily mislead the model's understanding of program semantic information.

[0050] As Figure 1a - 1b shown, it is the difference between two code segments with the same semantics between the high-level language domain and the assembly language domain. Above the demarcation line is the high-level language domain, and below is the corresponding assembly language domain. "num << 3" in Code A can be modified to "a * 8" in Code B, making it difficult to detect semantic cloning problems at the high-level language level. With the help of compilers and various code optimization tools, many "semantic cloning" problems can be effectively eliminated at the assembly level. As can be seen from the bottom of Figure 1a and Figure 1b , when processing the assembly code generated by the compiler, there are only slight differences in the rectangular area.

[0051] Assembly language-level code is a feature representation that provides code interaction with the memory system (through load / store instructions) and information about program execution. Architecture-dependent assembly code can provide more opportunities to understand program semantics.

[0052] First, the diversity of Complex Instruction Set Computers (CISCs) provides more information for understanding programs. For example, Figure 2aShows the arithmetic operations of multiplication and addition using different codes with the same semantics. Above the horizontal demarcation line is Code 1. The left side is the high-level language domain, and the right side is the corresponding assembly language domain (i.e., the assembly language generated for the CISC instruction set); below the horizontal demarcation line is Code 2. The left side is the high-level language domain, and the right side is the corresponding assembly language domain (i.e., the assembly language generated for the CISC instruction set). The leal instruction can standardize the semantic differences of high-level languages in code behavior, which helps code clone detection. However, multiple instruction optimization techniques make it more challenging to understand the semantics. Since there are multiple instructions with similar semantics in CISC ISAs, during the instruction generation process, high-level language descriptions with minor differences may result in significant differences at the assembly language level. For example, using a temporary variable in Figure 2b does not change the semantics, but the optimizations triggered in CISC lead to different instruction generations. Above the horizontal demarcation line is Code a. The left side is the high-level language domain, and the right side is the corresponding assembly language domain (where the rectangular area is the assembly language generated using the CISC instruction set, and outside the rectangular area is the assembly language generated using the RISC instruction set); below the horizontal demarcation line is Code b. The left side is the high-level language domain, and the right side is the corresponding assembly language domain (where the rectangular area is the assembly language generated using the CISC instruction set, and outside the rectangular area is the assembly language generated using the RISC instruction set). However, although RISC instructions are relatively simple, they can better reflect the core logic of program execution, and compiler optimization techniques (such as data flow analysis) can eliminate this difference and improve clone detection.

[0053] It can be seen that the execution trace in the assembly code can provide program behavior semantic information, which is very helpful for code clone detection. Single instruction set architectures (ISAs) have their own performances due to different characteristics, and ISA combinations can complement each other to strengthen the understanding of code semantics. Therefore, the behavior semantics from the assembly languages of multiple architectures can be used to help align high-level languages in the semantic space.

[0054] Therefore, the present invention proposes a clone code detection method and system for mining code behavior characteristics from the assembly languages of multiple architectures.

[0055] Example 1:

[0056] A clone code detection method for mining code behavior characteristics from the assembly languages of multiple architectures, using multiple language domains, including high-level languages and assembly languages. Referring to Figure 3 as shown, the specific steps include:

[0057] S1. Obtain the target code and the comparison code; generate their respective corresponding target assembly code sets and comparison assembly code sets based on multiple target compilation platforms; use the Asm2vec model for the target assembly code set and the comparison assembly code set respectively to generate the corresponding target semantic embedding feature set and comparison semantic embedding feature set;

[0058] S2. For the target code and the comparison code, use the BERT model to generate the target text embedding feature and comparison text embedding feature of the corresponding high-level source code respectively;

[0059] S3. Concatenate the target semantic embedding feature set and the target text embedding feature to form a target code feature representation; concatenate the comparison semantic embedding feature set and the comparison text embedding feature to form a comparison code feature representation;

[0060] S4. Input the target code feature representation and the comparison code feature representation into a classification model based on a DNN deep neural network to output a detection result.

[0061] In this embodiment, for the target code and the comparison code, this method extracts behavioral semantics from the low-level assembly code and lexical features from the high-level source code respectively, and then combines the information from multiple domains for code clone detection. Specifically, it consists of a semantic perception step, a text perception step, a concatenation step, and a classification step, corresponding to steps S1 to S4 above respectively; among them, the execution order of steps S1 and S2 is not sequential and can also be executed simultaneously, and the embodiments of the present disclosure do not make any limitations in this regard.

[0062] The clone code detection method for mining code behavior features from assembly languages with multiple architectures has a general principle framework as Figure 4 shown. For the processing of the target code or the comparison code, after they are processed separately, finally, the code feature representations corresponding to them are input into the classification model together to output a detection result.

[0063] In step S1, based on the assembly code generated by the compiler, use the asm2vec model to generate semantic embedding features. In specific implementation, multiple versions of assembly languages can be accepted, and the generated embedding features are respectively marked as E CISC , E RISC . In step S2, use the BERT model to generate the text embedding feature of the high-level source code, marked as E LAM . In step S3, the semantic embedding feature and the text embedding feature are concatenated to form a feature representation of the source code, that is, ψ(C s ).

[0064] ψ(C s ) = concat[E CISC ||E RISC ||ELAM (1)

[0065] In step S4, a classification model based on a deep neural network (DNN) is used to evaluate the code clone detection effect. The overall code clone detection model can be represented in the following form:

[0066] classify(C s ,C s ') = F(ψ(C s ), ψ(C s '),) (2)

[0067] where C s ,C s ' are two source code segments, F is a binary classification module, which is a linear model, and ψ() is the feature embedding representation framework generated using high-level language and assembly language introduced above.

[0068] The following is a detailed description of each of the above steps:

[0069] In step S1, the Asm2vec method is used to obtain behavioral semantic embedding features from the assembly code. Since the statements in the assembly code are always very short and cannot easily convey effective information, the statements are combined with context information. In the Asm2Vec model, each function in the assembly code training data is regarded as a set of sequences S(f) = seq[1:m]. The assembly code is converted into a sequence by means of edge sampling and random walk, and then the model performs semantic embedding on the sequence. During the model training process, the semantic embedding is the vector to be learned. In the Asm2vec neural model, just like in the PV-DM model, the sliding window starts from the beginning of the sequence and moves one step at a time within a single sequence. For each step, the model predicts the central sequence vector based on the adjacent sequences and the semantic embedding, and updates the sequence vector and the semantic embedding according to the loss function until the training ends. The vectors provided by the adjacent sequences can capture the behavioral semantic relationships and enhance the semantic representation ability.

[0070] The specific process is as follows: First, the corresponding compiler is used to generate the target assembly code for the high-level language source code, and then the semantic embedding features are extracted from the assembly code. The semantic embeddings of the assembly code based on different architectures are concatenated for program feature representation, such as X86, ARM, RISC-V, and MIPS. The data distributions of the assembly code of different architectures are different, and this simple feature fusion enables them to represent more comprehensive semantic information. Note that this embodiment can be flexibly designed to support any number of assembly program types.

[0071] For example, taking the ARM and X86 architectures as examples, the specific implementation methods are as follows. Data preprocessing is performed on the selected target code and comparison code, including three parts:

[0072] (1) Remove irrelevant information such as comments and blank lines in the original program.

[0073] (2) Select ARM and X86 as the target assembly platforms, the GCC cross-compiler tool as the assembly code generator, and use the parameter -S to dump the assembly code.

[0074] (3) Use the Asm2vec model to obtain the vector representations of all functions in the target assembly code set and the vector representations of all functions in the comparison assembly code set respectively; perform average pooling operations on the vector representations of all functions in the target assembly code sets under the ARM and X86 target compilation platforms to obtain the average values as the target semantic embedding feature sets; perform average pooling operations on the vector representations of all functions in the comparison assembly code sets under the ARM and X86 target compilation platforms to obtain the average values as the comparison semantic embedding feature sets;

[0075] The training process of the Asm2Vec model is as follows:

[0076] First, extract the function names in the corresponding assembly program as the training parameters of the Asm2Vec model. Finally, input the processed file into Asm2Vec for semantic embedding. Since Asm2Vec is a function-level method, using Asm2Vec for training can obtain the vector representations of all functions in the assembly code set. And since some assembly codes contain more than one function, average pooling operations (taking the average value) need to be performed on the function vector representations in such assembly codes to establish the semantic embedding based on the target program.

[0077] For example, the dataset used in the experiment is OJClone. This open-source dataset is a collection of c / c++ program codes submitted by different users for the same programming task. 35 programming tasks are randomly selected, and 100 original programs are extracted for each task. The total number of programs is 3500. That is, two C / C++ language files under the same programming task are clone pairs, and two C / C++ language files under different programming tasks are non-clone pairs.

[0078] When performing classification training at the end, the input is: The three extracted features combined into an overall feature represent a C language file. Since we are comparing whether two files are similar codes, the input is <ECISC1, ERISC1, ELAM1, ECISC2, ERISC2, ELAM2, 0 / 1>. Among these inputs, the ratio of cloned pairs to non-cloned pairs is 1:1, with a total of approximately 700,000 entries. 80% of these 700,000 entries are the training set train, and 20% are the test set. The test set does not participate in model training and is used to test the effectiveness of the trained classification model.

[0079] In step S2, the BERT model is used to extract text embedding features from the source code. Specifically, the bert-base-uncased model is used, which is a pre-trained general representation model provided by Google to eliminate specific high-level programming language restrictions and enhance the generality of the model.

[0080] In addition, in this embodiment, a linear layer is added on the basis of the original BERT model to change the output dimension, as Figure 5 shown. In this way, the output dimension of the BERT model can be adjusted to balance the ratio of text feature embedding to assembly feature embedding. Compared with the traditional BERT model that generates high-dimensional vectors (768 dimensions), this embodiment also reduces the computational cost of downstream tasks.

[0081] Inspired by the text classification task in natural language processing (NLP, Natural Language Processing) technology, this embodiment performs task-level fine-tuning in step S2. Two marked code snippets from the downstream task are input into LAM (a linear layer is added on the basis of the original BERT model to change the output dimension, and LAM represents the BERT model plus the linear layer) for training (still the source code of the OJClone dataset, and the input format is <source code 1,source code2,label> ), to maximize the classification accuracy and enable LAM to better represent the code and adapt to the classification task; that is: <sourcecode 1, source code 2, label> is input into the model of LAM + a text classification block and a softmax classifier for training and fine-tuning. During the fine-tuning process, only the linear layer and the text classification block are updated, and the parameters of the BERT model are frozen. The training framework is as Figure 6 shown, that is: in Figure 5On the basis of [the original model], a text classification block is added. This model can accept the supervision and guidance of downstream tasks, indicating that the model has learned downstream task knowledge to generate task-level embedding features. After training is completed, the linear layer has learned the target task knowledge. By removing the text classification block and a softmax classifier, the text embedding vector representation can be generated. In addition, this method completes the domain knowledge transfer from natural language (NL) to programming language (PL) using only a small amount of labeled data in the target domain.

[0082] During the fine-tuning process, only the linear layer and the text classification block can be updated, and the parameters of the BERT model are frozen. The linear layer and the text classification block have been trained under the pure language setting of the target task. Since it is not necessary to update all the parameters of the network, the data distributions of NL and PL can be effectively aligned, and the computational cost is also greatly reduced.

[0083] The specific implementation method is as follows: To generate task-level feature embeddings, the linear layer of the model is fine-tuned using the OJClone dataset. A text classification block and a softmax classifier are added to the LAM. The input is the output of the LAM, and the output is the classification result. After fine-tuning, the LAM model can strip the softmax classifier and directly generate the feature embedding for the source code. (In this way, task knowledge can be learned based on OJClone, making the generated embedding features [0.123, 0.354,...] of the source code more accurate).

[0084] Similar to the classic BERT text classification dataset MRPC, the data format is <C s , C s ', label>. To maintain the data independent and identically distributed, the ratio of cloned pairs to non-cloned pairs is 1:1. According to the overall data volume of the MRPC dataset, for example, 7000 training data are used, and the loss function is as shown in formula (3):

[0085]

[0086] Among them, N represents the total number of samples, y [i] represents the true value of the i-th sample, represents the predicted value of the i-th sample.

[0087] In step S3, the target semantic embedding feature set in step S1 and the target text embedding features in step S2 are concatenated to form the target code feature representation; similarly, the contrast semantic embedding feature set in step S1 and the contrast text embedding features in step S2 are concatenated to form the contrast code feature representation.

[0088] In step S4, a classification model based on DNN is designed, as Figure 7 shown. In this embodiment, the finally obtained code representation is a high-dimensional dense vector, and the DNN model performs well on this vector. A Dropout layer can be added to the classification model to avoid the overfitting problem caused by the parameters being too dependent on the training data, thereby improving the generalization ability of the model for different data sets. The classification model has a total of 5 hidden layers, and the number of neurons in each hidden layer is 576, 1152, 2304, 1152, and 576 respectively.

[0089] Finally, the clone detection task is processed as a binary classification task through the classification model. The input of the model is the code representations of two source codes, and the output is the clone detection result. Two programs under the same task can be regarded as a clone pair, while programs under different tasks are regarded as non-clone pairs.

[0090]

[0091] According to formula (4), where T represents the task domain, when two source codes come from the same task domain, the output of the classification module is 1; when two source codes come from different task domains, the output of the classification module is 0.

[0092] An embodiment of the present invention proposes a clone code detection method for mining code behavior characteristics from assembly languages of multiple architectures. CISC is used to normalize the semantic differences of high-level languages and capture different characteristics of programs in different scenarios; RISC is used to determine the actual execution trace of the program. Mining behavioral semantic information from RISC and CISC effectively solves the "semantic clone" problem caused by the diversity and flexibility of source codes. At the same time, the NLP model is used to effectively capture syntax and semantic information from high-level languages and assembly languages, greatly simplifying the code representation process and saving time. Finally, the code representations of the source code and the assembly code are concatenated and sent into the classification model to complete code clone detection and improve the performance of code clone detection. Among them, the present invention also conducts comprehensive experiments based on the OJClone data set. The results show that this method is superior to the latest methods in the prior art in terms of clone detection performance and time cost. Specifically, the accuracy of clone code detection reaches 0.990, the recall rate reaches 0.984, and the F1 value reaches 0.987. At the same time, the time consumption of training and inference is reduced by 44.30% and 59.06% respectively.

[0093] Embodiment 2:

[0094] Based on the same inventive concept, the present invention also provides a clone code detection system for mining features from assembly languages, as Figure 8 shown, including:

[0095] A semantic perception module, configured to obtain a target code and a comparison code; generate respective corresponding target assembly code sets and comparison assembly code sets based on multiple target compilation platforms; use the Asm2vec model for the target assembly code set and the comparison assembly code set respectively to generate corresponding target semantic embedding feature sets and comparison semantic embedding feature sets;

[0096] A text perception module, configured to use the BERT model for the target code and the comparison code respectively to generate a target text embedding feature and a comparison text embedding feature of a corresponding high-level source code;

[0097] A concatenation module, configured to concatenate the target semantic embedding feature set and the target text embedding feature to form a target code feature representation; concatenate the comparison semantic embedding feature set and the comparison text embedding feature to form a comparison code feature representation;

[0098] A classification module, configured to input the target code feature representation and the comparison code feature representation into a classification model based on a DNN deep neural network and output a detection result.

[0099] Since the principle of the problem solved by this system is similar to the foregoing clone code detection method for mining features from assembly language, the implementation of this system can refer to the implementation of the foregoing method, and the repeated parts will not be described again.

[0100] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) containing computer-usable program code.

[0101] The present invention is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, and the combination of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in one process or multiple processes in the flowchart and / or one block or multiple blocks in the block diagram.

[0102] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to operate in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means embodying the functionality specified in one or more blocks of the flowchart and / or one or more blocks of the block diagram.

[0103] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functionality specified in one or more blocks of the flowchart and / or one or more blocks of the block diagram.

[0104] It will be apparent to those skilled in the art that various modifications and variations can be made to the present invention without departing from the spirit and scope of the invention. Thus, if these modifications and variations of the present invention come within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these modifications and variations.

Claims

1. A method for detecting cloned code by mining features from assembly language, characterized in that, It includes the following steps: S1. Obtain the target code and the comparison code; Generate their respective corresponding target assembly code sets and comparison assembly code sets based on multiple target compilation platforms; use the Asm2vec model for the target assembly code set and the comparison assembly code set respectively to generate the corresponding target semantic embedding feature set and comparison semantic embedding feature set; S2. For the target code and the comparison code, use the BERT model respectively to generate the target text embedding feature and comparison text embedding feature of the corresponding high-level source code; S3. Concatenate the target semantic embedding feature set and the target text embedding feature to form a target code feature representation; Concatenate the comparison semantic embedding feature set and the comparison text embedding feature to form a comparison code feature representation; S4. Input the target code feature representation and the comparison code feature representation into a classification model based on a DNN deep neural network to output a detection result.

2. The clone code detection method for mining features from assembly language according to claim 1, wherein In the Asm2vec model of step S1, each function of the assembly code is regarded as a set of sequences, and the assembly code is converted into a sequence by means of edge sampling and random walk to obtain the vector representation of all functions in the assembly code.

3. A method for detecting cloned code by mining features from assembly language according to claim 1, characterized in that The step S1 includes: S11. Obtain the target code and the comparison code, and remove irrelevant information; S12. Select multiple target compilation platforms, use the GCC cross-compiler tool as the assembly code generator, and use the parameter -S to dump the assembly code to generate their respective corresponding target assembly code sets and comparison assembly code sets; S13. Use the Asm2vec model for each of the target assembly code set and the comparison assembly code set to obtain the vector representation of all functions in the target assembly code set and the vector representation of all functions in the comparison assembly code set; S14. Perform an average pooling operation on the vector representations of all functions in the target assembly code set under each target compilation platform to obtain an average value; use the average values corresponding to multiple target compilation platforms respectively as the target semantic embedding feature set; Perform an average pooling operation on the vector representations of all functions in the comparison assembly code set under each target compilation platform to obtain an average value; use the average values corresponding to multiple target compilation platforms respectively as the comparison semantic embedding feature set.

4. A method for detecting cloned code by mining features from assembly language according to claim 1, characterized in that, The BERT model in step S2 has a linear layer and a text classification block; the linear layer is used to adjust the output dimension of the BERT model and balance the ratio of the target text embedding feature and the comparison text embedding feature; the text classification block is used to generate task-level feature embeddings.

5. A method for detecting cloned code by mining features from assembly language according to claim 1, characterized in that, In step S4, the classification model based on the DNN deep neural network is used to handle the clone detection task as a binary classification task; The input of the model is the feature representations of two source codes, and the clone detection result is output; the clone detection result includes: two source codes under the same task are regarded as a clone pair, while two source codes under different tasks are regarded as a non-clone pair.

6. A clone code detection system for mining features from assembly language, characterized in that, It includes: A semantic perception module, which is used to obtain the target code and the comparison code; Generate their respective corresponding target assembly code sets and comparison assembly code sets based on multiple target compilation platforms; The target assembly code set and the comparison assembly code set are respectively used with the Asm2vec model to generate corresponding target semantic embedding feature sets and comparison semantic embedding feature sets; A text perception module is used to respectively adopt a BERT model for the target code and the comparison code to generate target text embedding features and comparison text embedding features of corresponding high-level source codes; A concatenation module is used to concatenate the target semantic embedding feature set and the target text embedding features to form a target code feature representation; The comparison semantic embedding feature set and the comparison text embedding features are concatenated to form a comparison code feature representation; A classification module is used to input the target code feature representation and the comparison code feature representation into a classification model based on a DNN (Deep Neural Network) to output a detection result.

7. A cloned code detection system for mining features from assembly language according to claim 6, characterized in that, In the Asm2vec model of the semantic perception module, each function of the assembly code is regarded as a set of sequences, and the assembly code is converted into a sequence by means of edge sampling and random walk to obtain vector representations of all functions in the assembly code.

8. A cloned code detection system for mining features from assembly language according to claim 6, characterized in that The semantic perception module includes: An acquisition unit is used to acquire the target code and the comparison code and remove irrelevant information; An assembly code generation unit is used to select multiple target compilation platforms, adopt a GCC cross-compiler tool as an assembly code generator, and use the parameter -S to dump the assembly code to generate respective corresponding target assembly code sets and comparison assembly code sets; A function vector representation unit is used to respectively use the Asm2vec model for each of the target assembly code set and the comparison assembly code set to obtain vector representations of all functions in the target assembly code set and obtain vector representations of all functions in the comparison assembly code set; A semantic embedding establishment unit is used to perform average pooling operation on the vector representations of all functions in the target assembly code set under each target compilation platform to obtain an average value; the average values corresponding to multiple target compilation platforms are used as the target semantic embedding feature set; It is also used to perform average pooling operation on the vector representations of all functions in the comparison assembly code set under each target compilation platform to obtain an average value; the average values corresponding to multiple target compilation platforms are used as the comparison semantic embedding feature set.

9. A clone code detection system for mining features from assembly language according to claim 6, characterized in that, The BERT model in the text perception module has a linear layer and a text classification block; the linear layer is used to adjust the output dimension of the BERT model to balance the ratio of the target text embedding features and the comparison text embedding features; the text classification block is used to generate task-level feature embeddings.

10. A cloned code detection system for mining features from assembly language according to claim 6, characterized in that, In the classification module, the classification model based on the DNN deep neural network is used to handle the clone detection task as a binary classification task; The input of the model is the feature representations of two source codes, and the clone detection result is output; the clone detection result includes: two source codes under the same task are regarded as a clone pair, while two source codes under different tasks are regarded as a non-clone pair.

Images