Virtualized code restoration method and device based on Transform neural network architecture and medium
Through the Transformer neural network method, the corresponding relationship between virtualized code and original code is learned, and the code reverse model is generated, which solves the problem of low accuracy in virtualized code restoration, and realizes the ability to automatically extract and adapt to new technologies.
Patent Information
- Application Number
- CN202410997400.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-24
- Publication Date
- 2024-11-15
AI Technical Summary
The existing virtualized code restoration technology has increased exponentially due to the exponential increase in the cost of writing feature templates, resulting in low accuracy of virtualized code restoration and is unable to effectively deal with changes in random features and the latest technologies.
Using a method based on Transformer neural network architecture, the virtualized code in the pre-trained code corpus is merged with the original code, and through word segmentation and vector space representation, the correspondence between the virtualized code and the original code is learned, and the code reverse model is generated. Automatically extract features to realize the restoration of virtualized code.
Improve the accuracy of virtualized code restoration, avoid exponential increase in writing costs caused by feature changes, and automatically adapt to new data sets and tasks without manual extraction.
Smart Images

Figure CN118963828A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a virtualized code restoration method, device and medium based on a Transformer neural network architecture. Background Art
[0002] Code virtualization can protect software copyrights, but virtualized code can also become an accomplice of malicious code. However, with the advent of the big model era, some simple but tedious tasks in the field of code reverse engineering can be quickly solved with today's powerful algorithms and computing power, reducing the time cost that originally took several months to minutes.
[0003] Existing virtualization code restoration technologies mostly rely on feature template engineering to manually write code to process and match virtualization code. In actual applications, virtualization features are constantly changing and even the latest technologies have random features. The cost of writing feature templates will increase exponentially, resulting in overly one-sided virtualization code restoration, resulting in low accuracy when performing virtualization code restoration. Summary of the invention
[0004] The present invention provides a virtualized code restoration method, device and medium based on a Transformer neural network architecture, the main purpose of which is to solve the problem of low accuracy when performing virtualized code restoration.
[0005] To achieve the above object, the present invention provides a virtualized code restoration method based on a Transformer neural network architecture, comprising:
[0006] Merge the virtualized code and original code in the pre-trained code corpus into a code corpus;
[0007] Dividing the code corpus into a training data set, an adjustment data set, and a verification data set;
[0008] Segmenting the code data in the training data set to generate multiple codes, each code segmentation corresponds to a unique identifier, and using a preset algorithm to generate a code vector space corresponding to the unique identifier;
[0009] According to the code vector space, the Transformer neural network is used to learn the corresponding relationship between the virtualized code and the original code to obtain a large code reverse model;
[0010] The parameters of the code reverse engineering model are adjusted according to the adjustment data set, the code reverse engineering model after parameter adjustment is verified by data induction according to the verification data set, and the target virtualized code is restored to the original code using the verified code reverse engineering model.
[0011] Optionally, merging the virtualized code and the original code in the pre-trained code corpus into a code corpus includes:
[0012] Divide each virtualized code in the pre-trained code corpus into code lines according to preset line identifiers, and use the code lines corresponding to each virtualized code after division as virtual code lines;
[0013] Divide each original code in the pre-trained code corpus into code lines according to preset line identifiers, and use the code lines corresponding to each divided original code as the original code lines;
[0014] Query all original code lines that match the virtual code lines;
[0015] Constructing code text pairs by combining the virtual code line with each of all the matched original code lines one by one;
[0016] All code-text pairs are collected into a code corpus.
[0017] Optionally, dividing the code corpus into a training data set, an adjustment data set, and a verification data set includes:
[0018] extracting code-text pairs from the code corpus;
[0019] Assigning index numbers to the code text pairs according to the storage order of the code text pairs in the code corpus, and dividing the code text pairs into code text pair subsets according to the index numbers based on a preset cross-folding threshold;
[0020] Select any subset from the code text pair subsets as an adjustment data set, select any subset from the remaining subsets as a verification data set, and finally use the remaining unselected subsets as a training data set;
[0021] The subset selected as the corresponding adjustment data set is deleted from the code text pair subset, and the process returns to the step of selecting any subset from the code text pair subset as the adjustment data set until all the subsets in the code text pair subset are selected as adjustment data sets.
[0022] Optionally, the segmenting the code data in the training data set includes:
[0023] Selecting segmentation characters corresponding to the code data in the training data set;
[0024] Segmenting the code data in the training data set according to the segmentation characters;
[0025] The segmented code data in the training data set is converted into numerical floating-point data.
[0026] Optionally, the generating the code vector space corresponding to the unique identifier by using a preset algorithm includes:
[0027] Assigning a content-random floating-point word vector to the unique identifier using a random floating-point number algorithm;
[0028] Add the position code corresponding to each floating-point word vector according to the position information of the unique identifier;
[0029] The floating-point word vector and the position code are vector-added to obtain a code vector space for each word segment.
[0030] Optionally, the learning of the correspondence between the virtualized code and the original code on the Transformer neural network according to the code vector space to obtain a large code reverse model includes:
[0031] Inputting the virtualized code vector in the code vector space into an encoder of a Transformer neural network and outputting an attention matrix;
[0032] Inputting the original code vector in the code vector space into the decoder of the Transformer neural network to obtain a value matrix corresponding to the original code vector;
[0033] The attention matrix is concatenated with the value matrix, and the concatenated matrix is input into the feedforward neural network of the Transformer neural network to output the probability distribution corresponding to the original code;
[0034] Calculating a loss value between a probability value of the original code in the probability distribution and a preset true value of the original code;
[0035] The weight matrix parameters of the Transformer neural network are updated according to the loss value until the loss value is less than a preset loss threshold, and the Transformer neural network is output as a code reverse large model.
[0036] Optionally, the adjusting parameters of the code reverse engineering large model according to the adjustment data set includes:
[0037] Using the virtualized code in the adjusted data set as a question data set, and using the original code in the adjusted data set as an answer data set;
[0038] The virtualized code in the question data set is matched one-to-one with the original code in the answer data set to obtain an instruction data set;
[0039] The weight matrix parameters in the code inverse model are adjusted according to the instruction data set.
[0040] Optionally, the using the verified code reverse engineering large model to restore the target virtualized code to the original code includes:
[0041] When the target virtualization code is a compiled language, disassembling the target virtualization code to obtain a disassembled code;
[0042] When the number of lines corresponding to the decompressed code is greater than a preset line number threshold, merging the number of lines of the decompressed code;
[0043] The combined de-assembly code is input into the verified code reverse engineering model, and the original code corresponding to the target virtualized code is output.
[0044] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:
[0045] at least one processor; and,
[0046] a memory communicatively connected to the at least one processor; wherein,
[0047] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned virtualization code restoration method based on the Transformer neural network architecture.
[0048] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned virtualization code restoration method based on the Transformer neural network architecture.
[0049] The embodiment of the present invention uses the Transformer neural network as the algorithm basis, and regards the virtualized code restoration task as a sequence-to-sequence text induction task. The virtualized code and the original code represent two texts of different lengths, but express the same meaning. By establishing a large number of control texts and inputting them into the neural network for pre-training, the neural network learns the subtle connection between the two texts to obtain a basic large model. Then, a fine-tuning method is used to make the basic large model understand the pattern of virtual code induction to original code and then form a reverse large model, which can automatically extract feature capabilities without manual feature extraction. Therefore, the virtualized code restoration method, device and medium based on the Transformer neural network architecture proposed in the present invention can solve the problem of low accuracy when performing virtualized code restoration. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 A flowchart of a virtualized code restoration method based on a Transformer neural network architecture provided by an embodiment of the present invention;
[0051] Figure 2 A schematic diagram of the overall process of the model pre-training stage provided by one embodiment of the present invention;
[0052] Figure 3 A schematic diagram of a flow chart of the reasoning phase of the adjusted code reverse engineering large model provided by an embodiment of the present invention;
[0053] Figure 4 A schematic diagram of the structure of an electronic device for implementing the virtualized code restoration method based on the Transformer neural network architecture provided by one embodiment of the present invention.
[0054] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION
[0055] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.
[0056] The embodiment of the present application provides a virtualized code restoration method based on the Transformer neural network architecture. The execution subject of the virtualized code restoration method based on the Transformer neural network architecture includes but is not limited to at least one of the electronic devices such as the server, the terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the virtualized code restoration method based on the Transformer neural network architecture can be executed by software or hardware installed on a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.
[0057] Reference Figure 1 FIG. 1 is a flow chart of a method for restoring virtualized code based on a Transformer neural network architecture according to an embodiment of the present invention. In this embodiment, the method for restoring virtualized code based on a Transformer neural network architecture includes:
[0058] S1. Merge the virtualized code and the original code in the pre-trained code corpus into a code corpus.
[0059] In an embodiment of the present invention, the code corpus refers to text pairs composed of Vcode (virtualized code) and Scode (original code), and a large number of text pairs constitute the corpus. Before constituting the code corpus, a large amount of pre-trained corpus needs to be prepared in advance, which can be a source code file. If it is a compiled language such as c, c++, c#, go, etc., a disassembly tool needs to be used to parse the corresponding disassembly language, because the purpose of binary reversal is to convert the virtualized disassembly language into an equivalent original disassembly language.
[0060] In the embodiment of the present invention, the step of merging the virtualized code and the original code in the pre-trained code corpus into a code corpus includes:
[0061] Divide each virtualized code in the pre-trained code corpus into code lines according to preset line identifiers, and use the code lines corresponding to each virtualized code after division as virtual code lines;
[0062] Divide each original code in the pre-trained code corpus into code lines according to preset line identifiers, and use the code lines corresponding to each divided original code as the original code lines;
[0063] Query all original code lines that match the virtual code lines;
[0064] Constructing code text pairs by combining the virtual code line with each of all the matched original code lines one by one;
[0065] All code-text pairs are collected into a code corpus.
[0066] In detail, the virtualized code and original code included in the pre-trained code corpus are taken as separate data, such as all code lines corresponding to the virtualized code A are taken as the virtual code lines of the virtualized code A, and all code lines corresponding to the original code B are taken as the original code of the original code A, and the corresponding relationship between the virtualized code and the original code is queried, that is, all the original codes corresponding to the virtualized code line A are queried. If the original code lines corresponding to the virtualized code line A are B, C, and D, then the virtual code lines are corresponded to all the original codes one by one to form code text pairs, that is, three code text pairs consisting of (A, B), (A, C), and (A, D), so that all the composed code text pairs are merged into a code corpus, that is, the code corpus is {(A, B), (A, C), (A, D), …}.
[0067] Furthermore, since it is necessary to extract subtle features of Vcode (virtualized code) and Scode (original code), the code corpus consisting of a large number of text pairs between Vcode (virtualized code) and Scode (original code) needs to be divided and analyzed to form different code data sets.
[0068] S2. Divide the code corpus into a training data set, an adjustment data set, and a verification data set.
[0069] In an embodiment of the present invention, the training data set is a data set used to train the code reverse engineering model to learn and adjust parameters. The training data set occupies the largest amount of data. The adjustment data set is used to guide the code reverse engineering model to complete the induction task of Vcode (virtualized code) and Scode (original code). The verification data set is used to verify the effect of the adjusted code reverse engineering model.
[0070] In the embodiment of the present invention, the dividing the code corpus into a training data set, an adjustment data set, and a verification data set includes:
[0071] extracting code-text pairs from the code corpus;
[0072] Assigning index numbers to the code text pairs according to the storage order of the code text pairs in the code corpus, and dividing the code text pairs into code text pair subsets according to the index numbers based on a preset cross-folding threshold;
[0073] Select any subset from the code text pair subsets as an adjustment data set, select any subset from the remaining subsets as a verification data set, and finally use the remaining unselected subsets as a training data set;
[0074] The subset selected as the corresponding adjustment data set is deleted from the code text pair subset, and the process returns to the step of selecting any subset from the code text pair subset as the adjustment data set until all the subsets in the code text pair subset are selected as adjustment data sets.
[0075] In detail, all code text pairs in the code corpus are assigned an index code, such as the index number of the code text pair (A, B) is 1, the index number of the code text pair (A, C) is 2, and the index number of the code text pair (A, D) is 3, that is, all code text pairs are assigned index numbers in ascending order according to the order of the code text pairs, and the code text pairs are divided into multiple code text pair subsets based on a preset cross-fold threshold. For example, if the cross-fold threshold is 5 and the number of code text pairs is 100, all code text pairs are divided into 5 code text pair subsets, that is, each code text pair subset contains 20 code text pair data.
[0076] Specifically, any one of the code text pair subsets after division is selected as the adjustment data set, and any one of the code text pair subsets that have not been selected is selected as the verification data set, and all the remaining subsets are used as training data sets, and the process is repeated until all subsets are used as adjustment data sets and training data sets. For example, if the five code text pair subsets include a, b, c, d, and e, a is selected as the adjustment data set, and any one of the code text pair subsets b, c, d, and e that have not been selected is selected, then b is selected as the verification data set, and the remaining c, d, and e are used as training data sets, and the selection is repeated. Since a has been selected as the adjustment data set, the next adjustment data set will be selected from b, c, d, and e, until all subsets are used as adjustment data sets and training data sets. The data is divided into multiple folds, and the sample distribution in each fold should be kept as similar to the distribution of the original data as possible, ensuring that the training data set, adjustment data set, and verification data set all contain code snippets of various types and styles, so as to better train and evaluate the generalization ability of the model.
[0077] Furthermore, since the data input into the model requires a specific input format, the data in the training data set, the adjustment data set, and the validation data set need to be adjusted to meet the input conditions of the model.
[0078] S3. Segment the code data in the training data set to generate multiple code segmentations, each of which corresponds to a unique identifier, and use a preset algorithm to generate a code vector space corresponding to the unique identifier.
[0079] In an embodiment of the present invention, the code data in the training data needs to be segmented one by one. Through word segmentation, the code data can be split into smaller units, such as words, identifiers, operators, etc., so as to extract the key information and structure in the code, provide clearer input data for the model, and help improve the performance and generalization ability of the model.
[0080] In the embodiment of the present invention, the word segmentation of the code data in the training data set includes:
[0081] Selecting segmentation characters corresponding to the code data in the training data set;
[0082] Segmenting the code data in the training data set according to the segmentation characters;
[0083] Convert the word segmentation code data in the training data set into numerical floating point data;
[0084] In detail, since the large model can only process numerical floating-point data, the text information needs to be converted into digital floating-point type to ensure the consistency and processability of the code data, and select the segmentation characters according to the grammatical rules of the code. The segmentation characters include but are not limited to spaces, operators, and punctuation marks. Then, according to the selected segmentation characters, the numerical floating-point code data is segmented, that is, the code text is split according to the selected segmentation characters to obtain basic units such as words, identifiers, and operators.
[0085] Specifically, after the code data in the corpus is segmented, a unique ID is generated for each word and stored in the vocabulary. Each number represents a word in the vocabulary. This can be done by traversing the text data and matching each word with an increasing integer. The result after segmentation is converted into a numerical representation to facilitate the processing of the machine learning model, which contains all the words and identifiers appearing in the code. The vocabulary will be used for subsequent feature extraction and model training. For example, the disassembled vocabulary contains push, jmp, lea, eax, ebx vocabulary, etc.
[0086] Furthermore, the meaning of a unique identifier is usually closely related to its context in the code. The word embedding model can consider the usage of the identifier in context to better understand its meaning. By learning the context of the identifier, the role and meaning of the identifier in the code can be more accurately represented. After the unique identifier is represented as a vector, these vectors can be input as features into the machine learning model, and the model can better utilize the semantic information of the identifier for modeling.
[0087] In an embodiment of the present invention, the code vector space refers to a space that represents code data as a vector, wherein each code word is mapped to a vector. The code vector captures the semantics, structure, and contextual information of the code, so that code understanding, analysis, and processing can be performed in the vector space.
[0088] In the embodiment of the present invention, the step of generating the code vector space corresponding to the unique identifier by using a preset algorithm includes:
[0089] Assigning a content-random floating-point word vector to the unique identifier using a random floating-point number algorithm;
[0090] Add the position code corresponding to each floating-point word vector according to the position information of the unique identifier;
[0091] The floating-point word vector and the position code are vector-added to obtain a code vector space for each word segment.
[0092] In detail, the preset algorithm is a random floating point number algorithm, and the random floating point number algorithm is a word embedding model such as BERT, GPT, etc., then a vector space containing 1024 dimensions is randomly generated for each id in the vocabulary for the first time through word embedding models such as BERT, GPT, etc., and a position code is added to each word in each sentence, mainly to solve the problem of text order, wherein the BERT model converts each word in the text into a one-dimensional vector by querying the word vector table as the model input, and converts the word vector into a text vector, then the value of the text vector is automatically learned during the model training process, which is used to characterize the global semantic information of the text and is integrated with the semantic information of the word segmentation, and because the semantic information carried by the characters / words appearing at different positions in the text is different, a position code is added to each word in each sentence, wherein the position code addition method includes but is not limited to rotated position coding (RoPE), fixed position coding, and position coding methods of sine and cosine functions, and finally, the BERT model uses the sum of the word vector, text vector and position vector as the model input to obtain the code vector space corresponding to the code data.
[0093] Furthermore, in order to establish a corresponding relationship between the virtualized code and the original code, it is necessary to train the model to capture the structural and semantic information in the code so as to better understand the contextual relationship in the code.
[0094] S4. According to the code vector space, the Transformer neural network is used to learn the corresponding relationship between the virtualized code and the original code to obtain a large code reverse model.
[0095] In an embodiment of the present invention, the code reverse large model refers to a model in which the 1024-dimensional vector randomly initialized for each id for the first time is optimized into a word context relationship after deep learning of the Transformer neural network.
[0096] In the embodiment of the present invention, the learning of the correspondence between the virtualized code and the original code on the Transformer neural network according to the code vector space to obtain a large code reverse model includes:
[0097] Inputting the virtualized code vector in the code vector space into an encoder of a Transformer neural network and outputting an attention matrix;
[0098] Inputting the original code vector in the code vector space into the decoder of the Transformer neural network to obtain a value matrix corresponding to the original code vector;
[0099] The attention matrix is concatenated with the value matrix, and the concatenated matrix is input into the feedforward neural network of the Transformer neural network to output the probability distribution corresponding to the original code;
[0100] Calculating a loss value between a probability value of the original code in the probability distribution and a preset true value of the original code;
[0101] The weight matrix parameters of the Transformer neural network are updated according to the loss value until the loss value is less than a preset loss threshold, and the Transformer neural network is output as a code reverse large model.
[0102] In detail, the code vector space is input into the multi-head attention mechanism in the Transformer neural network to obtain the relevance score of each code vector in the code vector space; the relevance score is normalized to obtain the probability distribution matrix corresponding to the code vector space, the probability distribution matrix is dot-producted with the value matrix, and the normalization operation is performed. After the matrix is normalized, the result is processed by a feedforward network, and the feedforward network includes a linear layer and a ReLU activation function layer to obtain the output attention matrix, and the attention matrix is used as the Query and Key matrices of one of the multi-head attention mechanisms in the decoder part, where the multi-head attention mechanism includes Query, Key and Value. Each matrix is obtained by multiplying different weight matrices with the transpose of the previously calculated matrix. This matrix is obtained by adding the word vector matrix and the position vector matrix. For example, for calculating the Query matrix, the number of rows of the weight matrix must be the same as the number of columns of the transposed matrix, while the number of columns of the weight matrix can be arbitrary, and the values in the weight matrix are between 0 and 1, randomly initialized, and when the Transformer begins to learn the meaning of the corpus, the weight matrix values will be continuously updated and learned.
[0103] Specifically, the decoder has two inputs, namely the attention matrix as one of the Query and Key matrices of the multi-head attention mechanism in the decoder part and the value matrix Value calculated by inputting the original code into the decoder of the Transformer neural network, and performing similar operations as the encoder to obtain the probability distribution corresponding to the original code, and calculate the loss value between the generated probability distribution of the original code and the true value. The difference between the predicted value and the true value can be measured by the cross entropy loss function, and the weight matrix parameters of the Transformer neural network are updated through the back propagation algorithm and optimizer (such as Adam, SGD, etc.), so that the loss value gradually decreases until the loss value is less than the preset loss threshold or reaches the preset training rounds, and the Transformer neural network is output as a large code inverse model.
[0104] Furthermore, the code reverse large model can establish subtle associations between words, that is, subtle associations between virtualized codes and original codes. The subtle associations are not very accurate associations between virtualized codes and original codes, but only indicate that there are subtle associations between virtualized codes and original codes. After outputting the code reverse large model, in order to make the model more accurate, the parameters of the model need to be adjusted.
[0105] S5. Adjust parameters of the code reverse engineering big model according to the adjustment data set, perform data induction verification on the code reverse engineering big model after parameter adjustment according to the verification data set, and use the adjusted code reverse engineering big model to restore the target virtualized code to the original code.
[0106] In an embodiment of the present invention, the learned code reverse engineering model also needs to be optimized by adjusting the data set and verifying the data set, and the model output effect of the code reverse engineering model needs to be verified, so as to help the code reverse engineering model better capture the semantic and structural information of the language and improve the generalization ability of the model.
[0107] In the embodiment of the present invention, the step of adjusting parameters of the code reverse engineering model according to the adjustment data set includes:
[0108] Using the virtualized code in the adjusted data set as a question data set, and using the original code in the adjusted data set as an answer data set;
[0109] The virtualized code in the question data set is matched one-to-one with the original code in the answer data set to obtain an instruction data set;
[0110] The weight matrix parameters in the code inverse model are adjusted according to the instruction data set.
[0111] In detail, the code reverse engineering large model is fine-tuned by adjusting the data set. The virtualized code in the adjustment data set can be used as the problem data set, and the original code can be used as the answer data set to ensure that each virtualized code in the problem data set has an accurately corresponding original code. The virtualized code in the problem data set is matched one by one with the original code in the answer data set to form an instruction data set for model training and tuning, and then the weight values in the weight matrix parameters in the code reverse engineering large model are adjusted through the instruction data set. In the fine-tuning stage, instructions, RLHF and other methods can be used for fine-tuning. If the instruction fine-tuning method is used, Vcode (virtualized code) is used as the question, Scode (original code) is used as the answer, and "induction" is used as the instruction to format the adjustment data set, thereby obtaining the adjusted code reverse engineering large model. After the adjusted code reverse engineering large model is generated, it is necessary to verify the inductive effect of the adjusted code reverse engineering large model according to the virtualized code in the verification data set, that is, the verification data set is input into the adjusted code reverse engineering large model to verify the inductive effect of Vcode (virtualized code) to Scode (original code) in the verification data set.
[0112] Specifically, in order to explain more clearly the generation of the adjusted code inverse large model, such as Figure 2 As shown in FIG. 1 , it is a schematic diagram of the overall process of the model pre-training stage. First, based on a large amount of source code corpus, such as Py source code corpus (Python program file), js source code corpus (JavaScript program file), so source code corpus (shared data file under Linux), exe source code corpus (Windows executable file), etc., the pre-training corpus is composed of text pairs between Vcode (virtualized code) and Scode (original code), and a large number of text pairs constitute the corpus. The corpus is divided into three parts, namely, the training set is used to train the code reverse large model, which has the largest amount of data; the data adjustment set (fine-tuning set) is used to guide the code reverse large model to complete the induction task from Vcode (virtualized code) to Scode (original code), and the verification set is used to train the code reverse large model. In order to verify the effect of the final reverse large model, the corpus is segmented, and a unique id is generated for each word and stored in the vocabulary library. After deep learning of the Transformer neural network, the 1024-dimensional vector of each id that is randomly initialized for the first time is optimized to a code reverse large model with word context relationship. This stage is mainly to allow the model to establish subtle associations between words. During the fine-tuning stage, instructions, RLHF and other methods can be used for fine-tuning. If the instruction fine-tuning method is used, Vcode (virtualized code) is used as the question, Scode (original code) is used as the answer, and "induction" is used as the instruction to format the fine-tuning data set, and finally a reverse large model is generated. After the reverse large model is generated, it is necessary to verify the induction effect of the large model according to the Vcode (virtualized code) in the verification set.
[0113] Furthermore, after generating the adjusted code reverse engineering model, a model reasoning phase is required to reason out the process of restoring the virtualized code to the original code.
[0114] In the embodiment of the present invention, the adjusted code reverse large model has a super strong ability to automatically extract features, and there is no need to extract features manually. The cost of encoding feature templates will not increase exponentially due to the constant change of features or even the emergence of random features in the latest technology, resulting in poor effect of restoring virtualized code to original code.
[0115] In the embodiment of the present invention, the method of restoring the target virtualized code to the original code by using the verified code reverse model includes:
[0116] When the target virtualization code is a compiled language, disassembling the target virtualization code to obtain a disassembled code;
[0117] When the number of lines corresponding to the decompressed code is greater than a preset line number threshold, merging the number of lines of the decompressed code;
[0118] The combined de-assembly code is input into the verified code reverse engineering model, and the original code corresponding to the target virtualized code is output.
[0119] In detail, obtain the content that needs to be summarized. Unlike the training stage, only the target Vcode (virtualized code) needs to be obtained here, that is, the problem part input to the big model. If it is a compiled language such as: c, c++, c#, go, etc., it is necessary to prepare a program compiled in the corresponding language, and use a disassembly tool to parse the anti-assembly code to obtain the anti-assembly code after disassembly; because there are multiple lines of Vcode (virtualized code) corresponding to one line of Scode (original code) in the training-fine-tuning stage, it is necessary to merge multiple lines of Vcode (virtualized code) in the reasoning stage. According to the start and end marks of different virtualization tools, it is possible to identify which lines need to be merged and sent to the reverse big model. The reverse big model outputs the summary result, which is Scode (original code), that is, through Scode=f(Scode), where f is the neural network algorithm, the target virtualized code is restored to the original code, and the reverse big model is the adjusted code reverse big model.
[0120] Specifically, model reasoning can make decisions automatically without human intervention and can be extended to new data sets or new tasks when needed without retraining. In order to verify the adjusted code, the reasoning process of the reverse large model is as follows: Figure 3As shown, it is a flow chart of the reasoning stage of the adjusted code reverse engineering model. The virtualized code Vcode, such as Py source code corpus (Python program file), js source code corpus (JavaScript program file), so source code corpus (shared data file under Linux), exe source code corpus (Windows executable file), etc. are input into the reverse engineering model and restored to Scode (original code) corresponding to Vcode (virtualized code).
[0121] The embodiment of the present invention uses the Transformer neural network as the algorithm basis, and regards the virtualized code restoration task as a sequence-to-sequence text induction task. The virtualized code and the original code represent two texts of different lengths, but express the same meaning. By establishing a large number of control texts and inputting them into the neural network for pre-training, the neural network learns the subtle connection between the two texts to obtain a basic large model. Then, a fine-tuning method is used to make the basic large model understand the pattern of virtualized code induction to original code and then form a reverse large model, which can automatically extract feature capabilities without manual feature extraction. Therefore, the virtualized code restoration method, device and medium based on the Transformer neural network architecture proposed in the present invention can solve the problem of low accuracy when performing virtualized code restoration.
[0122] like Figure 4 , is a schematic diagram of the structure of an electronic device for implementing a virtualized code restoration method based on a Transformer neural network architecture provided by an embodiment of the present invention.
[0123] The electronic device may include a processor 10, a memory 11, a communication bus 12 and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as a virtualized code restoration program based on a Transformer neural network architecture.
[0124] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (ControlUnit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, executing a virtualized code restoration program based on a Transformer neural network architecture, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.
[0125] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, mobile hard disk, multimedia card, card-type memory (for example: SD or DX memory, etc.), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 can also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 can also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of a virtualized code restoration program based on the Transformer neural network architecture, but also can be used to temporarily store data that has been output or is to be output.
[0126] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.
[0127] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.
[0128] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.
[0129] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.
[0130] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.
[0131] The virtualized code restoration program based on the Transformer neural network architecture stored in the memory 11 in the electronic device is a combination of multiple instructions. When running in the processor 10, it can achieve:
[0132] Merge the virtualized code and original code in the pre-trained code corpus into a code corpus;
[0133] Dividing the code corpus into a training data set, an adjustment data set, and a verification data set;
[0134] Segmenting the code data in the training data set to generate multiple code segmentations, each of which corresponds to a unique identifier, and using a preset algorithm to generate a code vector space corresponding to the unique identifier;
[0135] According to the code vector space, the Transformer neural network is used to learn the corresponding relationship between the virtualized code and the original code to obtain a large code reverse model;
[0136] The parameters of the code reverse engineering model are adjusted according to the adjustment data set, the code reverse engineering model after parameter adjustment is verified by data induction according to the verification data set, and the target virtualized code is restored to the original code using the verified code reverse engineering model.
[0137] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.
[0138] Furthermore, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).
[0139] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:
[0140] Merge the virtualized code and original code in the pre-trained code corpus into a code corpus;
[0141] Dividing the code corpus into a training data set, an adjustment data set, and a verification data set;
[0142] Segmenting the code data in the training data set to generate multiple code segmentations, each of which corresponds to a unique identifier, and using a preset algorithm to generate a code vector space corresponding to the unique identifier;
[0143] According to the code vector space, the Transformer neural network is used to learn the corresponding relationship between the virtualized code and the original code to obtain a large code reverse model;
[0144] The parameters of the code reverse engineering model are adjusted according to the adjustment data set, the code reverse engineering model after parameter adjustment is verified by data induction according to the verification data set, and the target virtualized code is restored to the original code using the verified code reverse engineering model.
[0145] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, media and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0146] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0147] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.
[0148] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.
[0149] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is not limited only according to the above description, and it is intended that all changes within the meaning and scope of equivalent elements within the scope of protection are included in the present invention.
[0150] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.
[0151] In addition, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the system can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any particular order.
[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.
Claims
1. A virtualized code restoration method based on Transformer neural network architecture, characterized in that: The method comprises: Merge the virtualized code and original code in the pre-trained code corpus into a code corpus; Dividing the code corpus into a training data set, an adjustment data set, and a verification data set; Segmenting the code data in the training data set to generate multiple code segmentations, each of which corresponds to a unique identifier, and using a preset algorithm to generate a code vector space corresponding to the unique identifier; According to the code vector space, the Transformer neural network is used to learn the corresponding relationship between the virtualized code and the original code to obtain a large code reverse model; The parameters of the code reverse engineering model are adjusted according to the adjustment data set, the code reverse engineering model after parameter adjustment is verified by data induction according to the verification data set, and the target virtualized code is restored to the original code using the verified code reverse engineering model.
2. The virtualized code restoration method based on the Transformer neural network architecture according to claim 1, characterized in that: The step of merging the virtualized code and the original code in the pre-trained code corpus into a code corpus includes: Divide each virtualized code in the pre-trained code corpus into code lines according to preset line identifiers, and use the code lines corresponding to each virtualized code after division as virtual code lines; Divide each original code in the pre-trained code corpus into code lines according to preset line identifiers, and use the code lines corresponding to each divided original code as the original code lines; Query all original code lines that match the virtual code lines; Constructing code text pairs by combining the virtual code line with each of all the matched original code lines one by one; All code-text pairs are collected into a code corpus.
3. The virtualized code restoration method based on the Transformer neural network architecture according to claim 1, characterized in that: The dividing the code corpus into a training data set, an adjustment data set, and a verification data set includes: extracting code-text pairs from the code corpus; Assigning index numbers to the code text pairs according to the storage order of the code text pairs in the code corpus, and dividing the code text pairs into code text pair subsets according to the index numbers based on a preset cross-folding threshold; Select any subset from the code text pair subsets as an adjustment data set, select any subset from the remaining subsets as a verification data set, and finally use the remaining unselected subsets as a training data set; The subset selected as the corresponding adjustment data set is deleted from the code text pair subset, and the process returns to the step of selecting any subset from the code text pair subset as the adjustment data set until all the subsets in the code text pair subset are selected as adjustment data sets.
4. The virtualized code restoration method based on the Transformer neural network architecture according to claim 1, characterized in that: The word segmentation of the code data in the training data set includes: Selecting segmentation characters corresponding to the code data in the training data set; Segmenting the code data in the training data set according to the segmentation characters; The segmented code data in the training data set is converted into numerical floating-point data.
5. The virtualized code restoration method based on the Transformer neural network architecture according to claim 1, characterized in that: The generating the code vector space corresponding to the unique identifier by using a preset algorithm includes: Assigning a content-random floating-point word vector to the unique identifier using a random floating-point number algorithm; Add the position code corresponding to each floating-point word vector according to the position information of the unique identifier; The floating-point word vector and the position code are vector-added to obtain a code vector space for each word segment.
6. The virtualized code restoration method based on the Transformer neural network architecture according to claim 1, characterized in that: The learning of the correspondence between the virtualized code and the original code on the Transformer neural network according to the code vector space to obtain a large code reverse model includes: Inputting the virtualized code vector in the code vector space into an encoder of a Transformer neural network and outputting an attention matrix; Inputting the original code vector in the code vector space into the decoder of the Transformer neural network to obtain a value matrix corresponding to the original code vector; The attention matrix is concatenated with the value matrix, and the concatenated matrix is input into the feedforward neural network of the Transformer neural network to output the probability distribution corresponding to the original code; Calculating a loss value between a probability value of the original code in the probability distribution and a preset true value of the original code; The weight matrix parameters of the Transformer neural network are updated according to the loss value until the loss value is less than a preset loss threshold, and the Transformer neural network is output as a code reverse large model.
7. The virtualized code restoration method based on the Transformer neural network architecture according to claim 1, characterized in that: The step of adjusting parameters of the code reverse engineering model according to the adjustment data set includes: Using the virtualized code in the adjusted data set as a question data set, and using the original code in the adjusted data set as an answer data set; The virtualized code in the question data set is matched one-to-one with the original code in the answer data set to obtain an instruction data set; The weight matrix parameters in the code inversion model are adjusted according to the instruction data set.
8. The virtualized code restoration method based on the Transformer neural network architecture according to claim 1, characterized in that: The method of using the verified code reverse engineering model to restore the target virtualized code to the original code includes: When the target virtualization code is a compiled language, disassembling the target virtualization code to obtain a disassembled code; When the number of lines corresponding to the decompressed code is greater than a preset line number threshold, merging the number of lines of the decompressed code; The combined de-assembly code is input into the verified code reverse engineering model, and the original code corresponding to the target virtualized code is output.
9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the virtualization code restoration method based on the Transformer neural network architecture as described in any one of claims 1 to 8.
10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the virtualized code restoration method based on the Transformer neural network architecture as described in any one of claims 1 to 8 is implemented.
