Method for realizing credential and credential adaptation based on decompilation Decom-Transform model
By decompiling the Decom-Transformer model based on the method, the problems of high decompilation difficulty and difficult code reading in the existing technology are solved, efficient and fast binary file conversion is realized, the CPU adaptation process is simplified, and the domestic information is promoted. Creating product development.
Patent Information
- Application Number
- CN202411294830.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-31
AI Technical Summary
The existing decompilation technology is based on rules, with high difficulty and low efficiency, lack of annotations for generating codes, difficult to understand, and high requirements for the expansion of new CPU structures or instruction sets, hindering the development of the ecological chain of domestic information innovation products.
Decom-Transformer model is adopted based on decompilation of Decom-Transformer model, and the CPU adaptation process is simplified by obtaining data, analyzing binary semantics, building corpus, training Glove pre-trained models and Decom-Transformer models. Lower costs and thresholds.
It realizes efficient and fast binary file decompilation, and the generated code is concise and easy to read, reducing CPU adaptation costs and promoting the development of the ecological chain of domestic information innovation products.
Smart Images
Figure CN119225739A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of program compilation, and in particular to a method for implementing information creation adaptation based on a decompiled Decom-Transformer model. Background Art
[0002] Existing decompilation technology generally proceeds through the following steps: First, parse the format of the binary file and extract the instructions, data and metadata. This usually involves the analysis of the file header, segment table and symbol table. Then, convert the binary instructions into assembly code. This step requires understanding the instruction set architecture (ISA) of the target processor to correctly interpret the meaning of each instruction. Analyze the control flow graph (CFG) of the program to identify structures such as functions, loops, conditional branches, etc. This helps to understand the logic and execution path of the program. Analyze the data flow in the program and track the definition and use of variables. This helps to recover the type and scope of the variables. Convert the assembly code into high-level language structures such as functions, loops, conditional statements, etc. This step usually involves pattern matching and syntax analysis to generate pseudocode or code close to the original source code. By analyzing how variables are used, their types are inferred. This helps to generate more accurate and readable high-level language code. Optimize the generated code to remove redundant and useless code to make it closer to the original source code. This method is basically a rule-based method.
[0003] The prior art has the following disadvantages:
[0004] 1. The rule-based approach has high requirements on people, is difficult to implement, and is inefficient.
[0005] 2. The code generated by decompilation is usually a low-level language such as C, which generally lacks comments and is difficult to understand.
[0006] 3. It is very troublesome to expand and modify the tools for the new CPU structure or instruction set.
[0007] 4. It requires relatively high software technical skills and has a high threshold, which is not accessible to ordinary people and hinders the development of the ecosystem. Summary of the invention
[0008] The purpose of the present invention is to overcome the shortcomings of the prior art and propose a method for implementing ICT adaptation based on decompiled Decom-Transformer models. The model trained by the present invention is small and can be run on the CPU, saving costs. In addition, the present invention directly converts binary files for different CPUs through artificial intelligence, which is simple to operate, reduces the cost and threshold of the underlying library and application multi-CPU adaptation, greatly improves the efficiency of localized adaptation of different applications, and promotes the development of the ecological chain of localized ICT products.
[0009] The present invention solves the technical problem by adopting the following technical solutions:
[0010] The method for implementing the ICT adaptation based on the decompiled Decom-Transformer model includes the following steps:
[0011] Step 1: Get data.
[0012] Step 2: Load the data obtained in step 1, parse the binary semantics, and assemble the instructions as word input;
[0013] Step 3: Build a corpus based on the data loaded in step 2, and use the corpus to train the Glove pre-training model for feature extraction;
[0014] Step 4: Use the Glove pre-trained model built in step 3 to extract features for feature embedding and build a Decom-Transformer model;
[0015] Step 5: Train the Decom-Transformer model to obtain the Encoder model and the Decoder model;
[0016] Step 6: Generate assembly language using the Encoder model and the Decoder model;
[0017] Step 7: Generate a high-level language using a large language model based on the generated assembly language.
[0018] Moreover, the data obtained in step 1 includes: assembly language files, instruction set specifications, ABI specifications, ELF files and auxiliary information, wherein the assembly language files, instruction set specifications, ABI specifications, ELF files are x86 binary files, and the auxiliary information is a variety of ARM files, including application programs, libraries and operating system binary files.
[0019] Moreover, the specific implementation method of step 2 is: reading the data obtained in step 1 into the memory.
[0020] Furthermore, the step 3 comprises the following steps:
[0021] Step 3.1, process the data loaded in step 2;
[0022] Use the libelf library to read and parse the ELF file and perform actions similar to word segmentation: read different sections of the ELF file, extract the opcodes and operands, and other symbol table binary file related information, and organize them into separate function calls and code segments for training to build a corpus;
[0023] Step 3.2, encode the data loaded in step 2;
[0024] Use the instruction set specification and ABI specification to establish a mapping between binary and instruction set, and construct the co-occurrence matrix of Glove;
[0025] Step 3.3, perform semantic training on the encoded data in step 3.2;
[0026] By performing matrix decomposition on the Glove co-occurrence matrix, the word vector of each word is learned to form a Glove pre-training model; the Glove model saves the word embedding vector that reflects the semantics of the binary file;
[0027] Step 3.4: Generate the Glove model as a file based on the data feature vector.
[0028] Moreover, the Decom-Transformer model structure constructed in the step 4 is the same as the traditional Transformer structure, but according to the characteristics of the ELF and PE file formats, a sparse Transformer structure is adopted, and sparse attention is applied to reduce the quadratic time and space complexity of self-attention to linear, and three types of attention are introduced: window attention, bridge attention and global attention.
[0029] Moreover, in step 5, the Encoder model is used to read the ELF file information and auxiliary information of the source CPU architecture, and the Decoder model is used to generate the assembler of the corresponding architecture, so as to realize the end-to-end conversion from the binary application to the assembly language program.
[0030] Moreover, the specific implementation method of step 6 is: using the trained Encoder model and Decoder model to generate a high-dimensional embedding vector, and then implementing linear transformation and Softmax function to map the high-dimensional vector to an assembly language space vector to generate assembly language.
[0031] Moreover, the specific implementation method of step 8 is: using the assembly language generated by the Decom-Transformer model as input, using the large language model to generate a high-level language of the specified language and adding comments.
[0032] The advantages and positive effects of the present invention are:
[0033] The present invention obtains data; loads and extracts features; embeds the extracted features, and constructs a Decom-Transformer model to obtain an Encoder model and a Decoder model; generates an assembly language using the Encoder model and the Decoder model; generates a large language model based on the generated assembly language; and finally generates a high-level language using the generated large language model. Since the semantic expression space of structures such as binary files ELF or PE is small, the basic form is an opcode and an operand, as well as keywords of the file structure, so the trained model is small and can be run on the CPU, saving costs. At the same time, the present invention has high decompilation efficiency and fast speed; it directly converts the binary file into the desired high-level language through artificial intelligence, and the operation is simple, which is conducive to learning the structure and development logic of the underlying software, and promoting the development of the ecological chain of domestically produced information and innovation products. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of the present invention;
[0035] Figure 2 It is a flow chart of feature extraction of the present invention;
[0036] Figure 3 This is the architecture diagram of the Decom-Transformer model of the present invention;
[0037] Figure 4 This is an illustration of the attention mechanism principle of the present invention. DETAILED DESCRIPTION
[0038] The present invention is further described in detail below with reference to the accompanying drawings.
[0039] The construction idea of the present invention is:
[0040] Artificial intelligence technology, especially Transformer technology, shows amazing results in cross-natural language translation. Later, the large model, multimodal and other technologies developed on its basis have achieved remarkable results in other fields. The present invention adopts similar Transformer technology, for instruction set, architecture specification, CPU level ABI specification and bottom ABI interface specification (libc, glibc, libc++, libstdc++, etc.), ELF file structure specification and other information, and the assembly language file corresponding to these binary files, through large data volume, model training is carried out, and Encoder and Decoder models are trained, Encoder is used to read the ELF file information and auxiliary information of the source CPU architecture, and Decoder is used to generate the assembler of the corresponding architecture, so that end-to-end conversion from binary application to assembly language program can be realized. Then, the assembly language program is used as input, and the code and code annotation of various high-level languages are generated using a large language model (LLM).
[0041] The method of implementing information innovation adaptation based on decompilation of Decom-Transformer model, such as Figure 1 As shown, the following steps are included:
[0042] Step 1: Get data.
[0043] The acquired data includes: assembly language files, instruction set specifications, ABI specifications, ELF files and auxiliary information. The assembly language files, instruction set specifications, ABI specifications, and ELF files are x86 binary files, and the auxiliary information is a variety of ARM files, including applications, libraries, and operating system binary files.
[0044] Step 2: Read the data obtained in step 1 into the memory.
[0045] Step 3: Build a corpus for the data loaded in step 2, and use the corpus to train the Glove pre-trained model for feature extraction.
[0046] Collect a variety of ARM and x86 binary files, including applications, libraries, and operating system binaries. And related auxiliary information such as ABI specifications, instruction sets, and specifications. Analyze the data set, including information about function boundaries, control flow, and memory access patterns. Use tools to decompile the collected programs and generate corresponding assembly language files. Extract features and use these features to generate binary machine code tokens, such as opcode sequences, function call graphs, and memory references. Use these features to form binary machine code tokens. Assembly language also uses instruction sets for tokenization. This stage is equivalent to the token vector expression of natural language. Training a set of machine language feature vectors is a prerequisite. These machine language feature vectors have the semantics of machine language. Semantic training is performed by creating a corpus of instruction sets and ELF-related content and performing new training through GloVe.
[0047] like Figure 2 As shown, step 3 includes the following steps:
[0048] Step 3.1, process the data loaded in step 2;
[0049] For binary files, we use the libelf library to read and parse ELF files, perform similar actions to word segmentation (read different sections of ELF, extract opcodes and operands, and other binary file related information such as symbol tables), and organize them into separate function calls and code segments that can be used for training to build a corpus. For assembly language, we use the CPU architecture instruction set and decompiled files as input, and read instructions and instruction parameters from them, and split them into structures such as function calls and code segments.
[0050] Step 3.2, encode the data loaded in step 2;
[0051] Use the instruction set specification and ABI specification to establish a mapping between binary and instruction set, and construct the co-occurrence matrix of Glove;
[0052] The constructed corpus is the basis for training the Glove pre-training model. Only after training the Glove model can we construct word vectors that can reflect the semantics of binary instruction sets and assembly instruction sets. These word vectors are the basis for training the Decom-Transformer model.
[0053] The co-occurrence matrix is constructed when training the Glove pre-trained model, which records the number of times words co-occur within a certain window. Each element of this matrix represents the frequency of co-occurrence of two words in the corpus.
[0054] Step 3.3, perform semantic training on the encoded data in step 3.2;
[0055] The collected prediction information is used to train the Glove pre-trained model, which stores word embedding vectors that reflect the semantics of binary files.
[0056] The Glove pre-trained model is trained based on this co-occurrence matrix. It learns the word vector (i.e., the word embedding representation) of each word by performing matrix decomposition on the co-occurrence matrix. These word vectors can capture the semantic relationship between words.
[0057] Step 3.4: Generate the Glove model as a file based on the data feature vector.
[0058] Step 4: Embed the features extracted in step 3 and build a Decom-Transformer model.
[0059] The Glove pre-trained model trained in steps 3.1 to 3.3 is used to directly generate binary and assembly language semantic word vectors, where the Glove pre-trained model input is the data loaded in step 2, and the output is binary instruction and assembly instruction word vectors.
[0060] like Figure 3 As shown in the figure, the constructed Decom-Transformer model structure is the same as the traditional Transformer structure, but according to the characteristics of ELF and PE file formats, the sparse Transformer structure is adopted, and sparse attention is applied to reduce the quadratic time and space complexity of self-attention to linear, as shown in Figure 4 As shown in the figure, three types of attention are introduced: window attention, bridge attention, and global attention. Each type is motivated by observations of the previous model. It is more suitable for the construction learning of binary code. The learning efficiency and convergence speed are greatly improved. In terms of details, since the semantics expressed by binary files are relatively simple (operands, opcodes, and symbols defined in the ELF specification, etc.), the vector length and parameters are adjusted according to the actual situation to make it more efficient and the model smaller.
[0061] At the same time, the evaluation and verification process of the Decom-Transformer model is the same as the general Transformer evaluation and verification process, except that the content of comparison is different. The content of comparison is the sequence of source and target binary files, and special programs need to be compiled for comparison.
[0062] Introducing three types of attention:
[0063] Introducing the Window Attention Mechanism: Considering the application scenario, the structure of the ELF class depends heavily on the local context (instruction data within the Section, etc.), while there are only a few (Section Header, etc.) long-distance dependency instances. This locality is used to sparse attention to achieve higher efficiency.
[0064] The attention mechanism formula is:
[0065]
[0066] Among them, Q, K and V represent query, key and value matrices respectively. They are linear transformations of the input sequence, K T is the transposed matrix of the key, D k ; is the dimension of the key vector, and M is the mask matrix.
[0067] By adjusting M in the softmax function, the algorithm of M is different under different attention mechanisms.
[0068] In the window attention mechanism, M is calculated as:
[0069]
[0070] Where w is the window size, and i and j are the subscripts of the token. This windowed attention mechanism reduces the complexity of the self-attention mechanism by limiting the receptive field size of each token to a small window of size w at each layer. The computational complexity of this mode is O(n×w), which scales linearly with the input sequence length n. After applying this sliding window attention mechanism for N Transformer layers, the receptive field size increases to N×w at the top layer. Since each token only pays attention to the w tokens to its left, rather than the entire previous sequence, this model can achieve faster inference speed.
[0071] Introducing the BridgeAttention mechanism: Window attention mainly processes local context (instruction data in Section, etc.), but in some structures such as ELF, it is necessary to refer to the contents of SectionHeader and SectionTable. Bridge tokens are introduced for these long-distance references to aggregate local information for access. Bridge tokens can process tokens of fixed length and participate from all subsequent tokens, which can be understood as learning about Section.
[0072] In the bridging attention mechanism, M is calculated as:
[0073]
[0074] Among them, S b is the set of bridge tokens, n is the length of the input token sequence, and m is the number of bridge tokens. The complexity of bridge attention is O(m×n)≈O(n), where m<<n. Compared with stacked window attention, bridge attention allows each token to participate in any previous token with at most 2 hops, which enables the model to effectively access remote context.
[0075] Introducing the Global Attention Mechanism: In structures such as ELF, global variables and functions have global scope identifiers. The previous two attention mechanisms cannot handle this situation. The introduction of the Global Attention Mechanism can be called from any location in the file. In order to inject attention into the code heuristics, the code structure with a tree setter is used to parse the code into an abstract syntax tree (AST) to help understand the relationship between each other.
[0076] In the global attention mechanism, M is calculated as:
[0077]
[0078] Among them, G is a set of locations with global significance, such as global variables, functions, section headers, symbol tables, etc. The complexity of global attention is O(kn)≈O(n), because we have k<<n, where n is the length of the sequence. The most important sentences are extracted and encoded in a fixed-length context window, and global attention requires less memory.
[0079] Considering all three types of attention together, M is calculated as:
[0080] M=max(M window ,M bridge ,M global )
[0081] Step 5: Train the Decom-Transformer model to obtain the Encoder model and the Decoder model.
[0082] Encoder model: reads the binary file of the conversion source CPU architecture (such as x86_64), and segments the binary file content according to the specification, calls the feature encoder to encode the binary Token, and generates a feature vector. The feature vector is the entire vector space content such as binary instructions, data, symbol table, character information, etc. Positional encoding is performed according to the ELF or PE file structure. The binary semantic expression feature vector Token and Positional encoding are the comprehensive features of the input.
[0083] Decoder model: reads the decompiled file corresponding to the same file, and splits the decompiled content according to the specification, splitting it into function calls or code snippets of instructions and parameters, and performing masking operations on the instructions or parameters, calling the feature encoder to encode the corresponding Token, and generating a feature vector. The feature vector is the entire vector space content such as binary instructions, data, symbol table, character information, assembly language instructions, etc., and is input with Positional encoding according to the ELF or PE file structure.
[0084] The present invention introduces a window attention mechanism (WindowAttention), a bridge attention mechanism (BridgeAttention), and a global attention mechanism (GlobalAttention) during training. For example, an operation code of an ELF file, multiple operands, such an instruction line and a code fragment of the entire Section content, as well as global information such as a symbol table will learn different weight parameters, so that the Encode model and the Decode model are more in line with the code call type business.
[0085] Step 6: Generate assembly language using the Encoder model and Decoder model.
[0086] The trained Encode model and Decode model are used to generate high-dimensional embedding vectors, and then linear transformation and Softmax function are implemented to map the high-dimensional vectors to assembly language space vectors to generate assembly language.
[0087] Step 7: Generate a high-level language using a large language model based on the generated assembly language.
[0088] The assembly language generated by the Decom-Transformer model is used as input, and the large language model is used to generate high-level language of the specified language and add annotations.
[0089] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.
Claims
1. A method for implementing information innovation adaptation based on decompilation of the Decom-Transformer model, characterized in that: The following steps are involved: Step 1: Get data. Step 2: Load the data obtained in step 1, parse the binary semantics, and assemble the instructions as word input; Step 3: Build a corpus based on the data loaded in step 2, and use the corpus to train the Glove pre-training model for feature extraction; Step 4: Use the Glove pre-trained model built in step 3 to extract features for feature embedding and build a Decom-Transformer model; Step 5: Train the Decom-Transformer model to obtain the Encoder model and the Decoder model; Step 6: Generate assembly language using the Encoder model and the Decoder model; Step 7: Generate a high-level language using a large language model based on the generated assembly language.
2. According to claim 1, the method for implementing information innovation adaptation based on decompilation of the Decom-Transformer model is characterized in that: The data obtained in step 1 includes: assembly language files, instruction set specifications, ABI specifications, ELF files and auxiliary information, wherein the assembly language files, instruction set specifications, ABI specifications, and ELF files are x86 binary files, and the auxiliary information is a variety of ARM files, including application programs, libraries, and operating system binary files.
3. According to claim 1, the method for implementing information innovation adaptation based on decompilation of the Decom-Transformer model is characterized in that: The specific implementation method of step 2 is: reading the data obtained in step 1 into the memory.
4. The method for implementing information innovation adaptation based on decompilation of the Decom-Transformer model as described in claim 1 is characterized in that: The step 3 comprises the following steps: Step 3.1, process the data loaded in step 2; Use the libelf library to read and parse the ELF file and perform actions similar to word segmentation: read different sections of the ELF file, extract the opcodes and operands, and other symbol table binary file related information, and organize them into separate function calls and code segments for training to build a corpus; Step 3.2, encode the data loaded in step 2; Use the instruction set specification and ABI specification to establish a mapping between binary and instruction set, and construct the co-occurrence matrix of Glove; Step 3.3, perform semantic training on the encoded data in step 3.2; By performing matrix decomposition on the Glove co-occurrence matrix, the word vector of each word is learned to form a Glove pre-training model; the Glove model saves the word embedding vector that reflects the semantics of the binary file; Step 3.4: Generate the Glove model as a file based on the data feature vector.
5. According to claim 1, the method for implementing information innovation adaptation based on decompilation of the Decom-Transformer model is characterized in that: The Decom-Transformer model structure constructed in the step 4 is the same as the traditional Transformer structure, but according to the characteristics of the ELF and PE file formats, a sparse Transformer structure is adopted, and sparse attention is applied to reduce the quadratic time and space complexity of self-attention to linear, and three types of attention are introduced: window attention, bridge attention and global attention.
6. According to claim 1, the method for implementing information innovation adaptation based on decompilation of the Decom-Transformer model is characterized in that: In step 5, the Encoder model is used to read the ELF file information and auxiliary information of the source CPU architecture, and the Decoder model is used to generate the assembler of the corresponding architecture, so as to realize the end-to-end conversion from the binary application to the assembly language program.
7. According to claim 1, the method for implementing information innovation adaptation based on decompilation of Decom-Transformer model is characterized in that: The specific implementation method of step 6 is: using the trained Encoder model and Decoder model to generate a high-dimensional embedding vector, and then implementing linear transformation and Softmax function to map the high-dimensional vector to an assembly language space vector to generate assembly language.
8. According to claim 1, the method for implementing information innovation adaptation based on decompilation of Decom-Transformer model is characterized in that: The specific implementation method of step 8 is: using the assembly language generated by the Decom-Transformer model as input, using the large language model to generate a high-level language of the specified language and adding comments.