Application adaptation method of end-to-end cross-CPU architecture
By building the BinCode-Transformer model, the application adaptation across CPU architecture is achieved, and the problems of low adaptation efficiency and high cost in the existing technology are solved, and a small model running on the CPU is realized, which simplifies operations and improves domestic production. The efficiency of adaptation has promoted the development of the ecological chain of information and innovation products.
Patent Information
- Application Number
- CN202411294828.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-14
- Publication Date
- 2024-12-31
AI Technical Summary
The application adaptation efficiency of the existing technology between different CPU architectures is low, high cost, high software technology requirements and large thresholds, which hinders the development of the ecological chain of domestic information innovation products.
Adopting an end-to-end cross-CPU architecture application method, by obtaining and parsing binary data, using Glove pre-trained model to extract features, and building a BinCode-Transformer model, training Encoder and Decoder models to realize cross-CPU architecture Binary-based application conversion reduces the cost and threshold for the underlying library and application multi-CPU adaptation.
It realizes a small model running on the CPU, saves costs, and directly converts binary files of different CPUs through artificial intelligence. It is simple to operate, greatly improving the efficiency of domestic adaptation and promoting the development of the ecological chain of domestic information innovation products.
Smart Images
Figure CN119225738A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of program compilation, and in particular to an end-to-end cross-CPU architecture application adaptation method. Background Art
[0002] There are generally the following methods for existing applications with different CPU architectures:
[0003] 1. Recompile:
[0004] Direct compilation: If the source code is available, the most direct approach is to recompile the source code on the target architecture. This usually involves using a compiler and possible libraries suitable for the target architecture. Cross-platform compilation tools: such as CMake, GNU Autotools, etc., these tools can help automate the compilation process and adapt it to different architectures.
[0005] 2. Binary Translation:
[0006] Use a binary translator (such as QEMU) to emulate the execution environment of the source architecture on the target architecture. This approach does not require source code, but may introduce performance overhead. Static binary translators can convert binary code of the source architecture into code of the target architecture, but this approach may not fully preserve the performance characteristics of the original program.
[0007] 3. Source code conversion tool:
[0008] Some tools (such as Clang's Cross-Translation Utility, CTU) can analyze source code and generate equivalent code for the target architecture. This approach requires source code.
[0009] The above-mentioned existing technologies have the following deficiencies: modifying the code or compiling options based on the source code, and processing different architectures in different ways, which is inefficient. At the same time, the performance of simulators such as QEMU is low and cannot be used in production environments. The software technology capabilities are relatively high, and the threshold is high. Ordinary people cannot do it, which hinders the development of the ecosystem. Summary of the invention
[0010] The purpose of the present invention is to overcome the shortcomings of the prior art and propose an end-to-end cross-CPU architecture application adaptation method. The model trained by the present invention is small and can be run on the CPU, saving costs. In addition, the present invention directly converts binary files for different CPUs through artificial intelligence, which is simple to operate, reduces the cost and threshold of the underlying library and application multi-CPU adaptation, greatly improves the efficiency of localized adaptation of different applications, and promotes the development of the ecological chain of localized information innovation products.
[0011] The present invention solves the technical problem by adopting the following technical solutions:
[0012] An end-to-end cross-CPU architecture application adaptation method comprises the following steps:
[0013] Step 1: Get data.
[0014] Step 2: Load the data obtained in step 1 and perform binary semantic analysis;
[0015] Step 3: Build a corpus based on the data loaded in step 2, and use the corpus to train the Glove pre-training model for feature extraction;
[0016] Step 4: Use the Glove pre-trained model trained in step 3 to extract features for feature embedding and build a BinCode-Transformer model;
[0017] Step 5: Train the BinCode-Transformer model to obtain the Encoder model and the Decoder model;
[0018] Step 6: Perform semantic restoration of the embedded vector through the trained BinCode-Transformer model;
[0019] Step 7: Generate a file based on the result of semantic restoration of the embedded vector in step 6.
[0020] Moreover, the data obtained in step 1 includes: instruction set specifications, ABI specifications, ELF files and auxiliary information, wherein the instruction set specifications, ABI specifications, and ELF files are x86 binary files, and the auxiliary information is a variety of ARM files, including application programs, libraries, and operating system binary files.
[0021] Moreover, the specific implementation method of step 2 is: reading the data obtained in step 1 into the memory.
[0022] Furthermore, the step 3 comprises the following steps:
[0023] Step 3.1, process the data loaded in step 2;
[0024] Use the libelf library to read and parse the ELF file and perform actions similar to word segmentation: read different sections of the ELF file, extract the opcodes and operands, and other symbol table binary file related information, and organize them into separate function calls and code segments for training to build a corpus;
[0025] Step 3.2, encode the data loaded in step 2;
[0026] Use the instruction set specification and ABI specification to establish a mapping between binary and instruction set, and construct the co-occurrence matrix of Glove;
[0027] Step 3.3, perform semantic training on the encoded data in step 3.2;
[0028] Decompose the co-occurrence matrix for building Glove, learn the word vector of each word, and form the Glove pre-training model; the Glove model saves the word embedding vector that reflects the semantics of the binary file;
[0029] Step 3.4: Generate the Glove model as a file based on the data feature vector.
[0030] Moreover, the BinCode-Transformer model structure constructed in step 4 is the same as the traditional Transformer structure, but according to the characteristics of ELF and PE file formats, a sparse Transformer structure is adopted, and sparse attention is applied to reduce the quadratic time and space complexity of self-attention to linear, and three types of attention are introduced: window attention, bridge attention and global attention.
[0031] Moreover, in step 5, the Encoder model is used to read the ELF file information and auxiliary information of the source CPU architecture, and the Decoder model is used to generate an application program of the target CPU architecture, thereby realizing end-to-end binary-based application program conversion across CPU architectures.
[0032] Moreover, the specific implementation method of step 6 is: restore the word vector generated by the Decode model into a semantic vector in the binary instruction set space through linear transformation and Softmax function.
[0033] Moreover, the specific implementation method of step 7 is: combining the obtained semantic vector of the binary instruction set space with the binary file format check to generate a specific binary file.
[0034] The advantages and positive effects of the present invention are:
[0035] The present invention obtains data; loads and extracts features from the data; embeds features based on the extracted features, and constructs and trains a BinCode-Transformer model; performs semantic restoration of embedded vectors through the trained BinCode-Transformer model; and generates files based on the results of semantic restoration of embedded vectors. Since the semantic expression space of binary files such as ELF or PE is small, the basic form is opcodes and operands, as well as keywords of the file structure, so the model trained by the present invention is small (much smaller than the complexity of natural language), and can be run on the CPU, saving costs. At the same time, the present invention directly converts binary files for different CPUs through artificial intelligence, which is simple to operate, reduces the cost and threshold of the underlying library and application of multi-CPU adaptation, greatly improves the efficiency of localized adaptation of different applications, and promotes the development of the ecological chain of localized information creation products. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flow chart of the present invention;
[0037] Figure 2 It is a flow chart of feature extraction of the present invention;
[0038] Figure 3 This is the architecture diagram of the BinCode-Transformer model of the present invention;
[0039] Figure 4 This is an illustration of the attention mechanism principle of the present invention. DETAILED DESCRIPTION
[0040] The present invention is further described in detail below with reference to the accompanying drawings.
[0041] The construction idea of the present invention is:
[0042] The present invention adopts similar Transformer technology, and performs model training on instruction sets, architecture specifications, CPU-level ABI specifications, underlying ABI interface specifications (libc, glibc, libc++, libstdc++, etc.), ELF file structure specifications and other information through large amounts of data to train Encoder and Decoder models. The Encoder is used to read the ELF file information and auxiliary information of the source CPU architecture, and the Decoder is used to generate an application program of the target CPU architecture, so that end-to-end cross-CPU architecture binary-based application conversion can be achieved.
[0043] An end-to-end cross-CPU architecture application adaptation method, such as Figure 1 As shown, the following steps are included:
[0044] Step 1: Get data.
[0045] The acquired data includes: instruction set specification, ABI specification, ELF file and auxiliary information. The instruction set specification, ABI specification and ELF file are x86 binary files, and the auxiliary information is a variety of ARM files, including application programs, libraries and operating system binary files.
[0046] Step 2: Read the data obtained in step 1 into the memory.
[0047] Step 3: Extract features from the data loaded in step 2.
[0048] Collect a variety of ARM and x86 binary files, including applications, libraries, and operating system binaries. And related auxiliary information such as ABI specifications, instruction sets, and specifications. Analyze the data set, including information about function boundaries, control flow, and memory access patterns. Extract features and use these features to generate binary machine code tokens, such as opcode sequences, function call graphs, and memory references. Use these features to form binary machine code tokens. This stage is equivalent to the token vector expression of natural language. Training a set of machine language feature vectors is a prerequisite. These machine language feature vectors have the semantics of machine language. Semantic training is performed by creating a corpus of instruction sets and ELF-related content, and performing new training through GloVe.
[0049] like Figure 2 As shown, step 3 includes the following steps:
[0050] Step 3.1, process the data loaded in step 2;
[0051] Use the libelf library to read and parse the ELF file and perform actions similar to word segmentation: read different sections of the ELF file separately, extract the opcodes and operands, as well as other symbol table binary file related information, and organize them into separate function calls and code segments for training to build a corpus.
[0052] The constructed prediction library is the basis for training the Glove pre-training model. Only after training the Glove model can we construct word vectors that can reflect the semantics of binary files. Word vectors that reflect binary semantics are the basis for training the BinCode-Transformer model.
[0053] Step 3.2: Encode the data loaded in step 2.
[0054] The instruction set specification and ABI specification are used to establish a mapping between binary and instruction set, and a co-occurrence matrix of Glove is constructed.
[0055] The co-occurrence matrix is constructed when training the Glove model, which records the number of times words co-occur within a certain window. Each element of this matrix represents the frequency of co-occurrence of two words in the corpus.
[0056] Step 3.3, perform semantic training on the encoded data in step 3.2;
[0057] The collected prediction information is used to train the Glove pre-trained model, which stores word embedding vectors that reflect the semantics of binary files.
[0058] The Glove model is trained based on this co-occurrence matrix. It learns the word vector (i.e., the word embedding representation) of each word by performing matrix decomposition on the co-occurrence matrix. These word vectors can capture the semantic relationship between words.
[0059] Step 3.4: Generate the Glove model as a file based on the data feature vector.
[0060] Step 4: Embed the features extracted in step 3 and build a BinCode-Transformer model.
[0061] The Glove pre-trained model trained in steps 3.1 to 3.3 can directly generate binary semantic word vectors (the Glove model input is the data loaded in step 2, and the output is a binary word vector)
[0062] like Figure 3 As shown in the figure, the constructed BinCode-Transformer model structure is the same as the traditional Transformer structure, but according to the characteristics of ELF and PE file formats, the sparse Transformer structure is adopted, and sparse attention is applied to reduce the quadratic time and space complexity of self-attention to linear, as shown in Figure 4 As shown in the figure, three types of attention are introduced: window attention, bridge attention, and global attention. Each type is motivated by observations of the previous model. It is more suitable for the construction learning of binary code. The learning efficiency and convergence speed are greatly improved. In terms of details, since the semantics expressed by binary files are relatively simple (operands, opcodes, and symbols defined in the ELF specification, etc.), the vector length and parameters are adjusted according to the actual situation to make it more efficient and the model smaller.
[0063] At the same time, the evaluation and verification process of the BinCode-Transformer model is the same as the general Transformer evaluation and verification process, except that the content of comparison is different. The content of comparison is the sequence of source and target binary files, and special programs need to be compiled for comparison.
[0064] Introducing three types of attention:
[0065] Introducing the Window Attention Mechanism: Considering the application scenario, the structure of the ELF class depends heavily on the local context (instruction data within the Section, etc.), while there are only a few (Section Header, etc.) long-distance dependency instances. This locality is used to sparse attention to achieve higher efficiency.
[0066] The attention mechanism formula is:
[0067]
[0068] Where Q, K, and V represent query, key, and value matrices, respectively. They are linear transformations of the input sequence; K T is the transposed matrix of the key; D k is the dimension of the key vector; M is the mask matrix.
[0069] By adjusting M in the softmax function, the algorithm of M is different under different attention mechanisms.
[0070] In the window attention mechanism, M is calculated as:
[0071]
[0072] Where w is the window size, and i and j are the subscripts of the token. This windowed attention mechanism reduces the complexity of the self-attention mechanism by limiting the receptive field size of each token to a small window of size w at each layer. The computational complexity of this mode is O(n×w), which scales linearly with the input sequence length n. After applying this sliding window attention mechanism for N Transformer layers, the receptive field size increases to N×w at the top layer. Since each token only pays attention to the w tokens to its left, rather than the entire previous sequence, this model can achieve faster inference speed.
[0073] Introducing the BridgeAttention mechanism: Window attention mainly processes local context (instruction data in Section, etc.), but in some structures such as ELF, it is necessary to refer to the contents of SectionHeader and SectionTable. Bridge tokens are introduced for these long-distance references to aggregate local information for access. Bridge tokens can process tokens of fixed length and participate from all subsequent tokens, which can be understood as learning about Section.
[0074] In the bridging attention mechanism, M is calculated as:
[0075]
[0076] Among them, S b is the set of bridge tokens, n is the length of the input token sequence, and m is the number of bridge tokens. The complexity of bridge attention is O(m×n)≈O(n), where m<<n. Compared with stacked window attention, bridge attention allows each token to participate in any previous token with at most 2 hops, which enables the model to effectively access remote context.
[0077] Introducing the Global Attention Mechanism: In structures such as ELF, global variables and functions have global scope identifiers. The previous two attention mechanisms cannot handle this situation. The introduction of the Global Attention Mechanism can be called from any location in the file. In order to inject attention into the code heuristics, the code structure with a tree setter is used to parse the code into an abstract syntax tree (AST) to help understand the relationship between each other.
[0078] In the global attention mechanism, M is calculated as:
[0079]
[0080] Among them, G is a set of locations with global significance, such as global variables, functions, section headers, symbol tables, etc. The complexity of global attention is O(kn)≈O(n), because we have k<<n, where n is the length of the sequence. The most important sentences are extracted and encoded in a fixed-length context window, and global attention requires less memory.
[0081] Considering all three types of attention together, M is calculated as:
[0082] M=max(M window ,M bridge ,M global )
[0083] Step 5: Train the BinCode-Transformer model to obtain the Encoder model and the Decoder model.
[0084] Among them, the Encoder model is used to read the ELF file information and auxiliary information of the source CPU architecture.
[0085] The Encoder model reads the binary file of the conversion source CPU architecture (such as x86_64), segments the binary file content according to the specification, calls the feature encoder to encode the binary Token, and generates a feature vector. The feature vector is the entire vector space content such as binary instructions, data, symbol table, character information, etc. Positional encoding is performed according to the ELF or PE file structure. The binary semantic expression feature vector Token and Positional encoding are the comprehensive features of the input.
[0086] The Decoder model is used to generate applications for the target CPU architecture and implement end-to-end binary-based application conversion across CPU architectures.
[0087] The Decoder model reads the binary file of the conversion target CPU architecture (such as ARM64) corresponding to the same file, segments the binary file content according to the specification, and performs masking operations on the opcodes or operands therein, calls the feature encoder to encode the binary Token, and generates a feature vector. The feature vector is the entire vector space content such as binary instructions, data, symbol table, character information, etc., and is input by Positional encoding according to the ELF or PE file structure.
[0088] The present invention introduces a window attention mechanism (WindowAttention), a bridge attention mechanism (BridgeAttention), and a global attention mechanism (GlobalAttention) during training. For example, an operation code of an ELF file, multiple operands, such an instruction line and a code fragment of the entire Section content, as well as global information such as a symbol table will learn different weight parameters, so that the trained Encode model and Decode model are more in line with code call-type businesses.
[0089] Step 6: Perform semantic restoration of the embedded vector through the trained BinCode-Transformer model.
[0090] The word vector generated by the Decode model is restored to a semantic vector in the binary instruction set space through linear transformation and Softmax function.
[0091] Step 7: Generate a file based on the result of semantic restoration of the embedded vector in step 6.
[0092] The semantic vector of the binary instruction set space is combined with the binary file format check to generate a specific binary file.
[0093] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.
Claims
1. An end-to-end cross-CPU architecture application adaptation method, characterized by: The following steps are involved: Step 1: Get data. Step 2: Load the data obtained in step 1 and perform binary semantic analysis; Step 3: Build a corpus based on the data loaded in step 2, and use the corpus to train the Glove pre-training model for feature extraction; Step 4: Use the Glove pre-trained model trained in step 3 to extract features for feature embedding and build a BinCode-Transformer model; Step 5: Train the BinCode-Transformer model to obtain the Encoder model and the Decoder model; Step 6: Perform semantic restoration of the embedded vector through the trained BinCode-Transformer model; Step 7: Generate a file based on the result of semantic restoration of the embedded vector in step 6.
2. The end-to-end cross-CPU architecture application adaptation method according to claim 1, characterized in that: The data obtained in step 1 includes: instruction set specifications, ABI specifications, ELF files and auxiliary information, wherein the instruction set specifications, ABI specifications, and ELF files are x86 binary files, and the auxiliary information is a variety of ARM files, including application programs, libraries, and operating system binary files.
3. The end-to-end cross-CPU architecture application adaptation method according to claim 1, characterized in that: The specific implementation method of step 2 is: reading the data obtained in step 1 into the memory.
4. The end-to-end cross-CPU architecture application adaptation method of claim 1, characterized in that: The step 3 comprises the following steps: Step 3.1, process the data loaded in step 2; Use the libelf library to read and parse the ELF file and perform actions similar to word segmentation: read different sections of the ELF file, extract the opcodes and operands, and other symbol table binary file related information, and organize them into separate function calls and code segments for training to build a corpus; Step 3.2, encode the data loaded in step 2; Use the instruction set specification and ABI specification to establish a mapping between binary and instruction set, and construct the co-occurrence matrix of Glove; Step 3.3, perform semantic training on the encoded data in step 3.2; Decompose the co-occurrence matrix for building Glove, learn the word vector of each word, and form the Glove pre-training model; the Glove model saves the word embedding vector that reflects the semantics of the binary file; Step 3.4: Generate the Glove model as a file based on the data feature vector.
5. The end-to-end cross-CPU architecture application adaptation method according to claim 1, characterized in that: The BinCode-Transformer model structure constructed in step 4 is the same as the traditional Transformer structure, but according to the characteristics of ELF and PE file formats, a sparse Transformer structure is adopted, and sparse attention is applied to reduce the quadratic time and space complexity of self-attention to linear, and three types of attention are introduced: window attention, bridge attention and global attention.
6. The end-to-end cross-CPU architecture application adaptation method according to claim 1, characterized in that: In step 5, the Encoder model is used to read the ELF file information and auxiliary information of the source CPU architecture, and the Decoder model is used to generate an application program of the target CPU architecture, thereby realizing end-to-end binary-based application program conversion across CPU architectures.
7. The end-to-end cross-CPU architecture application adaptation method according to claim 1, characterized in that: The specific implementation method of step 6 is: restore the word vector generated by the Decode model into a semantic vector in the binary instruction set space through linear transformation and Softmax function.
8. The end-to-end cross-CPU architecture application adaptation method according to claim 1, characterized in that: The specific implementation method of step 7 is: combining the obtained semantic vector of the binary instruction set space with the binary file format check to generate a specific binary file.