A performance prediction method, system and computing device based on machine instructions

Abstract

A performance prediction method based on machine instructions, comprising: determining a basic instruction block according to an input test case; the basic instruction block includes one or more instructions; the basic instruction block has a unique input and output; determining multiple key characters in each instruction, the multiple key characters including an operator and an operation object; for instructions written in different machine languages, arranging multiple key characters in each instruction in the same order, encoding and processing them, and obtaining an encoding vector of the basic instruction block; the machine language includes one of the following categories: assembly language, ARM language or X86 language; according to the encoding vector of the basic instruction block, performing self-supervised learning on a BBI-BERT pre-trained model to generate a first representation vector; determining hardware configuration parameters, and predicting the performance indicators of a CPU chip according to the first representation vector and the hardware configuration parameters.

Description

Technical Field

[0001] The present application relates to the field of natural language processing (NLP), and in particular to a performance prediction method, system and computing device based on machine instructions. Background Art

[0002] In the field of natural language processing, pre-trained models have achieved great success. Pre-trained models are pre-trained on large-scale general domain data by designing unsupervised training tasks, and then the pre-trained models are applied to downstream target tasks, which can greatly improve the efficiency of downstream target tasks.

[0003] Both natural language and machine language are representations of symbol sequences, and the function of the pre-trained model is to generate samples of symbol sequences to obtain better vectorized feature representations of symbol sequences, thereby affecting downstream tasks.

[0004] However, the existing technology vectorizes the machine language instructions for designated tasks under a specific CPU chip, which is strongly coupled to the task, has poor generalization, and cannot achieve migration between parameters. It is also impossible to extract learning-based representation vectors for very long instruction sequences, resulting in incomplete semantics, a large loss of information and precision, and low reasoning efficiency and poor results for downstream tasks. Summary of the invention

[0005] The embodiment of the present application proposes a performance prediction method, system and computing device based on machine instructions, which performs structured processing on the original binary instructions in the test application written in machine language, and uses the BBI-BERT pre-trained model to generate representation vectors, which are decoupled from the task and have strong generalization. Among them, the BBI-BERT pre-trained model is a basic block instruction-bidirectional encoder representations from transformers (BBI-BERT) pre-trained model.

[0006] In the first aspect, an embodiment of the present application proposes a performance prediction method based on machine instructions, including: determining a basic instruction block according to an input test case; the basic instruction block includes one or more instructions; the basic instruction block has a unique input and output; determining multiple key characters in each instruction, the multiple key characters include an operator and an operand; for instructions written in different machine languages, arranging multiple key characters in each instruction in the same order, and encoding and processing them to obtain an encoding vector of the basic instruction block; the machine language includes one of the following categories: assembly language, ARM language or X86 language; according to the encoding vector of the basic instruction block, self-supervising learning is performed on the BBI-BERT pre-trained model to generate a first representation vector; determining hardware configuration parameters, and predicting the performance indicators of the CPU chip according to the first representation vector and the hardware configuration parameters.

[0007] In this way, when performing vectorized representation and cross-platform recognition coding of test cases, the embodiment of the present application does not need to embed hardware configuration information. The original binary instructions in the test cases written in machine language are uniformly coded and processed, and the basic block instructions are self-supervised by using the BBI-BERT pre-training model. The generated representation vector can fully express the semantic information and jump information of the instruction, decoupled from the task, and has strong generalization. The BBI-BERT method is used to realize the vectorized representation of instruction block coding, and the CPU performance is predicted under different programs in combination with the hardware configuration information, supporting better modeling performance and multi-task generalization.

[0008] In a possible implementation, the test case includes multiple basic instruction blocks, and there is a jump relationship between the multiple basic instruction blocks. The method also includes: determining the encoding vectors of the multiple basic instruction blocks based on the multiple basic instruction blocks; determining the relationship matrix based on the jump relationship between the multiple basic instruction blocks; performing self-supervised learning on the BBI-BERT pre-trained model based on the encoding vectors and the relationship matrix of the multiple basic instruction blocks to generate a second representation vector; the second representation vector includes the first representation vector of each basic instruction block, and the connection between the first representation vectors of each basic instruction block conforms to the jump relationship.

[0009] Therefore, the embodiment of the present application uses the BBI-BERT pre-trained model to perform self-supervised learning on the ultra-long instruction sequences in the test cases written in machine language, and the generated second representation vector can fully express the semantic information and jump information of the test cases with high accuracy.

[0010] In one possible implementation, the BBI-BERT pre-trained model includes multiple Transformer encoders, which perform multi-attention mechanism processing on the encoding vector of the basic instruction block, and then transform and extract features through a feedforward neural network to output the first representation vector of the basic instruction block.

[0011] In this way, the embodiment of the present application performs multi-head attention processing on each key character in the instruction unit through a multi-layer Transformer encoder, which can characterize the upper and lower adjacent coding information. The first representation vector predicted by the feedforward neural network is applied to downstream task reasoning, which can greatly improve the reasoning efficiency and reduce the error, and support better modeling performance and multi-task generalization.

[0012] In one possible implementation, self-supervised learning is performed on the BBI-BERT pre-trained model, including random mask learning for the basic instruction block, including: randomly masking the codes in the coding vector of the basic instruction block; based on the coding vector of the masked basic instruction block, the BBI-BERT pre-trained model generates a third representation vector; normalization is performed based on the third representation vector and a multi-category classifier, and the normalization result is the conditional probability that the restored code belongs to the original category; the value of the first loss function is determined based on the normalization result, and the parameters of the BBI-BERT pre-trained model are iteratively trained until the first loss function converges.

[0013] Therefore, for a given coding vector of a basic instruction block, the embodiment of the present application randomly erases a code in the instruction unit, and can predict the conditional probability that the restored code belongs to the original category based on the remaining code. Based on self-supervised learning of random masks and multiple classifiers, the parameters of the BBI-BERT pre-trained model are iteratively trained to optimize the BBI-BERT pre-trained model.

[0014] In one possible implementation, self-supervised learning is performed on the BBI-BERT pre-trained model, and also includes random replacement learning for the basic instruction block, including: randomly replacing the encoding of the encoding vector of the basic instruction block; based on the encoding vector of the replaced basic instruction block, the BBI-BERT pre-trained model generates a fourth representation vector; based on the fourth representation vector and the multi-category classifier, predicts the probability of random replacement; determines the value of the second loss function based on the probability of random replacement, and iteratively trains the parameters of the BBI-BERT pre-trained model until the value of the second loss function converges.

[0015] Therefore, for a given encoding vector of a basic instruction block, the embodiment of the present application randomly replaces a code in an instruction unit, requires predicting whether there is a random replacement, and iteratively trains the parameters of the BBI-BERT pre-trained model based on random replacement and self-supervised learning of multiple classifiers to optimize the BBI-BERT pre-trained model.

[0016] The performance prediction method based on machine instructions provided in the embodiment of the present application constructs the smallest instruction unit for BB-based instruction sequence learning, and utilizes the attention structure of Transformer-Encoder and the prediction method of random mask and random replacement to realize self-supervised learning of BB instruction blocks.

[0017] In one possible implementation, the BBI-BERT pre-trained model is subjected to self-supervised learning, and also includes contrastive learning, including: determining a basic block jump graph based on a second representation vector; each node in the basic block jump graph corresponds to a first representation vector of each basic instruction block; the connection between each node conforms to a jump relationship; randomly replacing a node in the basic block jump graph to determine positive samples and negative samples; predicting whether the replaced node conforms to the jump relationship in the basic block jump graph through contrastive learning of positive and negative samples; determining a third loss function value based on the prediction result; calculating the value of the total loss function based on the first loss function, the second loss function and the third loss function, and iteratively training the parameters of the BBI-BERT pre-trained model until the value of the total loss function converges.

[0018] Therefore, the embodiment of the present application puts the representation vector learned from the BB instruction into the jump graph of the ultra-long instruction sequence as node information, and then obtains the structural information of the jump graph by randomly replacing the context of the node for comparative learning, optimizes the BBI-BERT pre-training model, and realizes the vectorized representation of the ultra-long sequence, which can fully express the semantic information and jump information of the ultra-long instruction sequence with high accuracy.

[0019] In a possible implementation, the hardware configuration parameters are concatenated or fused with the first representation vector or the second representation vector, and the parameters of the BBI-BERT pre-trained model are transferred and trained based on the concatenated or fused representation vectors.

[0020] In this way, the present application can perform performance prediction on a processor with different parameter configurations under the same instruction set. It only needs to fine-tune the BBI-BERT model to achieve transfer learning and reuse it. It can also fine-tune the transformer under different parameter configurations of the processor to achieve transfer learning.

[0021] In a possible implementation, each instruction unit is processed in the same arrangement, including: arranging the operators and operands in each instruction in the set arrangement, and performing OneHot encoding and concatenation processing.

[0022] In this way, the embodiments of the present application can be arranged in the same order, can unify the grammatical structure, decouple from the grammatical category of the instructions, improve generalization, and be applicable to the vectorized representation of various categories of machine language instructions.

[0023] In one possible implementation, the splicing process includes embedding a first hint at the beginning of the encoding vector of the basic instruction block, and embedding a second hint at the end of each instruction.

[0024] In this way, the embodiment of the present application vectorizes multiple instructions according to the control flow and embeds prompts for distinction.

[0025] In one possible implementation, the hardware configuration parameters include parameters of a processor, a CPU, a memory, and a hard disk. Determining the hardware configuration parameters and predicting the performance indicators of the CPU chip based on the first characterization vector and the hardware configuration parameters include: updating the hardware configuration parameters and predicting the performance indicators of the CPU chip based on the first characterization vector and the updated hardware configuration parameters.

[0026] In this way, the method provided in the embodiment of the present application vectorizes the basic instruction blocks in the field of chip design, and then combines the use of chip architecture parameters for target (IPC / Power, etc.) modeling, which can provide better accuracy and generalization characterization capabilities and provide development tools for automated chip design; performance prediction can be performed on a processor with different parameter configurations under the same instruction set, and performance prediction can also be performed under different parameter configurations of the processor.

[0027] In a possible implementation, the method further includes: predicting a performance indicator of the CPU according to the second characterization vector and the hardware configuration parameter.

[0028] In this way, the method provided in the embodiment of the present application vectorizes the ultra-long instruction sequences of the test program use cases in the field of chip design, and then combines the use of chip architecture parameters for target (IPC / Power, etc.) modeling, which can provide better accuracy and generalization characterization capabilities and provide development tools for automated chip design.

[0029] In the second aspect, the embodiment of the present application proposes a system for generating a characterization vector for machine instructions, the system comprising: an instruction processing module for determining a basic instruction block according to an input test case; a basic instruction block comprises one or more instructions; a basic instruction block has a unique input and output; determining multiple key characters in each instruction, the multiple key characters comprising an operator and an operand; for instructions written in different machine languages, multiple key characters in each instruction are arranged in the same order, and encoded and processed to obtain an encoding vector of the basic instruction block; the machine language comprises one of the following categories: assembly language, ARM language or X86 language; and a BBI-BERT pre-training model for self-supervised learning according to the encoding vector of the basic instruction block to generate a first characterization vector; the BBI-BERT pre-training model comprises multiple Transformer encoders, each Transformer encoder comprises a multi-attention mechanism and a feedforward neural network; a configuration parameter acquisition module for determining hardware configuration parameters; a performance prediction network for predicting the performance indicators of the CPU according to the first characterization vector and the hardware configuration parameters. Its beneficial effects are as in the first aspect, and will not be repeated here.

[0030] In a possible implementation, the test case includes multiple basic instruction blocks, and there is a jump relationship between the multiple basic instruction blocks. The device also includes: an ultra-long instruction encoding module, used to determine the encoding vectors of the multiple basic instruction blocks based on the multiple basic instruction blocks; determine the relationship matrix according to the jump relationship between the multiple basic instruction blocks; the BBI-BERT pre-training model is also used to perform self-supervised learning based on the encoding vectors and the relationship matrix of the multiple basic instruction blocks to generate a second representation vector; the second representation vector includes the first representation vector of each basic instruction block, and the connection between the first representation vectors of each basic instruction block conforms to the jump relationship.

[0031] In one possible implementation, the BBI-BERT pre-trained model includes multiple Transformer encoders, which perform multi-attention mechanism processing on the encoding vector of the basic instruction block, and then transform and extract features through a feedforward neural network to output the first representation vector of the basic instruction block.

[0032] In a third aspect, an embodiment of the present application proposes a computing device, comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory, wherein when the programs stored in the memory are executed, the processor is used to execute the method as described in any one of the first aspects.

[0033] In a fourth aspect, an embodiment of the present application proposes a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method as described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solutions of the multiple embodiments disclosed in this specification, the drawings required for the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only the multiple embodiments disclosed in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] The following is a brief introduction to the drawings required for describing the embodiments or prior art.

[0036] Figure 1 Ithemal network diagram for the first solution;

[0037] Figure 2 The PalmTree model diagram provided for the second solution;

[0038] Figure 3 A schematic diagram of an application system architecture of a performance prediction method based on machine instructions provided in an embodiment of the present application;

[0039] Figure 4 An architectural diagram of the BBI-BERT pre-training model in the method provided in an embodiment of the present application;

[0040] Figure 5 A processing flow chart of a basic instruction block in the method provided in an embodiment of the present application;

[0041] Figure 6 An example diagram of the processing of a basic instruction block in the method provided in an embodiment of the present application;

[0042] Figure 7 An example diagram of self-supervised training of the BBI-BERT pre-training model in the method provided in an embodiment of the present application;

[0043] Figure 8a A schematic diagram of mapping a very long instruction sequence and a jump map in the method provided in an embodiment of the present application;

[0044] Figure 8b A schematic diagram of randomly replacing nodes in a jump graph in the method provided in an embodiment of the present application;

[0045] Figure 8c A schematic diagram of self-supervised learning of a BBI-BERT pre-trained model for an extremely long instruction sequence in the method provided in an embodiment of the present application;

[0046] Fig. 9 A system schematic diagram of a performance prediction method based on machine instructions is provided for Example 1 of the present application;

[0047] Fig.10 A flowchart of a performance prediction method based on machine instructions is provided for Example 1 of the present application;

[0048] Fig.11 A flowchart of a performance prediction method based on machine instructions is provided for Embodiment 2 of the present application;

[0049] Fig.12 The following is a schematic diagram of the verification results of this application under three Intel chips: Ivy Bridge, Haswell and Skylake. DETAILED DESCRIPTION

[0050] In the description of the embodiments of the present application, words such as "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a concrete way.

[0051] In the description of the embodiments of the present application, the term "and / or" is merely a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the term "multiple" means two or more. For example, multiple systems refers to two or more systems, and multiple terminals refers to two or more terminals.

[0052] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. The terms "include", "comprises", "has" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0053] In the description of the embodiments of the present application, reference is made to “some embodiments”, which describe a subset of all possible embodiments, but it can be understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0054] In the description of the embodiments of the present application, the terms "first\second\third, etc." or module A, module B, module C, etc. are only used to distinguish similar objects and do not represent a specific ordering of the objects. It can be understood that the specific order or sequence can be interchanged where permitted so that the embodiments of the present application described here can be implemented in an order other than that illustrated or described here.

[0055] Relevant terms involved in the embodiments of this application:

[0056] Assembly instructions are some operators and mnemonics used in assembly language, and also include some pseudo-instructions (such as assume, end).

[0057] A basic block is a sequence of statements executed sequentially in an application or test case. It has only one entry and one exit. The entry is the first statement, and the exit is the last statement. For a basic block, execution only enters from the first statement and exits from the last statement. Specifically, a basic block has only one entry, which means that there is no other place in the program that can enter this basic block through jump instructions; it has only one exit, which means that only the last instruction in the program can lead to entering other basic blocks for execution. A typical feature of a basic block is that as long as the first instruction in the basic block is executed, all executions in the basic block will be executed only once in sequence. Basic blocks can be represented by source code, assembly, instructions, etc.

[0058] An extremely long instruction sequence refers to a partial or complete application or test case, which can be tens of millions of statements, including many basic instruction blocks and jumps between basic instruction blocks. An example of an extremely long instruction sequence is the standard SSD solid state drive transfer speed test tool benchmark.

[0059] OneHot encoding, also known as one-bit effective encoding, uses an N-bit state register to encode N states, each state has an independent register bit, and at any time, only one of them is valid. For example, in machine language processing, if an instruction includes n operators and m operands, each operator can be represented by an m-dimensional one-hot vector, each operand can be represented by an m-dimensional one-hot vector, and the length of the instruction one-hot encoding is m+n.

[0060] The instruction per clock (IPC) metric refers to the number of instructions executed by the CPU in each clock cycle. The IPC metric determines the computing speed performance of the CPU architecture.

[0061] In the description of the embodiments of the present application, the numbers representing the steps, such as S110, S120, etc., do not necessarily mean that the steps will be executed in this manner. If permitted, the order of the previous and next steps can be interchanged, or they can be executed simultaneously.

[0062] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of this application and are not intended to limit this application.

[0063] There are many training tasks for language models, and the most widely used one is the random masked language model (MLM). It first randomly masks the symbol sequence with a certain probability, and then trains the deep network model to predict the masked symbols, so that the model can learn the relationship between symbols in the sequence and obtain the vector representation of symbols and sequences.

[0064] Figure 1 The Ithemal network diagram provided for the first solution. Figure 1 As shown in the figure, this solution targets the CPU static performance prediction task. The assembly instructions to be tested are run on a real CPU. The time required to execute the test code is the performance of the CPU. In order to achieve vectorized encoding of the instruction block of the test code, in the feature layer of the Ithemal network, each input instruction is represented by a OneHot encoding method, and the encoding vector of the instruction block is obtained through feature embedding lookup table. In the instruction layer of the Ithemal network, a multi-level long short-term memory (LSTM) network is used to extract the representation of the encoding vectors of multiple instructions respectively, and the representation h of each encoding is obtained. mov …h add In the prediction layer of the Ithemal network, according to each encoded representation h mov …h add Get the vectorized representation h of the entire instruction block block , the vector representation h block Input a neural network model Regression training is performed on the target time overhead to enable the neural network model to predict the performance of instruction blocks under specific CPU chips.

[0065] The first solution uses OneHot encoding to vectorize the assembly instruction blocks, but there is no instruction semantic representation, resulting in loss of meaning. The vectorized representation of the instruction blocks of the test code is strongly coupled with the prediction task and cannot achieve migration between parameters. The LSTM method is used to extract the representation vector, which has a strong timing dependency problem, resulting in incomplete semantic information obtained by the representation vector and low reasoning efficiency. It is strongly coupled with the task and is difficult to generalize to more downstream tasks.

[0066] Figure 2 The PalmTree model diagram provided for the second solution. Figure 2 As shown in the figure, specifically, a DFG jump graph is constructed based on the assembly instruction stream, and pairs of two adjacent assembly instructions related to the assembly instruction stream in the DFG jump graph are constructed. The vector representation of the instruction pair is obtained by using the context-aligned statement pair construction method. According to the vector representation of the instruction pair, the target time overhead of the downstream task is predicted, classified, or regressed. The learning of vector representation can be carried out in the following three ways: ① Randomly mask the operators or operands in the instructions and infer the masked operators or operands; ② Randomly exchange the order of instructions to determine whether there is a change in order; ③ Randomly replace instructions to determine whether they belong to adjacent instructions in the control flow graph.

[0067] The PalmTree model is trained on pairs of instructions, ignoring the relationship between more instruction sequences. For downstream tasks based on Basic Block, the PalmTree model cannot directly obtain the vector representation of Basic Block, which will lose accuracy. If it is extended to the representation of instruction sequences for the entire application or test case, for very long sequences of assembly instructions with more than 10 million lines, the potential semantics predicted by summing and averaging pooling methods will lose a lot of information, especially the information of jump instructions within the program.

[0068] In view of this, an embodiment of the present application provides a method for generating a machine instruction representation vector, which performs random mask prediction learning on instruction sequences based on a pre-trained model of bidirectional encoder representations from transformers (BERT) to obtain a better representation vector, thereby affecting downstream tasks and supporting better prediction performance and multi-task generalization.

[0069] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0070] Figure 3 A schematic diagram of the application system architecture of the performance prediction method based on machine instructions provided in the embodiment of the present application. Figure 3 As shown, the application system architecture of the present application includes an instruction processing module 31, a BBI-BERT pre-trained model (BBI-BERT model) 32, a configuration parameter acquisition module 33, a feature fusion module 34 and a performance prediction network 35.

[0071] The instruction processing module 31 determines a basic instruction block according to an input test case; the basic instruction block includes one or more instructions; the basic instruction block has a unique input and output; multiple key characters in each instruction are determined, and the multiple key characters include operators and operands; for instructions written in different machine languages, multiple key characters in each instruction are arranged in the same order, and encoded and processed to obtain an encoding vector of the basic instruction block; the machine language includes one of the following categories: assembly language, ARM language or X86 language.

[0072] The BBI-BERT pre-trained model 32 performs self-supervised learning based on the encoding vector of the basic instruction block to generate a first representation vector.

[0073] Figure 4 This is a diagram of the architecture of the BBI-BERT pre-training model in the method provided in the embodiment of the present application. Figure 4 As shown, the BBI-BERT pre-trained model 32 is composed of multiple layers of Transformer encoders stacked together. Each Transformer encoder consists of multiple multi-head attention mechanisms + feed-forward neural networks (FFN).

[0074] Specifically, the multi-layer Transformer encoder takes the instruction unit as input, performs multi-head attention on each key character in the instruction unit, and outputs the intermediate value of the representation vector corresponding to each key character. The intermediate value of the representation vector can represent the adjacent encoding information above and below.

[0075] A feed-forward neural network (FFN) includes an input layer, several hidden layers, and an output layer. The input layer receives the input of the intermediate value of the representation vector, the hidden layer performs certain transformations and feature extraction on the input, and the output layer outputs the predicted first representation vector based on the results of the hidden layer.

[0076] The encoding vector of the basic instruction block of the input Transformer-Encoder network has the same dimension as the first representation vector output by the Transformer-Encoder network. For example, the encoding vector of the input basic instruction block and the output representation vector can both be 256-dimensional.

[0077] This embodiment does not intend to impose any limitation on the Transformer-Encoder network structure. Any model based on the attention mechanism can be used, and the specific needs need to be determined in combination with actual conditions.

[0078] In some feasible implementations, the Transformer-Encoder network is trained in a self-supervised manner, the encoding vectors of the basic instruction blocks are randomly masked and randomly replaced, and the predicted first representation vector is output.

[0079] The configuration parameter acquisition module 33 determines the hardware configuration parameters.

[0080] In some implementations, configuration parameters may be obtained according to a stored hardware configuration parameter file; the hardware includes chips, CPU, memory, hard disk, etc. The configuration parameters include parameter indicators such as chips, CPU, memory, hard disk, etc.

[0081] The feature fusion module 34 is used to concatenate or fuse the hardware configuration parameters and the first characterization vector to obtain a mixed characterization vector.

[0082] In some practicable implementations, the configuration parameters may be concatenated to the tail dimension of the first representation vector to obtain the sum of the two.

[0083] The performance prediction network 35 predicts the performance index of the CPU chip according to the first characterization vector and the hardware configuration parameters.

[0084] In some implementations, the concatenated or fused mixed representation vector may be input into a performance prediction network 35 to predict the IPC performance index of the CPU; the performance prediction network may be an MLP neural network.

[0085] The training method of the BBI-BERT pre-training model proposed in this application is explained below with reference to a flowchart and specific embodiments.

[0086] Figure 5 This is a processing flow chart of the basic instruction block in the method provided in the embodiment of the present application. It can be understood that the method can be executed by any model, device, platform, or device cluster with computing and processing capabilities. Figure 5 As shown, this application adopts a unified syntax structure, extracts the instruction sequence of the test code, encodes each arranged operator and operand, and realizes the conversion from string to digital vector representation. Among them, the unified syntax structure includes operator + source operand + destination operand, and the encoding method is OneHot encoding. Specifically, it includes the following steps:

[0087] S401, determining the basic block instructions to be tested according to the input test case.

[0088] The test case may be a test code written in machine language, which includes assembly language, ARM language, or X86 language.

[0089] like Figure 6 As shown, the basic block (Basic Block) instruction 53 can be determined according to the basic instruction block 51 of ARM or the basic instruction block 52 of X86. The basic block (Basic Block) instruction 53 includes one or more test instructions of the instruction control flow and data flow, has a unique input and output, and has no jump characteristics in the middle.

[0090] S402, slicing each instruction in the basic instruction block, extracting key characters therein, where the key characters include operators and operands.

[0091] The code slices of the test instructions are shown in Table 1:

[0092] Table 1

[0093]

[0094] In Table 1, the first column is the test instruction code set of the hexadecimal code corresponding to the basic instruction block, the second column is the operator expressed by the hexadecimal test code, and the third column is the operand expressed by the hexadecimal test instruction code. The operand is a memory operand, including a source operand and a target operand. The source operand is a base address, and the target operand is an optional offset and displacement. <mem> and< / mem> The operands also include constants, such as integer constants, memory addresses, etc. <const>Express.

[0095] Exemplarily, the operator may include control words such as mov, sub, cmp, jbe, etc., where jbe is the jump instruction control word, which is the last sentence in the basic instruction block; the source operand is a base address, including rcx, rax, 0xffdf, etc.; the target operand is an optional offset and displacement, such as r8, rax, 0xf.

[0096] In some embodiments, based on the differences in the syntax of machine languages ​​such as assembly, ARM and / or X86, each test instruction in the assembly basic instruction block, ARM basic instruction block or X86 basic instruction block is sliced ​​to extract key characters therein, where the key characters include operators, source operands and destination operands.

[0097] In this way, redundant codes and different grammatical components in the test instruction code set can be removed.

[0098] S403, arranging the key characters according to the set structure to obtain the instruction unit corresponding to each instruction.

[0099] like Figure 6 As shown, the set structure can be a set character arrangement method; the operator, source operand, and destination operand are arranged in a unified grammatical structure to obtain the instruction unit 54 corresponding to each instruction, and each instruction unit 54 includes the following grammatical structure: operator+source operand+destination operand.

[0100] In this way, the syntax structure can be unified according to the set structural arrangement, decoupled from the syntax category of the instruction, and improved generalization, and it is also applicable to the vectorized representation of ARM basic instruction blocks and X86 basic instruction blocks.

[0101] S404, OneHot encoding is performed on each key character in the instruction unit to obtain a one-hot encoding sequence.

[0102] OneHot encoding means that for each instruction unit, each operator, source operand, and target operand is encoded into an N-dimensional one-hot vector to obtain the corresponding one-hot encoding sequence.

[0103] For the one-hot encoding sequence of the instruction unit, since each instruction unit includes M types of encoding, illustratively, M=the number of operator categories M1+the number of source operand categories M2+the number of target operand categories M3, the length of the one-hot encoding sequence of each instruction unit is M.

[0104] S405 , arranging the one-hot encoding sequence according to the control flow of the basic instruction block, embedding the prompt, and obtaining the encoding vector of the basic block.

[0105] In some implementations, the one-hot encoding sequence corresponding to each instruction unit may be arranged according to the control flow, and prompts may be embedded between adjacent sequences to obtain an encoding vector stream of a basic block.

[0106] For example, Figure 6 As shown, at the beginning of the encoding vector 55 of the basic block is embedded <cls>Prompt, embedded at the end of each instruction unit <sep>prompt.

[0107] For an encoding vector of a basic instruction block including N instruction units, since each instruction unit includes M types of encoding and 1 prompt, the vector length of the basic block encoding category is N*(M+1).

[0108] The machine instruction-based performance prediction method provided in the embodiment of the present application, through the above steps S401-S405, adopts a unified grammatical structure to realize the encoding representation of instruction sequences of different machine languages ​​such as x86 / ARM.

[0109] In an embodiment of the present application, the training of the BBI-BERT pre-trained model 32 adopts self-supervised learning. Among them, self-supervised learning can be regarded as an "ideal state" of machine learning, in which the model learns directly from unlabeled data without labeling data. In addition, the BBI-BERT pre-trained model 32 is trained in a self-supervised learning manner, usually through a self-supervised training task. The self-supervised training tasks related to machine language include random masking (opcode and operand masking, OAOM) learning of the encoding vectors of the basic instruction blocks, random replacement (replaced instruction detection model, RID) learning and comparative learning of the encoding vectors of the basic instruction blocks corresponding to the ultra-long (tens of millions +) instruction sequences.

[0110] The following first introduces the random masking (opcode and operandmasking, OAOM) task and the random replacement (replaced instruction detection model, RID) task for the encoding vector of the basic instruction block.

[0111] For the OAOM task, the task description is: given an encoding vector of a basic instruction block, randomly erase a code in the instruction unit, and predict the conditional probability that the restored code belongs to the original category based on the remaining code.

[0112] Figure 7 Schematic diagram of self-supervised learning of the BBI-BERT pre-training model provided in the embodiment of the present application. Figure 7 As shown, taking the mask of the target operand as an example, when implementing the OAOM task, a certain instruction unit is randomly selected for prediction in the encoding vector of the basic instruction block with a certain probability, such as a probability of 30%. If the encoding in the i-th instruction unit is selected as the mask, it is processed as follows:

[0113] [MASK] the encoding of the destination operand with 85% probability. For example, Mov Rcx r8——>Mov Rcx[MASK]

[0114] The masked ith instruction unit is input into the BBI-BERT pre-trained model 32 to generate a mask representation vector. The mask representation vector can be recorded as the third representation vector.

[0115] Exemplarily, the encoding of the character r8 in the instruction “Mov Rcx r8” is masked, and the generated third characterization vector is the characterization vector of “Mov Rcx”.

[0116] The third representation vector is input into a multi-category classifier. The multi-category classifier classifies the third representation vector using the encoded category as a classification target. The classification result includes the masked encoded category.

[0117] It should be understood that the multi-class classifier uses the coding category as the classification target. For the coding vector of the basic instruction block including N instruction units, without considering the prompt, the number of multi-class classifiers required is the same as the vector length of the basic block coding category. Therefore, the vector length of the category output by the multi-class classifier is N*(M+1).

[0118] Then, normalization is performed based on the mask representation vector and the multi-category classifier to predict the conditional probability that the restored code belongs to the original category.

[0119] In some implementations, the N*(M+1) classification results can be added together for normalization, and the probability of belonging to the original instruction can be predicted based on the normalized result. If the normalized result is smaller, it means that the probability that the restored code belongs to the original code is higher, that is, L MLM The value of is close to 0, and the mask representation vector is consistent with the original instruction. If the normalized result is large, it means that the probability that the restored code belongs to the original code is small, that is, L MLM The larger the value of , the greater the difference between the mask representation vector and the original instruction.

[0120] Calculate the first loss function value based on the normalized result. The first loss function L MLM The calculation of is as follows:

[0121]

[0122] In formula (1), x \m(x) The category of the masked encoding, is the category of the restored code, is the conditional probability that the restored code belongs to the original category.

[0123] Based on the first loss function value, iteratively train the parameters of the BBI-BERT pre-trained model until L MLM convergence.

[0124] For the RID task, the task description is: given an encoding vector of a basic instruction block, randomly replace an encoding in the instruction unit, and predict whether there is a random replacement.

[0125] Taking operator replacement as an example, when implementing this task, a certain instruction unit is randomly selected for prediction in the encoding vector of the basic instruction block with a certain probability, such as a probability of 50%. If the jth instruction unit is selected, it is processed as follows:

[0126] Replace the encoding of the operator with a probability of 10%, such as Mov Rcx r8 -> Sub Rcx r8.

[0127] The jth instruction unit after replacement is input into the BBI-BERT pre-training model 32 to generate a replacement representation vector. The replacement representation vector can be recorded as the fourth representation vector.

[0128] The fourth representation vector is input into a multi-category classifier. The multi-category classifier classifies the fourth representation vector with the encoded category as a classification target. The classification result includes the category of the replaced operator.

[0129] Then, based on the classification results, the probability of random replacement p(t|x, y) is predicted, where x is the category of the replaced operator and y is the category of the original operator. If the value of p(t|x, y) is large, the fourth representation vector is far away from the original representation vector, and there is random replacement; if the value of p(t|x, y) is small, the replacement representation vector is far away from the original representation vector, and there is no random replacement.

[0130] Determine the second loss function L according to the probability of random replacement RID Values:

[0131] L RID = -∑log p(t|x,y) (2)

[0132] Based on the second loss function value, the parameters of the BBI-BERT pre-trained model are trained until the second loss function value converges.

[0133] Based on the above principles of random masking task and random replacement task for basic instruction blocks, the self-supervised learning of BBI-BERT pre-training model includes the following steps:

[0134] S701, randomly mask the encoding vector of the basic instruction block.

[0135] In some embodiments, random masking includes performing random masking (OAOM) on a first set proportion of instructions in the encoded vector stream of the base instruction block.

[0136] For example, Mov Rcx r8 --> Mov Rcx [MASK]

[0137] S702, based on the encoding vector of the masked basic instruction block, the BBI-BERT pre-trained model 32 generates a third representation vector.

[0138] In some embodiments, the encoding vector of the masked basic instruction block can be input into the BBI-BERT pre-trained model 32 to output a third representation vector.

[0139] S703, performing normalization based on the third representation vector and the multi-category classifier, wherein the normalization result is a conditional probability that the restored code belongs to the original code.

[0140] In some embodiments, the third representation vector can be input into a multi-category classifier, the number of which is the same as the vector length of the basic block coding vector; the multi-category classifier classifies the third representation vector using the coded category as the classification target, and the classification result includes the masked coded category; a normalization calculation is performed based on the classification result, and the normalized result is the conditional probability that the restored code belongs to the original category.

[0141] S704, determining a first loss function according to the normalization result, wherein the first loss function L MLM As shown in formula (1):

[0142]

[0143] S705, if the value of the first loss function does not converge, adjust the parameters of the BBI-BERT pre-trained model 32, return to execute steps S701-S705, and iteratively train the parameters of the BBI-BERT pre-trained model until the value of the first loss function converges.

[0144] S706, randomly replace the encoding vector of the basic instruction block.

[0145] In some implementations, a second set ratio of instructions in the basic block encoding sequence may be randomly replaced (replaced instruction detection model, RID).

[0146] Exemplarily, Mov Rcx r8 -> Sub Rcx r8.

[0147] S707, according to the encoding vector of the replaced basic instruction block, the BBI-BERT pre-trained model generates a fourth representation vector.

[0148] In some embodiments, the replaced basic block encoding sequence can be input into the BBI-BERT pre-trained model 32 to output a fourth representation vector.

[0149] S708: Perform normalization based on the fourth representation vector and the multi-category classifier, where the normalization result is a probability of random replacement.

[0150] In some embodiments, the fourth characterization vector can be input into a multi-category classifier, which uses the encoded category as a classification target to classify the fourth characterization vector, and the classification result includes the category of the replaced operator; a normalization calculation is performed based on the classification result, and the normalization result is the probability of the existence of random replacement.

[0151] S709, determining the value of a second loss function according to the probability of random replacement, wherein the second loss function L RID As shown in formula (2):

[0152] L RID = -∑log p(t|x,y) (2)

[0153] S710, if the value of the second loss function does not converge, adjust the parameters of the BBI-BERT pre-trained model 32, return to execute steps S706-710, and iteratively train the parameters of the BBI-BERT pre-trained model until the value of the second loss function converges.

[0154] The performance prediction method based on machine instructions provided in the embodiment of the present application constructs the smallest instruction unit for instruction sequence learning based on BB, and utilizes the attention structure of Transformer-Encoder and the prediction method of random mask and random replacement to realize self-supervised learning of BB.

[0155] Therefore, for a given encoding vector of a basic instruction block, the embodiment of the present application randomly replaces a code in the instruction unit, predicts whether there is a random replacement, iteratively trains the parameters of the BBI-BERT pre-trained model based on random replacement and self-supervised learning of multiple classifiers, and optimizes the BBI-BERT pre-trained model.

[0156] Next, we will introduce contrastive learning training for very long (tens of millions+) instruction sequences.

[0157] For very long instruction sequences, the BBI-BERT pre-trained model performs self-supervised learning based on the encoding vectors and relationship matrices of multiple basic instruction blocks in the very long instruction sequence to generate a second representation vector. This application determines the basic block jump graph based on the second representation vector, puts the first representation vector of each basic instruction block into the basic block jump graph as node information, and then performs comparative learning by randomly replacing the context of the node to obtain the structural information of the jump graph, thereby realizing the vectorized representation of the very long sequence. It can be processed as follows:

[0158] Firstly, multiple basic instruction blocks in the super-long instruction sequence and the jump relationship between the basic instruction blocks are determined.

[0159] Exemplarily, the multiple basic instruction blocks include a first basic instruction block, a second basic instruction block, a third basic instruction block, ..., a Kth basic instruction block.

[0160] The relationship matrix can be determined according to the jump relationship between multiple basic instruction blocks. For example, the next basic instruction block with a jump relationship can be determined according to the jump instruction control word "jbe" and the corresponding operand in the last sentence of each basic instruction block, and the relationship matrix can be obtained according to the jump relationship between multiple basic instruction blocks.

[0161] Then, a basic block jump graph is generated according to multiple basic instruction block sequences and a relationship matrix.

[0162] like Figure 8a As shown, the basic block jump graph includes multiple nodes and edges connecting the multiple nodes, wherein each node is a representation vector of a basic instruction block, and the connecting edges between each node conform to the jump relationship.

[0163] like Figure 8b As shown in Figure 1, a node is randomly selected in the basic block jump graph for replacement to obtain positive samples and negative samples.

[0164] Among them, the original node is replaced by a representation vector that is the same as or similar to the original node to obtain a positive sample of the jump graph, and the original node is replaced by a representation vector that is different from or dissimilar to the original node to obtain a negative sample of the jump graph.

[0165] Next, based on the similarity values ​​calculated for the positive and negative samples, the similarity s(x, y′) is calculated as follows:

[0166] s(x, y′)=||xy′||2 (3)

[0167] In formula (3), x is the characterization vector of the original node, and y′ is the characterization vector of the node with or without replacement.

[0168] It should be understood that the value of s(x, y′) is used to determine whether the representation vector of the replaced node conforms to the jump relationship between the representation vector of the original node and the context node in the jump graph. The larger the value of s(x, y'), the less the replaced representation vector conforms to the jump relationship in the jump graph; if the value of s(x, y') is extremely low, close to 0, the replaced representation vector conforms to the context relationship in the jump graph.

[0169] Determine the third loss function value L according to the similarity value cn , the third loss function value L cn The goal is to make the features of samples of the same type closer to each other and the features of samples of different types farther away from each other by comparing positive and negative samples.

[0170] The third loss function value L cn The calculation formula is:

[0171]

[0172] In formula (4), y + is the representation vector that is the same / similar to the original node in the positive sample; y - It is the representation vector that is different / dissimilar to the original node replaced in the negative sample.

[0173] The value of the total loss function is determined based on the values ​​of the first loss function, the second loss function, and the third loss function:

[0174] L=L MLM +L RID +L cn

[0175] like Figure 8c As shown, according to the value of the total loss function L, determine whether it converges. If not, adjust the parameters of the BBI-BERT pre-trained model 32, and iteratively train the parameters of the BBI-BERT pre-trained model until the value of the total loss function converges.

[0176] Based on the principle of the above contrastive learning task for very long instruction sequences, the self-supervised learning of the BBI-BERT pre-training model also includes the following steps:

[0177] S801, determining a basic block jump graph according to the second characterization vector; each node in the basic block jump graph corresponds to the first characterization vector of each basic instruction block; and the connection between each node conforms to the jump relationship.

[0178] S802, randomly replace a node in the basic block jump graph to determine positive samples and negative samples.

[0179] In some implementations, a representation vector identical or similar to the original node may be used to replace the original node to obtain a positive sample, and a representation vector different or dissimilar to the original node may be used to replace the original node to obtain a negative sample.

[0180] In some implementations, the original jump graph may be used as a positive sample, and the jump graph with replaced nodes may be used as a negative sample.

[0181] S803, predicting whether the replaced node conforms to the jump relationship in the basic block jump graph through comparative learning of positive samples and negative samples.

[0182] In some implementations, similarity can be calculated based on positive samples and negative samples; the value of the similarity is used to determine whether the replaced node conforms to the jump relationship between the upper and lower nodes in the jump graph. The value of the similarity is as shown in formula (3):

[0183] s(x, y′)=||xy′||2

[0184] The larger the s(x, y') value is, the less the replaced representation vector conforms to the jump relationship in the jump graph; if the s(x, y') value is extremely low, close to 0, the replaced representation vector conforms to the context relationship in the jump graph.

[0185] S804: Determine a third loss function value according to the prediction result. cn The calculation of is as follows:

[0186]

[0187] The third loss function value L cn The smaller the value, the positive and negative samples are of the same type, and their features are close to each other. The third loss function value L cn The larger the value of is, the positive and negative samples are different types of samples, and their features are far away from each other.

[0188] S805, determining the value of the total loss function according to the value of the first loss function, the value of the second loss function, and the value of the third loss function:

[0189] L=L MLM +L RID +L cn

[0190] S806, judging whether it converges according to the value of the total loss function, if not, adjusting the parameters of the BBI-BERT pre-trained model 32, returning to execute steps S801-S806, iteratively training the parameters of the BBI-BERT pre-trained model until the value of the total loss function converges.

[0191] Therefore, the embodiment of the present application puts the representation vector learned from the BB instruction into the jump graph of the ultra-long instruction sequence as node information, and then obtains the structural information of the jump graph by randomly replacing the context of the node for comparative learning, optimizes the BBI-BERT pre-training model, and realizes the vectorized representation of the ultra-long sequence, which can fully express the semantic information and jump information of the ultra-long instruction sequence with high accuracy.

[0192] Example 1

[0193] Fig. 9 A system diagram of a performance prediction method based on machine instructions is provided for Example 1 of the present application. Fig. 9 As shown, in this application scenario, the input basic instruction block is encoded, and the BBI-BERT method is used to realize the vectorized representation of the instruction block encoding. Finally, a neural network layer is connected to predict the CPU performance under different programs.

[0194] It is understood that the method can be executed by any instance, device, platform, or device cluster that has computing and processing capabilities. In addition, some or all steps of the performance prediction method based on machine instructions can refer to the aforementioned Figure 3-Figure 7 Related description in .

[0195] Fig.10 A flowchart of a performance prediction method based on machine instructions is provided for Example 1 of the present application. Fig.10 As shown, the following steps are included:

[0196] S901, determining a basic instruction block according to an input test case; the basic instruction block includes one or more instructions; the basic instruction block has a unique input and output.

[0197] In some embodiments, a test case can be obtained according to the user's input, and the test case is a test instruction set written in ARM language, and the machine language includes assembly language, ARM language or X86 language, etc. One or more basic instruction blocks to be tested are determined according to the test case. Among them, the basic block (Basic Block) instruction includes one or more instructions written in machine language, which has the characteristics of unique input and output, and no jump in the middle. The basic instruction block can be a basic instruction block of assembly, a basic instruction block of ARM or a basic instruction block of X86.

[0198] S902, determining multiple key characters in each instruction, where the multiple key characters include an operator and an operand.

[0199] In some implementations, each test instruction in the basic instruction block is sliced ​​to extract key characters therein, where the key characters include an operator, a source operand, and a destination operand.

[0200] S903, for instructions written in different machine languages, multiple key characters in each instruction are arranged in the same order, and encoded and processed to obtain the encoding vector of the basic instruction block.

[0201] In some implementations, S903 may be implemented by the following steps:

[0202] S9031, arrange the key characters in a unified grammatical structure to obtain an instruction unit corresponding to each test instruction code.

[0203] S9032, OneHot encode each key character in each instruction unit to obtain a one-hot encoding sequence.

[0204] S9033, concatenate each one-hot encoding sequence according to the control flow of the basic instruction block to obtain the encoding vector of the basic block.

[0205] In some embodiments, the concatenation process includes embedding a first hint at the beginning of the encoding vector of the basic instruction block and embedding a second hint at the end of each instruction.

[0206] S904, performing self-supervised learning on the BBI-BERT pre-trained model according to the encoding vector of the basic instruction block to generate a first representation vector h block .

[0207] The BBI-BERT pre-trained model 32 is composed of multiple layers of Transformer encoders. Each Transformer encoder consists of multiple multi-head attention mechanisms + feed-forward neural networks (FFN).

[0208] In the BBI-BERT pre-training model, a multi-head attention mechanism is used to process each key character in the instruction unit, and the intermediate value of the representation vector corresponding to each key character is output. The intermediate value of the representation vector can represent the upper and lower adjacent coding information; the feed-forward network (FFN) receives the input of the intermediate value of the representation vector, performs certain transformations and feature extraction on the intermediate value of the representation vector, and outputs the predicted first representation vector.

[0209] The parameter training of the BBI-BERT pre-trained model 32 adopts self-supervised learning. The specific steps can refer to steps S701-S712, which will not be repeated here.

[0210] S905: Determine hardware configuration parameters.

[0211] In some implementations, configuration parameters may be obtained according to a stored hardware configuration parameter file; the hardware includes chips, CPU, memory, hard disk, etc. The configuration parameters include parameter indicators such as chips, CPU, memory, hard disk, etc.

[0212] S906: predict the performance index of the CPU chip according to the first characterization vector and the hardware configuration parameters.

[0213] In some implementations, the performance index of the CPU chip includes an IPC performance index, which can be obtained by combining the hardware configuration parameters and the first characterization vector h block The mixed representation vector is obtained by concatenation or fusion; the concatenated or fused mixed representation vector can be input into a performance prediction network to predict the IPC performance index of the CPU, and the performance prediction network can be an MLP neural network.

[0214] In some implementations, the hardware configuration parameters may be updated, and the performance indicators of the CPU chip may be predicted based on the first characterization vector and the updated hardware configuration parameters.

[0215] In some embodiments, the hardware configuration parameters and the first representation vector are concatenated or fused, and the parameters of the BBI-BERT pre-trained model are transferred and trained based on the concatenated or fused representation vector.

[0216] The machine instruction-based performance prediction method provided in Example 1 of the present application uniformly encodes x86 / ARM instructions and uses the BBI-BERT pre-trained model to vectorize the application. When performing vectorization representation and cross-platform recognition encoding of the application, there is no need to embed hardware architecture information.

[0217] The machine instruction-based performance prediction method provided in the embodiment of the present application, the BBI-BERT pre-training model can optimize the modeling of the machine instruction sequence of the target task and obtain an optimized representation vector.

[0218] In the field of chip design, the method of the embodiment of the present application can provide better characterization accuracy and generalization capability, provide development tools for automated chip design, and accelerate the chip design process.

[0219] Example 2

[0220] Embodiment 2 of the present application provides an application scenario of a performance prediction method based on machine instructions. The application scenario is an automatic design and development scenario of a CPU chip. During the design and development process of the CPU chip, sufficient simulation and verification are required, and the time overhead of the test case used for simulation is very huge. Embodiment 2 of the present application provides a performance prediction method based on machine instructions, combining the characterization vector of the test case with the hardware architecture parameters to achieve the prediction of the CPU performance IPC.

[0221] In Example 2 of the present application, the test case is a standard benchmark, and each case is an ultra-long instruction sequence. The BBI-BERT pre-training model is used to realize vectorized representation of the benchmark ultra-long instruction sequence, thereby improving the prediction accuracy.

[0222] It is understood that the method can be executed by any instance, device, platform, or device cluster that has computing and processing capabilities. In addition, some or all steps of the performance prediction method based on machine instructions can refer to the aforementioned Figure 3-Figure 7 Related description in .

[0223] Fig.11 A flowchart of a performance prediction method based on machine instructions is provided for Example 2 of the present application. Fig.11 As shown, the following steps are included:

[0224] S1001, for an ultra-long instruction sequence of a benchmark, determine a plurality of basic instruction blocks that need to be tested, and determine encoding vectors of the plurality of basic instruction blocks according to the plurality of basic instruction blocks.

[0225] The generation of the encoding vector of each basic instruction block may refer to part or all of the implementation methods of steps S401-S405.

[0226] S1002, determining a relationship matrix according to the jump relationship between multiple basic instruction blocks.

[0227] Exemplarily, the next basic instruction block with a jump relationship can be determined based on the jump instruction control word jbe and the corresponding operand in the last sentence of each basic instruction block, and the relationship matrix can be obtained based on the jump relationship between multiple basic instruction blocks.

[0228] S1003, based on the encoding vectors and relationship matrices of multiple basic instruction blocks, perform self-supervised learning on the BBI-BERT pre-trained model to generate a second representation vector; the second representation vector includes the first representation vector of each basic instruction block, and the connection between the first representation vectors of each basic instruction block conforms to the jump relationship.

[0229] Among them, the self-supervised learning of the BBI-BERT pre-training model can refer to steps S801-S806, which will not be repeated here.

[0230] S1004, obtaining hardware configuration parameters according to the stored hardware configuration parameter file; the hardware includes chips, CPU, memory, hard disk, etc.; the configuration parameters include parameter indicators of the performance of chips, CPU, memory, hard disk, etc.

[0231] S1005, concatenating the hardware configuration parameters and the second characterization vector to obtain a mixed feature vector.

[0232] In some achievable implementations, the configuration parameters can be concatenated at the end of the second representation vector, with the dimension being the sum of the two, and a prompt at the end <cls>, to characterize the extremely long instruction sequence.

[0233] In some feasible implementations, the configuration parameters and the second characterization vector are fused to obtain a mixed feature vector. The fusion method may be to add multiple vectors without changing the dimension.

[0234] S906: predict the performance index of the CPU chip according to the second characterization vector and the hardware configuration parameters.

[0235] In some implementations, the hardware configuration parameters and the second characterization vector may be concatenated or fused to obtain a hybrid characterization vector; the concatenated or fused hybrid characterization vector may be input into a performance prediction network to predict the IPC performance index of the CPU; the performance prediction network may be an LMP neural network.

[0236] In some implementations, the hardware configuration parameters may be updated, and the performance indicators of the CPU chip may be predicted based on the second characterization vector and the updated hardware configuration parameters.

[0237] In some embodiments, the hardware configuration parameters and the second representation vector are concatenated or fused, and the parameters of the BBI-BERT pre-trained model are transferred and trained based on the concatenated or fused representation vector.

[0238] The performance prediction method based on machine instructions provided in Example 2 of the present application uses the BBI-BERT pre-trained model to generate a representation vector of an ultra-long instruction sequence, and performs CPU performance prediction based on the representation vector. It can perform CPU performance prediction on processors with different configuration parameters under the same instruction set, and test cases can be reused by simply fine-tuning the configuration parameters.

[0239] The performance prediction method based on machine instructions of Example 2 of the present application is adopted to combine the representation vector generated by the BBI-BERT pre-trained model with the parameter configuration of the hardware. The transformer parameters of the BBI-BERT pre-trained model can be fine-tuned under different configuration parameters of the processor to achieve transfer learning. Through iterative optimization, downstream performance prediction tasks are supported, the prediction accuracy is improved, and the simulation and verification overhead in the design and development process of the CPU chip is reduced.

[0240] The method of the embodiment of the present application realizes vectorized feature extraction of the instruction sequence of the application based on the BBI-BERT model, and optimizes the test target through the prediction layer neural network, and the test target includes target time and target power consumption. The embodiment of the present application supports the generalization of multiple tasks (supporting multiple downstream tasks).

[0241] Fig.12 The following is a schematic diagram of the verification results of this application under three Intel chips: Ivy Bridge, Haswell and Skylake. Fig.12 As shown, for the same instruction set sequence, the embodiment of the present application provides a performance prediction method based on machine instructions, using BBI-BERT-M and BBI-BERT-MR pre-trained models to generate representation vectors. In the two chip scenarios of Ivy Bridge and Skylake, the prediction accuracy and performance are better than Ithemal, IACA, and Iivm-mca solutions. Among them, BBI-BERT-M is a pre-trained model obtained by random mask self-supervised learning, and BBI-BERT-MR is a pre-trained model obtained by random mask and random replacement self-supervised learning.

[0242] It is understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0243] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0244] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid state disk (SSD)), etc.

[0245] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.< / cls> < / sep> < / cls> < / const>

Claims

1. A performance prediction method based on machine instructions, characterized in that: The method comprises: Determine a basic instruction block according to an input test case; the basic instruction block includes one or more instructions; the basic instruction block has a unique input and output; Determine a plurality of key characters in each of the instructions, wherein the plurality of key characters include an operator and an operand; For instructions written in different machine languages, multiple key characters in each instruction are arranged in the same order, and encoded and processed to obtain an encoding vector of the basic instruction block; the machine language includes one of the following categories: assembly language, ARM language or X86 language; According to the encoding vector of the basic instruction block, self-supervised learning is performed on the BBI-BERT pre-trained model to generate a first representation vector; Determine hardware configuration parameters, and predict performance indicators of the CPU chip based on the first characterization vector and the hardware configuration parameters.

2. The method according to claim 1, characterized in that: The test case includes a plurality of basic instruction blocks, and there is a jump relationship between the plurality of basic instruction blocks. The method further includes: Determine encoding vectors of the multiple basic instruction blocks according to the multiple basic instruction blocks; Determine a relationship matrix according to the jump relationship between the multiple basic instruction blocks; According to the encoding vectors of the multiple basic instruction blocks and the relationship matrix, the BBI-BERT pre-trained model is self-supervised learning to generate a second representation vector; the second representation vector includes the first representation vector of each basic instruction block, and the connection between the first representation vectors of each basic instruction block conforms to the jump relationship.

3. The method according to claim 1 or 2, characterized in that: The BBI-BERT pre-training model includes multiple Transformer encoders, which perform multi-attention mechanism processing on the encoding vector of the basic instruction block, and then transform and extract features through a feedforward neural network to output the first representation vector of the basic instruction block.

4. The method according to any one of claims 1 to 3, characterized in that: The self-supervised learning of the BBI-BERT pre-trained model, including random mask learning for the basic instruction block, includes: Performing random masking on the codes in the code vector of the basic instruction block; The BBI-BERT pre-trained model generates a third representation vector according to the encoding vector of the masked basic instruction block; Normalizing based on the third representation vector and the multi-category classifier, the normalization result being a conditional probability that the restored code belongs to the original code; Determine the value of the first loss function according to the normalization result, and iteratively train the parameters of the BBI-BERT pre-trained model until the value of the first loss function converges.

5. The method according to any one of claims 1 to 4, characterized in that: The self-supervised learning of the BBI-BERT pre-trained model also includes random replacement learning for the basic instruction block, including: Randomly replacing the codes in the code vector of the basic instruction block; According to the encoding vector of the replaced basic instruction block, the BBI-BERT pre-trained model generates a fourth representation vector; Normalizing based on the fourth representation vector and the multi-category classifier, wherein the normalization result is a probability of random replacement; Determine the value of the second loss function according to the probability of the random replacement, and iteratively train the parameters of the BBI-BERT pre-trained model until the value of the second loss function converges.

6. The method according to any one of claims 2 to 5, characterized in that: The self-supervised learning of the BBI-BERT pre-trained model also includes contrastive learning, including: Determine a basic block jump graph according to the second characterization vector; each node in the basic block jump graph corresponds to the first characterization vector of each basic instruction block; and the connection between each node conforms to the jump relationship; Randomly replace a node in the basic block jump graph to determine positive samples and negative samples; Predicting whether the replaced node conforms to the jump relationship in the basic block jump graph by comparing the positive samples with the negative samples; Determine the third loss function value according to the prediction result; The value of the total loss function is calculated according to the first loss function, the second loss function and the third loss function, and the parameters of the BBI-BERT pre-trained model are iteratively trained until the value of the total loss function converges.

7. The method according to claims 1-6, characterized in that: The self-supervised learning of the BBI-BERT pre-trained model includes: The hardware configuration parameters are spliced ​​or fused with the first representation vector or the second representation vector, and the parameters of the BBI-BERT pre-trained model are migrated and trained according to the spliced ​​or fused representation vector.

8. The method according to any one of claims 2 to 7, characterized in that: The step of arranging the multiple key characters in each instruction in the same order, encoding and processing the key characters includes: The operators and operands in each instruction are arranged in the same order, and OneHot encoding and concatenation processing are performed.

9. The method according to any one of claim 10, characterized in that: The splicing process includes embedding a first hint at the beginning of the encoding vector of the basic instruction block and embedding a second hint at the end of each instruction.

10. The method according to claims 1-9, characterized in that The hardware configuration parameters include parameters of a processor, a CPU, a memory, and a hard disk. The determining of the hardware configuration parameters and predicting the performance index of the CPU chip according to the first characterization vector and the hardware configuration parameters include: The hardware configuration parameters are updated, and the performance indicators of the CPU chip are predicted according to the first characterization vector and the updated hardware configuration parameters.

11. The method according to claims 2-8, characterized in that: The hardware configuration parameters include parameters of a processor, a CPU, a memory, and a hard disk. The method further includes: A performance indicator of the CPU is predicted according to the second characterization vector and the hardware configuration parameter.

12. A system for generating a representation vector for a machine instruction, characterized in that: The system comprises: An instruction processing module is used to determine a basic instruction block according to an input test case; the basic instruction block includes one or more instructions; the basic instruction block has a unique input and output; determine multiple key characters in each instruction, the multiple key characters include an operator and an operation object; for instructions written in different machine languages, arrange the multiple key characters in each instruction in the same order, encode and process them, and obtain an encoding vector of the basic instruction block; the machine language includes one of the following categories: assembly language, ARM language or X86 language; and A BBI-BERT pre-trained model, used for performing self-supervised learning according to the encoding vector of the basic instruction block to generate a first representation vector; the BBI-BERT pre-trained model includes a plurality of Transformer encoders, each Transformer encoder includes a multi-attention mechanism and a feedforward neural network; Configuration parameter acquisition module, used to determine hardware configuration parameters; The performance prediction network is used to predict the performance index of the CPU according to the first characterization vector and the hardware configuration parameters.

13. The system according to claim 12, characterized in that The test case includes a plurality of basic instruction blocks, and there is a jump relationship between the plurality of basic instruction blocks. The device also includes: A super-long instruction encoding module, used to determine encoding vectors of the multiple basic instruction blocks according to the multiple basic instruction blocks; and determine a relationship matrix according to the jump relationship between the multiple basic instruction blocks; The BBI-BERT pre-trained model is also used to perform self-supervised learning based on the encoding vectors of the multiple basic instruction blocks and the relationship matrix to generate a second representation vector; the second representation vector includes the first representation vector of each basic instruction block, and the connection between the first representation vectors of each basic instruction block conforms to the jump relationship.

14. The system according to claim 12, characterized in that The BBI-BERT pre-training model includes multiple Transformer encoders, which perform multi-attention mechanism processing on the encoding vector of the basic instruction block, and then transform and extract features through a feedforward neural network to output the first representation vector of the basic instruction block.

15. A computing device, characterized in that: include: at least one memory for storing a program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 11.

16. A computer storage medium, wherein instructions are stored in the computer storage medium, and when the instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 11.

Images