Model program compiling method, electronic equipment, program product and medium

Abstract

The invention discloses a model program compiling method, electronic equipment, a program product and a medium, relates to the technical field of artificial intelligence, and can set a corresponding block-level instruction for a model unit in a neural network model, and in the process of compiling the neural network model from a source program to an executable program, the executable program can be executed. The block-level instruction can be used for recording the unit type of the model unit, the memory position where the operation data input into the model unit is located and the memory position where the operation result output by the model unit is stored, so that the number of instructions required for compiling the model unit can be reduced, and the operation efficiency is improved. And meanwhile, the processor is helped to know the calculation operation, the calculation data storage position and the calculation result storage position corresponding to the model unit at one time, so that the size and the complexity of an executable program can be reduced, and the processing efficiency is also helped to be improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model program compilation method, an electronic device, a program product and a medium. Background Art

[0002] As the functions of neural network models continue to improve, their operating complexity is also increasing. Therefore, how to reduce the operating complexity of neural network models is an important part of improving the operating performance of the model.

[0003] In the related art, the neural network model is generally compiled using a conventional instruction set. However, the conventional instruction set is not set in combination with the characteristics of the neural network model, which easily leads to a larger size, higher complexity and lower execution efficiency of the compiled executable program. Summary of the invention

[0004] The present application provides a model program compilation method, an electronic device, a program product and a medium, so as to at least solve the technical problem in the related art that using conventional instruction sets to compile neural network models easily leads to large executable program size and low execution efficiency.

[0005] In order to solve the above technical problems, the present application may provide a model program compilation method, comprising:

[0006] In the process of compiling the neural network model from a source program to an executable program, a block-level instruction corresponding to a program block in the source program is matched in a preset instruction set; wherein the program block corresponds to a model unit pre-divided in the neural network model; the block-level instruction is a machine instruction corresponding to the model unit, including a unit type, a calculation data location and a calculation result location corresponding to the model unit, wherein the calculation data location is a memory location corresponding to the calculation data input to the model unit, and the calculation result location is a memory location corresponding to the calculation result output by the model unit;

[0007] Allocate operation data location and operation result location for the program block and fill them into the block-level instructions;

[0008] Add block-level instructions to complete the padding to the executable program.

[0009] Optionally, the block-level instruction includes a unit type, an operation data address, an operation result address, an operation data scale and a bias identifier, wherein the operation data address is the memory address corresponding to the operation data, the operation result address is the memory address corresponding to the operation result, the operation data scale is the operation data size corresponding to the operation data, and the bias identifier is the usage information corresponding to the bias data in the operation data.

[0010] Optionally, after adding the filled block-level instructions to the executable program, the method further includes:

[0011] When executing a block-level instruction, the unit type in the block-level instruction is read to determine the combined operation corresponding to the model unit according to the unit type; wherein the combined operation includes the operation and operation sequence corresponding to the model unit;

[0012] Assigning values ​​to the operation data register according to the operation data address and the offset identifier, assigning values ​​to the operation result register according to the operation result address, and assigning values ​​to the operation data scale register according to the operation data scale;

[0013] The operation data is extracted from the memory according to the operation data register and the operation data scale register, the combined operation operation is performed using the operation data to obtain the operation result, and the operation result is written into the memory according to the operation result register and the operation data scale register.

[0014] Optionally, the combined operation has been set in the processor based on hardware code.

[0015] Optionally, the operation data scale is composed of a plurality of bits, and the bits correspond to the operation data;

[0016] Extract operation data from the memory according to the operation data register and the operation data size register, including:

[0017] Determine the operation data size of each operation data according to each bit in the operation data size register;

[0018] Extracting operation data from the memory according to the operation data register corresponding to each operation data and the operation data size;

[0019] Write the operation result into the memory according to the operation result register and the operation data size register, including:

[0020] Determine the size of the operation result according to each bit in the operation data size register;

[0021] Write the operation result into the memory according to the operation result register and the operation result size.

[0022] Optionally, assigning a value to the operation data register according to the operation data address and the offset identifier includes:

[0023] Determine whether the bias flag indicates the use of bias data;

[0024] If the bias mark indicates the use of bias data, the operation data register corresponding to the bias data is assigned a value using the operation data address of the bias data;

[0025] If the bias flag indicates that the bias data is not used, no value is assigned to the operation data register corresponding to the bias data.

[0026] Optionally, after matching the block-level instruction corresponding to the program block in the source program in the preset instruction set, the method further includes:

[0027] If there is no block-level instruction corresponding to the program block in the preset instruction set, then determining the operation type corresponding to the code line in the program block; the operation type includes a matrix operation type and a scalar operation type;

[0028] If the operation type of the code line is a matrix operation type, a memory location is allocated for the matrix operation data and the matrix operation result corresponding to the code line, and the memory location and the matrix operation operation information of the code line are filled into the operator-level instruction; wherein the operator-level instruction is a machine instruction corresponding to the matrix operation type, including matrix operation operation information, matrix operation data location and matrix operation result location, the matrix operation data location is the memory location corresponding to the matrix operation data, and the matrix operation result location is the memory location corresponding to the matrix operation result;

[0029] If the operation type of the code line is a scalar operation type, a memory location is allocated for the scalar operation data and the scalar operation result corresponding to the code line, and the memory location and the scalar operation information of the code line are filled into the basic instruction; wherein the basic instruction is a machine instruction corresponding to the scalar operation type, including scalar operation operation information, scalar operation data location and scalar operation result location, the scalar operation data location is the memory location corresponding to the scalar operation data, and the scalar operation result location is the memory location corresponding to the scalar operation result;

[0030] Add operator-level instructions or basic instructions to the executable program.

[0031] Optionally, the operator-level instruction includes matrix operation operation information, matrix operation data address, matrix operation result address and matrix operation data scale, the matrix operation data address is the memory address corresponding to the matrix operation data, the matrix operation result address is the memory address corresponding to the matrix operation result, and the matrix operation data scale is the matrix operation data size corresponding to the matrix operation data.

[0032] Optionally, after adding the filled block-level instructions to the executable program, the method further includes:

[0033] When executing an operator-level instruction, reading matrix operation information in the operator-level instruction to determine the matrix operation to be executed;

[0034] Assign values ​​to the operation data register according to the matrix operation data address, assign values ​​to the operation result register according to the matrix operation result address, and assign values ​​to the operation data scale register according to the matrix operation data scale;

[0035] Matrix operation data is extracted from the memory according to the operation data register and the operation data scale register, matrix operation operation is performed using the matrix operation data to obtain the matrix operation result, and the matrix operation result is written into the memory according to the operation result register and the operation data scale register.

[0036] Optionally, the basic instruction includes scalar operation information, scalar operation data address, scalar operation result address and scalar operation data identifier, the scalar operation data address is the memory address corresponding to the scalar operation data, the scalar operation result address is the memory address corresponding to the scalar operation result, and the scalar operation data identifier is the number of scalar operation data involved in the scalar operation operation.

[0037] Optionally, after adding the filled block-level instructions to the executable program, the method further includes:

[0038] When executing a basic instruction, scalar operation information is read to determine the scalar operation to be performed;

[0039] Assigning values ​​to the operation data register according to the scalar operation data address and the scalar operation data identifier, and assigning values ​​to the operation result register according to the scalar operation result address;

[0040] The scalar operation data is extracted from the memory according to the operation data register, the scalar operation operation is performed using the scalar operation data to obtain the scalar operation result, and the scalar operation result is written into the memory according to the operation result register.

[0041] Optionally, assigning a value to an operation data register according to a scalar operation data address and a scalar operation data identifier includes:

[0042] Determine whether the number of scalar operation data indicated by the scalar operation data identifier is 1;

[0043] If yes, assigning a value to the operation data register of the first scalar operation data according to the scalar operation data address of the first scalar operation data;

[0044] If not, assign values ​​to the operation data register of the first scalar operation data and the operation data register of the second scalar operation data respectively according to the scalar operation data addresses of the first scalar operation data and the second scalar operation data.

[0045] Optionally, memory locations are allocated for matrix operation data corresponding to the code line, including:

[0046] Determine whether the matrix operation result of the previous code line is the matrix operation data of the current code line;

[0047] If yes, the memory location of the matrix operation result of the previous code line is used as the memory location of the matrix operation data of the current code line;

[0048] If not, allocate memory locations for the matrix operation data of the current code line;

[0049] Allocate memory locations for scalar operation data corresponding to the code line, including:

[0050] Determine whether the scalar operation result of the previous code line is the scalar operation data of the current code line;

[0051] If yes, the memory location of the scalar operation result of the previous code line is used as the memory location of the scalar operation data of the current code line;

[0052] If not, allocate memory locations for the scalar operation data for the current line of code;

[0053] Add operator-level instructions or basic instructions to the executable program, including:

[0054] The operator-level instructions or basic instructions corresponding to the code lines in the program block are organized into an instruction chain in the order of the code lines, and the instruction chain is added to the executable program.

[0055] The present application also provides an electronic device, including:

[0056] Memory for storing computer programs;

[0057] A processor is used to implement the above-mentioned model program compilation method when executing a computer program.

[0058] The present application also provides a computer program product, including a computer program or instructions, which implements the above-mentioned model program compilation method when the computer program or instructions are executed by a processor.

[0059] The present application also provides a non-volatile computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the above-mentioned model program compilation method is implemented.

[0060] Through the present application, since corresponding block-level instructions can be set for the pre-divided model units in the neural network model, and the block-level instructions are used to record the unit type of the model unit, the memory location where the operation data input to the model unit is located, and the memory location where the operation result output by the model unit is stored, when the neural network model is compiled from the source program into an executable program using the block-level instructions, a block-level instruction can be used to represent a complex model unit, which can reduce the number of instructions required to compile the model unit; at the same time, it helps the processor to understand the operation operations, operation data storage locations, and operation result storage locations corresponding to the model unit at one time. In this way, the size and complexity of the executable program can be reduced, and the operation efficiency can also be improved, which can avoid the defects of large executable program size and low execution efficiency that are easily caused by compiling the neural network model with conventional instruction sets. The present application also provides an electronic device, a program product, and a medium, which have the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0062] Figure 1 A flowchart of a model program compiling method provided in an embodiment of the present application;

[0063] Figure 2 A schematic diagram of a multi-layer perceptron block-level instruction provided in an embodiment of the present application;

[0064] Figure 3 A schematic diagram of an attention mechanism block-level instruction provided in an embodiment of the present application;

[0065] Figure 4 A schematic diagram of a binary variable operator-level instruction provided by an embodiment of the present invention;

[0066] Figure 5 A schematic diagram of a unary variable operator-level instruction provided by an embodiment of the present invention;

[0067] Figure 6 A schematic diagram of a basic instruction provided by an embodiment of the present invention;

[0068] Figure 7 A structural block diagram of a model program compiling device provided in an embodiment of the present application;

[0069] Figure 8 The present invention provides a structural block diagram of an electronic device. DETAILED DESCRIPTION

[0070] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0071] It should be noted that, in the description of this application, the terms "include", "comprise" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence.

[0072] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0073] In the related art, the neural network model is generally compiled using a conventional instruction set during the compilation process, such as using a reduced instruction set (RISC, Reduced Instruction Set Computer) to compile the neural network model. However, the machine instructions in the conventional instruction set contain a small number of fields, which are suitable for controlling the execution of small tasks, while the neural network model contains a large number of calculation operations. Therefore, when using a conventional instruction set to compile the executable program of the neural network model, it is easy to cause the executable program to be large, complex, and inefficient. For example, when dealing with complex tasks in the neural network model, the above-mentioned executable program will have an increased demand for cache and is prone to cache misses; for another example, the execution of the above-mentioned executable program requires more hardware resources and more optimization designs, such as adding proprietary instructions, instruction pipelines, etc.

[0074] In view of this, in order to solve the technical problem in the related art that using conventional instruction sets to compile neural network models easily leads to large executable program size and low execution efficiency, the present application can provide a model program compilation method, which can start from the instruction set perspective and set corresponding block-level instructions for the model units in the neural network model, and can use the block-level instructions to record the unit type of the model unit, the memory location of the operation data input to the model unit, and the memory location where the operation result output by the model unit is stored. This can not only reduce the number of instructions required to compile the model unit, but also help the processor to understand the operation operations, operation data storage locations and operation result storage locations corresponding to the model unit at one time, thereby reducing the size and complexity of the executable program and improving processing efficiency.

[0075] It should be noted that the present application does not limit the hardware device and hardware architecture for executing the present method, and can be set according to actual application requirements. For example, the present method can be executed by a personal computer, a server, etc.

[0076] The following is an introduction to the model program compilation method provided by this method. For easy understanding, please refer to Figure 1 , Figure 1 A flowchart of a model program compilation method provided in an embodiment of the present application. The method may include:

[0077] S101. In the process of compiling a neural network model from a source program to an executable program, block-level instructions corresponding to program blocks in the source program are matched in a preset instruction set; wherein the program blocks correspond to model units pre-divided in the neural network model; the block-level instructions are machine instructions corresponding to the model units, including the unit type, operation data location and operation result location corresponding to the model units, the operation data location is the memory location corresponding to the operation data input to the model unit, and the operation result location is the memory location corresponding to the operation result output by the model unit.

[0078] The following first introduces the relevant terms in this step:

[0079] Neural network model and model unit: Neural network model refers to a mathematical model based on artificial neurons. Common neural network models may include image processing models, language processing models, etc., among which image processing models are neural network models that perform image processing tasks such as image analysis and image generation; language processing models are neural network models that perform text processing tasks such as text analysis and text generation. Model units are pre-set and divided model structures, which are the basic components of neural network models, and neural network models can be constructed by connecting model units in series or in parallel. Common model units may include multilayer perceptrons (MLP), attention mechanism modules (such as self-attention mechanism modules, cross-attention mechanism modules), transformer modules, etc. It is worth noting that neural network models are usually composed of multiple identical model units, for example, language processing models are usually composed of multiple transformer modules.

[0080] Source program, executable program and compilation: Source program refers to an uncompiled text file written in a certain programming language specification. For example, a neural network model can be developed using Python. Executable program refers to the binary code generated after the source program is compiled by a compiler. Compilation refers to the translation of the human-oriented programming language in the source program into machine-oriented machine instructions in the executable program. Compilation requires the use of instruction sets.

[0081] Program block: In this embodiment, a program block in the source program corresponds to a model unit and contains all the codes of the model unit. As mentioned above, a neural network model can usually be composed of multiple identical model units, so the program block will appear repeatedly in the source program of the neural network model.

[0082] Preset instruction set and block-level instructions: In this embodiment, in order to reduce the compilation complexity of the neural network model, a preset instruction set including block-level instructions is specially provided. Among them, the block-level instruction is the machine instruction corresponding to the model unit, including the unit type, operation data location and operation result location corresponding to the model unit. The unit type is used to prompt the processor which operation operations need to be performed on the corresponding model unit; the operation data location is the memory location corresponding to the operation data input to the model unit, which is used to prompt the processor from which memory location to obtain the operation data; the operation result location is the memory location corresponding to the operation result output by the model unit, which is used to prompt the processor to which memory location to write the operation result. It should be noted that the operation data not only includes the input data of the model unit, but also includes the data involved in the operation inside the model unit (such as bias data). It should be noted that this embodiment does not limit the form of the operation data, for example, it can be a matrix or a scalar. It is worth noting that a model unit may include several operation operations, and these operation operations need to be executed sequentially. This embodiment can set the combination operation corresponding to the model unit (including the operation and operation order corresponding to the model unit) and the correspondence between the combination operation and the unit type in the processor based on the hardware code, so that the processor can directly determine the combination operation to be executed according to the unit type in the block-level instruction.

[0083] Based on the above-mentioned term introduction, it can be seen that the present application can set block-level instructions corresponding to the model unit in the neural network model, and use the block-level instructions to record the unit type of the model unit, the memory location where the operation data input to the model unit is located, and the memory location where the operation result output by the model unit is stored. In this way, in the process of compiling the neural network model from the source program to the executable program, the present application can match the block-level instructions corresponding to the program block in the source program in the preset instruction set to translate the program block into the block-level instruction. In this way, the present application can not only use one block-level instruction to represent a complex model unit, but also does not require several conventional instructions to represent a model unit, which can reduce the complexity of compilation; at the same time, the processor can understand the operation operation, operation data storage location and operation result storage location corresponding to the model unit at one time by parsing a block-level instruction. In this way, the volume and complexity of the executable program can be reduced, and it also helps to improve the computing efficiency.

[0084] It should be noted that the present embodiment does not limit the specific form of the block-level instruction, and can be set according to actual application requirements. In order to record the above-mentioned unit type, operation data location, operation result location, and at the same time be closer to the common operations in the model unit, the block-level instruction may include the unit type, operation data address, operation result address, operation data scale, and bias identification. Among them, the operation data address is the memory address corresponding to the operation data, the operation result address is the memory address corresponding to the operation result, and the operation data scale is the operation data size corresponding to the operation data. The memory location corresponding to the operation data can be determined based on the operation data address and the operation data scale. In addition, since the operations in the model unit are usually matrix operations and scalar operations, and the scale of the matrix operation result can be determined according to the scale of the matrix involved in the matrix operation, the memory location corresponding to the operation data can also be determined based on the operation result address and the operation data scale. The bias identification is the usage information corresponding to the bias data in the operation data, and the usage information includes using bias data and not using bias data.

[0085] Furthermore, block-level instructions can also be divided into two parts: instruction prefix and instruction number. The instruction prefix contains the above unit type, and the instruction number contains the above operation data address, operation result address, operation data size and offset identifier. For easy understanding, please refer to Figure 2 and Figure 3 . Figure 2 This is a schematic diagram of a multilayer perceptron block-level instruction provided in an embodiment of the present application, and the block-level instruction includes an instruction prefix and 8 instruction numbers. Among them, the instruction prefix includes the unit type of the multilayer perceptron; "registers 0 to 6" are the 7 instruction numbers that need to be assigned to registers 0 to 6, which are the matrix A address, matrix B1 address, bias C1 address, matrix B2 address, bias C2 address, result address and data size; "identifier" is the bias identifier. Figure 3 This is a schematic diagram of an attention mechanism block-level instruction provided in an embodiment of the present application, and the block-level instruction includes 6 instruction numbers. Among them, the instruction prefix includes the unit type of the attention mechanism; "registers 0~5" are the 6 instruction numbers that need to be assigned to registers 0~5, which are matrix Q address, matrix K address, scalar data, matrix V address, result address and data scale.

[0086] It should also be noted that this embodiment does not limit how to match the block-level instructions corresponding to the program block in the source program in the preset instruction set, and reference may be made to the relevant techniques of compilation. For example, a mapping relationship may be established between the code content of the program block and the block-level instructions, and the mapping relationship may be saved in the preset instruction set. In this way, when the program block is compiled, the corresponding block-level instructions may be matched according to the above mapping relationship and the code content of the program block.

[0087] S102, allocating operation data locations and operation result locations to the program block and filling them into block-level instructions.

[0088] In this step, after completing the block-level instruction matching, memory space can be allocated for the operation data and operation results in the program block, that is, the operation data location and the operation result location are allocated, and the allocation result is filled into the block-level instruction to complete the compilation of the program block.

[0089] It can be seen that since the present embodiment can perform memory allocation during the compilation process and can perform memory allocation in units of one program block (i.e. one model unit), the memory space can be reasonably allocated according to the model unit to make reasonable use of the memory and reduce the probability of memory miss events.

[0090] S103: Add the filled block-level instructions to the executable program.

[0091] In this step, the filled block-level instructions can be added to the executable program to complete the compilation of the neural network model. It is worth noting that, since the present embodiment can use a block-level instruction to represent a model unit, and the neural network model can usually be composed of multiple model units with the same structure, there are fewer block-level instructions in the executable program, which can effectively reduce the size and complexity of the executable program.

[0092] Based on the above embodiments, the present application can set corresponding block-level instructions for the pre-divided model units in the neural network model, and use the block-level instructions to record the unit type of the model unit, the memory location where the operation data input to the model unit is located, and the memory location where the operation result output by the model unit is stored. Therefore, when the neural network model is compiled from the source program to the executable program using the block-level instructions, a block-level instruction can be used to represent a complex model unit, which can reduce the number of instructions required to compile the model unit; at the same time, it helps the processor to understand the operation operation, operation data storage location and operation result storage location corresponding to the model unit at one time. In this way, the volume and complexity of the executable program can be reduced, and the operation efficiency can also be improved, which can avoid the defects of large executable program volume and low execution efficiency that are easily caused by compiling the neural network model with conventional instruction sets.

[0093] Based on the above embodiment, the specific process of the processor executing the block-level instruction is introduced below. In one possible case, after adding the filled block-level instruction to the executable program, the method may also include:

[0094] S201. When executing a block-level instruction, read the unit type in the block-level instruction to determine the combination operation corresponding to the model unit according to the unit type; wherein the combination operation includes the operation operation and operation sequence corresponding to the model unit.

[0095] In this step, when executing a block-level instruction, the processor may first read the unit type in the block-level instruction to determine the combined operation corresponding to the model unit according to the unit type, wherein the combined operation includes all the operations and operation sequences corresponding to the model unit.

[0096] It should be noted that this embodiment does not limit the specific combination operation, and can be set according to the specific model unit.

[0097] S202, assigning values ​​to the operation data register according to the operation data address and the offset identifier, assigning values ​​to the operation result register according to the operation result address, and assigning values ​​to the operation data scale register according to the operation data scale.

[0098] In this step, the processor needs to assign values ​​to the corresponding registers according to the operation data address, operation result address, operation data size and offset identifier in the block-level instruction, as follows:

[0099] First, the operation data register needs to be assigned according to the operation data address and bias identifier. Among them, the operation data can be divided into biased data and non-biased data according to the function. For biased data, before assigning a value, it is necessary to first determine whether the bias identifier indicates the use of biased data. If the bias identifier indicates the use of biased data, the register corresponding to the biased data can be assigned a value; if the bias identifier indicates that the biased data is not used, there is no need to assign a value to the register corresponding to the biased data. For non-biased data, the corresponding operation data address can be directly used to assign a register.

[0100] Based on this, assigning values ​​to the operation data register according to the operation data address and the offset identifier may include:

[0101] Step 11: Determine whether the bias flag indicates the use of bias data;

[0102] Step 12: If the bias identifier indicates that the bias data is used, the operation data register corresponding to the bias data is assigned a value using the operation data address of the bias data;

[0103] Step 13: If the bias flag indicates that the bias data is not used, then the calculation data register corresponding to the bias data is not assigned a value.

[0104] Secondly, the calculation result register needs to be assigned a value according to the calculation result address.

[0105] Finally, the operation data scale register needs to be assigned a value according to the operation data scale. It is worth pointing out that in order to use an operation data scale to record the scale of each operation data, the operation data scale can be composed of multiple bits, and the bit corresponds to the operation data, which is used to record the scale of the corresponding operation data. For example, in the block-level instructions of the multi-layer perceptron, the operation data scale can be divided into left, middle and right bits. The left bit stores the number of rows of matrix A, the middle bit stores the number of columns of matrix A (equal to the number of rows of matrix B1), and the right bit stores the number of columns of matrix B1. In this way, the data scale of bias C1, matrix B2, bias C2 and the result matrix can also be obtained through the data of this bit, thereby realizing that one register can obtain the data length information of all matrices.

[0106] It should be noted that the above operation data register, operation result register, and operation scale register are all general registers.

[0107] S203, extracting operation data from the memory according to the operation data register and the operation data scale register, performing a combined operation using the operation data to obtain an operation result, and writing the operation result into the memory according to the operation result register and the operation data scale register.

[0108] In this step, after completing the register assignment, the processor can extract the operation data from the memory according to the operation data register and the operation data scale register, use the operation data to perform combined operation operations to obtain the operation results, and write the operation results to the memory according to the operation result register and the operation data scale register, thereby efficiently completing the operation of the model unit.

[0109] It should be noted that since the scale of calculation data can be composed of multiple bits, and the bits correspond to the calculation data to record the scale of the corresponding calculation data, when extracting the calculation data and writing the calculation results, it is necessary to determine the size of each calculation data and the size of the calculation result according to the different bits in the calculation data scale.

[0110] Based on this, extracting operation data from the memory according to the operation data register and the operation data size register may include:

[0111] Step 21: determining the operation data size of each operation data according to each bit in the operation data size register;

[0112] Step 22: extract the operation data from the memory according to the operation data register corresponding to each operation data and the operation data size.

[0113] Write the operation result into the memory according to the operation result register and the operation data size register, which may include:

[0114] Step 31: Determine the size of the operation result according to each bit in the operation data size register;

[0115] Step 32: Write the operation result into the memory according to the operation result register and the operation result size.

[0116] The following uses two block-level instructions as examples to introduce the process of the processor executing block-level instructions.

[0117] 1. For a multi-layer perceptron, it usually includes two matrix multiplications and activation function calculations, which can be specifically expressed as:

[0118] ;

[0119] Where D represents the result of the operation, A represents matrix A, B1 represents matrix B1, C1 represents bias C1, B2 represents matrix B2, C2 represents bias C2, and gelu() represents the activation function. The block-level instruction form of the multilayer perceptron can be referred to Figure 2 . The fields of register 0, register 1, register 2, register 3, register 4, and register 5 indicate the addresses of matrix A, B1, offset C1, matrix B2, offset C2, and result matrix D respectively; the identification field indicates whether the offset data is included in the two matrix calculations; the data scale field can use multiple bits, divided into left, middle, and right bits, the left bit stores the number of rows of matrix A, the middle bit stores the number of columns of matrix A (equal to the number of rows of matrix B1), and the right bit stores the number of columns of matrix B1. In this way, the data scale of offset C1, matrix B2, offset C2, and result matrix can also be obtained through the data of this bit, thereby realizing that one register can obtain the data length information of all matrices. In hardware calculations, when the instruction is parsed to the block-level instruction, the calculations of matrix A multiplication with B1, the addition of the intermediate result matrix with offset C1, the calculation of the activation function on the result, the calculation of the multiplication of the intermediate result matrix with matrix B2, and the calculation of the addition of the intermediate result matrix with offset C2 will be calculated in sequence.

[0120] In a practical scenario, the calculation scale of the multilayer perceptron is the product of the matrix A of token_length×896 and the matrix B1 of 896×4864, and then the matrix result is activated. The activation function result is multiplied by the matrix of 4864×896 to obtain the result of the matrix scale of token_length×896. Among them, token_length represents the input length of the multilayer perceptron. When generating this block-level instruction, execute:

[0121] 1) The instruction prefix contains a bit indicating that the instruction is a multilayer perceptron block-level instruction;

[0122] 2) Give the starting addresses of matrices A and B1 and assign them to registers 0 and 1;

[0123] 3) At this time, there is no bias C1 and C2, and the assigned flag is 0, indicating that bias C does not participate in the calculation;

[0124] 4) This process performs activation function calculation;

[0125] 5) Give the starting address of matrix B2 and assign it to register 3;

[0126] 6) Give the storage address of the result and assign it to register 5 to store the result of the matrix product;

[0127] 7) The data size field is set so that the upper 16 bits store the value of token_length, the middle 16 bits store the value of 896, and the lower 32 bits store the value of 4864, and the final value is assigned to register 6.

[0128] The identification bit is 2 bits, which respectively indicate whether the product of matrix A and B1 and the product of activation function and B2 need to be added with bias matrices C1 and C2 to obtain the final result matrix; the advantage of such block-level instructions is that the field value can be configured once to assign values ​​to a series of related instructions, eliminating the trouble of assigning values ​​one by one. During hardware operation, the instruction parsing process can also be executed continuously according to the block-level instructions to avoid parsing instructions one by one, thereby increasing the actual execution time.

[0129] 2. For the attention mechanism calculation, the specific calculation formula is:

[0130] ;

[0131] Where D represents the result of the operation, Q, K, and V represent the matrix Q (query matrix), matrix K (key matrix), and matrix V (value matrix), respectively, and T represents transpose. Represents the dimension of matrix K. softmax() represents the softmax function (normalization function). The block-level instruction form of the attention mechanism can be referred to Figure 3 . Similar to the block-level instructions of the multi-layer perceptron, in actual use, the values ​​of registers 0, 1, 3, and 4 corresponding to the addresses of Q, K, V, and the result matrix are configured in sequence, and register 2 is assigned a value. , register 5 is assigned a data size value, and the instruction prefix contains a bit indicating that the instruction is an attention mechanism. Therefore, this series of instructions are included as a block-level instruction to achieve the purpose of continuous instruction generation, thereby reducing the instruction configuration work and instruction parsing work during hardware operation. In real hardware operation, when the instruction is parsed, the hardware will calculate the transpose of matrix K, the product of matrix Q and K transpose, multiply each element of the intermediate result matrix by scalar data, perform softmax calculation on each row of data, and then multiply it with matrix V according to the address, data size and other information in the instruction.

[0132] Based on the above embodiments, considering that the model units of the neural network model are replaced quickly, there may be a situation where block-level instructions cannot completely cover all model units in the neural network model. To this end, considering that most operations in the neural network model are matrix operations and scalar operations, operator-level instructions and basic instructions can also be set in the preset instruction set. Operator-level instructions can cover matrix operations, and basic instructions can cover scalar operations. In this way, when block-level instructions cannot completely cover model units, operator-level instructions and basic instructions can be used instead. Based on this, after matching the block-level instructions corresponding to the program blocks in the source program in the preset instruction set, the method can also include:

[0133] S301. If there is no block-level instruction corresponding to the program block in the preset instruction set, determine the operation type corresponding to the code line in the program block; the operation type includes a matrix operation type and a scalar operation type.

[0134] In this step, if there is no block-level instruction matching the program block in the preset instruction set, each code line in the program block needs to be compiled separately. As described above, this embodiment can also provide operator-level instructions corresponding to matrix operations and basic instructions corresponding to scalar operations, so before compiling a code line, it is first necessary to determine the type of operation corresponding to the code line.

[0135] It should be noted that this embodiment does not limit how to determine the type of operation corresponding to a code line. For example, it can be determined based on the content of the code line (such as parameter type, operation type).

[0136] S302. If the operation type of the code line is a matrix operation type, a memory location is allocated for the matrix operation data and the matrix operation result corresponding to the code line, and the memory location and the matrix operation operation information of the code line are filled into the operator-level instruction; wherein the operator-level instruction is a machine instruction corresponding to the matrix operation type, including matrix operation operation information, matrix operation data location and matrix operation result location, the matrix operation data location is the memory location corresponding to the matrix operation data, and the matrix operation result location is the memory location corresponding to the matrix operation result.

[0137] In this step, when it is determined that the operation type of the code line is a matrix operation type, a memory location can be allocated for the matrix operation data and matrix operation result corresponding to the code line, and the memory location and the matrix operation information of the code line are filled into the operator-level instruction, thereby completing the compilation of the code line. The following is an introduction to the operator-level instruction:

[0138] The operator-level instruction is a machine instruction corresponding to the matrix operation type, and the instruction may include matrix operation information, matrix operation data location, and matrix operation result location. The matrix operation information is the specific information of the matrix operation, which is used to prompt the processor which matrix operation operation needs to be performed. Common matrix operation operations include: normalization calculation, softmax, activation function calculation, matrix multiplication by vector, matrix addition to matrix, matrix addition to vector, matrix transposition, matrix left and right bisection, swapping positions and right inversion operation, vector summation, vector exp (exponential function operation on vector), vector activation function, etc. The matrix operation data location is the memory location corresponding to the matrix operation data, which is used to prompt the processor from which memory location to obtain the matrix operation data. The matrix operation result location is the memory location corresponding to the matrix operation result, which is used to prompt the processor to which memory location to write the matrix operation result. It is worth noting that in this embodiment, the matrix operation operation can be set in the processor based on the hardware code, so that the processor can directly determine the matrix operation operation to be performed according to the matrix operation operation information in the operator-level instruction.

[0139] It should be noted that the present embodiment does not limit the specific form of the operator-level instruction, and it can be set according to actual application requirements. For example, the operator-level instruction may include matrix operation information, matrix operation data address, matrix operation result address and matrix operation data scale, the matrix operation data address is the memory address corresponding to the matrix operation data, the matrix operation result address is the memory address corresponding to the matrix operation result, and the matrix operation data scale is the size of the matrix operation data corresponding to the matrix operation data.

[0140] Furthermore, since some matrix operations occur between matrices, such as matrix multiplication by vector, matrix addition, matrix addition to vector, and vector sum; some matrix operations are performed only on a single matrix, such as normalization calculation, softmax, activation function calculation, matrix transposition, matrix bisection and swapping positions and right inversion, vector exp, and vector activation function, this embodiment can provide two forms of operator-level instructions. For easy understanding, please refer to Figure 4 and Figure 5 . Figure 4A schematic diagram of a binary variable operator-level instruction provided in an embodiment of the present invention, the operator-level instruction includes an instruction prefix and 5 instruction numbers. Among them, "registers 0 to 3" are 4 instruction numbers that need to be assigned to registers 0 to 3, which are matrix / vector A address, matrix / vector V address, data size and result address respectively; "identifier" is matrix calculation operation information. Figure 5 A schematic diagram of a univariate operator-level instruction provided in an embodiment of the present invention, the operator-level instruction includes an instruction prefix and four instruction numbers. Among them, "registers 0~2" are three instruction numbers that need to be assigned to registers 0~2, which are matrix / vector A address, data size and result address respectively; "identifier" is matrix calculation operation information. It can be seen that the main difference between binary variable operator-level instructions and univariate operator-level instructions is the different number of matrix operation data.

[0141] Furthermore, since the use of operator-level instructions increases the number of instructions, in order to minimize the negative impact on the model's running performance, the coherence between instructions can be considered when allocating memory. Specifically, when allocating memory locations for the matrix operation data of the current code line, it can be determined whether the matrix operation result of the previous code line is the matrix operation data of the current code line, that is, whether the current code line will use the matrix operation result of the previous code line. If the matrix operation result of the previous code line is the matrix operation data of the current code line, the memory location of the matrix operation result of the previous code line can be used as the memory location of the matrix operation data of the current code line to avoid additional allocation of memory locations. If the matrix operation result of the previous code line is not the matrix operation data of the current code line, the memory location can be reallocated for the matrix operation data of the current code line. In this way, the memory location can be reasonably allocated, and the probability of memory miss events can also be reduced.

[0142] Based on this, memory locations are allocated for matrix operation data corresponding to the code line, which may include:

[0143] Step 41: Determine whether the matrix operation result of the previous code line is the matrix operation data of the current code line;

[0144] Step 42: If yes, the memory location of the matrix operation result of the previous code line is used as the memory location of the matrix operation data of the current code line;

[0145] Step 43: If not, allocate memory locations for the matrix operation data of the current code line.

[0146] S303. If the operation type of the code line is a scalar operation type, a memory location is allocated for the scalar operation data and the scalar operation result corresponding to the code line, and the memory location and the scalar operation information of the code line are filled into the basic instruction; wherein the basic instruction is a machine instruction corresponding to the scalar operation type, including scalar operation operation information, scalar operation data location and scalar operation result location, the scalar operation data location is the memory location corresponding to the scalar operation data, and the scalar operation result location is the memory location corresponding to the scalar operation result.

[0147] In this step, when it is determined that the operation type of the code line is a scalar operation type, a memory location can be allocated for the scalar operation data and scalar operation result corresponding to the code line, and the memory location and the scalar operation information of the code line are filled into the basic instruction, thereby completing the compilation of the code line. The basic instructions are introduced below:

[0148] The basic instruction is a machine instruction corresponding to the scalar operation type, and the instruction may include scalar operation information, scalar operation data location, and scalar operation result location. The scalar operation operation information is the specific information of the scalar operation, which is used to prompt the processor which scalar operation operation needs to be performed. Common scalar operation operations include: finding the inverse, scalar addition, multiplication, square root, finding exp (performing exponential function operation), finding activation function, etc. The scalar operation data location is the memory location corresponding to the scalar operation data, which is used to prompt the processor from which memory location to obtain the scalar operation data. The scalar operation result location is the memory location corresponding to the scalar operation result, which is used to prompt the processor to which memory location to write the scalar operation result. It is worth noting that in this embodiment, the scalar operation operation can be set in the processor based on the hardware code, so that the processor can directly determine the scalar operation operation to be executed according to the scalar operation operation information in the basic instruction.

[0149] It should be noted that this embodiment does not limit the specific form of the basic instruction, and it can be set according to actual application requirements. For example, the basic instruction includes scalar operation information, scalar operation data address, scalar operation result address and scalar operation data identifier. The scalar operation data address is the memory address corresponding to the scalar operation data, the scalar operation result address is the memory address corresponding to the scalar operation result, and the scalar operation data identifier is the number of scalar operation data involved in the scalar operation operation. For ease of understanding, please refer to Figure 6 . Figure 6A schematic diagram of a basic instruction provided by an embodiment of the present invention, the basic instruction includes an instruction prefix and 5 instruction numbers. Among them, "registers 0~2" are 3 instruction numbers that need to be assigned to registers 0~2, which are data a address, data b address and result address respectively; "identifier 1" is scalar calculation operation information; "identifier 2" is a scalar operation data identifier, which is used to determine the number of scalar operation data. When the number of scalar operation data is 1, only register 0 needs to be copied. When the number of scalar operation data is 2, registers 0 and 1 need to be copied to distinguish between unary variables and binary variables.

[0150] In addition, it is worth noting that the present embodiment can also use basic instructions to achieve the effect of operator-level instructions, and it is only necessary to convert the matrix operation into a scalar operation for each element in the matrix. For example, the operator-level instruction for multiplying vector a by vector b can be obtained by assembling the following basic instructions: 1) Configure Figure 6 In the basic instructions shown, register 0 and register 1 indicate the first element address of vector a and vector b respectively, register 2 indicates the address of the result, identifier 1 indicates that multiplication is performed, and identifier 2 is assigned a value of 1 to indicate that data b participates in the calculation; 2) repeat the calculation of step 1, the specific number of times is the number of times the vector length is repeated; 3) then use the basic instructions to perform scalar addition calculations; the specific number of repetitions is the vector length minus 1. According to a series of scalar multiplication and scalar addition basic instructions, the calculation effect of vector-vector multiplication of operator-level instructions can be achieved.

[0151] Furthermore, since the use of basic instructions increases the number of instructions, in order to minimize the negative impact on the model's running performance, the coherence between instructions can be considered when allocating memory. Specifically, when allocating memory locations for the scalar operation data of the current code line, it can be determined whether the scalar operation result of the previous code line is the scalar operation data of the current code line, that is, whether the current code line will use the scalar operation result of the previous code line. If the scalar operation result of the previous code line is the scalar operation data of the current code line, the memory location of the scalar operation result of the previous code line can be used as the memory location of the scalar operation data of the current code line to avoid additional allocation of memory locations. If the scalar operation result of the previous code line is not the scalar operation data of the current code line, the memory location can be reallocated for the scalar operation data of the current code line. In this way, the memory location can be reasonably allocated, and the probability of memory miss events can also be reduced.

[0152] Based on this, memory locations are allocated for the scalar operation data corresponding to the code line, including:

[0153] Step 51: Determine whether the scalar operation result of the previous code line is the scalar operation data of the current code line;

[0154] Step 52: If yes, the memory location of the scalar operation result of the previous code line is used as the memory location of the scalar operation data of the current code line;

[0155] Step 53: If not, allocate memory locations for the scalar operation data of the current code line.

[0156] S304: Add operator-level instructions or basic instructions to the executable program.

[0157] In this step, operator-level instructions and basic instructions may be added to the executable program, so that the block-level instruction compilation model unit may be replaced by the operator-level instructions and basic instructions to improve compilation flexibility.

[0158] Furthermore, it is worth noting that since the program block corresponds to the model unit, that is, the code lines in the program block have a coherent execution process, in order to ensure that the corresponding operator-level instructions / basic instructions also have a smooth execution order, the operator-level instructions or basic instructions corresponding to the code lines in the program block can be organized into an instruction chain in the order of the code lines, and the instruction chain is added to the executable program. In this way, since the instructions in the instruction chain are related to each other through the memory location, the execution efficiency of the instruction chain can be improved.

[0159] Based on this, operator-level instructions or basic instructions are added to the executable program, which may include:

[0160] Step 61: The operator-level instructions or basic instructions corresponding to the code lines in the program block are organized into an instruction chain in the order of the code lines, and the instruction chain is added to the executable program.

[0161] The following will introduce the use of operator-level instructions instead of block-level instructions to compile model units. In one case, the multi-layer perceptron can be compiled by the following chain assembly method to achieve the same computational effect:

[0162] 1) Configure matrices A and B1 Figure 4 In the binary variable operator-level instruction shown, the fields register 0, register 1, and register 3 are assigned the starting addresses of matrices A, B1, and result matrix D1, respectively; the data scale field is assigned the row and column dimensions required in the matrix calculation; the identification field is configured with a numerical value indicating the type of matrix multiplication calculation to be performed;

[0163] 2) If there is a bias C1, configure Figure 4 The operator-level instruction of the binary variable shown in the figure, the fields register 0, register 1, and register 3 are assigned the starting addresses of the matrix D1, the bias, and the result matrix D2 respectively, the data scale field is assigned the row and column dimensions required in the calculation, and the identification field is configured with a numerical value indicating the type of calculation of matrix plus matrix / vector; if there is no bias C1, this step is omitted;

[0164] 3) Configuration Figure 5 The operator-level instruction of the unary variable shown performs activation function calculation on the matrix D2, and the field register 0 and register 2 are respectively assigned the starting addresses of the matrix D2 and the result matrix D3, the data scale field is assigned the row and column dimensions required in the calculation, and the identification field is configured with a numerical value indicating the calculation type of the activation function;

[0165] 4) Configure matrices D3 and B2 Figure 4 The operator-level instruction of the binary variable shown, the fields register 0, register 1, and register 3 are assigned the starting addresses of the matrices D3, B2, and the result matrix D4 respectively, the data scale field is assigned the row and column dimensions required in the matrix calculation, and the identification field is configured with a numerical value indicating the type of calculation to be performed for matrix multiplication;

[0166] 5) If there is bias C2, configure Figure 4 As shown in the operator-level instruction of the binary variable, the fields register 0, register 1, and register 3 are assigned the starting addresses of the matrix D4, the bias C2, and the result matrix D5 respectively. The data scale field is assigned the row and column dimensions required in the calculation. The identification field is configured with a numerical value indicating the type of calculation of matrix addition / matrix / vector. If there is no bias C2, this step is omitted.

[0167] Based on the above embodiment, the specific process of the processor executing the operator-level instruction is introduced below. In one possible case, after adding the filled block-level instruction to the executable program, the method may also include:

[0168] S401. When executing an operator-level instruction, read matrix operation information in the operator-level instruction to determine the matrix operation to be executed.

[0169] In this step, when executing an operator-level instruction, the processor may first read the matrix operation information in the operator-level instruction to determine the matrix operation to be executed.

[0170] S402, assigning values ​​to the operation data register according to the matrix operation data address, assigning values ​​to the operation result register according to the matrix operation result address, and assigning values ​​to the operation data scale register according to the matrix operation data scale.

[0171] In this step, the processor needs to assign values ​​to the corresponding registers according to the matrix operation data address, matrix operation result address and matrix operation data scale in the operator-level instruction. For a one-variable operator-level instruction, only a single operation data register needs to be assigned; for a two-variable operator-level instruction, two operation data registers need to be assigned.

[0172] S403, extracting matrix operation data from the memory according to the operation data register and the operation data scale register, performing matrix operation operation using the matrix operation data to obtain the matrix operation result, and writing the matrix operation result into the memory according to the operation result register and the operation data scale register.

[0173] In this step, after completing the register assignment, the processor can extract matrix operation data from the memory according to the operation data register and the operation data scale register, use the matrix operation data to perform matrix operation operations to obtain matrix operation results, and write the matrix operation results into the memory according to the operation result register and the operation data scale register, thereby efficiently completing the matrix operation operation.

[0174] Based on the above embodiment, the specific process of the processor executing the basic instruction is introduced below. In one possible case, after adding the filled block-level instruction to the executable program, the method may also include:

[0175] S501. When executing a basic instruction, read scalar operation information to determine a scalar operation to be executed.

[0176] In this step, when executing a basic instruction, the processor may first read the scalar operation information in the basic instruction to determine the scalar operation to be executed.

[0177] S502 . Assign values ​​to the operation data register according to the scalar operation data address and the scalar operation data identifier, and assign values ​​to the operation result register according to the scalar operation result address.

[0178] In this step, the processor needs to assign values ​​to the corresponding registers according to the scalar operation data address, scalar operation result address and scalar operation data identifier in the basic instruction. It is necessary to pay special attention to the value of the scalar operation data identifier. If the scalar operation data identifier indicates that the number of scalar operation data is 1, only one scalar operation data register needs to be assigned; if the scalar operation data identifier indicates that the number of scalar operation data is 2, only two scalar operation data registers need to be assigned.

[0179] Based on this, assigning a value to the operation data register according to the scalar operation data address and the scalar operation data identifier may include:

[0180] Step 71: determine whether the number of scalar operation data indicated by the scalar operation data identifier is 1;

[0181] Step 72: If yes, assigning a value to the operation data register of the first scalar operation data according to the scalar operation data address of the first scalar operation data;

[0182] Step 73: If not, assign values ​​to the operation data register of the first scalar operation data and the operation data register of the second scalar operation data respectively according to the scalar operation data addresses of the first scalar operation data and the second scalar operation data.

[0183] S503, extracting scalar operation data from the memory according to the operation data register, performing a scalar operation operation using the scalar operation data to obtain a scalar operation result, and writing the scalar operation result into the memory according to the operation result register.

[0184] In this step, after completing the register assignment, the processor can extract scalar operation data from the memory according to the operation data register and the operation data scale register, use the scalar operation data to perform scalar operation operations to obtain scalar operation results, and write the scalar operation results to the memory according to the operation result register and the operation data scale register, thereby efficiently completing the scalar operation operation.

[0185] Through the above-mentioned multi-level instruction design improvements, for the typical model reasoning process, block-level instructions can be used to directly generate the instructions required for the typical program block. This modular instruction generation method can not only reduce the complexity of decoding multiple instructions in the later stage, but also improve the hit rate of the instruction cache and speed up the calculation process. For the large model that has changed, there is no ready-made block-level instruction. The operator-level instructions can be used to chain and assemble the process of generating the corresponding changed block-level instructions. This makes it possible to flexibly obtain customized high-level task instructions through operator-level instruction chain generation. Similarly, for the transformed operator, if there is no ready-made operator-level instruction, the corresponding operator-level instruction requirements can be generated by chaining the basic instruction combination. Therefore, the multi-level instruction design achieves a good balance between instruction flexibility and high-performance computing, supporting both instruction generation of existing typical program blocks and flexible instruction generation after the model has changed. By providing modularity and ready-made chain assembly, this instruction design simplifies the complexity of instruction generation. Developers can focus more on high-level task logic without spending a lot of time and energy on the underlying instruction technical details.

[0186] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0187] The following is an introduction to the model program compiling device, electronic device, computer program product, and non-volatile computer-readable storage medium provided in the embodiments of the present application.

[0188] Please refer to Figure 7 , Figure 7A structural block diagram of a model program compiling device provided in an embodiment of the present application, the device may include:

[0189] A matching module 701 is used to match the block-level instructions corresponding to the program blocks in the source program in the preset instruction set in the process of compiling the neural network model from the source program to the executable program; wherein the program blocks correspond to the model units pre-divided in the neural network model; the block-level instructions are machine instructions corresponding to the model units, including the unit type, operation data location and operation result location corresponding to the model units, the operation data location is the memory location corresponding to the operation data input to the model unit, and the operation result location is the memory location corresponding to the operation result output by the model unit;

[0190] A memory allocation module 702, for allocating operation data locations and operation result locations for program blocks and filling them into block-level instructions;

[0191] The instruction adding module 703 is used to add the filled block-level instructions to the executable program.

[0192] Optionally, the device may further include:

[0193] The unit type parsing module is used to read the unit type in the block-level instruction when executing the block-level instruction, so as to determine the combination operation corresponding to the model unit according to the unit type; wherein the combination operation includes the operation operation and operation sequence corresponding to the model unit;

[0194] A register assignment module, used to assign a value to an operation data register according to an operation data address and an offset identifier, assign a value to an operation result register according to an operation result address, and assign a value to an operation data scale register according to an operation data scale;

[0195] The operation module is used to extract operation data from the memory according to the operation data register and the operation data scale register, use the operation data to perform combined operation operations to obtain operation results, and write the operation results into the memory according to the operation result register and the operation data scale register.

[0196] Optionally, the combined operation has been set in the processor based on hardware code.

[0197] Optionally, the operation data scale is composed of a plurality of bits, and the bits correspond to the operation data;

[0198] The operation module may include:

[0199] The operation data size analysis submodule is used to determine the operation data size of each operation data according to each bit in the operation data size register;

[0200] The operation data extraction submodule is used to extract the operation data from the memory according to the operation data register corresponding to each operation data and the operation data size;

[0201] The operation result size analysis submodule is used to determine the operation result size according to each bit in the operation data size register;

[0202] The operation result writing submodule is used to write the operation result into the memory according to the operation result register and the operation result size.

[0203] Optionally, the register assignment module may include:

[0204] The bias assignment submodule is used to determine whether the bias identifier indicates the use of bias data; if the bias identifier indicates the use of bias data, the operation data register corresponding to the bias data is assigned using the operation data address of the bias data; if the bias identifier indicates that the bias data is not used, the operation data register corresponding to the bias data is not assigned.

[0205] Optionally, the device may further include:

[0206] An operation type determination module is used to determine the operation type corresponding to the code line in the program block if there is no block-level instruction corresponding to the program block in the preset instruction set; the operation type includes a matrix operation type and a scalar operation type;

[0207] An operator-level instruction translation module is used to allocate a memory location for the matrix operation data and the matrix operation result corresponding to the code line if the operation type of the code line is a matrix operation type, and fill the memory location and the matrix operation operation information of the code line into the operator-level instruction; wherein the operator-level instruction is a machine instruction corresponding to the matrix operation type, including matrix operation operation information, matrix operation data location and matrix operation result location, the matrix operation data location is the memory location corresponding to the matrix operation data, and the matrix operation result location is the memory location corresponding to the matrix operation result;

[0208] A basic instruction translation module is used for allocating a memory location for the scalar operation data and the scalar operation result corresponding to the code line if the operation type of the code line is a scalar operation type, and filling the memory location and the scalar operation information of the code line into the basic instruction; wherein the basic instruction is a machine instruction corresponding to the scalar operation type, including scalar operation operation information, scalar operation data location and scalar operation result location, the scalar operation data location is the memory location corresponding to the scalar operation data, and the scalar operation result location is the memory location corresponding to the scalar operation result;

[0209] The instruction adding module 703 is further used to add operator-level instructions or basic instructions to the executable program.

[0210] Optionally, the device may further include:

[0211] An operator parsing module is used to read the matrix operation information in the operator-level instruction when executing the operator-level instruction to determine the matrix operation to be executed;

[0212] The register assignment module is further used to assign values ​​to the operation data register according to the matrix operation data address, assign values ​​to the operation result register according to the matrix operation result address, and assign values ​​to the operation data scale register according to the matrix operation data scale;

[0213] The operation module is also used to extract matrix operation data from the memory according to the operation data register and the operation data scale register, use the matrix operation data to perform matrix operation operations to obtain matrix operation results, and write the matrix operation results into the memory according to the operation result register and the operation data scale register.

[0214] Optionally, the device may further include:

[0215] A scalar operation analysis module, used to read scalar operation information when executing basic instructions to determine the scalar operation to be performed;

[0216] The register assignment module is further used to assign a value to the operation data register according to the scalar operation data address and the scalar operation data identifier, and to assign a value to the operation result register according to the scalar operation result address;

[0217] The operation module is also used to extract scalar operation data from the memory according to the operation data register, perform scalar operation operations using the scalar operation data to obtain scalar operation results, and write the scalar operation results into the memory according to the operation result register.

[0218] Optionally, the register assignment module may include:

[0219] The scalar operation data assignment submodule is used to determine whether the number of scalar operation data represented by the scalar operation data identifier is 1; if so, the operation data register of the first scalar operation data is assigned according to the scalar operation data address of the first scalar operation data; if not, the operation data register of the first scalar operation data and the operation data register of the second scalar operation data are assigned according to the scalar operation data addresses of the first scalar operation data and the second scalar operation data respectively.

[0220] Optionally, the operator-level instruction translation module may include:

[0221] The first memory allocation submodule is used to determine whether the matrix operation result of the previous code line is the matrix operation data of the current code line; if so, the memory location of the matrix operation result of the previous code line is used as the memory location of the matrix operation data of the current code line; if not, a memory location is allocated for the matrix operation data of the current code line;

[0222] Basic instruction translation module, which can include:

[0223] The first memory allocation submodule is used to determine whether the scalar operation result of the previous code line is the scalar operation data of the current code line; if so, the memory location of the scalar operation result of the previous code line is used as the memory location of the scalar operation data of the current code line; if not, a memory location is allocated for the scalar operation data of the current code line;

[0224] The instruction adding module 703 may include:

[0225] The instruction chain adding submodule is used to organize the operator-level instructions or basic instructions corresponding to the code lines in the program block into an instruction chain in the order of the code lines, and add the instruction chain to the executable program.

[0226] The description of the features in the embodiment corresponding to the model program compiling device can refer to the relevant description of the embodiment corresponding to the model program compiling method, which will not be repeated here.

[0227] An embodiment of the present application further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above-mentioned model program compilation method embodiments.

[0228] Please refer to Figure 8 , Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The embodiment of the present invention provides an electronic device 10, including a processor 11 and a memory 12; wherein the memory 12 is used to store computer programs; the processor 11 is used to execute the model program compilation method provided in the aforementioned embodiment when executing the computer program.

[0229] For the specific process of the above-mentioned model program compilation method, reference may be made to the corresponding contents provided in the aforementioned embodiments, which will not be elaborated here.

[0230] Furthermore, the memory 12, as a carrier for storing resources, may be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc., and the storage method may be temporary storage or permanent storage.

[0231] In addition, the electronic device 10 also includes a power supply 13, a communication interface 14, an input / output interface 15 and a communication bus 16; wherein the power supply 13 is used to provide working voltage for each hardware device on the electronic device 10; the communication interface 14 can create a data transmission channel between the electronic device 10 and an external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present invention, and is not specifically limited here; the input / output interface 15 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs, and is not specifically limited here.

[0232] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above-mentioned model program compilation method embodiments when running.

[0233] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0234] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps in any one of the above-mentioned model program compilation method embodiments are implemented.

[0235] An embodiment of the present application also provides another computer program product, including a non-volatile computer-readable storage medium, the non-volatile computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the steps in any of the above-mentioned model program compilation method embodiments are implemented.

[0236] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0237] The above is a detailed introduction to a model program compilation method, electronic device, program product and medium provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application, and the description of the above embodiments is only used to help understand the method and core idea of ​​the present application. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A model program compiling method, characterized in that: include: In the process of compiling the neural network model from a source program to an executable program, a block-level instruction corresponding to a program block in the source program is matched in a preset instruction set; wherein the program block corresponds to a model unit pre-divided in the neural network model; the block-level instruction is a machine instruction corresponding to the model unit, including a unit type, a calculation data position and a calculation result position corresponding to the model unit, wherein the calculation data position is a memory position corresponding to the calculation data input to the model unit, and the calculation result position is a memory position corresponding to the calculation result output by the model unit; Allocating the operation data location and the operation result location to the program block and filling them into the block-level instructions; Block-level instructions that complete the padding are added to the executable program.

2. The model program compiling method according to claim 1, characterized in that: The block-level instruction includes the unit type, operation data address, operation result address, operation data scale and offset identifier, the operation data address is the memory address corresponding to the operation data, the operation result address is the memory address corresponding to the operation result, the operation data scale is the operation data size corresponding to the operation data, and the offset identifier is the usage information corresponding to the offset data in the operation data.

3. The model program compiling method according to claim 2, characterized in that: After adding the filled block-level instructions to the executable program, the method further includes: When executing the block-level instruction, the unit type in the block-level instruction is read to determine the combined operation corresponding to the model unit according to the unit type; wherein the combined operation includes the operation and operation sequence corresponding to the model unit; Assigning a value to an operation data register according to the operation data address and the offset identifier, assigning a value to an operation result register according to the operation result address, and assigning a value to an operation data scale register according to the operation data scale; The operation data is extracted from the memory according to the operation data register and the operation data scale register, the combined operation operation is performed using the operation data to obtain the operation result, and the operation result is written into the memory according to the operation result register and the operation data scale register.

4. The model program compiling method according to claim 3, characterized in that: The operation data scale is composed of a plurality of bits, and the bits correspond to the operation data; Extracting the operation data from the memory according to the operation data register and the operation data size register comprises: Determine the operation data size of each operation data according to each bit in the operation data size register; Extracting the operation data from the memory according to the operation data register corresponding to each operation data and the operation data size; Writing the operation result into the memory according to the operation result register and the operation data size register comprises: Determine the size of the operation result according to each bit in the operation data size register; The operation result is written into a memory according to the operation result register and the operation result size.

5. The model program compiling method according to claim 3, characterized in that: Assigning a value to an operation data register according to the operation data address and the offset identifier, comprising: Determining whether the bias identifier indicates using the bias data; If the bias identifier indicates that the bias data is used, then the operation data register corresponding to the bias data is assigned a value using the operation data address of the bias data; If the offset identifier indicates that the offset data is not used, no value is assigned to the operation data register corresponding to the offset data.

6. The model program compiling method according to any one of claims 1 to 5, characterized in that: After matching the block-level instructions corresponding to the program blocks in the source program in the preset instruction set, the method further includes: If there is no block-level instruction corresponding to the program block in the preset instruction set, determining the type of operation corresponding to the code line in the program block; the operation type includes a matrix operation type and a scalar operation type; If the operation type of the code line is the matrix operation type, a memory location is allocated for the matrix operation data and the matrix operation result corresponding to the code line, and the memory location and the matrix operation operation information of the code line are filled into the operator-level instruction; wherein the operator-level instruction is a machine instruction corresponding to the matrix operation type, comprising the matrix operation operation information, the matrix operation data location and the matrix operation result location, the matrix operation data location is the memory location corresponding to the matrix operation data, and the matrix operation result location is the memory location corresponding to the matrix operation result; If the operation type of the code line is the scalar operation type, a memory location is allocated for the scalar operation data and the scalar operation result corresponding to the code line, and the memory location and the scalar operation information of the code line are filled into the basic instruction; wherein the basic instruction is a machine instruction corresponding to the scalar operation type, comprising the scalar operation information, the scalar operation data location and the scalar operation result location, the scalar operation data location is the memory location corresponding to the scalar operation data, and the scalar operation result location is the memory location corresponding to the scalar operation result; The operator-level instruction or the basic instruction is added to the executable program.

7. The model program compiling method according to claim 6, characterized in that: The operator-level instruction includes the matrix operation operation information, the matrix operation data address, the matrix operation result address and the matrix operation data scale. The matrix operation data address is the memory address corresponding to the matrix operation data, the matrix operation result address is the memory address corresponding to the matrix operation result, and the matrix operation data scale is the matrix operation data size corresponding to the matrix operation data.

8. The model program compiling method according to claim 7, characterized in that: After adding the filled block-level instructions to the executable program, the method further includes: When executing the operator-level instruction, reading the matrix operation information in the operator-level instruction to determine the matrix operation to be performed; Assigning a value to an operation data register according to the matrix operation data address, assigning a value to an operation result register according to the matrix operation result address, and assigning a value to an operation data scale register according to the matrix operation data scale; The matrix operation data is extracted from the memory according to the operation data register and the operation data scale register, the matrix operation operation is performed using the matrix operation data to obtain the matrix operation result, and the matrix operation result is written into the memory according to the operation result register and the operation data scale register.

9. The model program compiling method according to claim 6, characterized in that: The basic instruction includes the scalar operation information, the scalar operation data address, the scalar operation result address and the scalar operation data identifier. The scalar operation data address is the memory address corresponding to the scalar operation data, the scalar operation result address is the memory address corresponding to the scalar operation result, and the scalar operation data identifier is the number of scalar operation data involved in the scalar operation operation.

10. The model program compiling method according to claim 9, characterized in that: After adding the filled block-level instructions to the executable program, the method further includes: When executing the basic instruction, reading the scalar operation information to determine the scalar operation to be performed; Assigning a value to an operation data register according to the scalar operation data address and the scalar operation data identifier, and assigning a value to an operation result register according to the scalar operation result address; The scalar operation data is extracted from the memory according to the operation data register, the scalar operation operation is performed using the scalar operation data to obtain the scalar operation result, and the scalar operation result is written into the memory according to the operation result register.

11. The model program compiling method according to claim 10, characterized in that: Assigning a value to an operation data register according to the scalar operation data address and the scalar operation data identifier includes: Determine whether the number of scalar operation data represented by the scalar operation data identifier is 1; If yes, assigning a value to the operation data register of the first scalar operation data according to the scalar operation data address of the first scalar operation data; If not, assign values ​​to the operation data register of the first scalar operation data and the operation data register of the second scalar operation data respectively according to the scalar operation data addresses of the first scalar operation data and the second scalar operation data.

12. The model program compiling method according to claim 6, characterized in that: Allocating memory locations for matrix operation data corresponding to the code line includes: Determine whether the matrix operation result of the previous code line is the matrix operation data of the current code line; If yes, the memory location of the matrix operation result of the previous code line is used as the memory location of the matrix operation data of the current code line; If not, allocate memory locations for the matrix operation data of the current code line; Allocating a memory location for the scalar operation data corresponding to the code line includes: Determine whether the scalar operation result of the previous code line is the scalar operation data of the current code line; If yes, the memory location of the scalar operation result of the previous code line is used as the memory location of the scalar operation data of the current code line; If not, allocate memory locations for the scalar operation data for the current line of code; Adding the operator-level instruction or the basic instruction to the executable program includes: Operator-level instructions or basic instructions corresponding to the code lines in the program block are combined into an instruction chain in the order of the code lines, and the instruction chain is added to the executable program.

13. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the model program compiling method as claimed in any one of claims 1 to 12 when executing the computer program.

14. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the model program compiling method according to any one of claims 1 to 12 is implemented.

15. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are loaded and executed by a processor, the model program compiling method according to any one of claims 1 to 12 is implemented.

Images