Data layout optimization method and device applied to NPU code compiling and medium

Abstract

The invention discloses a data layout optimization method and device applied to NPU code compilation and a medium. The method comprises the steps that an intermediate representation IR input by an upper layer is split into an operation type OP and an operand, and the operand input by the upper layer is divided into logic data and a logic mask; converting the logic data of the upper layer into data vector representation of the abstraction layer, and converting the logic mask of the upper layer into mask vector representation of the abstraction layer; obtaining a corresponding bottom hardware instruction capability according to an operation type OP input by an upper layer, and performing legalization operation on the operation type OP; and performing legalization processing on the converted abstract data vector type representation and mask vector representation according to the target hardware capability, performing instruction mapping on the processed abstract instruction according to the underlying hardware pair operation type OP, and generating a target LLVM instruction compatible with the underlying hardware. Register layout abstraction and conversion can be carried out on the NPU code under mixed precision input, and the utilization efficiency of the NPU bottom layer register is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of compiler, in particular to a data layout optimization method, device and medium applied to NPU code compilation. BACKGROUND

[0002] With the progress of science and technology, the demand for computing resources in the fields of artificial intelligence, high-performance computing, Internet of Things, etc. is increasing. Hardware resources provide the necessary computing power, and the traditional single computing architecture based on CPU as the main computing core has low efficiency in processing parallel computing tasks such as matrix multiplication of neural network core computing operations, which cannot meet the high-throughput and high-efficiency computing requirements. At the same time, with the slowing down of Moore's Law, the computing performance of a single CPU core is gradually slow.

[0003] GPU is gradually applied to the above high-computing-demand scenarios due to its high performance in processing parallel tasks. CPU and GPU form a typical heterogeneous computing platform, in which CPU is responsible for processing complex control logic, task scheduling and serialization operations, and GPU is responsible for executing large-scale parallel computing tasks. Heterogeneous computing architecture provides better performance by dividing computing tasks into multiple subtasks and assigning them to different processing units, can support high-throughput computing and low-latency computing at the same time, and significantly reduces energy consumption by selecting appropriate processors to execute specific tasks.

[0004] NPU and traditional CPU and GPU have obvious differences in design ideas. It is specially optimized for neural networks, adopts self-developed Da Vinci architecture, uses AICore as the computing core, and uses data reuse, multi-level cache and pipeline parallelism to mask delays. The new heterogeneous parallel computing system composed of CPU and NPU provides more energy-efficient support for the above scenarios. Building an NPU compiler as a core task to perfect the NPU ecosystem needs to ensure the correctness and efficiency of its code generation, especially the utilization rate of its registers. However, in the case of mixed-precision input, multiple widths of data types exist at the same time, and the compiler faces the problem of incompatible upper and lower abstractions. According to the traditional method, the full bandwidth of the hardware cannot be utilized and the full computing power performance of the hardware cannot be achieved, so a reasonable and efficient abstraction method and conversion method are needed to improve the overall utilization rate and performance of the system. SUMMARY

[0005] To solve the technical problems existing in the prior art, the present application provides a data layout optimization method, device and medium for NPU code compilation. The present application can perform register layout abstraction and conversion on NPU code under mixed-precision input, which can improve the utilization efficiency of NPU bottom-layer registers.

[0006] The object of the application can be achieved by adopting the following technical solutions:

[0007] The data layout optimization method applied to NPU code compilation comprises the following steps:

[0008] S1, split the intermediate representation IR input from the upper layer into an operation type OP and an operand, and divide the operand input from the upper layer into logical data and a logical mask;

[0009] S2, convert the logical data from the upper layer into a data vector representation in a continuous arrangement form located at the 0th channel of the abstract layer, and convert the logical mask from the upper layer into a mask vector representation in a continuous arrangement form located at the 0th channel of the abstract layer;

[0010] S3, obtain the corresponding bottom-layer hardware instruction capability according to the operation type OP input from the upper layer, and perform a legalization operation on the operation type OP when the bottom-layer hardware instruction capability corresponding to the operation type OP does not support the combination of the OP and the type;

[0011] S4, perform a legalization processing on the converted abstract data vector type representation and the mask vector representation according to the target hardware capability, so that the combination of the operation type OP and the abstract data vector layout and the abstract mask layout corresponding to the operation type OP can be effectively mapped to the instruction form supported by the bottom-layer hardware.

[0012] S5, perform an instruction mapping on the processed abstract instruction according to the operation type OP of the bottom-layer hardware, and generate a target LLVM instruction compatible with the bottom-layer hardware.

[0013] Further, the conversion of the logical data from the upper layer into the data vector representation in the continuous arrangement form located at the 0th channel of the abstract layer comprises:

[0014] extracting the element type and the element number of the logical data, and calculating the lane number and the channel number of the element type on the bottom-layer hardware according to the bit width of the element type and the fixed length of the bottom-layer vector register;

[0015] constructing an abstract layer data vector based on the lane number and the channel number of the element type on the bottom-layer hardware, wherein the element type of the abstract layer data vector is consistent with the logical data from the upper layer, the actual length of the data vector is the calculated lane number, and the effective data length of the data vector is the element number of the logical data from the upper layer;

[0016] embed the logical data from the upper layer into the abstract layer data vector, and divide the logical data arrangement into a continuous arrangement according to the data spacing relationship between different channels, to obtain the mask vector representation in the continuous arrangement form located at the 0th channel of the abstract layer.

[0017] Further, the element type and the element number of the extracted logical data are used to calculate the lane number and the channel number of the element type on the underlying hardware according to the bit width of the element type, the element number and the fixed length of the underlying vector register, including:

[0018] The lane number of the element type on the underlying hardware is obtained according to the ratio of the bit number occupied by the underlying vector register to the bit number occupied by the element type; and the channel number of the element type on the underlying vector register is obtained by dividing the lane number of the element type on the underlying hardware by the element number of the logical data.

[0019] Further, the conversion of the logical mask of the upper layer into the mask vector representation of the abstract layer in which the logical mask is arranged continuously and located in the 0th channel includes:

[0020] The length of the logical mask of the upper layer is extracted; the lane number and the channel number of the element type on the underlying mask register are calculated according to the length of the logical mask and the length of the underlying mask register; and the mask vector representation of the abstract layer is constructed in the mapping mode in which one logical mask bit is represented by the actual mask bit of the underlying hardware, the number of mask bits of the mask vector representation is consistent with the logical mask of the upper layer, the valid bit type of the mask vector representation is 1bit type, and the channel number is 0;

[0021] Then, the logical mask of the upper layer is embedded into the mask vector representation, and the logical mask type of the upper layer is mapped into the mask vector representation in which the logical mask is arranged continuously, the channel number is 0, and the length of the logical mask is 1bit.

[0022] Further, the calculation of the lane number and the channel number of the element type on the underlying mask register according to the length of the logical mask and the length of the underlying mask register includes:

[0023] The total lane number that can be accommodated in the underlying mask register is obtained by dividing the total bit number of the underlying mask register by the single mask bit number occupied by the logical mask; and the channel number of the logical mask corresponding to the underlying mask register of the hardware is obtained by dividing the total lane number that can be accommodated in the underlying mask register by the actual mask bit number corresponding to the logical mask.

[0024] Further, according to the arrangement requirement of the data vector representation of the abstract instruction of the abstract layer, the arrangement type and the channel number are adjusted to be legal, and the corresponding abstract instruction is used to make the adjusted data vector representation conform to the data input specification of the underlying instruction in the physical structure;

[0025] According to the requirement of the mask control of the abstract instruction of the abstract layer, the arrangement type, the valid bit type and the channel number of the mask vector representation are adjusted to be legal, and the corresponding abstract instruction is used to make the mask control granularity and structure align with the data path;

[0026] The abstract instructions introduced in the legalization adjustment process are folded and eliminated or are redundantly combined to obtain the legalized abstract instructions.

[0027] Further, the processed abstract instructions are mapped to operation types OP according to the underlying hardware to generate target LLVM instructions compatible with the underlying hardware.

[0028] An instruction list of the underlying ISA is obtained, a mapping relationship between the upper operation types OP and the interface of the underlying hardware is established, an abstract instruction is selected as a target instruction for legalization according to the mapping relationship between the upper operation types OP and the interface of the underlying hardware, and the legalized target instruction is obtained.

[0029] According to the size relationship between the operation type OP capability and the underlying instruction capability, different mappings are made when the legalized target instruction is converted to the underlying hardware to generate target LLVM instructions compatible with the underlying hardware.

[0030] The second object of the application can be achieved by adopting the following technical solution:

[0031] A computer device comprises a processor and a memory for storing a program executable by the processor, and the processor implements the above-mentioned data layout optimization method applied to NPU code compilation when executing the program stored in the memory.

[0032] The third object of the application can be achieved by adopting the following technical solution:

[0033] A storage medium stores a program, and the program is executed by a processor to implement the above-mentioned data layout optimization method applied to NPU code compilation.

[0034] Compared with the prior art, the application has the following advantages and beneficial effects:

[0035] The application provides a data layout optimization method, device and medium applied to NPU code compilation, which splits an intermediate representation IR input by an upper layer into an operation type OP and an operand, converts logical data of the upper layer into a data vector representation of an abstract layer, converts a logical mask of the upper layer into a mask vector representation of the abstract layer, and performs a legalization operation on the operation type OP; the converted abstract data vector type representation and the mask vector representation are subjected to legalization processing according to a target hardware capability, the processed abstract instructions are subjected to instruction mapping on the operation type OP according to a bottom layer hardware, and target LLVM instructions compatible with the bottom layer hardware are generated. The NPU code under mixed precision input can be subjected to register layout abstraction and conversion, the correctness and efficiency of register allocation in the case of multiple precision codes are realized, the cost of mapping the instructions to corresponding instructions of the back end is quantified by using a cost model, so that the optimal instruction of the back end is selected, which can completely correspond to the bottom layer instructions of the bottom layer and the requirements of the corresponding instructions on the format and content of the bottom layer vector register and the bottom layer mask register, thereby improving the utilization efficiency of the bottom layer register. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained according to the structures shown in the drawings without creative labor.

[0037] Figure 1 is a flow chart of the data layout optimization method applied to NPU code compilation in the embodiments of the present application;

[0038] Figure 2 is a type description schematic diagram of the upper layer, the intermediate abstract layer and the bottom layer in the embodiments of the present application;

[0039] Figure 3 is a description diagram of the process of converting the logical data from the upper layer to the data vector of the abstract layer in the embodiments of the present application;

[0040] Figure 4 is a description diagram of the process of converting the logical mask from the upper layer to the logical mask of the abstract layer in the embodiments of the present application;

[0041] Figure 5 is a description diagram of the process of participating in the instruction legalization of the converted data vector of the abstract layer and the logical mask of the abstract layer in the embodiments of the present application. DETAILED DESCRIPTION

[0042] The technical solutions of the present application will be described in further detail below with reference to the drawings and embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments of the present application. The embodiments of the present application are not limited to the described embodiments. Based on the embodiments of the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of the present application.

[0043] Embodiment 1

[0044] The present application provides a data layout optimization method applied to NPU code compilation, which can perform register layout abstraction and conversion on NPU code under mixed precision input, from upper layer logical instructions, logical data, logical masks, through abstract instructions, abstract masks, abstract data of the intermediate abstraction layer, legalizing the arrangement type, element type, data actual length and channel number of the data, so that it can completely correspond to the lower layer instructions, and the requirements of the corresponding instructions on the format and content of the lower layer vector register and lower layer mask register, thereby improving the utilization efficiency of the lower layer register.

[0045] As shown in Figure 1 The data layout optimization method applied to NPU code compilation of the present application optimizes the compilation process of NPU code from MLIR to LLVM. The specific implementation steps are as follows:

[0046] Step 1, split the intermediate representation (Intermediate Representation, IR) input from the upper layer into operation type (OP) and operands, and divide the operands input from the upper layer into logical data and logical masks.

[0047] IR is usually composed of a series of instructions, each instruction contains: operation type (OP) and operands (Operands). The operation type (OP) represents a specific computing operation type, which implements a specific operation on data, defines the semantic behavior of the calculation, such as arithmetic operation, logical judgment or data transmission, etc. The operands represent the data participating in the calculation, which are input parameters of the operation type OP. The operands can be divided into data and masks according to the purpose: the data represents the data set participating in the actual calculation, which usually corresponds to continuous or structured tensor data; the mask is used to indicate which data elements in the data are actually operated when performing the calculation operation, so as to realize the semantic control of bit-selective calculation, and realize conditional execution or selective calculation by specifying the enable relationship bit by bit.

[0048] After separating the operation type OP and the operand, different optimization strategies can be adopted for different types of operations (such as arithmetic operations, memory access, control flow, etc.), and different data layout strategies can be adopted for different types of operands (such as registers, variables, constants, or memory addresses). Separating the operation type (OP) and the operand allows independent optimization of data layout and operation execution mode, providing a basic framework for subsequent data layout optimization and enabling the use of the most suitable layout strategy for different types of operands. Data layout optimization: focuses on the organization of operands in memory or registers, such as the storage order of arrays (row-major or column-major), the arrangement of fields in structure bodies, cache alignment, etc. These optimizations can significantly affect the memory access performance of the program. Operation execution optimization: focuses on the execution efficiency of the operation type itself, such as instruction selection, instruction scheduling, pipeline optimization, etc.

[0049] As shown in Figure 2 the type description diagram of the upper layer, the intermediate abstraction layer, and the bottom layer, the operation type OP in the upper layer input is called a logical instruction, the data in the operand of the upper layer input is called logical data, and the mask in the upper layer input is called a logical mask. The logical instruction is designed to be compatible with all hardware backends and has a single function without complex patterns. The logical data and the logical mask use an ideal data layout, i.e., the actual data and mask length used under the premise of considering the tail block of the mask processing, i.e., the length of the logical data and the logical mask.

[0050] The operation type of the intermediate abstraction layer is called an abstract instruction, which is designed for NPU and has rich on-the-fly functions such as on-the-fly data processing and mask merging patterns. The abstract instruction set is orthogonal to the bottom layer instruction set (when an instruction set has corresponding instructions for any data type, it is called orthogonal, and if a data type T is supported by an instruction OP_A but not by OP_B, the instruction set is not orthogonal). The data of the abstract layer is used to describe the logical representation mapped to the fixed-length data register of the bottom layer, which includes four attributes: arrangement type (cross arrangement or continuous arrangement), element type (Type), actual data length (Length), and channel number (Slot). The mask of the abstract layer is used to describe the logical representation mapped to the fixed-length mask register of the bottom layer, which includes four attributes: arrangement type (cross arrangement or continuous arrangement), valid bit type (Type), number of logical mask bits (Length), and channel number (Slot); where the valid bit type refers to a logical mask bit that may correspond to 1 bit, 2 bits, or 4 bits of actual mask bits, and its specific width can be determined by the bottom layer mask register length and the number of logical mask bits, without the need for explicit storage.

[0051] The bottom layer hardware instruction is referred to as a bottom layer instruction, the bottom layer actual data is referred to as a bottom layer vector register, and the bottom layer actual mask is referred to as a bottom layer mask register. The bottom layer instruction is completely designed for the NPU, has rich on-the-way functions, and has advanced functions such as on-the-way data processing and mask merging mode. The bottom layer data has different lane numbers and layout conditions according to different types, and the bottom layer mask has different enable modes according to the bottom layer data type matched therewith, thereby also having different lane numbers and layout conditions.

[0052] The arrangement type, element type, data actual length, and channel number of the converted data are legalized, so that they can completely correspond to the requirements of the bottom layer instruction and the format and content of the corresponding instruction on the bottom layer actual data and mask (i.e., the bottom layer vector register and the bottom layer mask register), thereby improving the utilization efficiency of the bottom layer register.

[0053] S2, converting the upper layer logical data into an abstract layer data vector representation in a continuous arrangement form located in the 0th channel, and converting the upper layer logical mask into an abstract layer mask vector representation in a continuous arrangement form located in the 0th channel.

[0054] Specifically, converting the upper layer logical data into an abstract vector representation in a continuous arrangement form located in the 0th channel includes:

[0055] First, the element type and the element number of the logical data are extracted, and the lane number and the supportable lane number of the element type on the bottom layer hardware are calculated according to the bit width of the element type and the fixed length of the bottom layer vector register;

[0056] Then, an abstract layer data vector is constructed based on the lane number and the supportable lane number of the element type on the bottom layer hardware, the element type of the abstract layer data vector is consistent with the upper layer logical data, the actual length of the data vector is the calculated lane number, the effective data length of the data vector is the element number of the upper layer logical data, and the channel number is set as needed;

[0057] Finally, the upper layer logical data is embedded into the abstract layer data vector, the logical data arrangement is divided into a continuous arrangement according to the data spacing relationship between different channels, and an abstract layer mask vector representation in a continuous arrangement form located in the 0th channel is obtained, which is used to uniformly describe the register mapping and scheduling constraints on the target hardware. The continuous arrangement means that the data corresponding to two adjacent lanes are distributed in the same channel.

[0058] Furthermore, the element type and number of elements in the logical data are extracted. Based on the bit width of the element type, the number of elements, and the fixed length of the underlying vector register, the number of lanes and channels for each element type on the underlying hardware are calculated, including:

[0059] The NPU architecture requires a fixed length of 2048 bits (256 bytes) for its underlying vector registers. First, the number of lanes for each element type on the underlying hardware can be obtained by dividing the fixed length of the underlying vector register (i.e., the number of bits occupied by the underlying vector register) by the bit width of the element type (i.e., the percentage of bits occupied by the element type). Then, the number of lanes for each element type on the underlying hardware can be divided by the number of logical data elements to obtain the number of channels for each element type in the underlying vector register.

[0060] A lane refers to the logical number of data elements stored in a given location. For example, if the logical data is 64 x f16 data, where xf16 is the bit width of the element type (16 bits), and the number of logical data elements is 64, and the underlying vector register has a fixed length of 2048 bits, then dividing the fixed length of the underlying vector register (i.e., the number of bits occupied by the underlying vector register) by the bit width of the element type: 2048 / 16 = 128, we get the number of lanes for the element type on the underlying hardware as 128. 128 / 64 = 2, therefore the number of channels for the underlying vector register is 2.

[0061] like Figure 3 The diagram illustrates the process of converting upper-level logical data into data vectors at the abstract layer. Given the differences in expressive power and execution mechanisms between upper-level logical instructions and lower-level hardware instructions, a vector type with layout abstraction and channel number abstraction is used to represent the data. During the type conversion process, the upper-level logical data is uniformly mapped to a continuously arranged abstract vector with channel number 0.

[0062] Specifically, the upper-layer logical mask is converted into a mask vector representation of the abstract layer located in channel 0 in a continuous arrangement, including:

[0063] First, extract the length of the upper-layer logical mask. Based on the length of the logical mask and the length of the lower-layer mask register, calculate the number of lanes and channels of the element type in the lower-layer mask register. Construct the mask vector representation of the abstract layer using the mapping method of the actual mask bits of the lower layer representing a logical mask bit. The number of mask bits represented by the mask vector is consistent with the upper-layer logical mask. The effective bit type of the mask vector representation is 1 bit (binary bit), and the channel number is 0.

[0064] Then, the upper-level logical mask is embedded into the mask vector representation, and the upper-level logical mask type is mapped to a mask vector representation with continuous arrangement, channel number 0 and logical mask length of 1 bit, to ensure that the data and mask after type conversion meet the requirements of the underlying instruction execution semantics.

[0065] For the NPU architecture, the length of the underlying mask register in the hardware is also required to be a fixed value. One bit of the mask register corresponds to one byte of data in the data vector register. When the data element type is greater than one byte, the first mask bit is used to represent the entire element's mask. The mask type can be uniquely determined based on the size of the masked data vector element. Specifically, the total number of bits in the underlying mask register is divided by the number of individual mask bits occupied by the logic mask to obtain the total number of lanes that the underlying mask register can hold. Then, the number of lanes is divided by the actual number of mask bits occupied by the logic mask to obtain the number of channels corresponding to the logic mask in the underlying mask register of the hardware. Finally, based on the mask bit spacing relationship between different channels, two adjacent logic mask bits are distributed in the same channel, mapping the upper-layer logic mask to a continuously arranged logic mask type with channel number 0 and a logic mask length of 1 bit.

[0066] like Figure 4 The diagram illustrates the conversion process from the upper-level logic mask to the abstract-level logic mask. Given the differences in expressive power and execution mechanisms between upper-level logic instructions and lower-level hardware instructions, a mask type possessing layout abstraction, channel number abstraction, and logic type attributes is used to represent the logic mask. The upper-level logic mask is uniformly mapped to a logic mask type with continuous arrangement, channel number 0, and a logic mask length of 1 bit, thereby ensuring that the data and mask after type conversion meet the semantic requirements of the lower-level instruction execution.

[0067] S3. Obtain the corresponding underlying hardware instruction capability based on the operation type OP input from the upper layer. If the underlying hardware instruction capability corresponding to the operation type OP does not support the combination of the operation type OP and the operand type, perform a legalization operation on the operation type OP.

[0068] Specifically, the hardware instruction capability table is a static configuration table maintained internally by the compiler, which is essentially a multi-dimensional mapping structure used to record the correspondence between the operation type (OP) supported by the hardware and the combination of operand types. The capability of the underlying hardware instruction is stored in the form of a table in the compiler. When encountering an operation type OP and a combination of operands, the hardware instruction capability table in the compiler is used to look up. If not found, it means that legalization operation is needed. When OP exists in the table but does not support this operand type, insert type conversion instructions to convert the unsupported operand type to a supported type and then back. If OP is not supported, use multiple other OPs to implement the function of this OP. In order to be compatible with the difference between generations of shielding chips, the same series of chips share a capability table, so different mapping needs to be done when converting to the underlying layer.

[0069] S4, the converted abstract data vector type representation and the mask vector representation are legalized according to the target hardware capability, so that the combination of the operation type OP and its corresponding abstract data vector layout and abstract mask layout can be effectively mapped to the instruction form supported by the underlying hardware.

[0070] In the NPU architecture, because the underlying instructions and the upper logical instructions have different requirements for the bit width of data, the format of the mask, the alignment method, the on-the-way processing, and the internal storage layout, etc., the instructions cannot be directly mapped to the hardware, and legalization needs to be done for the underlying instructions, so that the data bit width, the mask format requirement, the alignment method, the on-the-way processing, and the internal storage layout of the IR are completely aligned with the underlying layer.

[0071] For the converted vector type, according to the requirements of the underlying instruction corresponding to the operation type OP for the bit width of data, the format of the mask, the alignment method, the on-the-way processing, and the internal storage layout, etc., the layout type and the channel number are legalized, so as to ensure that the data can meet the strict requirements of the instruction on the format when executed on the hardware;

[0072] For the converted mask type, according to the requirements of the underlying instruction corresponding to the operation type OP for the actual number of occupied bits of the mask, the enabled bit layout, and the internal storage method, etc., the layout type and the channel number are legalized, so as to ensure the effectiveness and correctness of the mask in the operation process.

[0073] S41, according to the arrangement requirements of the abstract instruction of the abstract layer for the data vector representation, the arrangement type, the channel number and other properties are legalized and adjusted, and the corresponding abstract instruction is used to adjust the data vector representation to meet the data input specification of the underlying instruction in the physical structure.

[0074] Specifically, taking the type conversion class operation as an example, the upper layer logic vector has an instruction of converting 64xf32 to 64xf16, which corresponds to the convert instruction of the underlying hardware. The underlying convert instruction needs 64xf16, i.e. a narrower type, which is arranged in an interval arrangement in the underlying register. Therefore, we need to legalize the continuously arranged data of the 0th channel of 64xf16. The arrangement type legalization manner is implemented by inserting an abstract layout change class op, for example, the function of intlv2planar is to realize the conversion of interval arrangement to continuous arrangement. They will be converted into interleave and deinterleave commands in the underlying layer. The channel number is the same. In the legalization, an abstract tochannel1 is inserted to represent the movement to the 1st channel. In the end, he will also be converted into the corresponding underlying instruction sequence.

[0075] S42, according to the requirements of the mask control of the abstract instruction of the abstract layer, the arrangement type, the valid bit type, and the channel number of the mask vector representation are legalized and adjusted, and the mask is aligned with the data path in the control granularity and structure using the corresponding abstract instruction, satisfying the constraints of the underlying mask register structure and instruction semantics.

[0076] Specifically, taking the type conversion class operation as an example, in the conversion of 64xf16 to 64xf32 type, not only the data type needs to be aligned, but also the mask needs to be aligned with the data. For example, the logical mask is a 64xi1 mask vector, but in the underlying layer, the mask must exist in the form of 256xi1. Among them, he guides the calculation of f16, and he needs to enable one bit every 2 bits. Therefore, the valid bit type is legalized to 2 bits. Then, the mask here is 128xi1. Then, the arrangement type of the mask is legalized. The data type of the upper layer is 64xf16, and the actual data layout is interval arrangement. Therefore, we need to convert the mask type to interval arrangement. Use the corresponding abstract instruction pregplanar2intlv to convert the arrangement of the mask to interval, and at the end of the flow, this abstract instruction will also be converted to the specific underlying instruction sequence with the same function. The channel number legalization is the same.

[0077] S43, the abstract instructions introduced in the legalization adjustment process are folded and eliminated or redundantly combined to obtain the legalized abstract instructions. The operations with equivalent functions and redundant calculation paths are combined, thereby reducing the number of instructions, improving the scheduling efficiency, and reducing the code generation complexity.

[0078] Specifically, for the abstract instructions introduced in the legalization adjustment process, through the introduction of planar2intlv and intlv2planar and some channel number conversion operations, these operations are reversible. For example, planar2intlv and intlv2planar can be folded to eliminate. And like continuous channel number conversion instructions, they can be fused into a conversion instruction from start to end to get the final abstract instruction and the basic ability of the underlying hardware to fully align. However, the underlying hardware instructions have more functions such as on-the-fly processing compared with the abstract instructions, so optimization needs to be done again for these hardware characteristics.

[0079] As shown in Figure 5 For the instruction description diagram of the converted abstract layer data vector and the abstract layer logical mask, the legalization processing of the abstract data vector type and the abstract mask type is the key link to ensure the correctness of the processor hardware execution. For the converted abstract data vector and the abstract mask type, according to the requirements of the defined abstract instruction on data bit width, mask format, alignment mode and internal storage layout, the layout type and channel number are legalized to ensure that they meet the format constraints of hardware execution. For the instruction itself, if the defined abstract instruction requires on-the-fly processing parameters, the corresponding no on-the-fly processing parameters need to be completed in the conversion process to ensure the integrity of the abstract instruction behavior and the consistency of the underlying instruction.

[0080] S5, the processed abstract instruction is mapped to the operation type OP according to the underlying hardware instruction mapping, and the target LLVM instruction (LLVM IR Instructions) compatible with the underlying hardware is generated.

[0081] In the compiler architecture, the process of converting intermediate representation (IR) into efficient machine code is the key link between abstract computing model and specific hardware execution by converting and optimizing the processed abstract instructions to operation type (OP) mapping according to the underlying hardware characteristics, generating LLVM instructions compatible with the target hardware. By deeply understanding the corresponding relationship between computing semantics and hardware execution model, the abstract instructions are mapped to hardware compatible LLVM instructions, completing the core task of the compiler backend. LLVM (Low Level Virtual Machine) is a modular and reusable compiler infrastructure that supports the development of compiler frontends, optimizers, and backends. LLVM provides a set of intermediate representations (IR, Intermediate Representation), called LLVM IR, which is the core of the LLVM compilation process, connecting source code and target machine code. In LLVM, "instructions" usually refer to instructions in LLVM IR (Instructions), which are the basic operation units that make up LLVM IR programs. These instructions are similar to instructions in assembly language, but are more advanced, type-safe, and oriented towards compiler optimization.

[0082] S51, obtain the instruction list of the underlying ISA (Instruction Set Architecture, Instruction Set Architecture), establish the mapping relationship between the upper operation type OP and the interface of the underlying hardware of the upper LLVM interface, select the abstract instruction as the target instruction according to the mapping relationship between the upper operation type OP and the interface of the underlying hardware of the upper LLVM interface, and get the legal target instruction. The mapping relationship includes: single mapping, one-to-many mapping and no mapping.

[0083] The ISA is an interface specification between the CPU and software, which defines: a set of executable instructions (such as addition, jump, load and store, etc.), registers (general registers, status registers, etc.), memory addressing mode, data types (such as byte, word, double word, etc.). One-to-many mapping refers to: when the abstract instruction of the current layer has no direct equivalent instruction in the underlying hardware, or needs to be realized by combining multiple underlying hardware instructions, such as the function of data conversion which can be realized by the underlying hardware supported instructions (such as vconvert, vsqueeze, vunsqueeze). For one-to-many mapping, according to the capabilities and characteristics of the underlying instructions, the cost of each underlying instruction to execute the upper OP is calculated, and the target instruction with the minimum cost is selected as the legal target instruction among multiple candidate instructions. The factors considered in the calculation of the cost include the execution period of the instruction, the occupation of the required resources, etc., to ensure that the selected target instruction can maximize the computing efficiency and resource utilization while meeting the functional requirements. Finally, the legalized instruction will be scheduled and executed according to the optimal scheme. One-to-one mapping refers to: the abstract instruction corresponds to a unique underlying instruction. For one-to-one mapping, direct execution translation is performed, the upper OP is directly mapped to the underlying corresponding instruction, and the upper logic is targeted at the corresponding abstract instruction for legalization. No mapping refers to: the abstract instruction has no ready implementation in the underlying instruction. For the case of no mapping, the current process is directly skipped, and no legalization operation is performed. After legalization, the legalized target instruction will participate in scheduling and generation according to the optimal target.

[0084] S52, according to the size relationship between the operation type OP capability and the capability of the underlying instruction, different mappings are made when the legalized target instruction is converted to the underlying layer, and the target LLVM instruction compatible with the underlying hardware is scheduled and generated.

[0085] In order to be compatible, the generational difference of the shielding chip is shielded, and chips of the same series share a capability table, so it is also necessary to make different mappings when converting to the underlying layer. After mapping, the target LLVM instruction compatible with the underlying hardware is handed over to the LLVM compiler back end for back end optimization.

[0086] Among them, the equal capability mapping means that the upper layer OP is consistent with the bottom layer instruction in function. For equal capability mapping, the corresponding instruction can be directly generated and scheduled. For example, the bottom layer provides a native FP32 matrix multiplication instruction, and the OP is defined as a standard FP32 GEMM. When it is detected that the upper layer OP and the bottom layer instruction are equal in capability (such as a standard FP32 convolution corresponding to a hardware native CONV2D instruction), the compiler directly executes instruction scheduling optimization. The degraded mapping means that the upper layer OP is stronger than the bottom layer instruction, and the function needs to be degraded. For degraded mapping, the semantics of the abstract OP is completed by combining multiple bottom layer instructions in the back end. For example, the OP needs to support dynamic convolution of INT4 quantization, but the bottom layer only provides INT8 fixed point convolution instructions. The upgrade mapping means that the upper layer OP is weaker than the bottom layer instruction, and the upper layer requirement is met by extending or combining the bottom layer instruction. For upgrade mapping, the upper layer OP function needs to be simulated by calling the hardware interface with specific parameters according to the hardware capability. For example, the OP is only defined as a simple element-by-element addition, but the bottom layer provides a fusion addition instruction supporting broadcast and mask.

[0087] Since the operation instruction legalized in the previous stage is consistent with the hardware, such instruction does not need additional rewriting or conversion, and can generate target LLVM instructions compatible with the bottom layer hardware in a direct mapping manner and participate in subsequent scheduling and code generation process, to realize semantic correct, structure legal, and performance controllable machine-level instruction output, thereby improving the utilization efficiency of the bottom layer register.

[0088] This paper proposes a code compilation time data layout expression form and optimization method for NPU, classifies the "OP-instruction" capability mapping relationship system (equal capability mapping, degraded mapping, and upgrade mapping) by constructing the "OP-instruction" capability mapping relationship system, designs differential processing strategies for different mapping types, and finally realizes efficient compilation combined with the LLVM back end. This paper proposes a new type of code compilation time data layout expression form and optimization method, which realizes accurate conversion from operator definition to hardware instruction by explicitly modeling the "OP-instruction" capability relationship and classifying processing, and finally generates high-performance executable code through the LLVM back end. From the logical instruction, logical data, and logical mask of the upper layer, the abstract instruction, abstract mask, and abstract data of the intermediate abstraction layer are abstracted, the arrangement type, element type, actual length, and channel number of the data are legalized, so that they can completely correspond to the bottom layer instruction and the requirements of the corresponding instruction to the format and content of the bottom layer vector register and bottom layer mask register, thereby improving the utilization efficiency of the bottom layer register.

[0089] Embodiment 2:

[0090] The embodiment provides a computer device, which can be a server, a computer or the like, comprising a processor, a memory, an input device, a display and a network interface connected through a system bus, the processor is used for providing computing and control capability, the memory comprises a non-volatile storage medium and an internal memory, the non-volatile storage medium stores an operating system, a computer program and a database, the internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium, and the processor implements the data layout optimization method applied to NPU code compilation of the above-mentioned embodiment 1 when executing the computer program stored in the memory.

[0091] Embodiment 3

[0092] The embodiment provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the program is executed by a processor, and the processor implements the data layout optimization method applied to NPU code compilation of the above-mentioned embodiment 1 when executing the computer program stored in the memory.

[0093] The above-mentioned embodiment is a preferred embodiment of the present application, but the embodiment of the present application is not limited by the above-mentioned embodiment, and any change, modification, replacement, combination, simplification made without departing from the spirit and principle of the present application should be an equivalent replacement mode, and all should be included in the protection scope of the present application.

Claims

1. A data layout optimization method applied to NPU code compilation, characterized in that, The method comprises the following steps: S1, splitting the intermediate representation IR input from the upper layer into an operation type OP and an operand, and dividing the operand input from the upper layer into logical data and a logical mask; S2, converting the logical data from the upper layer into a data vector representation in a continuous arrangement form in the 0th channel of the abstract layer, and converting the logical mask from the upper layer into a mask vector representation in a continuous arrangement form in the 0th channel of the abstract layer; S3, obtaining the corresponding bottom-layer hardware instruction capability according to the operation type OP input from the upper layer, and performing a legalization operation on the operation type OP when the bottom-layer hardware instruction capability corresponding to the operation type OP does not support the combination of the OP and the type; S4, performing a legalization processing on the converted abstract data vector type representation and the mask vector representation according to the target hardware capability, so that the combination of the operation type OP and the abstract data vector layout and the abstract mask layout corresponding to the operation type OP can be effectively mapped to the instruction form supported by the bottom-layer hardware; S5, performing an instruction mapping on the processed abstract instruction according to the operation type OP of the bottom-layer hardware, and generating a target LLVM instruction compatible with the bottom-layer hardware.

2. The data layout optimization method for NPU code compilation of claim 1, wherein, In the step S2, the logical data from the upper layer is converted into a data vector representation in a continuous arrangement form in the 0th channel of the abstract layer, comprising: extracting the element type and the element number of the logical data, and calculating the lane number and the channel number of the element type on the bottom-layer hardware according to the bit width of the element type and the fixed length of the bottom-layer vector register; constructing an abstract-layer data vector based on the lane number and the channel number of the element type on the bottom-layer hardware, the element type of the abstract-layer data vector being consistent with the logical data from the upper layer, the actual length of the data vector being the calculated lane number, and the effective data length of the data vector being the element number of the logical data from the upper layer; embedding the logical data from the upper layer into the abstract-layer data vector, and dividing the logical data arrangement into a continuous arrangement according to the data spacing relationship between different channels, to obtain a mask vector representation in a continuous arrangement form in the 0th channel of the abstract layer.

3. The data layout optimization method for NPU code compilation of claim 2, wherein, The extraction of the element type and the element number of the logical data, and the calculation of the lane number and the channel number of the element type on the bottom-layer hardware according to the bit width of the element type, the element number, and the fixed length of the bottom-layer vector register, comprises: dividing the lane number of the element type on the bottom-layer hardware by the element number of the logical data to obtain the channel number of the element type in the bottom-layer vector register.

4. The data layout optimization method for NPU code compilation of claim 2, wherein, In the step S2, the logical mask from the upper layer is converted into a mask vector representation in a continuous arrangement form in the 0th channel of the abstract layer, comprising: The length of the upper layer logical mask is extracted, the number of lanes and the number of channels of the element type in the bottom layer mask register are calculated according to the length of the logical mask and the length of the mask register of the bottom layer, the mapping mode of the actual mask bit of the bottom layer is used to construct the mask vector representation of the abstract layer, the number of mask bits of the mask vector representation is consistent with the logical mask of the upper layer, the valid bit type of the mask vector representation is 1bit type, and the channel number is 0; Then, the logical mask of the upper layer is embedded into the mask vector representation, and the logical mask type of the upper layer is mapped into the mask vector representation which is arranged continuously, the channel number is 0, and the logical mask length is 1bit.

5. The data layout optimization method for NPU code compilation of claim 4, wherein, The number of lanes and the number of channels of the element type in the bottom layer mask register are calculated according to the length of the logical mask and the length of the mask register of the bottom layer, including: The total number of lanes that can be accommodated in the bottom layer mask register is obtained by dividing the total number of bits of the bottom layer mask register by the number of single mask bits occupied by the logical mask; the number of channels corresponding to the logical mask in the bottom layer mask register is obtained by dividing the total number of lanes that can be accommodated in the bottom layer mask register by the actual number of mask bits occupied by the logical mask.

6. The data layout optimization method for NPU code compilation of claim 1, wherein, In step S4, the converted abstract data vector type representation and the mask vector representation are legalized according to the target hardware capability, including: According to the arrangement requirement of the abstract instruction of the abstract layer on the data vector representation, the arrangement type and the channel number thereof are legalized and adjusted, and the corresponding abstract instruction is used to make the adjusted data vector representation conform to the data input specification of the bottom layer instruction in the physical structure; According to the requirement of the abstract instruction of the abstract layer on the mask control, the arrangement type, the valid bit type and the channel number of the mask vector representation are legalized and adjusted, and the corresponding abstract instruction is used to make the mask control granularity and structure align with the data path; The abstract instructions introduced in the legalization adjustment process are folded and eliminated or redundantly combined to obtain the legalized abstract instructions.

7. The data layout optimization method for NPU code compilation of claim 1, wherein, In step S5, the processed abstract instructions are mapped to the operation type OP according to the bottom layer hardware to generate target LLVM instructions compatible with the bottom layer hardware, including: An instruction list of the bottom layer ISA is obtained, the mapping relationship between the operation type OP of the upper layer and the interface of the bottom layer of the hardware is established by providing the LLVM interface of the bottom layer to the upper layer, the abstract instruction is selected as the target instruction for legalization according to the mapping relationship between the operation type OP of the upper layer and the interface of the bottom layer of the hardware, and the legalized target instruction is obtained; According to the size relationship between the operation type OP capability and the capability of the bottom layer instruction, different mappings are made based on the legalized target instruction when converting to the bottom layer to schedule the bottom layer instruction to generate target LLVM instructions compatible with the bottom layer hardware.

8. A computer device comprising a processor and a memory for storing a processor executable program, characterized in that, The processor executes the program stored in the memory to implement the data layout optimization method for NPU code compilation in any one of claims 1-7.

9. A storage medium storing a program, characterized by comprising: The program is executed by the processor to implement the data layout optimization method for NPU code compilation in any one of claims 1-7.

Images