Computer-implemented method for generating instructions, and instruction execution method and device

Abstract

The embodiment of the invention discloses a computer implementation method for generating an instruction, and an instruction execution method and equipment. The method comprises the following steps: determining a first intermediate representation corresponding to a neural network model to be compiled; determining a first input tensor of a first operator in the first intermediate representation, a first data arrangement mode of the first input tensor, and a preset data arrangement mode supported by a calculation unit for executing the first operator; determining a data arrangement conversion operation sequence based on the first input tensor, the first data arrangement mode and a preset data arrangement mode; generating an arrangement conversion instruction of the first input tensor based on the data arrangement conversion operation sequence; the arrangement conversion instruction is used for converting the first input tensor into a second input tensor stored according to a preset data arrangement mode; and determining a target instruction sequence of the to-be-compiled neural network model based on the arrangement conversion instruction and a calculation instruction corresponding to the first operator. Therefore, the data arrangement conversion efficiency can be improved.

Description

Technical Field

[0001] This disclosure relates to compilation technology, and in particular to a computer implementation method for generating instructions, an instruction execution method, and an apparatus. Background Technology

[0002] In chip design for neural network model inference, data arrangement is an important design issue that requires comprehensive consideration of hardware architecture, computing mode, memory access mode and parallel computing characteristics. Different computing tasks may have different requirements for data storage and access. Therefore, it is necessary to determine the appropriate data arrangement method for specific tasks.

[0003] In related technologies, during data layout transformation, each transformation operation generates a layout transformation instruction. Since the chip executing the model instructions supports a limited number of data layout methods, multiple layout transformation instructions are often required to convert the tensor to be transformed into a data layout method supported by the computing unit. Multiple layout transformation instructions require multiple configurations, and each data layout transformation requires changing the data structure in the memory, resulting in multiple data interactions and data transformations with the memory, as well as causing model inference delays, leading to low model inference efficiency. Summary of the Invention

[0004] The embodiments of this disclosure provide a computer implementation method, instruction execution method, and apparatus for generating instructions, thereby reducing the number of data interactions between the computing unit and external memory (hereinafter referred to as external memory), reducing data conversion consumption, improving data arrangement and conversion efficiency, and thus improving model inference efficiency.

[0005] A first aspect of this disclosure provides a computer-implemented method for generating instructions, comprising: determining a first intermediate representation corresponding to a neural network model to be compiled; determining a first input tensor of a first operator in the first intermediate representation, a first data arrangement of the first input tensor, and a preset data arrangement supported by a computing unit executing the first operator; determining a data arrangement transformation operation sequence based on the first input tensor, the first data arrangement, and the preset data arrangement, the data arrangement transformation operation sequence including multiple transformation operations; generating an arrangement transformation instruction for the first input tensor based on the data arrangement transformation operation sequence; the arrangement transformation instruction being used to convert the first input tensor into a second input tensor stored according to the preset data arrangement; and determining a target instruction sequence for the neural network model to be compiled based on the arrangement transformation instruction and a computing instruction corresponding to the first operator.

[0006] A second aspect of this disclosure provides an instruction execution method, comprising: acquiring a target instruction sequence of a pre-configured neural network model; the target instruction sequence being obtained based on a computer implementation method for generating instructions as described in any of the above embodiments; and executing the instructions in the target instruction sequence to obtain an instruction execution result.

[0007] A third aspect of this disclosure provides a computer implementation apparatus for generating instructions, comprising: a processor configured to: determine a first intermediate representation corresponding to a neural network model to be compiled; determine a first input tensor of a first operator in the first intermediate representation, a first data arrangement of the first input tensor, and a preset data arrangement supported by a computing unit executing the first operator; determine a data arrangement transformation operation sequence based on the first input tensor, the first data arrangement, and the preset data arrangement, the data arrangement transformation operation sequence including at least one transformation operation; generate an arrangement transformation instruction for the first input tensor based on the data arrangement transformation operation sequence; the arrangement transformation instruction is used to convert the first input tensor into a second input tensor stored according to the preset data arrangement; and determine a target instruction sequence for the neural network model to be compiled based on the arrangement transformation instruction and a computing instruction corresponding to the first operator.

[0008] A fourth aspect of this disclosure provides a chip, comprising: a memory configured to store a target instruction sequence of a neural network model, the target instruction sequence being obtained based on a computer implementation method for generating instructions as described in any of the preceding embodiments; a controller configured to retrieve the target instruction sequence from the memory and control a computing unit to execute the instructions in the target instruction sequence to obtain an instruction execution result; and a computing unit configured to execute the instructions in the target instruction sequence based on a control signal from the controller to obtain an instruction execution result.

[0009] A fifth aspect of this disclosure provides a computer-readable storage medium storing a computer program that is executed by a processor to perform a computer implementation method for generating instructions or an instruction execution method as described in any of the foregoing embodiments of this disclosure.

[0010] A sixth aspect of this disclosure provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory, the processor executing the executable instructions to implement the computer implementation method for generating instructions or the instruction execution method described in any of the foregoing embodiments of this disclosure.

[0011] A fifth aspect of this disclosure provides a computer program product that, when instructions in the computer program product are executed by a processor, performs a computer implementation method for generating instructions provided in any of the above embodiments of this disclosure or an instruction execution method described in any of the above embodiments.

[0012] The computer implementation method, instruction execution method, and apparatus for generating instructions provided in the above embodiments of this disclosure, after determining the first intermediate representation corresponding to the neural network model to be compiled, can determine the first input tensor of the first operator in the first intermediate representation, the first data arrangement of the first input tensor, and the preset data arrangement supported by the computing unit executing the first operator. Based on the first input tensor, the first data arrangement, and the preset data arrangement, a data arrangement transformation operation sequence is determined. Based on the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated. Based on the arrangement transformation instruction and the computing instruction corresponding to the first operator, a target instruction sequence for the neural network model to be compiled is determined. Since the arrangement transformation instruction in the target instruction sequence can integrate multiple transformation operations of the data arrangement transformation operation sequence, multiple transformation operations can be achieved by configuring the arrangement transformation instruction once during instruction execution, so as to convert the first input tensor into a second input tensor stored according to the preset data arrangement supported by the computing unit. This reduces the number of configurations and data interactions with the memory during the transformation process, thereby improving the data arrangement transformation efficiency and thus helping to improve the execution efficiency of the target instruction sequence of the model. Attached Figure Description

[0013] Figure 1 This is an exemplary application scenario of the computer implementation method for generating instructions provided in this disclosure;

[0014] Figure 2 This is a flowchart illustrating a computer implementation method for generating instructions provided in an exemplary embodiment of this disclosure;

[0015] Figure 3 This is a flowchart illustrating a computer implementation method for generating instructions provided in another exemplary embodiment of this disclosure;

[0016] Figure 4 This is a flowchart illustrating a computer implementation method for generating instructions provided in yet another exemplary embodiment of this disclosure;

[0017] Figure 5 This is a flowchart illustrating the generation of layout conversion instructions provided in yet another exemplary embodiment of this disclosure;

[0018] Figure 6 This is a flowchart illustrating an exemplary embodiment of the present disclosure of an instruction execution method;

[0019] Figure 7 This is a flowchart illustrating an instruction execution method provided in another exemplary embodiment of this disclosure;

[0020] Figure 8 This is a schematic diagram of the structure of a computer implementation apparatus for generating instructions provided in an exemplary embodiment of the present disclosure;

[0021] Figure 9 This is a schematic diagram of the structure of a computer implementation apparatus for generating instructions provided in another exemplary embodiment of this disclosure;

[0022] Figure 10 This is a schematic diagram of the structure of a chip provided in an exemplary embodiment of this disclosure;

[0023] Figure 11 This is a structural diagram of an electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0024] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0025] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0026] This disclosure outlines

[0027] In the process of realizing this disclosure, the inventors discovered that in the process of generating instructions corresponding to a neural network model, it is necessary to perform data arrangement transformation on the data in the neural network model according to the data arrangement method supported by the chip executing the model instructions. During the data arrangement transformation process, since the data arrangement method supported by the chip executing the model instructions is limited, multiple transformation operations are often required to convert the tensor to be converted into the data arrangement method supported by the computing unit. Each transformation operation corresponds to generating an arrangement transformation instruction. When executing the instruction sequence of the model, multiple arrangement transformation instructions need to be configured multiple times, and each data arrangement transformation requires changing the data structure in the memory. For example, each time an arrangement transformation instruction is executed, the tensor to be converted needs to be read from the memory. After the arrangement transformation instruction is executed, the transformed tensor needs to be written to the memory. When executing the next arrangement transformation instruction, it is also necessary to read the transformed tensor of the previous instruction from the memory. Thus, during the transformation process, multiple data interactions and data transformation costs with the memory are required, as well as model inference delays, resulting in low model inference efficiency.

[0028] Exemplary Overview

[0029] Figure 1 This is an exemplary application scenario of the computer implementation method for generating instructions provided in this disclosure. For example... Figure 1 As shown, the neural network model to be compiled can be any neural network model to be deployed in chip 10. The neural network model to be compiled can be, but is not limited to, one or more of the following: object detection model, object tracking model, semantic segmentation model, trajectory prediction model, and speech recognition model. The computer implementation device 11 for generating instructions can execute the computer implementation method for generating instructions disclosed herein to obtain the target instruction sequence of the neural network model to be compiled, and then deploy the target instruction sequence into chip 10 and store the target instruction sequence in the memory of chip 10. Chip 10 is, for example, but not limited to, a chip in an in-vehicle terminal, mobile phone, smart wearable device, robot, etc. During operation, when chip 10 acquires input data for the input neural network model, it can read the target instruction sequence from the memory, and based on the input data, obtain the instruction execution result corresponding to the input data by executing the instructions in the target instruction sequence. Input data includes, for example, but not limited to, one or more of the following: image data, text data, voice data, and point cloud data. For example, in an intelligent driving scenario, chip 10 can obtain environmental information around the vehicle by executing a target instruction sequence based on acquired image data and / or point cloud data, and use the environmental information for planning and controlling the vehicle's driving path. Specifically, the computer implementation device 11 for generating instructions can determine the first intermediate representation corresponding to the neural network model to be compiled; determine the first input tensor of the first operator in the first intermediate representation, the first data arrangement of the first input tensor, and the preset data arrangement supported by the computing unit executing the first operator; determine a data arrangement transformation operation sequence based on the first input tensor, the first data arrangement, and the preset data arrangement; generate an arrangement transformation instruction for the first input tensor based on the data arrangement transformation operation sequence; the arrangement transformation instruction is used to convert the first input tensor into a second input tensor stored according to the preset data arrangement; and determine the target instruction sequence for the neural network model to be compiled based on the arrangement transformation instruction and the computation instruction corresponding to the first operator. Since the layout transformation instructions in the target instruction sequence can integrate multiple transformation operations of the data layout transformation operation sequence, multiple transformation operations can be achieved by configuring the layout transformation instructions once during instruction execution. This converts the first input tensor into a second input tensor stored according to the preset data layout supported by the computing unit, reducing the number of configurations and data interactions with the memory during the transformation process. This improves the data layout transformation efficiency, which in turn helps to improve the execution efficiency of the target instruction sequence of the model, and further improves the inference speed of the chip for the neural network model.

[0030] Exemplary methods

[0031] Figure 2 This is a schematic flowchart illustrating a computer implementation method for generating instructions provided in an exemplary embodiment of this disclosure. The computer implementation method for generating instructions provided in this embodiment can be applied to electronic devices, which include, but are not limited to, servers and terminal devices. Figure 2 As shown, the computer implementation method for generating instructions according to embodiments of this disclosure may include the following steps:

[0032] Step 210: Determine the first intermediate representation corresponding to the neural network model to be compiled.

[0033] The intermediate representation (IR) is a computational graph representation that lies between the neural network model and the executable instructions. The first intermediate representation is the intermediate representation corresponding to the neural network model to be compiled.

[0034] Step 220: Determine the first input tensor of the first operator in the first intermediate representation, the first data arrangement of the first input tensor, and the preset data arrangement supported by the computation unit executing the first operator.

[0035] Here, the first operator can be any operator in the first intermediate representation. The first input tensor of the first operator is the input tensor to be calculated by the first operator. The first data arrangement of the first input tensor is the data arrangement of the first input tensor in memory. The data arrangement is to divide the data into smaller data blocks and store the data blocks in memory.

[0036] For example, the native arrangement of an N×H×W×C tensor in memory is represented as follows:

[0037] {T 0,0,0,0 ,T 0,0,0,1 …,T 0,0,0,C-1 ,T 0,0,1,0 ,T 0,0,1,1 …,T N-1,H-1,W-1,C-1 Equation (1)

[0038] Data layout involves dividing the aforementioned data lines into smaller data blocks. The size of each data block is called blocksize (BS), which can be represented as p*x*y*z, and denoted by T. pn,xh,yw,zc This represents a Tensor with data arranged as pn, xh, yw, zc, and its memory layout as follows:

[0039] {B 0,0,0,0 B 0,0,0,1 ,…,B 0,0,0,cbc-1 B 0,0,1,0 ,…,B nbc-1,hbc-1,wbc-1,cbc-1 Equation (2)

[0040] Where nbc = ceil(N / p), hbc = ceil(H / x), wbc = ceil(W / y), cbc = ceil(C / z), and ceil() represents rounding up. Data block B... i,j,k,l It is expressed as follows:

[0041] B i,j,k,l ={T bn,bh,bw,bc ,T bn,bh,bw,bc+1 ,…,T bn,bh,bw,bc+(z-1) ,T bn,bh,bw+1,bc ,…,T bn+(p-1),bh+(x-1),bw+(y-1),bc+(z-1) Equation (3)

[0042] Where i = 0, 1, ..., nbc, j = 0, 1, ..., hbc, k = 0, 1, ..., wbc, l = 0, 1, ..., cbc, bn = (i-1)*p, bh = (j-1)*x, bw = (k-1)*y, bc = (l-1)*z.

[0043] When bn+(p-1)≥N or bh+(x-1)≥H or bw+(y-1)≥W or bc+(z-1)>=C, then T bn+(p-1),bh+(x-1),bw+(y-1),bc+(z-1) The default padding is 0. A data block can include one or more data groups. For example, a data block of size 8w16c represents 8 data groups along the w direction, and each data group includes 16 data points along the c direction.

[0044] The computing unit executing the first operator is the computing unit in the terminal device or chip where the model is to be deployed, used to execute the first operator. The computing unit includes, but is not limited to, one or more of the following: Central Processing Unit (CPU), Neural Network Processing Unit (NPU), Graphics Processing Unit (GPU), Image Signal Processor (ISP), Accelerator Control Unit (ACU), Vector Processing Unit (VPU), Scalar Processing Unit (SPU), and Digital Signal Processor (DSP). The preset data arrangement supported by the computing unit refers to the data arrangement of the input tensors supported by the computing unit when executing the first operator. The first data arrangement of the first input tensor in the first intermediate representation may differ from the preset data arrangement supported by the computing unit. In this case, the first data arrangement of the first input tensor needs to be transformed to adapt to the computing unit. The preset data arrangement supported by the computing unit can be determined and stored in advance based on the actual situation of the computing unit.

[0045] Optionally, for each operator in the first intermediate representation, based on the data arrangement of the operator's input tensor and the preset data arrangement supported by the computation unit executing the operator, the operator that needs to undergo data arrangement transformation can be determined and designated as the first operator. For operators that do not require data arrangement transformation, the original data arrangement of the operator's input tensor can be maintained.

[0046] Step 230: Determine the data arrangement transformation operation sequence based on the first input tensor, the first data arrangement method, and the preset data arrangement method.

[0047] The data arrangement transformation operation sequence includes multiple transformation operations. By applying multiple transformation operations in the data arrangement transformation operation sequence to the first data arrangement of the first input tensor, the first input tensor can be transformed into a second input tensor stored according to a preset data arrangement.

[0048] In some optional embodiments, the data layout transformation operation sequence may include one or more types of transformation operations, and each type of transformation operation may include one or more transformation operations. The types of transformation operations include, but are not limited to, one or more of the following: data layout transformation operation types between data blocks and data groups, data layout transformation operation types between data groups, data layout transformation operation types between a data group and data within the group, and data layout transformation operation types within a data group. The number and type of transformation operations included in the data layout transformation operation sequence are determined by the actual situation of the first data layout method and the preset data layout method of the first input tensor. Different first data layout methods and preset data layout methods result in different data layout transformation operation sequences. A data block is the smallest logical unit for data storage. A data group is a grouping of data within a data block. For example, a data block size of 8w16c represents 8 data groups along the w direction, and each data group includes 16 data items along the c direction.

[0049] For example, the data arrangement transformation operation sequence includes two transformation operations, Op1 and Op2. Op1 is 8w16c-2h4w16c, and Op2 is 2h4w16c-4w2h16c. Op1 is used to convert a tensor arranged in 8w16c data blocks into a tensor arranged in 2h4w16c data blocks. Op2 is used to convert a tensor arranged in 2h4w16c data blocks into a tensor arranged in 4w2h16c data blocks. Thus, through these two transformation operations, a first input tensor arranged in 8w16c data blocks (i.e., the first data arrangement method) can be converted into a second input tensor arranged in 4w2h16c data blocks (i.e., the preset data arrangement method). 2h4w16c indicates that the data block includes 2*4 data groups, each data group includes 16 data points along the c direction. Within the data block, the data groups are first arranged along the w direction, and then along the h direction. 4w2h16c indicates that the data block consists of 4*2 data groups, and each data group contains 16 data points along the c direction.

[0050] In some optional embodiments, a preset transformation path determination algorithm can be used to determine the data arrangement transformation operation sequence based on the first input tensor, the first data arrangement, and a preset data arrangement. The transformation path determination algorithm includes, but is not limited to, shortest path search algorithms or other path search algorithms.

[0051] Step 240: Based on the data arrangement transformation operation sequence, generate the arrangement transformation instruction for the first input tensor.

[0052] The arrangement conversion instruction is used to convert the first input tensor into a second input tensor stored according to a preset data arrangement. Since the data arrangement conversion operation sequence includes multiple conversion operations to convert the first input tensor from a first data arrangement to a preset data arrangement, the sequence is converted into executable instructions according to the conversion relationship from intermediate representation to executable instructions. This generates the arrangement conversion instruction for the first input tensor. When the terminal device or chip executes this instruction, it triggers multiple conversion operations on the first input tensor, converting it into a second input tensor that adapts to the preset data arrangement of the computing unit. Therefore, the data arrangement conversion of the first input tensor can be completed with a single configuration using the arrangement conversion instruction, reducing the number of configuration steps during the conversion process. Furthermore, the conversion process does not require data interaction with external memory, further reducing the number of interactions with external memory and effectively improving the efficiency of data arrangement conversion.

[0053] Step 250: Determine the target instruction sequence of the neural network model to be compiled based on the arrangement conversion instruction and the calculation instruction corresponding to the first operator.

[0054] The calculation instruction corresponding to the first operator is the executable instruction corresponding to the first operator. This calculation instruction is used to perform the calculation corresponding to the first operator on the second input tensor after the arrangement conversion instruction, so as to obtain the calculation result corresponding to the first operator.

[0055] In some optional embodiments, for each first operator in the first intermediate representation that requires data arrangement transformation, the arrangement transformation instruction of the first input tensor of the first operator and the corresponding calculation instruction of the first operator can be generated according to the above steps. For the second operator in the first intermediate representation that does not require data arrangement transformation, there is no need to generate the arrangement transformation instruction of the input tensor of the second operator, or the arrangement transformation instruction of the input tensor of the second operator is empty. The target instruction sequence of the neural network model to be compiled can be determined based on the arrangement transformation instruction of the first input tensor of the first operator, the calculation instruction corresponding to the first operator, and the calculation instruction corresponding to the second operator. Optionally, the arrangement transformation instruction corresponding to the first operator, the calculation instruction of the first operator, and the calculation instruction of the second operator can form the target instruction sequence according to the dependency relationship between the first operator and the second operator.

[0056] The computer implementation method for generating instructions provided in this embodiment, after determining the first intermediate representation corresponding to the neural network model to be compiled, can determine the first input tensor of the first operator in the first intermediate representation, the first data arrangement of the first input tensor, and the preset data arrangement supported by the computing unit executing the first operator. Based on the first input tensor, the first data arrangement, and the preset data arrangement, a data arrangement transformation operation sequence is determined. Based on the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated. Based on the arrangement transformation instruction and the computing instruction corresponding to the first operator, a target instruction sequence for the neural network model to be compiled is determined. Since the arrangement transformation instruction in the target instruction sequence can integrate multiple transformation operations of the data arrangement transformation operation sequence, multiple transformation operations can be achieved by configuring the arrangement transformation instruction once during instruction execution, so as to convert the first input tensor into a second input tensor stored according to the preset data arrangement supported by the computing unit. This reduces the number of configurations and data interactions with the memory during the transformation process, thereby improving the data arrangement transformation efficiency, and thus helping to improve the execution efficiency of the target instruction sequence of the model.

[0057] Figure 3 This is a flowchart illustrating a computer implementation method for generating instructions, provided in another exemplary embodiment of this disclosure.

[0058] In some alternative embodiments, in the above... Figure 2 Based on the illustrated embodiments, as Figure 3 As shown, step 230, which determines the data arrangement transformation operation sequence based on the first input tensor, the first data arrangement method, and the preset data arrangement method, may include:

[0059] Step 2310: Based on the size of the first input tensor and the first data arrangement, determine the initial arrangement of the data arrangement transformation operation sequence.

[0060] The size of the first input tensor includes the data dimensions of the first input tensor and the size of each data dimension. For example, the first input tensor is a four-dimensional tensor, represented as an N×H×W×C tensor. N, H, W, and C are all positive integers, representing the size of each dimension respectively. N represents the batch size, H represents the height of the tensor, W represents the width of the tensor, and C represents the depth (or number of channels) of the tensor. The first data arrangement method is the data arrangement method of the first input tensor in memory. The first data arrangement method is, for example, but not limited to, the data line arrangement method as shown in equation (1) above or the data block arrangement method as shown in equation (2) or (3) above.

[0061] Optionally, the size of the first input tensor and the first data arrangement can be determined as the initial arrangement of the data arrangement transformation operation sequence. Based on the size of the first input tensor and the first data arrangement, the number of data blocks arranged in the first data arrangement and the arrangement order of the data blocks can be determined. For example, the first input tensor is an H×W×C tensor, the first data arrangement is arranged in 8w16c data blocks, the first input tensor includes H*(w / 8)*(c / 16) data blocks, the initial arrangement includes H×W×C and 8w16c, and the size of the first input tensor affects the convertible dimension of the transformation operation. For example, a certain transformation operation needs to convert a tensor arranged in 8w16c to a tensor arranged in 2h4w16c. Combining the data arrangement transformation principle of the above equations (1)-(3), it can be determined that it is equivalent to exchanging the data of the 2h dimension and the data of the 2w1 dimension in the tensor (2h, [2w1, 4w0, (16c)]) to obtain the tensor (2w1, [2h, 4w0, (16c)]). (2h, [2w1, 4w0, (16c)]) is determined according to 8w16c and 2h4w16c. Since it is necessary to change the data block from 8w16c to 2h4w16c, it involves the transformation of the h dimension and w dimension in the tensor. Therefore, (2h, [2w1, 4w0, (16c)]) is determined according to 2h and [8w, (16c)] in 2h4w16c. [2w1,4w0,(16c)] represents splitting the 8w dimension of 8w16c into {2w1,4w0}. w1 and w0 are only used to distinguish the two dimensions of the 8w split. Since the dimension order of w and c has not changed, the actual storage order of the tensor arranged in [2w1,4w0,(16c)] and the tensor arranged in 8w16c in memory has not changed. The transformation process from (2h,[2w1,4w0,(16c)]) to (2w1,[2h,4w0,(16c)]) involves data exchange between multiple data blocks (i.e., 2 data blocks in the h direction) and multiple data groups (i.e., the data of 2 {4w0,(16c)} in the w1 direction, each {4w0,(16c)} data includes 4 data groups (16c) in the w0 direction). The tensor after the data exchange is the tensor arranged in 2h4w16c.

[0062] Step 2320: Based on the size of the first input tensor and the preset data arrangement method, determine the ending arrangement method of the data arrangement transformation operation sequence.

[0063] Specifically, the size of the first input tensor and the preset data arrangement can be determined as the ending arrangement of the data arrangement transformation operation sequence, so that tensors arranged according to the ending arrangement can be supported by the computing unit.

[0064] The execution order of steps 2310 and 2320 is not important.

[0065] Step 2330: Based on the pre-configured transformation operation sub-sequence, determine multiple transformation paths from the initial layout mode to the final layout mode.

[0066] The transformation operation subsequence includes a combination of various preset transformation operation types. These preset types include, but are not limited to, data arrangement transformation operation types between data blocks and data groups, data arrangement transformation operation types between data groups, data arrangement transformation operation types between data groups and data within a group, and data arrangement transformation operation types within a data group.

[0067] In some optional embodiments, the transformation operation subsequence can be represented as:

[0068] FuseOp={SOp1,SOp2,SOp3,SOp4,SOp3,SOP2,SOp1} Formula (4)

[0069] Among them, SOp i This represents a preset type of transformation operation, where i = 1, 2, 3, 4, with different values ​​of i representing different types of transformation operations. The transformation operation subsequence is set as a symmetrical structure to allow for any combination of the four preset types of transformation operations to achieve the transformation from the starting arrangement to the ending arrangement. Each preset type of transformation operation in the subsequence can be set as a null operation or a valid operation. A null operation indicates that the operation is not used, or skipped, thereby controlling the various combinations of preset types of transformation operations. Through multiple combinations, multiple transformation paths can be determined.

[0070] Optionally, SOP1 = GroupInnerRotate or NULL, SOP2 = GroupExchange or NULL, SOP3 = GroupRotate or NULL, and SOP4 = BlockExchange or NULL. GroupInnerRotate represents a data layout transformation operation within a data group, GroupExchange represents a data layout transformation operation between data groups and data within a group, GroupRotate represents a data layout transformation operation between data groups, and BlockExchange represents a data layout transformation operation between a data block and a data group.

[0071] Optionally, each preset type of transformation operation can include transformation operations with multiple parameters, each parameter corresponding to a specific transformation operation. For example, the GroupInnerRotate transformation operation of SOp1 can include transformation operations with three different parameters (or subtypes): GroupInnerRotate(α1), GroupInnerRotate(α2), and GroupInnerRotate(α3). GroupInnerRotate(α1) is used to convert tensors arranged in 8h2w8c to tensors arranged in 8h8c2w. GroupInnerRotate(α2) is used to convert tensors arranged in 8h4w4c to tensors arranged in 8h4c4w. GroupInnerRotate(α3) is used to convert tensors arranged in 8h8w2c to tensors arranged in 8h2c8w. Parameters α1, α2, and α3 are used to distinguish transformation operations of different subtypes within the same preset type of transformation operation. This is only one example of parameters; in actual applications, the parameters are not limited to those in this example. Each preset type of transformation operation offers a variety of selectable parameters, which can be set according to actual needs and are not limited to the parameters in the examples above. For instance, more different GroupInnerRotate transformation operations with varying parameters can be set to meet different data arrangement transformation requirements within a data group.

[0072] For example, a conversion path is represented as follows:

[0073] {GroupInnerRotate(α1),GroupExchange(β1),NULL,NULL,GroupRotate(γ1),GroupExchange(β2),NULL} Formula (5)

[0074] The transformation path includes combinations of GroupInnerRotate(α1), GroupExchange(β1), GroupRotate(γ1), and GroupExchange(β2). β1 and γ1 are the parameters of the GroupExchange and GroupRotate type transformation operations, respectively. Through the various transformation operations included in the transformation path, the first input tensor is progressively transformed into a second input tensor arranged according to a preset data layout.

[0075] In some optional embodiments, any transformation path may include one or more subsequences of transformation operations. For example, a transformation path may include {FuseOp1, FuseOp2, ..., FuseOp...} n}, where n represents the number of subsequences of transformation operations, and n is a positive integer. FuseOp j ={SOp j1 ,SOp j2 ,SOp j3 ,SOp j4 ,SOp j3 ,SOp j2 ,SOp j1}, j = 1, 2, ..., n, SOp ji See the aforementioned SOp i Different transformation operation subsequences can include different combinations of various preset transformation operations.

[0076] Step 2340: Based on the initial and final layout methods, determine the data layout transformation operation sequence from multiple transformation paths using the shortest path search algorithm.

[0077] Among them, the shortest path search algorithm refers to the algorithm that searches for the path with the fewest transformation operations from the initial layout to the final layout from multiple transformation paths.

[0078] In some optional embodiments, the optimal target transformation path is determined from multiple transformation paths using a shortest path search algorithm, and the multiple transformation operations included in the target transformation path are determined as a data arrangement transformation operation sequence.

[0079] For example, the target transformation path includes {FuseOp1, FuseOp2, ..., FuseOp...} n}, FuseOp j ={SOp j1 ,SOp j2 ,SOp j3 ,SOp j4 ,SOp j3 ,SOp j2 ,SOp j1}, FuseOp j If the target transformation path contains one or more valid transformation operations and the others are null operations (NULL), then all valid transformation operations in the target transformation path can be used to form a data arrangement transformation operation sequence.

[0080] In the embodiments of this disclosure, the starting arrangement of the data arrangement transformation operation sequence is determined based on the size of the first input tensor and the first data arrangement method, and the ending arrangement of the data arrangement transformation operation sequence is determined based on the size of the first input tensor and the preset data arrangement method. The data arrangement transformation operation is transformed into a path search problem from the starting point to the ending point. Then, the optimal data arrangement transformation operation sequence is searched through the shortest path search algorithm, thereby improving the accuracy and effectiveness of the data arrangement transformation operation sequence and further improving the model inference efficiency.

[0081] In some optional embodiments, the transformation operation subsequence includes at least one of the following: data arrangement transformation operation between data blocks and data groups, data arrangement transformation operation between data groups, data arrangement transformation operation between data groups and data within a group, and data arrangement transformation operation within a data group, wherein a data block includes at least two data groups of different dimensions.

[0082] The data arrangement transformation operation between data blocks and data groups represents the data exchange between multiple data blocks and multiple data groups. This operation can be, but is not limited to, represented as `BlockExchange`. The data arrangement transformation operation between data groups represents the data exchange between multiple data groups. This operation can be, but is not limited to, represented as `GroupRotate`. The data arrangement transformation operation between a data group and its data within the group represents the data exchange between data groups and their data within the group. This operation can be, but is not limited to, represented as `GroupExchange`. The data arrangement transformation operation within a data group represents the data exchange within the data group. For example, but not limited to, `GroupInnerRotate`, can be used. These different preset types of data arrangement transformation operations are used to implement data arrangement transformations of different dimensions, enabling the conversion of a tensor with one data arrangement into tensors with multiple different data arrangements through various data arrangement transformation operations. A data block includes at least two data groups. Optionally, the size of the data group can be determined based on the storage space size of the storage unit. For example, if each storage unit can store 16 data items, then the size of the data group is 16. Data block 8w16c includes 8 data groups, each data group includes 16 data items in the c direction. Data block 8h8c2w includes 8 data groups, each data group includes C1×W1=8×2 data items, or 8 sub-data groups, each sub-data group includes 2 data items along the w direction.

[0083] In some optional embodiments, each preset type of conversion operation may include multiple different subtypes of conversion operations.

[0084] In some optional examples, for instance, taking a data block size of 128 bytes and a data block storage method of 8 16-byte data groups, such as data blocks [8w(16c)], [2h4w(16c)], [8w(2h8c)], etc., in [8w(16c)], 16 data in the c direction are treated as one data group, so 8w represents 8 data groups. In [8w(2h8c)], (2h8c) is treated as one data group, so 8w represents 8 data groups. Based on this, the transformation principle of various preset data arrangement conversion operations is explained as follows:

[0085] The first type: Data arrangement transformation operation between data blocks and data groups. BlockExchange can include BlockExchange(θ1), BlockExchange(θ2), BlockExchange(θ3), ..., and some transformation operations are represented as follows:

[0086] (1) BlockExchange(θ1) transforms the corresponding block into 8w16c<->2h4w16c. The transformation principle is: 2h,[2w1,4w0,(16c)]<->2w1,[2h,4w0,(16c)], which means that multiple data blocks (i.e., two data blocks in the h dimension) exchange data with multiple data groups (two {4w0,(16c)} data in the w1 dimension, each {4w0,(16c)} data includes four data groups in the w0 direction).

[0087] (2) BlockExchange(θ2) transforms the corresponding block into 8w16c<->4h2w16c. The transformation principle is: 4h,[4w1,2w0,(16c)]<->4w1,[4h,2w0,(16c)], which means that multiple data blocks (i.e., the four data blocks in the h direction) exchange data with multiple data groups (i.e., the four {2w0,(16c)} data in the w1 direction).

[0088] (3) BlockExchange(θ3) transforms the corresponding block into 8w16c<->8h16c. The transformation principle is: 8h,[8w,(16c)]<->8w,[8h,(16c)], which means that multiple data blocks (i.e., 8 data blocks in the h direction) exchange data with multiple data groups (i.e., 8 data groups in the w direction).

[0089] Here, parameters θ1, θ2, and θ3 are different parameters or subtypes used to identify different transformation operations between data blocks and data groups. For example, θ1, θ2, and θ3 can be, but are not limited to, 2, 4, and 8 respectively. In practical applications, more transformation operations with different parameters can be set according to actual needs, not limited to the three parameter transformation operations mentioned above.

[0090] The second type involves data arrangement transformation operations between data groups. GroupRotate can include GroupRotate(γ1), GroupRotate(γ2), ..., and some transformation operations are represented as follows:

[0091] (1) GroupRotate(γ1) The corresponding block transformation is 2h4w16c<->4w2h16c. The transformation principle is: [2h,4w,(16c)]<->[4w,2h,(16c)], that is, multiple data groups (i.e., the two {4w,(16c)} data in the h direction, each {4w,(16c)} data includes 4 data groups in the w direction) exchange data with multiple data groups (i.e., 4 data groups in the w direction).

[0092] (2) GroupRotate(γ2) The corresponding block transformation is 4h2w16c<->2w4h16c. The transformation principle is: [4h,2w,(16c)]<->[2w,4h,(16c)], that is, multiple data groups (i.e., the four {2w,(16c)} data in the h direction, each {2w,(16c)} data includes two data groups in the w direction) exchange data with multiple data groups (i.e., the two data groups in the w direction).

[0093] Here, parameters γ1 and γ2 are different parameters or subtypes used to identify different transformation operations between data groups. For example, γ1 and γ2 can be, but are not limited to, 2 and 4 respectively. In practical applications, more transformation operations with different parameters can be set according to requirements, not limited to the two parameter operations in the example above.

[0094] The third type is the data arrangement transformation operation between data groups and data within a group. GroupExchange can include GroupExchange(β1), GroupExchange(β2), GroupExchange(β3), ..., and some transformation operations are represented as follows:

[0095] (1) GroupExchange(β1) The corresponding block transformation is 8w16c<->4w2c2w8c. The transformation principle is: [4w1,2w0,(2c1,8c0)]<->[4w1,2c1,(2w0,8c0)], that is, data is exchanged between the data group (2 data groups in the w0 direction) and the data within the group (2 groups in the c1 direction and 8 data in the c0 direction).

[0096] (2) GroupExchange(β2) The corresponding block transformation is 8w16c<->2w4c4w4c. The transformation principle is: [2w1,4w0,(4c1,4c0)]<->[2w1,4c1,(4w0,4c0)], that is, the data group (4 data groups in the w0 direction) exchanges data with the data in the group (i.e., 4 groups of 4c0 in the c1 direction).

[0097] (3) GroupExchange(β3) The corresponding block transformation is 8w16c<->8c8w2c. The transformation principle is: [8w,(8c1,2c0)]<->[8c1,(8w,2c0)], that is, data is exchanged between the data group (8 data groups in the w direction) and the data in the group (8 2c0s in the c1 direction).

[0098] Here, parameters β1, β2, and β3 are different parameters or subtypes used to identify different transformation operations between data groups and data within groups. For example, β1, β2, and β3 can be, but are not limited to, 2, 4, and 8 respectively. In practical applications, more transformation operations with different parameters can be set according to actual needs, not limited to the three parameter transformation operations mentioned above.

[0099] The fourth type of data arrangement transformation operation within a data group, GroupInnerRotate, can include GroupInnerRotate(α1), GroupInnerRotate(α2), GroupInnerRotate(α3), ..., with some transformation operations represented as follows:

[0100] (1) GroupInnerRotate(α1) The corresponding block transformation is 8h2w8c<->8h8c2w. The transformation principle is: [8h,(2w,8c)]<->[8h,(8c,2w)], that is, the data in the data group (i.e., the two 8c in the w direction) and the data in the data group (i.e., the eight data in the c direction) exchange data.

[0101] (2) GroupInnerRotate(α2) The corresponding block transformation is 8h4w4c<->8h4c4w. The transformation principle is: [8h,(4w,4c)]<->[8h,(4c,4w)], that is, the data in the data group (i.e., the four 4c in the w direction) and the data in the data group (i.e., the four data in the c direction) exchange data.

[0102] (3) GroupInnerRotate(α3) The corresponding block transformation is 8h8w2c<->8h2c8w. The transformation principle is: [8h,(8w,2c)]<->[8h,(2c,8w)], that is, the data in the data group (i.e., the 8 2c in the w direction) and the data in the data group (i.e., the 2 data in the c direction) exchange data.

[0103] Here, parameters α1, α2, and α3 are different parameters or subtypes used to identify different transformation operations within the data set. For example, α1, α2, and α3 can be, but are not limited to, 2, 4, and 8 respectively. In practical applications, more transformation operations with different parameters can be set according to actual needs, not limited to the three parameter transformation operations mentioned above.

[0104] In the embodiments of this disclosure, a sub-sequence of conversion operations, incorporating at least one of the following: data arrangement conversion operations between data blocks and data groups, data arrangement conversion operations between data groups, data arrangement conversion operations between data groups and data within a group, and data arrangement conversion operations within a data group, is used to determine the data arrangement conversion operation sequence from the initial arrangement mode to the final arrangement mode. This can flexibly meet the conversion needs from various initial arrangement modes to various final arrangement modes, reducing data copy overhead and improving conversion efficiency while satisfying more diverse data arrangement conversion requirements, thereby improving the completeness and flexibility of data arrangement.

[0105] Figure 4 This is a flowchart illustrating a computer implementation method for generating instructions, provided in yet another exemplary embodiment of this disclosure.

[0106] In some alternative embodiments, based on any of the above embodiments, such as Figure 4 As shown, step 240, which generates the arrangement transformation instruction for the first input tensor based on the data arrangement transformation operation sequence, may include:

[0107] Step 2410: Iterate through each transformation operation in the data arrangement transformation operation sequence one by one, take the currently iterated transformation operation as the current transformation operation, and determine the source data arrangement method of the current transformation operation.

[0108] The data arrangement transformation operation sequence includes multiple ordered transformation operations. Each transformation operation is taken as the current transformation operation being traversed. For the current transformation operation, the source data arrangement method of the current transformation operation is determined. The source data arrangement method represents the data arrangement method of the tensor to be transformed supported by the current transformation operation. The source data arrangement method of each determined transformation operation is preset. For example, the source data arrangement method of BlockExchange(θ1) above is [2w1,4w0,(16c)]. The data arrangement of the input tensor of the current transformation operation may be the same as or different from the data arrangement of the source tensor. For example, if the current transformation operation is the first transformation operation in a data arrangement transformation operation sequence, then the input tensor of the current transformation operation is the first input tensor, but the data arrangement of the first input tensor is the initial arrangement, which may be different from the source data arrangement of the current transformation operation. For example, the data arrangement of the input tensor of BlockExchange(θ1) is 8w16c, and the source data arrangement is [2w1,4w0,(16c)]. If the current transformation operation is not the first transformation operation, then the input tensor of the current transformation operation is the tensor after the previous transformation operation, and the source data arrangement is the data arrangement of the tensor to be transformed supported by the current transformation operation.

[0109] Step 2420: Based on the current input tensor of the current transformation operation, the current transformation operation, and the source data arrangement, generate the intermediate instructions corresponding to the current transformation operation.

[0110] Specifically, when the current transformation operation is the first transformation operation in the data arrangement transformation operation sequence (i.e., the first transformation operation), the current input tensor of the current transformation operation is the first input tensor. When the current transformation operation is not the first transformation operation, the current input tensor of the current transformation operation is the tensor transformed by the previous transformation operation. The intermediate instruction corresponding to the current transformation operation is an instruction in a language format between the first intermediate representation and executable instructions.

[0111] Optionally, the intermediate instructions (or intermediate operators) corresponding to the current transformation operation may include, but are not limited to, one or more of the view and transpose instructions. The view instruction is used to reshape the tensor shape, specifically by sharing stored data; that is, it treats the stored data from one tensor shape to another without changing the data's storage order in memory, and does not require data copying. The transpose instruction is used to swap the dimension order of a tensor, achieving data rearrangement. For example, it can convert an N×C×H×W tensor to an N×H×C×W tensor. By adjusting the dimension order, the tensor's data arrangement continuously changes towards a preset data arrangement. Referring to BlockExchange(θ1) above, the corresponding block transformation 8w16c<->2h4w16c is based on the principle of treating the tensor arranged in 8w16c as a tensor arranged in [2w1,4w0,(16c)]. This treating operation does not incur data conversion overhead. Then, multiple data blocks (i.e., two data blocks in the h dimension) of the tensor arranged in [2w1,4w0,(16c)] interact with multiple data groups (i.e., two {4w0,(16c) data} in the w1 dimension) to achieve a conversion of a data arrangement method.

[0112] For each transformation operation encountered, the intermediate instruction corresponding to the transformation operation can be determined by following the steps described above, thus obtaining the intermediate instruction corresponding to each transformation operation in the data arrangement transformation operation sequence.

[0113] Step 2430: In response to the completion of the traversal of the transformation operations in the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated based on the intermediate instructions corresponding to each transformation operation.

[0114] In this process, once all transformation operations in the data arrangement transformation operation sequence have been traversed, it indicates that the intermediate instructions corresponding to each transformation operation in the data arrangement transformation operation sequence have been obtained. Subsequently, through a compilation process (such as assembly processing), the intermediate instructions corresponding to each transformation operation are compiled into executable instructions, resulting in the arrangement transformation instructions for the first input tensor. The arrangement transformation instructions are executable instructions that integrate multiple transformation operations from the data arrangement transformation operation sequence. After the target instruction sequence is deployed to the terminal device or chip, the terminal device or chip can trigger multiple transformation operations on the first input tensor by executing the arrangement transformation instructions, converting the first input tensor into a second input tensor arranged according to a preset data arrangement method.

[0115] In the embodiments of this disclosure, by traversing each conversion operation in the data arrangement conversion operation sequence, an intermediate instruction corresponding to each conversion operation is generated. Based on the intermediate instructions corresponding to each conversion operation, a first input tensor arrangement conversion instruction is generated. The data arrangement conversion instruction integrates the executable instructions of multiple conversion operations in the data arrangement conversion operation sequence. By configuring the arrangement conversion instruction once, multiple conversion operations on the first input tensor can be triggered, converting the first input tensor into a second input tensor arranged according to a preset data arrangement method to meet the data arrangement requirements of the computing unit. Moreover, the tensor is transformed in memory during the conversion process, without needing to be copied to external storage, reducing the number of accesses to external storage and improving the data arrangement conversion efficiency.

[0116] In some optional embodiments, step 2420, based on the current input tensor of the current transformation operation, the current transformation operation, and the source data arrangement, generates intermediate instructions corresponding to the current transformation operation, which may include:

[0117] Based on the current input tensor and source data arrangement of the current transformation operation, a first intermediate instruction is generated; the first intermediate instruction is used to treat the current input tensor as a first tensor arranged according to the source data arrangement; based on the first tensor and the current transformation operation, a second intermediate instruction is generated; the second intermediate instruction is used to perform data arrangement transformation on the first tensor through the current transformation operation to obtain a second tensor; in response to the current transformation operation not being the last transformation operation in the data arrangement transformation operation sequence, the first and second intermediate instructions are determined as the intermediate instructions corresponding to the current transformation operation; in response to the current transformation operation being the last transformation operation in the data arrangement transformation operation sequence, based on the second tensor and a preset data arrangement, a third intermediate instruction is generated, and the first, second, and third intermediate instructions are determined as the intermediate instructions corresponding to the current transformation operation; the third intermediate instruction is used to treat the second tensor as a second input tensor arranged according to the preset data arrangement.

[0118] The first intermediate instruction is generated based on the data arrangement of the current input tensor and the source data arrangement of the current transformation operation. The data arrangement of the current input tensor may differ from that of the source data. For example, the current input tensor may be represented as T. curDL =T srcDL T srcDL This represents a tensor arranged according to the srcDL data layout method, and the current transformation operation is represented as Op. i Get the current conversion operation Op i Source data arrangement method Op i SrcDL = getSrcDL(Op i ), getSrcDL(Op i ) indicates obtaining Op iA function that describes the arrangement of the source data. It treats the current input tensor as an array arranged according to the source data's Op format. i The first tensor T′ of the ScrcDL arrangement curDL , represented as viewData(T curDL Op i SrcDL)–>T′ curDL The first intermediate instruction is obtained, which is the view instruction, used to treat the current input tensor as the first tensor arranged according to the source data arrangement.

[0119] Applying the current transformation operation to the first tensor converts its data arrangement to the desired arrangement of the current transformation operation, resulting in the second tensor T. curDL , represented as T curDL =applyOp(Op i ,T′ curDL This generates a second intermediate instruction, which instructs the operation on the first tensor T′. curDL Apply the current transformation operation Op iIf the current transformation operation is not the last transformation operation in the data arrangement transformation operation sequence, it means the transformation process is not yet complete. In this case, the first and second intermediate instructions are used as the intermediate instructions corresponding to the current transformation operation. If the current transformation operation is the last transformation operation in the data arrangement transformation operation sequence, it means all transformation operations in the sequence have been traversed. The data arrangement of the tensor after the current transformation operation may differ from the preset data arrangement. Therefore, a third intermediate instruction needs to be generated based on the second tensor and the preset data arrangement to treat the second tensor as a second input tensor arranged according to the preset data arrangement. The third intermediate instruction is a view instruction, similar to the first intermediate instruction. For example, if the data arrangement of the second tensor after the current transformation operation is [2w1,4w0,(4c1,4c0)], and the preset data arrangement is 8w16c, then based on the second tensor and the preset data arrangement, a third intermediate instruction is generated to treat the second tensor arranged in [2w1,4w0,(4c1,4c0)] as the second input tensor arranged in the preset data arrangement 8w16c. The [2w1,4w0,(4c1,4c0)] arrangement is equivalent to splitting the 8w16c arrangement in terms of dimensions, that is, splitting the 16c dimension into the 4c1 dimension and the 4c0 dimension, and splitting the 8w dimension. Divided into 2w1 and 4w0 dimensions, since the order of the dimensions is not changed—for example, splitting a 16c dimension into 4c1 and 4c0 dimensions—is equivalent to treating 16 sequentially stored data [c1,c2,c3,c4,c5,…,c16] as a matrix [c1,c2,c3,c4; c5,c6,c7,c8;…;c13,c14,c15,c16] from a length 16 vector as a matrix of height and width 4 [c1,c2,c3,c4; c5,c6,c7,c8;…;c13,c14,c15,c16]. Therefore, the arrangement order of the tensor in memory is not changed, and no data arrangement conversion overhead is generated. The preset data arrangement can be achieved by changing the reading order. Thus, through multiple conversion operations in the data arrangement conversion operation sequence, the first input tensor is converted into a second input tensor arranged according to the preset data arrangement.

[0120] Optionally, if the data arrangement of the current input tensor is consistent with the data arrangement of the source data in the current transformation operation, then there is no need to generate the first intermediate instruction.

[0121] Optionally, if the data arrangement of the second tensor after the last transformation operation is consistent with the preset data arrangement, then there is no need to generate a third intermediate instruction.

[0122] In the embodiments of this disclosure, a first intermediate instruction is generated based on the current input tensor and the source data arrangement of the current transformation operation. This instruction treats the current input tensor as a first tensor arranged according to the source data arrangement of the current transformation operation, enabling data arrangement transformation using the current transformation operation. A second intermediate instruction is generated based on the first tensor and the current transformation operation. This instruction performs data arrangement transformation on the first tensor using the current transformation operation to obtain a second tensor. When traversing to the last transformation operation, a third intermediate instruction is generated based on the second tensor and a preset data arrangement. This instruction treats the second tensor as a second input tensor arranged according to the preset data arrangement. Thus, by generating arrangement transformation instructions for the first input tensor through the intermediate instructions of multiple transformation operations in the data arrangement transformation operation sequence, it is ensured that the first input tensor can be accurately and effectively converted into a second input tensor arranged according to the preset data arrangement.

[0123] In some optional embodiments, step 2430, which generates the arrangement transformation instruction of the first input tensor based on the intermediate instructions corresponding to each transformation operation, may include:

[0124] For any intermediate instruction corresponding to any transformation operation in each transformation operation, this intermediate instruction is taken as the current intermediate instruction. In response to the current intermediate instruction including a first intermediate instruction and a second intermediate instruction, based on the first intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement corresponding to the current intermediate instruction are taken as the input parameters corresponding to the second intermediate instruction. Based on the input parameters and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. In response to the current intermediate instruction including a first intermediate instruction, a second intermediate instruction, and a third intermediate instruction, based on the first intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement corresponding to the current intermediate instruction are taken as the input parameters corresponding to the second intermediate instruction. Based on the third intermediate instruction, the second tensor and the preset data arrangement are taken as the output parameters corresponding to the second intermediate instruction. Based on the input parameters, the output parameters, and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. Based on the executable instructions corresponding to each intermediate instruction, the arrangement transformation instruction of the first input tensor is determined.

[0125] Specifically, the current input tensor and source data arrangement of the current intermediate instruction are used as input parameters for the second intermediate instruction. This allows the terminal device or chip to use the current input tensor as a first tensor arranged according to the source data arrangement when executing the executable instruction corresponding to the second intermediate instruction. By executing the executable instruction corresponding to the second intermediate instruction, the data arrangement of the current input tensor is transformed to obtain the second tensor. If the current intermediate instruction also includes a third intermediate instruction, the second tensor and a preset data arrangement are used as output parameters for the second intermediate instruction. This allows the terminal device or chip to determine, based on the output parameters, that the output tensor is the second input tensor arranged according to the preset data arrangement when executing the executable instruction corresponding to the second intermediate instruction. Therefore, the first and third intermediate instructions do not perform data arrangement transformations and do not generate executable instructions, eliminating the need for data copying and thus avoiding resource overhead.

[0126] In some optional embodiments, for a first input tensor of a first operator, the parameters of an arrangement transformation instruction can be determined based on the executable instructions or the storage addresses of the executable instructions corresponding to each intermediate instruction, generating an arrangement transformation instruction for the first input tensor. By executing this arrangement transformation instruction, multiple ordered transformation operations on the first input tensor can be triggered based on the parameters of this instruction, outputting a second input tensor arranged according to a preset data arrangement. The second input tensor is then used as the input tensor for the calculation instruction corresponding to the first operator. The calculation unit executes the calculation instruction corresponding to the first operator to complete the calculation of the second input tensor and obtain the calculation result.

[0127] In the embodiments of this disclosure, the first intermediate instruction in each conversion operation does not perform data layout conversion, nor does it need to generate an executable instruction. Instead, based on the first intermediate instruction, the current input tensor and the source data layout are used as input parameters for the second intermediate instruction, and only the second intermediate instruction generates the corresponding executable instruction. Similarly, if the intermediate instruction includes a third intermediate instruction, based on the third intermediate instruction, the second tensor and the preset data layout are used as output parameters for the second intermediate instruction. This allows the executable instruction corresponding to the second intermediate instruction to directly use the current input tensor as the second tensor arranged according to the source data layout, and directly use the output tensor of the last executable instruction as the second input tensor arranged according to the preset data layout. Multiple executable instructions are merged into a layout conversion instruction, thereby completing multiple conversion operations on the first input tensor by executing a single layout conversion instruction, quickly converting the first input tensor into a second input tensor arranged according to the preset data layout, and improving the data layout conversion efficiency.

[0128] In some optional embodiments, the method of this disclosure further includes storing the target instruction sequence in a memory.

[0129] The memory can be, but is not limited to, external storage. The type of external storage can be any type that can be used as external storage.

[0130] In some alternative embodiments, Figure 5 This is a flowchart illustrating the generation of layout conversion instructions provided in yet another exemplary embodiment of this disclosure. For example... Figure 5 As shown, the method of this disclosure includes the following steps:

[0131] Step 310: Obtain the transformation operation sequence transOps from srcDL to dstDL, and record TcurDL = TsrcDL.

[0132] Where srcDL represents the initial layout, dstDL represents the final layout, and transOps is the sequence of data layout transformation operations from the initial layout to the final layout. TcurDL represents the current input tensor, and TsrcDL represents the first input tensor arranged according to the initial layout. Recording TcurDL = TsrcDL indicates initializing the current input tensor as the first input tensor arranged according to the initial layout.

[0133] Step 320: Determine whether transOps is empty or has been traversed completely. If transOps is empty or has been traversed completely, proceed to step 360; if transOps is not empty or has not been traversed completely, proceed to step 330.

[0134] If transOps is empty, it means there are no valid transformation operations, or if transOps has been traversed completely, it means that all valid transformation operations have been completed, and step 360 is executed. If transOps is not empty, it means there are valid transformation operations, but they have not been traversed completely, and step 330 is executed.

[0135] Step 330: Select transformation operations Opi sequentially from transOps. i = 1, 2, ... That is, traverse each transformation operation in the data arrangement transformation operation sequence one by one, and take the encountered Opi as the current operation.

[0136] Step 340: Obtain the source data layout of operation Opi: OpSrcDL = getSrcDL(Opi). Treat the current data layout as OpSrcDL, i.e., viewData(TcurDL, TOpiSrcDL)->TcurDL'. This generates the first intermediate instruction, which treats the current data layout of the current input tensor as the source data layout of Opi, i.e., treats the current input tensor as the first tensor TcurDL' arranged according to the source data layout, so that operation Opi can be applied to the current input tensor.

[0137] Step 350: Apply the transformation of operation Opi and update the current data TcurDL = applyOp(Opi, TcurDL'). This generates a second intermediate instruction to convert the first tensor TcurDL' into the second tensor TcurDL by applying operation Opi to the first tensor, thus updating the current input tensor TcurDL. Return to step 320.

[0138] Step 360: Treat the current data layout as TdstDL, i.e., viewData(TcurDL,TdstDL), to obtain the data TdstDL of the target layout.

[0139] If transOps is empty or has been traversed, then TcurDL is the second tensor after the last transformation operation, and the current data layout is the data layout of the second tensor. TdstDL represents the second input tensor arranged according to the preset data layout or the final layout method dstDL. viewData(TcurDL, TdstDL) means that the second tensor TcurDL arranged according to the current data layout is regarded as the second input tensor TdstDL arranged according to dstDL, that is, the third intermediate instruction is generated.

[0140] Step 370: Generate layout conversion instructions from srcDL to dstDL. Based on the intermediate instructions corresponding to each conversion operation, generate the layout conversion instructions.

[0141] Step 380, End.

[0142] After generating the arrangement transformation instructions for each first operator, the target instruction sequence for the neural network model to be compiled can be generated based on the arrangement transformation instructions and the corresponding computation instructions for the first operators. Alternatively, the target instruction sequence can be generated based on the arrangement transformation instructions and the corresponding computation instructions for the first operators, as well as the computation instructions for the second operators. The second operators are those that do not require data arrangement transformation.

[0143] The computer implementation method for generating instructions provided in this disclosure reduces data copying overhead during data layout transformation through a 7-level fused pipeline approach, effectively improving transformation efficiency. The 7-level fused pipeline can include four preset types of transformation operations. By combining various preset types of transformation operations, and by allowing each preset type of transformation operation to include multiple transformation operations with different parameters, efficient transformation between any two data layouts can be achieved, improving the completeness and flexibility of data layout methods and enhancing data layout transformation performance.

[0144] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0145] Any of the computer-implemented methods for generating instructions provided in this disclosure can be executed by any suitable electronic device with data processing capabilities, including but not limited to: terminal devices and servers. Alternatively, any of the computer-implemented methods for generating instructions provided in this disclosure can be executed by a processor, such as by a processor executing any of the computer-implemented methods for generating instructions mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated upon below.

[0146] Figure 6 This is a schematic flowchart illustrating an exemplary embodiment of the present disclosure of an instruction execution method. The instruction execution method provided in this embodiment can be applied to chips, including but not limited to intelligent driving chips, intelligent cockpit chips, and integrated cockpit chips. Figure 6 As shown, the instruction execution method of this disclosure embodiment may include the following steps:

[0147] Step 510: Obtain the target instruction sequence of the pre-configured neural network model.

[0148] The target instruction sequence is obtained based on the computer implementation method for generating instructions provided in any of the above embodiments.

[0149] Alternatively, the target instruction sequence may be pre-written into memory, in which case the target instruction sequence can be retrieved from memory.

[0150] Step 520: Execute the instructions in the target instruction sequence to obtain the instruction execution result.

[0151] The processor or controller, in conjunction with the computing unit, can execute the instructions in the target instruction sequence to obtain the instruction execution result.

[0152] Optionally, the target instruction sequence may include, but is not limited to, instructions executed by the central processing unit (CPU) and / or instructions executed by various computing units (or accelerators). Instructions executed by the computing units can be dispatched by the CPU to the controller that controls the computing units, and the controller controls the computing units to execute the corresponding instructions. The processor, controller, and computing units work together to obtain the instruction execution result of the target instruction sequence.

[0153] The instruction execution method provided in this embodiment obtains the instruction execution result by executing a target instruction sequence. Since the target instruction sequence is obtained by the computer implementation method for generating instructions provided in the foregoing embodiment, the target instruction sequence includes the arrangement conversion instruction of the first operator. For each first operator, multiple conversion operations on the first input tensor of the first operator can be completed by executing one arrangement conversion instruction, and the first input tensor is quickly and efficiently converted into a second input tensor that meets the preset data arrangement method supported by the computing unit. During the conversion process, there is no need to interact with external memory, which effectively improves the data arrangement conversion efficiency.

[0154] Figure 7 This is a flowchart illustrating an instruction execution method provided in another exemplary embodiment of this disclosure.

[0155] In some alternative embodiments, in Figure 6 Based on the illustrated embodiment, step 520, which involves executing instructions in the target instruction sequence to obtain the instruction execution result, may include:

[0156] Step 5210: Take each instruction in the target instruction sequence as the current instruction, and in response to the current instruction being an arrangement transformation instruction, determine the first input tensor to be transformed and the executable instruction sequence for performing data arrangement transformation on the first input tensor based on the arrangement transformation instruction.

[0157] Specifically, the executable instruction sequence can be determined based on the parameters of the layout transformation instruction. For example, the parameters of the layout transformation instruction may include the storage address of the executable instruction sequence, and the executable instruction sequence is retrieved from the corresponding storage space based on the storage address. The layout transformation instruction and its function are described in the foregoing embodiments and will not be repeated here.

[0158] Step 5220: Based on the first input tensor, execute the executable instruction sequence to obtain the transformed second input tensor corresponding to the first input tensor.

[0159] After obtaining the second input tensor, calculations can be performed on the second input tensor based on the second input tensor and the calculation instructions in the target instruction sequence to obtain the calculation result corresponding to the calculation instruction. Optionally, the second input tensor is input into the corresponding calculation unit, and the calculation unit executes the calculation instructions based on the second input tensor to obtain the calculation result corresponding to the calculation instruction. In response to the completion of the execution of the instructions in the target instruction sequence, the instruction execution result of the target instruction sequence is obtained. Optionally, the instruction execution result of the target instruction sequence may include, but is not limited to, one or more of the following: object detection result, semantic segmentation result, trajectory prediction result, speech recognition result, etc. The specific instruction execution result is determined based on the task function of the model corresponding to the target instruction sequence, and this embodiment of the disclosure does not limit it.

[0160] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0161] Any instruction execution method provided in this disclosure can be executed by any suitable electronic device or chip with data processing capabilities. Electronic devices include, but are not limited to, various terminal devices, and chips include, but are not limited to, intelligent driving chips, intelligent cockpit chips, cockpit-driver integrated chips, and other chips. Alternatively, any instruction execution method provided in this disclosure can be executed by a processor, which includes, but is not limited to, a central processing unit, a neural network processor, and a graphics processing unit. The processor executes any instruction execution method mentioned in this disclosure by calling corresponding instructions stored in memory. Further details will not be elaborated upon below.

[0162] Exemplary device

[0163] Figure 8 This is a schematic diagram of a computer implementation apparatus for generating instructions provided in an exemplary embodiment of this disclosure. The computer implementation apparatus for generating instructions provided in this embodiment can be used to implement the computer implementation method for generating instructions provided in any of the above embodiments of this disclosure, such as... Figure 8 The computer implementation apparatus 61 shown for generating instructions may include a processor 611. The processor 611 is configured to:

[0164] The process involves: determining a first intermediate representation corresponding to the neural network model to be compiled; determining the first input tensor of the first operator in the first intermediate representation, the first data arrangement of the first input tensor, and the preset data arrangement supported by the computation unit executing the first operator; determining a data arrangement transformation operation sequence based on the first input tensor, the first data arrangement, and the preset data arrangement, wherein the data arrangement transformation operation sequence includes at least one transformation operation; generating an arrangement transformation instruction for the first input tensor based on the data arrangement transformation operation sequence; using the arrangement transformation instruction to convert the first input tensor into a second input tensor stored according to the preset data arrangement; and determining the target instruction sequence for the neural network model to be compiled based on the arrangement transformation instruction and the computation instruction corresponding to the first operator.

[0165] In some alternative embodiments, in the above... Figure 8 Based on the illustrated embodiment, the processor is specifically configured as follows:

[0166] Based on the size of the first input tensor and the first data arrangement, the initial arrangement of the data arrangement transformation operation sequence is determined. Based on the size of the first input tensor and the preset data arrangement, the final arrangement of the data arrangement transformation operation sequence is determined. Based on the pre-configured transformation operation sub-sequences, multiple transformation paths from the initial arrangement to the final arrangement are determined. These transformation operation sub-sequences include combinations of various preset types of transformation operations. Based on the initial and final arrangement methods, a shortest path search algorithm is used to determine the data arrangement transformation operation sequence from these multiple transformation paths.

[0167] In some optional embodiments, the transformation operation subsequence includes at least one of the following: data arrangement transformation operation between data blocks and data groups, data arrangement transformation operation between data groups, data arrangement transformation operation between data groups and data within a group, and data arrangement transformation operation within a data group, wherein a data block includes at least two data groups of different dimensions.

[0168] In some alternative embodiments, based on any of the above embodiments, the processor is specifically configured as follows:

[0169] Each transformation operation in the data arrangement transformation operation sequence is iterated one by one. The currently iterated transformation operation is taken as the current transformation operation, and the source data arrangement method of the current transformation operation is determined. Based on the current input tensor of the current transformation operation, the current transformation operation, and the source data arrangement method, the intermediate instruction corresponding to the current transformation operation is generated. Where the current transformation operation is the first transformation operation in the data arrangement transformation operation sequence, the current input tensor of the current transformation operation is the first input tensor. In response to the completion of the traversal of transformation operations in the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated based on the intermediate instructions corresponding to each transformation operation.

[0170] In some alternative embodiments, the processor is specifically configured as follows:

[0171] Based on the current input tensor and source data arrangement of the current transformation operation, a first intermediate instruction is generated; the first intermediate instruction is used to treat the current input tensor as a first tensor arranged according to the source data arrangement; based on the first tensor and the current transformation operation, a second intermediate instruction is generated; the second intermediate instruction is used to perform data arrangement transformation on the first tensor through the current transformation operation to obtain a second tensor; in response to the current transformation operation not being the last transformation operation in the data arrangement transformation operation sequence, the first and second intermediate instructions are determined as the intermediate instructions corresponding to the current transformation operation; in response to the current transformation operation being the last transformation operation in the data arrangement transformation operation sequence, based on the second tensor and a preset data arrangement, a third intermediate instruction is generated, and the first, second, and third intermediate instructions are determined as the intermediate instructions corresponding to the current transformation operation; the third intermediate instruction is used to treat the second tensor as a second input tensor arranged according to the preset data arrangement.

[0172] In some alternative embodiments, the processor 611 is specifically configured as follows:

[0173] For any intermediate instruction corresponding to any transformation operation in each transformation operation, this intermediate instruction is taken as the current intermediate instruction. In response to the current intermediate instruction including a first intermediate instruction and a second intermediate instruction, based on the first intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement corresponding to the current intermediate instruction are taken as the input parameters corresponding to the second intermediate instruction. Based on the input parameters and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. In response to the current intermediate instruction including a first intermediate instruction, a second intermediate instruction, and a third intermediate instruction, based on the first intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement corresponding to the current intermediate instruction are taken as the input parameters corresponding to the second intermediate instruction. Based on the third intermediate instruction, the second tensor and the preset data arrangement are taken as the output parameters corresponding to the second intermediate instruction. Based on the input parameters, the output parameters, and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. Based on the executable instructions corresponding to each intermediate instruction, the arrangement transformation instruction of the first input tensor is determined.

[0174] Figure 9 This is a schematic diagram of the structure of a computer implementation apparatus for generating instructions provided in another exemplary embodiment of this disclosure.

[0175] In some optional embodiments, based on any of the above embodiments, the apparatus of this disclosure further includes a memory 612 coupled to a processor 621. The processor 621 is also configured to store a target instruction sequence in the memory 612. The memory 612 can be any type of external memory.

[0176] The embodiments described above can be implemented individually or in any combination without conflict. The specific implementation can be set according to actual needs, and this disclosure does not limit them.

[0177] The beneficial technical effects corresponding to the exemplary embodiments of this device can be found in the corresponding beneficial technical effects of the exemplary method section above, and will not be repeated here.

[0178] Exemplary chip

[0179] Figure 10 This is a schematic diagram of the structure of a chip provided in an exemplary embodiment of this disclosure. The chip provided in the embodiments of this disclosure can be used to implement the instruction execution method provided in any of the above embodiments of this disclosure. Figure 10 The chip 71 shown may include:

[0180] The memory 711 is configured to store a target instruction sequence for a neural network model, the target instruction sequence being obtained based on a computer implementation method for generating instructions provided in any of the above embodiments.

[0181] The controller 712, coupled to the memory 711, is configured to retrieve the target instruction sequence from the memory 711, control the computing unit to execute the instructions in the target instruction sequence, and obtain the instruction execution result;

[0182] The calculation unit 713 is configured to execute instructions in the target instruction sequence according to the control signal of the controller, and obtain the instruction execution result.

[0183] Optionally, the controller 712 can be implemented using any control logic circuit with corresponding control functions, for example, it can be a CPU. The memory can be any type of memory, and the computing unit includes, but is not limited to, one or more of the following: CPU, NPU, GPU, ISP, ACU, VPU, SPU, DSP, etc.

[0184] The beneficial technical effects corresponding to the exemplary embodiments of this chip can be found in the corresponding beneficial technical effects in the exemplary method section above, and will not be repeated here.

[0185] Exemplary electronic devices

[0186] Figure 11 This is a structural diagram of an electronic device provided in an embodiment of the present disclosure, including at least one processor 91 and a memory 92.

[0187] The processor 91 may be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and may control other components in the electronic device 90 to perform desired functions.

[0188] The memory 92 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 91 may execute one or more computer program instructions to implement the computer implementation methods for generating instructions, instruction execution methods, and / or other desired functions of the various embodiments of this disclosure described above.

[0189] In one example, the electronic device 90 may also include an input device 93 and an output device 94, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0190] The input device 93 may also include, for example, a touchscreen, a microphone, various sensors, etc. Sensors may include, for example, image sensors (e.g., cameras, webcams), LiDAR, millimeter-wave radar, ultrasonic radar, positioning sensors, pressure sensors, air quality sensors, temperature sensors, etc. Image sensors, LiDAR, millimeter-wave radar, ultrasonic radar, etc., can be used for environmental perception, i.e., detecting moving and static objects in the surrounding environment. Moving and static objects may include, for example, static objects such as lane lines, curbs, arrows, signs, trees, and buildings, as well as dynamic objects such as surrounding vehicles, pedestrians, and cyclists. Positioning sensors are used to locate the mobile device (e.g., a bicycle, a robot, etc.) where the electronic device is located. Positioning sensors may include, for example, an Inertial Measurement Unit (IMU), a Global Positioning System (GPS), etc. Pressure sensors can be used to detect seat pressure. Temperature sensors can be used to detect the temperature inside the vehicle cabin. Air quality sensors can be used to detect the air quality inside the vehicle cabin.

[0191] The output device 94 can output various information to the outside, including, for example, a display, a speaker, a communication network and its connected remote output devices, etc.

[0192] Of course, for the sake of simplicity, Figure 11 Only some of the components of the electronic device 90 relevant to this disclosure are shown, omitting components such as buses, input / output interfaces, etc. In addition, the electronic device 90 may include any other suitable components depending on the specific application.

[0193] Exemplary computer program products and computer-readable storage media

[0194] In addition to the methods and apparatus described above, embodiments of this disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, cause the processor to perform the steps of the computer implementation method for generating instructions and the instruction execution method of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0195] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0196] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps of the computer implementation method for generating instructions and the instruction execution method of the various embodiments of this disclosure described in the "Exemplary Methods" section above.

[0197] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, but is not limited to, systems, apparatuses, or devices that are electrical, magnetic, optical, electromagnetic, infrared, or semiconductor, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0198] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the aforementioned specific details for implementation.

[0199] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. A computer implementation method for generating instructions, comprising: Determine the first intermediate representation corresponding to the neural network model to be compiled; Determine the first input tensor of the first operator in the first intermediate representation, the first data arrangement of the first input tensor, and the preset data arrangement supported by the computing unit that executes the first operator; Based on the first input tensor, the first data arrangement method, and the preset data arrangement method, a data arrangement transformation operation sequence is determined, which includes multiple transformation operations; Based on the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated; the arrangement transformation instruction is used to convert the first input tensor into a second input tensor stored according to the preset data arrangement method. Based on the arrangement conversion instruction and the calculation instruction corresponding to the first operator, the target instruction sequence of the neural network model to be compiled is determined.

2. The method according to claim 1, wherein, The step of generating the arrangement transformation instruction for the first input tensor based on the data arrangement transformation operation sequence includes: Each transformation operation in the data arrangement transformation operation sequence is traversed one by one, and the currently traversed transformation operation is taken as the current transformation operation to determine the source data arrangement method of the current transformation operation; Based on the current input tensor of the current transformation operation, the current transformation operation, and the source data arrangement, an intermediate instruction corresponding to the current transformation operation is generated; wherein, when the current transformation operation is the first transformation operation in the data arrangement transformation operation sequence, the current input tensor of the current transformation operation is the first input tensor; In response to the completion of the traversal of the transformation operations in the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated based on the intermediate instructions corresponding to each transformation operation.

3. The method according to claim 2, wherein, The intermediate instructions corresponding to the current transformation operation are generated based on the current input tensor of the current transformation operation, the current transformation operation, and the source data arrangement, including: Based on the current input tensor of the current transformation operation and the source data arrangement, a first intermediate instruction is generated; the first intermediate instruction is used to treat the current input tensor as a first tensor arranged according to the source data arrangement. Based on the first tensor and the current transformation operation, a second intermediate instruction is generated; the second intermediate instruction is used to perform data arrangement transformation on the first tensor through the current transformation operation to obtain the second tensor; In response to the fact that the current conversion operation is not the last conversion operation in the data layout conversion operation sequence, the first intermediate instruction and the second intermediate instruction are determined as the intermediate instructions corresponding to the current conversion operation; In response to the fact that the current transformation operation is the last transformation operation in the data arrangement transformation operation sequence, a third intermediate instruction is generated based on the second tensor and the preset data arrangement method, and the first intermediate instruction, the second intermediate instruction, and the third intermediate instruction are determined as the intermediate instructions corresponding to the current transformation operation; the third intermediate instruction is used to regard the second tensor as the second input tensor arranged according to the preset data arrangement method.

4. The method according to claim 3, wherein, The step of generating the arrangement transformation instruction for the first input tensor based on the intermediate instructions corresponding to each transformation operation includes: For any intermediate instruction corresponding to any of the aforementioned conversion operations, that intermediate instruction shall be used as the current intermediate instruction. In response to the current intermediate instruction including the first intermediate instruction and the second intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement method corresponding to the current intermediate instruction are used as the input parameters corresponding to the second intermediate instruction based on the first intermediate instruction. Based on the input parameters and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. In response to the current intermediate instruction including a first intermediate instruction, a second intermediate instruction, and a third intermediate instruction, based on the first intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement corresponding to the current intermediate instruction are used as the input parameters corresponding to the second intermediate instruction; based on the third intermediate instruction, the second tensor and the preset data arrangement are used as the output parameters corresponding to the second intermediate instruction; based on the input parameters, the output parameters, and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. Based on the executable instructions corresponding to each of the intermediate instructions, the arrangement and transformation instructions of the first input tensor are determined.

5. The method according to any one of claims 1-4, wherein, The step of determining the data arrangement transformation operation sequence based on the first input tensor, the first data arrangement method, and the preset data arrangement method includes: Based on the size of the first input tensor and the first data arrangement, determine the initial arrangement of the data arrangement transformation operation sequence; Based on the size of the first input tensor and the preset data arrangement, determine the ending arrangement of the data arrangement transformation operation sequence; Based on a pre-configured sequence of transformation operations, multiple transformation paths are determined from the initial arrangement to the final arrangement; the sequence of transformation operations includes a combination of various preset types of transformation operations. Based on the initial arrangement method and the ending arrangement method, the data arrangement transformation operation sequence is determined from the multiple transformation paths using the shortest path search algorithm.

6. The method according to claim 5, wherein, The transformation operation subsequence includes at least one of the following: data arrangement transformation operation between data blocks and data groups, data arrangement transformation operation between data groups, data arrangement transformation operation between data groups and data within a group, and data arrangement transformation operation within a data group, wherein a data block includes at least two data groups of different dimensions.

7. The method according to any one of claims 1-4, wherein, The method further includes: The target instruction sequence is stored in memory.

8. A method for executing instructions, comprising: Obtain a target instruction sequence of a pre-configured neural network model; the target instruction sequence is obtained based on the computer implementation method for generating instructions as described in any one of claims 1-7. Execute the instructions in the target instruction sequence to obtain the instruction execution result.

9. The method according to claim 8, wherein, The execution of instructions in the target instruction sequence to obtain the instruction execution result includes: Each instruction in the target instruction sequence is taken as the current instruction. In response to the current instruction being an arrangement transformation instruction, the first input tensor to be transformed and the executable instruction sequence for performing data arrangement transformation on the first input tensor are determined based on the arrangement transformation instruction. Based on the first input tensor, the executable instruction sequence is executed to obtain the transformed second input tensor corresponding to the first input tensor.

10. A computer implementation apparatus for generating instructions, comprising a processor configured to: Determine the first intermediate representation corresponding to the neural network model to be compiled; Determine the first input tensor of the first operator in the first intermediate representation, the first data arrangement of the first input tensor, and the preset data arrangement supported by the computing unit that executes the first operator; Based on the first input tensor, the first data arrangement method, and the preset data arrangement method, a data arrangement transformation operation sequence is determined, wherein the data arrangement transformation operation sequence includes at least one transformation operation; Based on the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated; the arrangement transformation instruction is used to convert the first input tensor into a second input tensor stored according to the preset data arrangement method. Based on the arrangement conversion instruction and the calculation instruction corresponding to the first operator, the target instruction sequence of the neural network model to be compiled is determined.

11. The apparatus according to claim 10, wherein, The processor is specifically configured as follows: Each transformation operation in the data arrangement transformation operation sequence is traversed one by one, and the currently traversed transformation operation is taken as the current transformation operation to determine the source data arrangement method of the current transformation operation; Based on the current input tensor of the current transformation operation, the current transformation operation, and the source data arrangement, an intermediate instruction corresponding to the current transformation operation is generated; wherein, when the current transformation operation is the first transformation operation in the data arrangement transformation operation sequence, the current input tensor of the current transformation operation is the first input tensor; In response to the completion of the traversal of the transformation operations in the data arrangement transformation operation sequence, an arrangement transformation instruction for the first input tensor is generated based on the intermediate instructions corresponding to each transformation operation.

12. The apparatus according to claim 11, wherein, The processor is specifically configured as follows: Based on the current input tensor of the current transformation operation and the source data arrangement, a first intermediate instruction is generated; the first intermediate instruction is used to treat the current input tensor as a first tensor arranged according to the source data arrangement. Based on the first tensor and the current transformation operation, a second intermediate instruction is generated; the second intermediate instruction is used to perform data arrangement transformation on the first tensor through the current transformation operation to obtain the second tensor; In response to the fact that the current conversion operation is not the last conversion operation in the data layout conversion operation sequence, the first intermediate instruction and the second intermediate instruction are determined as the intermediate instructions corresponding to the current conversion operation; In response to the fact that the current transformation operation is the last transformation operation in the data arrangement transformation operation sequence, a third intermediate instruction is generated based on the second tensor and the preset data arrangement method, and the first intermediate instruction, the second intermediate instruction, and the third intermediate instruction are determined as the intermediate instructions corresponding to the current transformation operation; the third intermediate instruction is used to regard the second tensor as the second input tensor arranged according to the preset data arrangement method.

13. The apparatus according to claim 12, wherein, The processor is specifically configured as follows: For any intermediate instruction corresponding to any of the aforementioned conversion operations, that intermediate instruction shall be used as the current intermediate instruction. In response to the current intermediate instruction including the first intermediate instruction and the second intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement method corresponding to the current intermediate instruction are used as the input parameters corresponding to the second intermediate instruction based on the first intermediate instruction. Based on the input parameters and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. In response to the current intermediate instruction including a first intermediate instruction, a second intermediate instruction, and a third intermediate instruction, based on the first intermediate instruction, the current input tensor of the current intermediate instruction and the source data arrangement corresponding to the current intermediate instruction are used as the input parameters corresponding to the second intermediate instruction; based on the third intermediate instruction, the second tensor and the preset data arrangement are used as the output parameters corresponding to the second intermediate instruction; based on the input parameters, the output parameters, and the second intermediate instruction, the executable instruction corresponding to the intermediate instruction is determined. Based on the executable instructions corresponding to each of the intermediate instructions, the arrangement and transformation instructions of the first input tensor are determined.

14. The apparatus according to any one of claims 10-13, wherein, The processor is specifically configured as follows: Based on the size of the first input tensor and the first data arrangement, determine the initial arrangement of the data arrangement transformation operation sequence; Based on the size of the first input tensor and the preset data arrangement, determine the ending arrangement of the data arrangement transformation operation sequence; Based on a pre-configured sequence of transformation operations, multiple transformation paths are determined from the initial arrangement to the final arrangement; the sequence of transformation operations includes a combination of various preset types of transformation operations. Based on the initial arrangement method and the ending arrangement method, the data arrangement transformation operation sequence is determined from the multiple transformation paths using the shortest path search algorithm.

15. The apparatus according to claim 14, wherein, The transformation operation subsequence includes at least one of the following: data arrangement transformation operation between data blocks and data groups, data arrangement transformation operation between data groups, data arrangement transformation operation between data groups and data within a group, and data arrangement transformation operation within a data group, wherein a data block includes at least two data groups of different dimensions.

16. The apparatus according to any one of claims 10-13, wherein, Also includes: Memory; The processor is also configured to: The target instruction sequence is stored in the memory.

17. A computer-readable storage medium storing a computer program that is executed by a processor to perform the computer implementation method for generating instructions as described in any one of claims 1-7 or the instruction execution method as described in any one of claims 8 and 9.

18. An electronic device, the electronic device comprising: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory, and execute the executable instructions to implement the computer implementation method for generating instructions as described in any one of claims 1-7 or the instruction execution method as described in any one of claims 8 and 9.

Images