Tensor instruction optimization method, device and storage medium

Abstract

Embodiments of the present application provide a tensor instruction optimization method and device and a storage medium, and relate to the technical field of artificial intelligence chips. The method comprises: sequentially traversing N tensor instructions in a target basic block; if there is a target instruction on which a currently traversed tensor instruction depends, and there are L intermediate instructions between the tensor instruction and the target instruction, obtaining total additional time based on the additional execution time of each of the L intermediate instructions; and if the total additional time is greater than the unit execution time of the currently traversed tensor instruction, it is indicated that the target instruction on which the currently traversed tensor instruction depends has been executed when the pipeline executes the currently traversed tensor instruction, so the dependency descriptor of the currently traversed tensor instruction is set to a first descriptor to represent no dependency relationship. In this way, the pipeline can directly execute the tensor instruction without waiting, thereby reducing the probability of pipeline blocking, reducing the idling of hardware resources in the pipeline, and improving the utilization rate of hardware resources.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of artificial intelligence chip, and particularly relate to a tensor instruction optimization method, device and storage medium. BACKGROUND

[0002] In the use of artificial intelligence chips, deep learning is a very important application scenario. Tensor is a very core data structure in deep learning, which can be used to store and process data. In artificial intelligence chips, Tensor Core, as an important module, can efficiently accelerate tensor calculation, which plays a very important role in deep learning framework training and inference, and can significantly improve the performance of tensor operations such as convolution and matrix multiplication.

[0003] In the architecture of artificial intelligence chips, Multiply-and-accumulate (MMA) instruction and Convolution (CONV) instruction are key tensor instructions in Tensor Core. MMA is mainly used to efficiently perform matrix multiplication and accumulation operation, and CONV is specially used to accelerate convolution operation.

[0004] Under the related technology, pipeline is used to complete tensor instructions such as MMA instruction or CONV instruction. If there is data dependency between tensor instructions (for example, the input of the current MMA instruction is the output of the previous MMA instruction), the current tensor instruction needs to wait for the dependent tensor instruction to end execution before starting execution, which will cause the pipeline of tensor instructions to be blocked, and the hardware resources in the pipeline to be idle, thereby causing a large waste of hardware resources. SUMMARY

[0005] Embodiments of the present application provide a tensor instruction optimization method, device and storage medium, which are used to reduce the probability of pipeline blocking, reduce the idle of hardware resources in the pipeline, and improve the utilization rate of hardware resources.

[0006] In one aspect, the present application provides a tensor instruction optimization method, which comprises:

[0007] sequentially traversing N tensor instructions in a target basic block, wherein, each time a tensor instruction is traversed, M tensor instructions that have been traversed are queried, wherein the one tensor instruction is a non-first traversed tensor instruction, and 1≤M<N;

[0008] if there is a target instruction depended by the one tensor instruction in the M tensor instructions, and there are L intermediate instructions between the one tensor instruction and the target instruction, obtaining a total extra time based on respective extra execution times of the L intermediate instructions, where L≥1;

[0009] if the total extra time is greater than a unit execution time of the one tensor instruction, setting a dependency descriptor of the one tensor instruction as a first descriptor, the first descriptor indicating no dependency.

[0010] In an aspect, an embodiment of the present application provides a tensor instruction optimization device, which comprises:

[0011] a traversal module, configured to traverse N tensor instructions in a target basic block one by one, wherein, each time a tensor instruction is traversed, M tensor instructions that have been traversed are queried, and the one tensor instruction is a non-first traversed tensor instruction, ;

[0012] a judgment module, configured to, if there is a target instruction depended by the one tensor instruction in the M tensor instructions, and there are L intermediate instructions between the one tensor instruction and the target instruction, obtain a total extra time based on respective extra execution times of the L intermediate instructions, where L≥1; ;

[0013] a setting module, configured to, if the total extra time is greater than a unit execution time of the one tensor instruction, set a dependency descriptor of the one tensor instruction as a first descriptor, the first descriptor indicating no dependency.

[0014] Optionally, the setting module is further configured to:

[0015] if there is no target instruction depended by the one tensor instruction in the M tensor instructions, set the dependency descriptor of the one tensor instruction as the first descriptor.

[0016] Optionally, the setting module is further configured to:

[0017] if there is a target instruction depended by the one tensor instruction in the M tensor instructions, and there is no intermediate instruction between the one tensor instruction and the target instruction, set the dependency descriptor of the one tensor instruction as a second descriptor, the second descriptor indicating that the one tensor instruction depends on a previous tensor instruction.

[0018] Optionally, the setting module is further configured to:

[0019] The dependency descriptor of the one tensor instruction is set as a third descriptor before the total extra time is obtained based on the extra execution time of each of the L intermediate instructions, the third descriptor indicating that the second tensor instruction is traced back in dependency;

[0020] The setting module is specifically configured to:

[0021] If the total extra time is greater than the unit execution time of the one tensor instruction, the dependency descriptor of the one tensor instruction is adjusted from the third descriptor to the first descriptor.

[0022] Optionally, the decision module is further configured to:

[0023] Based on the input data of the one tensor instruction, the output data of each of the M tensor instructions is queried;

[0024] When there is output data matching the input data in the output data of each of the M tensor instructions, the tensor instruction corresponding to the output data is taken as a target instruction dependent on the one tensor instruction.

[0025] Optionally, the decision module is further configured to:

[0026] The first instruction sequence number corresponding to the one tensor instruction and the second instruction sequence number corresponding to the target instruction are obtained.

[0027] Based on the first instruction sequence number and the second instruction sequence number, the number of intermediate instructions between the one tensor instruction and the target instruction is determined, the number of intermediate instructions being 0 or L.

[0028] Optionally, the one tensor instruction and the target instruction are original instructions in the target basic block; the first instruction sequence number is the positive sequence number of the one tensor instruction in the target basic block; and the second instruction sequence number is the positive sequence number of the target instruction in the target basic block.

[0029] The decision module is specifically configured to:

[0030] Based on a target difference value between the first instruction sequence number and the second instruction sequence number, the number of intermediate instructions between the one tensor instruction and the target instruction is determined.

[0031] Optionally, the one tensor instruction is an original instruction in the target basic block, and the target instruction is a tensor instruction poured from another basic block; the first instruction sequence number is the positive sequence number of the one tensor instruction in the target basic block, and the second instruction sequence number is the reverse sequence number of the target instruction in the other basic block.

[0032] The decision module is specifically configured to:

[0033] determine an intermediate instruction quantity between the one tensor instruction and the target instruction based on a summation result of the first instruction sequence number and the second instruction sequence number.

[0034] In an aspect, an embodiment of the present application provides a computer device, comprising a memory, an artificial intelligence chip, and a computer program stored on the memory and running on the artificial intelligence chip, wherein the artificial intelligence chip implements the steps of the above-mentioned tensor instruction optimization method when executing the computer program.

[0035] In an aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program executable by a computer device, and when the computer program runs on the computer device, the computer device executes the steps of the above-mentioned tensor instruction optimization method.

[0036] In an aspect, an embodiment of the present application provides a computer program product, comprising a computer program stored on a computer readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer device, the computer device executes the steps of the above-mentioned tensor instruction optimization method.

[0037] In an embodiment of the present application, in the compilation phase of the target basic block, N tensor instructions in the target basic block are traversed in turn, wherein, each time a tensor instruction is traversed, M traversed tensor instructions are queried. If there is a target instruction depended by the currently traversed tensor instruction in the M tensor instructions, and there are L intermediate instructions between the tensor instruction and the target instruction, then based on the additional execution time of each of the L intermediate instructions, the total additional time (i.e. the time interval between the currently traversed tensor instruction and the depended target instruction) is obtained. If the total additional time is greater than the unit execution time of the currently traversed tensor instruction, it means that when the pipeline executes the currently traversed tensor instruction, the target instruction depended by the currently traversed tensor instruction has been executed, so the currently traversed tensor instruction can be executed directly without additional waiting, and therefore the dependency descriptor of the currently traversed tensor instruction is set to the first descriptor to represent that there is essentially no dependency between the currently traversed tensor instruction and other tensor instructions.

[0038] In this way, in the execution phase of the target basic block, when the pipeline executes each tensor instruction, when the pipeline executes the tensor instruction with the first descriptor, the tensor instruction can be executed directly by using the hardware resources without additional waiting, so as to greatly reduce the probability of pipeline blocking, reduce the idling of hardware resources in the pipeline, and further improve the utilization rate of hardware resources on the artificial intelligence chip. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0040] Figure 1 A structural schematic diagram of an artificial intelligence chip provided for an embodiment of the present application is shown in the figure.

[0041] Figure 2 A flowchart of a tensor instruction optimization method provided for an embodiment of the present application is shown in the figure.

[0042] Figure 3 A schematic diagram of a dependency relationship provided for an embodiment of the present application is shown in the figure.

[0043] Figure 4 A schematic diagram of a dependency relationship provided for an embodiment of the present application is shown in the figure.

[0044] Figure 5 A schematic diagram of a dependency relationship provided for an embodiment of the present application is shown in the figure.

[0045] Figure 6 A schematic diagram of a dependency relationship provided for an embodiment of the present application is shown in the figure.

[0046] Figure 7 A flowchart of a pipeline parallel processing method provided for an embodiment of the present application is shown in the figure.

[0047] Figure 8 A schematic diagram of a control flow graph provided for an embodiment of the present application is shown in the figure.

[0048] Figure 9 A schematic diagram of a control flow graph provided for an embodiment of the present application is shown in the figure.

[0049] Figure 10 A structural schematic diagram of a tensor instruction optimization device provided for an embodiment of the present application is shown in the figure.

[0050] Figure 11 A structural schematic diagram of a computer device provided for an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0051] In order to make the objects, technical solutions and beneficial effects of the present application more clear, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0052] REFERENCE Figure 1Fig. 1 is a structural diagram of an artificial intelligence chip 100 applicable to an embodiment of the present application. The artificial intelligence chip 100 at least includes a display memory 101 and a plurality of computing units 102. The computing unit 102 can be a Streaming Processing Cluster (SPC). The display memory 101 can be a High Bandwidth Memory (HBM), or other types of memory.

[0053] The computing unit 102 includes a plurality of execution units 103. Each execution unit 103 includes a tensor core 104 and a vector core 105. The tensor core 104 is used to execute tensor instructions such as MMA instructions and CONV instructions. The vector core 105 is usually used to execute activation (softmax) instructions, random dropout (dropout) instructions, etc.

[0054] In an embodiment of the present application, in the compilation phase of a target basic block, the central processing unit sequentially traverses N tensor instructions in the target basic block. For each traversed tensor instruction, the M traversed tensor instructions are queried, where the currently traversed tensor instruction is not the first traversed tensor instruction, .

[0055] If there is a target instruction dependent on the currently traversed tensor instruction among the M traversed tensor instructions, and there are L intermediate instructions between the tensor instruction and the target instruction, the total additional time (i.e. the time interval between the currently traversed tensor instruction and the dependent target instruction) is obtained based on the additional execution time of each of the L intermediate instructions. If the total additional time is greater than the unit execution time of the currently traversed tensor instruction, it means that when the pipeline executes the currently traversed tensor instruction, the dependent target instruction of the currently traversed tensor instruction has been executed. Therefore, the currently traversed tensor instruction can be executed directly without additional waiting, and thus the dependency descriptor of the currently traversed tensor instruction is set to the first descriptor to indicate that there is essentially no dependency between the currently traversed tensor instruction and other tensor instructions.

[0056] In this way, in the execution phase of the target basic block, when the tensor core 104 executes each tensor instruction in the pipeline, when the pipeline executes a tensor instruction with the first descriptor, the tensor instruction can be executed directly using hardware resources without additional waiting, thereby greatly reducing the probability of pipeline blocking, reducing the idling of hardware resources in the pipeline, and further improving the utilization rate of hardware resources on the artificial intelligence chip.

[0057] In addition to the above structure, the artificial intelligence chip 100 in the present application can also include other structures, which are not limited in the present application.

[0058] Artificial intelligence chips 100 can be: Graphics Processing Unit (GPU), General-Purpose Graphics Processing Units (GPGPU), Domain Specific Architecture (DSA), etc.

[0059] The following is based on Figure 1 The diagram shown illustrates the architecture of an AI chip and details a process for providing a tensor instruction optimization method. (See attached image.) Figure 2 This method is performed by a computer device, which includes... Figure 1 The artificial intelligence chip shown includes the following steps:

[0060] Step 201: Iterate through the N tensor instructions in the target basic block in sequence. For each tensor instruction, query the M tensor instructions that have been traversed.

[0061] Specifically, tensor instruction optimization methods can be applied to various scenarios, such as image processing, speech processing, and text processing. In different application scenarios, the physical meaning of the input data processed by tensor instructions can be different.

[0062] For example, in a text processing scenario, the input data can be text data used in tasks such as text generation and text recognition.

[0063] For example, in a speech processing scenario, the input data can be speech data used in tasks such as speech enhancement, speech recognition, and speech synthesis.

[0064] For example, in image processing scenarios, the input data can be image data used in tasks such as image preprocessing, image segmentation, and object detection.

[0065] A basic block is a set of instructions executed sequentially, characterized by a single entry point and a single exit point. In other words, the instructions within a basic block are executed in a predetermined order, without any jumps or other operations altering the execution flow. Jump instructions may only exist at the end of the basic block to change the program's execution path.

[0066] In this embodiment, the target basic block includes N tensor instructions, which are executed sequentially in a preset order. During the instruction encoding stage, a preset number of bits (e.g., 2 bits) are reserved as description fields for the dependencies between tensor instructions. The description fields for the dependencies include: a first descriptor dep0, a second descriptor dep1, and a third descriptor dep2.

[0067] The first descriptor dep0 represents no dependency; the second descriptor dep1 represents dependency on a previous tensor instruction; and the third descriptor dep2 represents dependency on a second tensor instruction that backtracks forward.

[0068] Since the size of the description bit field is limited, it is difficult to set a corresponding descriptor for each case of dependency on a tensor instruction that backtracks forward; based on this, in the embodiments of the present application, if the currently traversed tensor instruction depends on an Rth (R is greater than 2) tensor instruction that backtracks forward, the dependency descriptor of the currently traversed tensor instruction is also set to the third descriptor.

[0069] For the case that the currently traversed tensor instruction depends on an Rth tensor instruction that backtracks forward, if the second tensor instruction that backtracks forward ends execution, the Rth tensor instruction that backtracks forward must also end execution, that is, the tensor instruction on which the currently traversed tensor instruction depends has ended execution; therefore, it is reasonable to set the dependency descriptor to the third descriptor for the above case.

[0070] In the compilation phase, the N tensor instructions in the target basic block are traversed in turn according to a preset order. For the first traversed tensor instruction, the dependency descriptor of the tensor instruction is directly set to the first descriptor.

[0071] For each other traversed tensor instruction (i.e., a non-first traversed tensor instruction), it is determined whether the M traversed tensor instructions include a target instruction on which the currently traversed tensor instruction depends. .

[0072] In some embodiments, based on the input data of the currently traversed tensor instruction, the output data of the M tensor instructions is queried; when there is output data that matches the input data in the output data of the M tensor instructions, the tensor instruction corresponding to the output data is taken as the target instruction on which the currently traversed tensor instruction depends.

[0073] Specifically, the attribute information records of the N tensor instructions in the target basic block are recorded in a local arrangement container (LocalMap), and the attribute information records include: a pointer of the target basic block, an instruction number of the tensor instruction, and a first identifier (denoted as D) of a register that saves output data.

[0074] The input data of the currently traversed tensor instruction includes: a plurality of use operands; a target operand on which a dependency relationship is likely to occur is selected from the plurality of use operands, and a second identifier of a register that saves the target operand. For example, an MMA instruction generally includes three use operands, and the third use operand is a target operand on which a dependency relationship is likely to occur; the second identifier of the register that saves the target operand is denoted as C.

[0075] Use the second identifier to query the local layout container. If there is no first identifier in the local layout container that matches the second identifier, it means that there is no target instruction that the currently traversed tensor instruction depends on among the M tensor instructions. In this case, set the dependency descriptor of the currently traversed tensor instruction to the first descriptor.

[0076] For example, see Figure 3 The target basic block includes two MMA instructions: Instruction I1: D1 = mma(a1, b1, c1) and Instruction I2: D2 = mma(a2, b2, c2). D1 represents the output data of instruction I1, and a1, b1, and c1 represent the input data of instruction I1; D2 represents the output data of instruction I2, and a2, b2, and c2 represent the input data of instruction I2. There is no dependency between instruction I1 and instruction I2; therefore, the dependency descriptor of instruction I2 is set to the first descriptor dep0.

[0077] If a first identifier that matches the second identifier exists in the local layout container, it means that the register corresponding to the first identifier and the register corresponding to the second identifier are the same register. That is, the input data of the tensor instruction being traversed depends on the output data stored in the register of the first identifier. Therefore, the tensor instruction corresponding to the output data is used as the target instruction that the tensor instruction being traversed depends on.

[0078] In some embodiments, if there is a target instruction that the currently traversed tensor instruction depends on among the M tensor instructions, and there is no intermediate instruction between the tensor instruction and the target instruction, then the dependency descriptor of the tensor instruction is set to the second descriptor.

[0079] For example, see Figure 4 The target basic block includes two MMA instructions: instruction I1: D1 = mma(a1, b1, c1) and instruction I2: D2 = mma(a2, b2, D1). The third operation used by instruction I2 is D1 (i.e., the output data of instruction I1); therefore, instruction I2 depends on instruction I1. There are no intermediate instructions between instruction I2 and instruction I1. In this case, the dependency descriptor of instruction I2 is set to the second descriptor dep1.

[0080] Step 202: If there is a target instruction that the currently traversed tensor instruction depends on among the M tensor instructions, and there are L intermediate instructions between the tensor instruction and the target instruction, then the total additional time is obtained based on the additional execution time of each of the L intermediate instructions.

[0081] Specifically, if among the M tensor instructions there exists a target instruction upon which the currently traversed tensor instruction depends, and there are L intermediate instructions between the tensor instruction and the target instruction, If the target instruction is the second or Rth tensor instruction that is backtracked from the tensor instruction, then the dependency descriptor of the tensor instruction is set to the third descriptor.

[0082] For example, see Figure 5 The target basic block includes three MMA instructions: Instruction I1: D1 = mma(a1, b1, c1), Instruction I2: D2 = mma(a2, b2, c2), and Instruction I3: D3 = mma(a3, b3, D1). The third operation used by instruction I3 is D1 (i.e., the output data of instruction I1); therefore, instruction I3 depends on instruction I1. There is an instruction I2 between instruction I3 and instruction I1, that is, instruction I1 is the second MMA instruction backtracking from instruction I3. Therefore, the dependency descriptor of instruction I3 is set to the third descriptor dep2.

[0083] For example, see Figure 6 The target basic block includes four MMA instructions: Instruction I1: D1 = mma(a1, b1, c1), Instruction I2: D2 = mma(a2, b2, c2), Instruction I3: D3 = mma(a3, b3, c3), and Instruction I4: D4 = mma(a4, b4, D1). The third operation used by instruction I4 is D1 (i.e., the output data of instruction I1); therefore, instruction I4 depends on instruction I1. There are two MMA instructions between instruction I4 and instruction I1, that is, instruction I1 is the third MMA instruction backtracking from instruction I4. Therefore, the dependency descriptor of instruction I4 is also set to the third descriptor deep2.

[0084] In this embodiment, when pipelined processing of tensor instructions, multiple stages of the tensor instruction are executed systematically according to a fixed arrangement of hardware resources. These stages include decoding, address fetching, execution, and write-back. Each stage has a different function and can be further subdivided internally; for example, the execution stage can be divided into internal sub-stages such as registering, type conversion, accumulation, rounding, write-back, and post-processing. In practical applications, the input data to be processed by the tensor instruction is divided into multiple data units according to the standard unit size. Then, the pipeline executes the above multiple stages sequentially on the multiple data units. The time required for each data unit to complete the above multiple stages is the unit execution time; the unit execution time is determined by the type of artificial intelligence chip.

[0085] In practical applications, the data unit can be an identity matrix or other data types; this application does not impose specific limitations on this. The following explanation uses the process of executing MMA instructions (i.e., unit-mma) on an identity matrix as an example:

[0086] The input data to be processed by MMA instructions is divided into multiple identity matrices (i.e., matrices of a standard unit size, such as 16x16x32 matrices). The time required for each identity matrix to complete multiple stages (i.e., unit execution time) is determined by the type of AI chip; for example, the unit execution time for each identity matrix is ​​38 clock cycles. Multiple identity matrices do not have data dependencies and are executed in parallel.

[0087] In practice, the decoding resources are first used to decode the first identity matrix, and then the decoding resources are released after decoding. Then, the addressing resources are used to dereference the first identity matrix, and the released decoding resources are used to decode the second identity matrix. This process is repeated in sequence, so that the hardware resources in the pipeline are reused continuously.

[0088] Extra Cycle refers to the difference between the total processing time required to complete the above multiple stages on the input data and the unit execution time mentioned above.

[0089] For example, see Figure 7 When processing input data using tensor instructions, the input data is split into four identity matrices: identity matrix 1, identity matrix 2, identity matrix 3, and identity matrix 4. Decoding resources are used to decode identity matrix 1, and the decoding resources are released after decoding. Addressing resources are used to fetch the address of identity matrix 1, and the released decoding resources are used to decode identity matrix 2.

[0090] After addressing identity matrix 1, release the addressing resources; then use the execution resources to execute identity matrix 1. After decoding identity matrix 2, release the decoding resources; use the released addressing resources to address identity matrix 2, and simultaneously use the released decoding resources to decode identity matrix 3.

[0091] This process continues until identity matrix 4 is written back. The unit execution time for each identity matrix is ​​38 clock cycles, with an additional 20 clock cycles. The sum of the unit execution time and the additional execution time represents the total processing time for processing the entire input data using tensor instructions.

[0092] It should be noted that when there is no dependency between two adjacent tensor instructions, after the previous tensor instruction finishes decoding the last identity matrix of the input data and releases the decoding resources, the next tensor instruction can use the released decoding resources to decode the first identity matrix of the input data.

[0093] When there is a dependency between two adjacent tensor instructions, after the preceding tensor instruction finishes decoding the last identity matrix of the input data and releases the decoding resources, the following tensor instruction cannot use the released decoding resources to decode the first identity matrix of the input data. Instead, it must wait for the preceding tensor instruction to complete the write-back of the last identity matrix of the input data before the following tensor instruction can use the released decoding resources to decode the first identity matrix of the input data. During the process from the release of decoding resources to the completion of the write-back by the preceding tensor instruction, the pipeline is blocked, and decoding resources, value fetching resources, and execution resources are all idle, resulting in wasted hardware resources.

[0094] In this embodiment, the additional execution time is determined based on the data type and size of the input data. Specifically, see Tables 1 and 2 below for the additional execution time schedule of MMA instructions.

[0095]

[0096]

[0097] Among them, bf16 represents a 16-bit floating-point number, fp16 represents a half-precision floating-point number, tf32 represents a 32-bit tensor floating-point number, fp8 represents an 8-bit floating-point number, bf8 represents an 8-bit floating-point number, s8 represents a signed 8-bit integer, and u8 represents an unsigned 8-bit integer.

[0098] In some embodiments, the attribute information record of each tensor instruction recorded in the local layout container may also include: the additional execution time of the tensor instruction.

[0099] After determining the L intermediate instructions between the current traversal tensor instruction and the target instruction, the additional execution time of each of the L intermediate instructions is obtained by querying the local layout container; then the L additional execution times are summed to obtain the total additional time.

[0100] Step 203: If the total additional time is greater than the unit execution time of the tensor instruction, then set the dependency descriptor of the tensor instruction as the first descriptor.

[0101] Specifically, as described above, when there are L intermediate instructions between the tensor instruction being traversed and its dependent target instruction, the dependency descriptor of the tensor instruction is first set to the third descriptor. If the total extra time of the L intermediate instructions is greater than the unit execution time of the tensor instruction being traversed, then the dependency descriptor of the tensor instruction is adjusted from the third descriptor to the first descriptor.

[0102] In this embodiment, the total extra time represents the time interval between the target instruction and the currently traversed tensor instruction; specifically, the total extra time represents the time interval between the start time of processing the last identity matrix that the target instruction needs to process and the start time of processing the first identity matrix that the currently traversed tensor instruction needs to process.

[0103] If the total extra time is greater than the unit execution time, it means that after the last identity matrix that the target instruction needs to process is issued (i.e., processing begins), the last identity matrix can be processed and completed (i.e., write-back is completed) within the total extra time.

[0104] In other words, after the last identity matrix that the target instruction needs to process is issued, the last identity matrix is ​​processed by a pipeline, as well as the input data that each of the L intermediate instructions needs to process. The input data that each intermediate instruction needs to process can be divided into multiple identity matrices.

[0105] For the last intermediate instruction executed out of L intermediate instructions, the decoding resources are released after the decoding of the last identity matrix of the input data is completed. At this point, the last identity matrix that the target instruction needs to process must have been written back, meaning the output data of the target instruction that the currently traversed tensor instruction depends on has been written back. Therefore, the released decoding resources can be used directly to decode the first identity matrix that the currently traversed tensor instruction needs to process. The currently traversed tensor instruction does not actually need to wait additionally, i.e., it has no dependency relationship. Therefore, the dependency descriptor of this tensor instruction is adjusted from the third descriptor to the first descriptor.

[0106] by Figure 6 Taking the target basic block shown as an example, the unit execution time is set to 38 clock cycles; the additional execution time of instructions I2 and I3 is obtained.

[0107] Case 1: If the extra execution time of instruction I2 is 8 clock cycles and the extra execution time of instruction I3 is 8 clock cycles, then the total extra time of instructions I2 and I3 is 16 clock cycles.

[0108] At this point, the total extra time is less than the unit execution time. Therefore, after the last identity matrix that instruction I1 needs to process is emitted, it cannot be processed within the total extra time. That is, when instruction I3 finishes decoding the last identity matrix of the input data, the last identity matrix that instruction I1 needs to process has not yet been written back. Instruction I4 depends on the output data written back by instruction I1. Therefore, instruction I4 cannot be executed immediately using the released decoding resources, but must continue to wait until the last identity matrix that instruction I1 needs to process is written back. In this case, the dependency descriptor of instruction I4 will not be adjusted to the first descriptor dep0.

[0109] Scenario 2: If the extra execution time for instruction I2 is 24 clock cycles and the extra execution time for instruction I3 is 16 clock cycles, then the total extra time for instructions I2 and I3 is 40 clock cycles. In this case, the total extra time is greater than the unit execution time. Therefore, after the last identity matrix that instruction I1 needs to process is emitted, it can be processed within the total extra time. That is, when instruction I3 finishes decoding the last identity matrix of the input data, the last identity matrix that instruction I1 needs to process has already been written back. At this point, the released decoding resources can be used immediately to execute instruction I4, i.e., no waiting is required. In this case, the dependency descriptor of instruction I4 is adjusted to the first descriptor dep0.

[0110] In this embodiment, during the compilation phase of the target basic block, N tensor instructions in the target basic block are traversed sequentially. For each tensor instruction traversed, the M traversed tensor instructions are queried. If one of the M tensor instructions is a target instruction that the currently traversed tensor instruction depends on, and there are L intermediate instructions between the tensor instruction and the target instruction, then the total additional time (i.e., the time interval between the currently traversed tensor instruction and its dependent target instruction) is obtained based on the additional execution time of each of the L intermediate instructions. If the total additional time is greater than the unit execution time of the currently traversed tensor instruction, it indicates that when the pipeline reaches the currently traversed tensor instruction, the target instruction that the currently traversed tensor instruction depends on must have already been executed. Therefore, the currently traversed tensor instruction can be executed directly without additional waiting. Thus, the dependency descriptor of the currently traversed tensor instruction is set to the first descriptor to indicate that there is no substantial dependency between the currently traversed tensor instruction and other tensor instructions.

[0111] In this way, during the execution phase of the target basic block, when the pipelined execution of each tensor instruction is used, when the pipeline executes a tensor instruction whose dependency descriptor is the first descriptor, the hardware resources can be used directly to execute the tensor instruction without additional waiting, thereby greatly reducing the probability of pipeline blockage, reducing the idle running of hardware resources in the pipeline, and thus improving the utilization rate of hardware resources on the artificial intelligence chip.

[0112] In some embodiments, the first instruction number corresponding to the tensor instruction being traversed and the second instruction number corresponding to the target instruction are obtained; then, based on the first instruction number and the second instruction number, the number of intermediate instructions between the tensor instruction being traversed and the target instruction is determined; the number of intermediate instructions is 0 or L.

[0113] Specifically, the N tensor instructions in the target basic block can all be the original instructions of the target basic block.

[0114] In practical applications, the relationships between basic blocks are recorded using a Control Flow Graph (CFG). Different basic blocks have both live-in and live-out information. The CFG is used to obtain the predecessor and successor basic blocks of the target basic block; that is, which predecessor basic blocks' outputs affect the target basic block. In this case, the outputs of these predecessor basic blocks (i.e., the live-out information) are merged to obtain the live-in information of the target basic block. Similarly, when the target basic block affects a successor basic block, the live-out information of the target basic block is also the live-in information of the successor basic block. Therefore, in this embodiment, the N tensor instructions in the target basic block may not only contain the original instructions of the target basic block but may also include tensor instructions imported from other basic blocks.

[0115] For each basic block, the raw instructions in the basic block are arranged according to a preset order (i.e., execution order) to obtain a raw instruction sequence. Each raw instruction in the raw instruction sequence is assigned a sequence number from beginning to end to obtain a forward sequence number; and each raw instruction in the raw instruction sequence is assigned a sequence number from end to beginning to obtain a reverse sequence number. The forward sequence number can be set to start counting from 1 and the reverse sequence number from 0; alternatively, the forward sequence number can start counting from 0 and the reverse sequence number from 1. This application does not specifically limit this.

[0116] In some embodiments, the tensor instruction currently being traversed and the target instruction it depends on are the original instructions in the target basic block; the first instruction number is the ascending sequence number of the tensor instruction currently being traversed in the target basic block; and the second instruction number is the ascending sequence number of the target instruction in the target basic block.

[0117] Based on the target difference between the first instruction number and the second instruction number, determine the number of intermediate instructions between the tensor instruction being traversed and the target instruction.

[0118] Specifically, obtain the target difference between the first instruction number and the second instruction number; subtract one from the target difference to obtain the number of intermediate instructions between the currently traversed tensor instruction and the target instruction.

[0119] When the target difference is 1, it means that the number of intermediate instructions between the tensor instruction being traversed and the target instruction is 0. In this case, the dependency descriptor of the tensor instruction being traversed is set to the second descriptor.

[0120] When the target difference is greater than or equal to 2, it means that the number of intermediate instructions between the currently traversed tensor instruction and the target instruction is L ( If the current tensor instruction is traversed, then the dependency descriptor of the tensor instruction will be set to the third descriptor.

[0121] In some embodiments, the tensor instruction currently being traversed is the original instruction in the target basic block; the target instruction is the tensor instruction imported from other basic blocks; the first instruction number is the ascending sequence number of the tensor instruction currently being traversed in the target basic block; and the second instruction number is the descending sequence number of the target instruction in other basic blocks.

[0122] Based on the sum of the first instruction number and the second instruction number, determine the number of intermediate instructions between the tensor instruction being traversed and the target instruction.

[0123] Specifically, the exported information of all predecessor basic blocks that affect the target basic block is used as the imported information of the target basic block, and the imported information of the target basic block is recorded in the local layout container; wherein, the exported information of the predecessor basic blocks includes: the attribute information record of the tensor instructions in the predecessor basic blocks that affect the target basic block.

[0124] For different predecessor basic blocks, if the same tensor instruction is defined (i.e., the output data is stored in the same register), then the tensor instruction with the smallest reverse sequence number is selected, and its attribute information is recorded and stored in the local layout container. If the reverse sequence numbers are the same, then the tensor instruction with the shortest additional execution time is selected, and its attribute information is recorded and stored in the local layout container.

[0125] If the tensor instruction being traversed is an original instruction in the target basic block, and the target instruction is a tensor instruction imported from another basic block, then obtain the sum of the first instruction number and the second instruction number; subtract one from the sum to obtain the number of intermediate instructions between the tensor instruction being traversed and the target instruction.

[0126] When the summation result is 1, it means that the number of intermediate instructions between the current tensor instruction and the target instruction is 0, and the dependency descriptor of the current tensor instruction is set to the second descriptor.

[0127] When the summation result is greater than or equal to 2, it indicates that the number of intermediate instructions between the currently traversed tensor instruction and the target instruction is L ( If the current tensor instruction is traversed, then the dependency descriptor of the tensor instruction will be set to the third descriptor.

[0128] For example, see Figure 8 Basic blocks 1 and 2 are the predecessor basic blocks of basic block 3. The output data of instruction I3 in both basic block 1 and basic block 2 affects instruction I0 in basic block 3. Simultaneously, the output data of instruction I3 in both basic block 1 and basic block 2 are stored in register D3. The forward sequence number starts from 1, and the reverse sequence number starts from 0. The reverse sequence number of instruction I3 in basic block 1 is 0, and the reverse sequence number of instruction I3 in basic block 2 is 1. Therefore, instruction I3 in basic block 1 (i.e., the instruction with the smaller reverse sequence number) is imported into basic block 3.

[0129] At this point, in basic block 3, the forward sequence number of instruction I0 is 1, the reverse sequence number of the imported instruction I3 is 0, and the sum of the forward and reverse sequence numbers is 1. Therefore, the dependency descriptor of instruction I0 is set to the second descriptor deep1.

[0130] For example, see Figure 9 Basic blocks 1 and 2 are the predecessor basic blocks of basic block 3. The output data of instruction I2 in basic block 1 and the output data of instruction I1 in basic block 2 both affect instruction I0 in basic block 3. Simultaneously, the output data of instruction I2 in basic block 1 and the output data of instruction I1 in basic block 2 are both stored in register D2 (i.e., the same MMA instruction). The forward sequence number starts from 1, and the reverse sequence number starts from 0. The reverse sequence number of instruction I2 in basic block 1 is 1, and the reverse sequence number of instruction I1 in basic block 2 is 2. Therefore, instruction I2 in basic block 1 (i.e., the instruction with the smaller reverse sequence number) is imported into basic block 3. Since executing instruction I2 in basic block 1 requires executing instruction I3 in basic block 1 before executing instruction I0 in basic block 3, instruction I3 in basic block 1 also affects basic block 3. Therefore, instruction I3 in basic block 1 is also imported into basic block 3.

[0131] At this point, in basic block 3, the forward sequence number of instruction I0 is 1, the reverse sequence number of instruction I2 is 1, and the sum of the forward and reverse sequence numbers is 2. Therefore, the dependency descriptor of instruction I0 is set to the third descriptor.

[0132] In basic block 3, the imported instruction I2 has an additional execution time of 10 clock cycles, and the imported instruction I3 has an additional execution time of 32 clock cycles. Thus, the total additional time is 42 clock cycles, which is greater than the unit execution time (i.e., 38 clock cycles). Therefore, the dependency descriptor of instruction I0 is optimized from the third descriptor dep2 to the first descriptor dep0.

[0133] In this embodiment, the dependencies between tensor instructions in the operator are determined by software algorithms, and corresponding descriptors are set for the dependencies. At the same time, relevant optimizations are performed on the third descriptor to maximize the pipeline parallelism of the artificial intelligence chip. This eliminates the need for hardware units to identify dependencies and control the execution order of instructions, thereby saving valuable chip area and a significant amount of cost. It also reduces the probability of pipeline blockage, improves pipeline utilization, and increases throughput.

[0134] Based on the same technical concept, this application provides a schematic diagram of the structure of a tensor instruction optimization device, such as... Figure 10 As shown, the tensor instruction optimization device 1000 includes:

[0135] The traversal module 1001 is used to sequentially traverse the N tensor instructions in the target basic block. For each tensor instruction traversed, the module queries the M traversed tensor instructions, where the traversed tensor instruction is not the first tensor instruction traversed. ;

[0136] The decision module 1002 is configured to, if among the M tensor instructions there exists a target instruction that the tensor instruction depends on, and there are L intermediate instructions between the tensor instruction and the target instruction, obtain the total additional time based on the additional execution time of each of the L intermediate instructions, wherein... ;

[0137] The setting module 1003 is configured to set the dependency descriptor of the tensor instruction to a first descriptor if the total additional time is greater than the unit execution time of the tensor instruction, wherein the first descriptor indicates no dependency relationship.

[0138] Optionally, the setting module 1003 is further configured to:

[0139] If none of the M tensor instructions are the target instruction that the tensor instruction depends on, then the dependency descriptor of the tensor instruction is set to the first descriptor.

[0140] Optionally, the setting module 1003 is further configured to:

[0141] If one of the M tensor instructions is a target instruction that the tensor instruction depends on, and there is no intermediate instruction between the tensor instruction and the target instruction, then the dependency descriptor of the tensor instruction is set to the second descriptor, which indicates that it depends on the previous tensor instruction.

[0142] Optionally, the setting module 1003 is further configured to:

[0143] Before obtaining the total additional time based on the additional execution time of each of the L intermediate instructions, the dependency descriptor of the tensor instruction is set to the third descriptor, which represents the second tensor instruction that depends on the backward lookup.

[0144] The setting module 1003 is specifically used for:

[0145] If the total additional time is greater than the unit execution time of the tensor instruction, then the dependency descriptor of the tensor instruction is changed from the third descriptor to the first descriptor.

[0146] Optionally, the decision module 1002 is further configured to:

[0147] Based on the input data of the single tensor instruction, query the output data of each of the M tensor instructions;

[0148] When there is output data in the output data of each of the M tensor instructions that matches the input data, the tensor instruction corresponding to the output data is taken as the target instruction that the tensor instruction depends on.

[0149] Optionally, the decision module 1002 is further configured to:

[0150] Obtain the first instruction number corresponding to the tensor instruction and the second instruction number corresponding to the target instruction;

[0151] Based on the first instruction number and the second instruction number, the number of intermediate instructions between the tensor instruction and the target instruction is determined, wherein the number of intermediate instructions is 0 or L.

[0152] Optionally, the tensor instruction and the target instruction are the original instructions in the target basic block; the first instruction number is the ascending sequence number of the tensor instruction in the target basic block; and the second instruction number is the ascending sequence number of the target instruction in the target basic block.

[0153] The decision module 1002 is specifically used for:

[0154] Based on the target difference between the first instruction number and the second instruction number, the number of intermediate instructions between the tensor instruction and the target instruction is determined.

[0155] Optionally, the tensor instruction is the original instruction in the target basic block, and the target instruction is the tensor instruction imported from other basic blocks; the first instruction sequence number is the forward sequence number of the tensor instruction in the target basic block, and the second instruction sequence number is the reverse sequence number of the target instruction in the other basic blocks;

[0156] The decision module 1002 is specifically used for:

[0157] Based on the sum of the first instruction number and the second instruction number, the number of intermediate instructions between the tensor instruction and the target instruction is determined.

[0158] In this embodiment, during the compilation phase of the target basic block, N tensor instructions in the target basic block are traversed sequentially. For each tensor instruction traversed, the M traversed tensor instructions are queried. If one of the M tensor instructions is a target instruction that the currently traversed tensor instruction depends on, and there are L intermediate instructions between the tensor instruction and the target instruction, then the total additional time (i.e., the time interval between the currently traversed tensor instruction and its dependent target instruction) is obtained based on the additional execution time of each of the L intermediate instructions. If the total additional time is greater than the unit execution time of the currently traversed tensor instruction, it indicates that when the pipeline reaches the currently traversed tensor instruction, the target instruction that the currently traversed tensor instruction depends on must have already been executed. Therefore, the currently traversed tensor instruction can be executed directly without additional waiting. Thus, the dependency descriptor of the currently traversed tensor instruction is set to the first descriptor to indicate that there is no substantial dependency between the currently traversed tensor instruction and other tensor instructions.

[0159] In this way, during the execution phase of the target basic block, when the pipelined execution of each tensor instruction is used, when the pipeline executes a tensor instruction whose dependency descriptor is the first descriptor, the hardware resources can be used directly to execute the tensor instruction without additional waiting, thereby greatly reducing the probability of pipeline blockage, reducing the idle running of hardware resources in the pipeline, and thus improving the utilization rate of hardware resources on the artificial intelligence chip.

[0160] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0161] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 11 As shown, it includes at least one artificial intelligence chip 100 and a memory 1101 connected to at least one artificial intelligence chip 100. In this embodiment, the specific connection medium between the artificial intelligence chip 100 and the memory 1101 is not limited. Figure 11Taking the connection between the AI ​​chip 100 and the memory 1101 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0162] In this embodiment of the application, the memory 1101 stores instructions that can be executed by at least one artificial intelligence chip 100. By executing the instructions stored in the memory 1101, the at least one artificial intelligence chip 100 can perform the steps of the tensor instruction optimization method described above.

[0163] The artificial intelligence chip 100 serves as the control center of the computer device, connecting to various parts of the device via various interfaces and lines. It achieves tensor instruction optimization by running or executing instructions stored in the memory 1101 and accessing data stored in the memory 1101. Optionally, the artificial intelligence chip 100 may include one or more processing units. It may integrate an application processor and a modem processor, where the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 1101 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0164] The artificial intelligence chip 100 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0165] Memory 1101, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 1101 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. Memory 1101 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. In the embodiments of this application, memory 1101 can also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0166] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the tensor instruction optimization method described above.

[0167] Based on the same inventive concept, this application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the tensor instruction optimization method described above.

[0168] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0169] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0170] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer device or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0171] These computer program instructions may also be loaded onto a computer device or other programmable data processing equipment to cause a series of operational steps to be performed on the computer device or other programmable equipment to produce a process implemented by the computer device, thereby providing instructions that execute on the computer device or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0172] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0173] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A tensor instruction optimization method, characterized in that, include: The N tensor instructions in the target basic block are traversed sequentially. For each tensor instruction traversed, the M traversed tensor instructions are checked, where the tensor instruction is not the first tensor instruction traversed. ; If one of the M tensor instructions is a target instruction that the tensor instruction depends on, and there are L intermediate instructions between the tensor instruction and the target instruction, then the total additional time is obtained based on the additional execution time of each of the L intermediate instructions, where... ; If the total additional time is greater than the unit execution time of the tensor instruction, then the dependency descriptor of the tensor instruction is set to the first descriptor, where the first descriptor indicates no dependency.

2. The method as described in claim 1, characterized in that, Also includes: If none of the M tensor instructions are the target instruction that the tensor instruction depends on, then the dependency descriptor of the tensor instruction is set to the first descriptor.

3. The method as described in claim 1, characterized in that, Also includes: If one of the M tensor instructions is a target instruction that the tensor instruction depends on, and there is no intermediate instruction between the tensor instruction and the target instruction, then the dependency descriptor of the tensor instruction is set to the second descriptor, which indicates that it depends on the previous tensor instruction.

4. The method as described in claim 1, characterized in that, Before obtaining the total additional time based on the additional execution time of each of the L intermediate instructions, the method further includes: Set the dependency descriptor of the tensor instruction to the third descriptor, where the third descriptor represents the second tensor instruction that depends on the backward lookup. If the total additional time is greater than the unit execution time of a tensor instruction, then setting the dependency descriptor of the tensor instruction as the first descriptor includes: If the total additional time is greater than the unit execution time of the tensor instruction, then the dependency descriptor of the tensor instruction is changed from the third descriptor to the first descriptor.

5. The method as described in claim 1, characterized in that, The target instruction that a tensor instruction depends on is determined in the following way: Based on the input data of the single tensor instruction, query the output data of each of the M tensor instructions; When there is output data in the output data of each of the M tensor instructions that matches the input data, the tensor instruction corresponding to the output data is taken as the target instruction that the tensor instruction depends on.

6. The method according to any one of claims 1 to 5, characterized in that, Also includes: Obtain the first instruction number corresponding to the tensor instruction and the second instruction number corresponding to the target instruction; Based on the first instruction number and the second instruction number, the number of intermediate instructions between the tensor instruction and the target instruction is determined, wherein the number of intermediate instructions is 0 or L.

7. The method as described in claim 6, characterized in that, The tensor instruction and the target instruction are the original instructions in the target basic block; the first instruction number is the ascending sequence number of the tensor instruction in the target basic block; the second instruction number is the ascending sequence number of the target instruction in the target basic block. Determining the number of intermediate instructions between a tensor instruction and the target instruction based on the first instruction number and the second instruction number includes: Based on the target difference between the first instruction number and the second instruction number, the number of intermediate instructions between the tensor instruction and the target instruction is determined.

8. The method as described in claim 7, characterized in that, The tensor instruction is the original instruction in the target basic block, and the target instruction is the tensor instruction imported from other basic blocks; the first instruction sequence number is the ascending sequence number of the tensor instruction in the target basic block, and the second instruction sequence number is the descending sequence number of the target instruction in the other basic blocks. Determining the number of intermediate instructions between a tensor instruction and the target instruction based on the first instruction number and the second instruction number includes: Based on the sum of the first instruction number and the second instruction number, the number of intermediate instructions between the tensor instruction and the target instruction is determined.

9. A computer device comprising a memory, an artificial intelligence chip, and a computer program stored in the memory and running on the artificial intelligence chip, characterized in that, When the artificial intelligence chip executes the computer program, it implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that is executed by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method as described in any one of claims 1 to 8.

Images