Artificial intelligence chip and operating method thereof

Abstract

Embodiments of the present disclosure provide an artificial intelligence chip and an operating method thereof. The artificial intelligence chip includes an instruction emitting unit, a memory access queue unit, a computing task queue unit, a crossbar, and a memory access and computing task scheduling unit. The crossbar is coupled to the instruction emitting unit, the memory access queue unit, and the computing task queue unit. The crossbar classifies a plurality of instructions emitted by the instruction emitting unit into memory access class instructions and computing task class instructions. The crossbar stores the memory access class instructions into the memory access queue unit. The crossbar stores the computing task class instructions into the computing task queue unit. The memory access and computing task scheduling unit controls outputs of the memory access queue unit and the computing task queue unit. The memory access and computing task scheduling unit executes the memory access class instructions output by the memory access queue unit.

Description

TECHNICAL FIELD

[0001] Embodiments of the present disclosure relate to the technical field of integrated circuits, and in particular, to an artificial intelligence (AI) chip and an operating method thereof. BACKGROUND

[0002] A matrix is a set of complex or real numbers arranged in a rectangular array, which has rows and columns. The number of elements contained in a matrix is equal to the number of rows multiplied by the number of columns. In the field of artificial intelligence (AI), much of the information processed by AI chips is multi-dimensional information. For example, a picture is a two-dimensional matrix composed of a large number of pixels. Using a matrix to represent this data is intuitive and convenient for mathematical processing. Matrix multiplication, or matrix multiplication and addition (MMA), is widely used in large-scale neural networks, image processing, scientific computing, and other applications.

[0003] A program containing MMA instructions is usually executed on one or more warps in an artificial intelligence chip or a general-purpose computing on GPU (GPGPU). Before executing the MMA instructions, the program needs to ensure that the data (operands) required by the MMA has been moved to the register file of the current warp or the shared memory. Before emitting the MMA instructions, the instruction emission unit needs to ensure that the operands required by the MMA have been read from the register file to the operand collector. Currently, the instruction emission unit first emits a memory access type instruction, then checks whether the operands have been returned to the operand collector, and then issues a matrix calculation instruction (a calculation task type instruction). The current instruction emission unit is a serial issue and execution scheme. The instruction emission unit cannot continuously issue matrix calculation instructions, and there is a bubble in the middle, which reduces the overall MMA execution efficiency. How to improve the efficiency of AI chips is one of the many technical issues in the field. SUMMARY

[0004] Embodiments of the present disclosure are directed to an artificial intelligence (AI) chip and an operating method thereof to improve execution efficiency.

[0005] In embodiments according to the present disclosure, an artificial intelligence chip includes a plurality of instruction launch units, a memory access queue unit, a compute task queue unit, a crossbar, an on-chip memory, an operand collection unit, and a memory access and compute task scheduling unit. The crossbar is coupled to the plurality of instruction launch units, the memory access queue unit, and the compute task queue unit. The plurality of instructions received by the crossbar launched by the plurality of instruction launch units include memory access class instructions and compute task class instructions. The crossbar stores the memory access class instructions in the memory access queue unit according to instruction type. The crossbar stores the compute task class instructions in the compute task queue unit according to instruction type. The memory access and compute task scheduling unit is coupled to the on-chip memory, the operand collection unit, the memory access queue unit, and the compute task queue unit. The memory access and compute task scheduling unit controls the output of the memory access queue unit and the compute task queue unit. The memory access and compute task scheduling unit executes the memory access class instructions output by the memory access queue unit to read data from the on-chip memory to the operand collection unit.

[0006] In embodiments according to the present disclosure, an operation method includes receiving, by a crossbar of an artificial intelligence chip, a plurality of instructions launched by a plurality of instruction launch units of the artificial intelligence chip, wherein the plurality of instructions include memory access class instructions and compute task class instructions, and the crossbar is coupled to the plurality of instruction launch units, a memory access queue unit of the artificial intelligence chip, and a compute task queue unit of the artificial intelligence chip; storing, by the crossbar, the memory access class instructions in the memory access queue unit according to instruction type; storing, by the crossbar, the compute task class instructions in the compute task queue unit according to instruction type; controlling, by a memory access and compute task scheduling unit of the artificial intelligence chip, the output of the memory access queue unit and the compute task queue unit, wherein the memory access and compute task scheduling unit is coupled to an on-chip memory of the artificial intelligence chip, an operand collection unit of the artificial intelligence chip, the memory access queue unit, and the compute task queue unit; and executing, by the memory access and compute task scheduling unit, the memory access class instructions output by the memory access queue unit to read data from the on-chip memory to the operand collection unit.

[0007] Based on the above, the AI ​​chip is configured with a crossbar switch, a memory access queue unit, a computation task queue unit, and a memory access and computation task scheduling unit. The instruction issuing unit can issue multiple instructions in parallel to the crossbar switch without considering the cooperative relationship between memory access instructions and computation task instructions. Therefore, the instruction issuing unit reduces "blocking caused by instruction order," thereby improving efficiency. The crossbar switch stores computation task instructions into the computation task queue unit and memory access instructions into the memory access queue unit according to the instruction type. The memory access and computation task scheduling unit can perform synchronization control on the memory access queue unit and the computation task queue unit, so that the memory access queue unit and the computation task queue unit output cooperative memory access instructions and computation task instructions according to a synchronized timing. For example, the time when the memory access instruction issued by the memory access queue unit completes execution (retrieves the corresponding operand) is earlier than the time when the computation task queue unit outputs the corresponding computation task instruction. Therefore, data synchronization between memory access instructions and computation task instructions is ensured.

[0008] To make the above features and advantages of the embodiments of this disclosure more apparent and understandable, specific embodiments are described below in conjunction with the accompanying drawings. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of a circuit block of an artificial intelligence (AI) chip according to an embodiment of the present disclosure.

[0010] Figure 2 This is a flowchart illustrating an operation method of an AI chip according to an embodiment of the present disclosure.

[0011] Figure 3 This is a schematic diagram of the circuit modules of a storage access queue unit, a computing task queue unit, and a storage access and computing task scheduling unit according to an embodiment of the present disclosure.

[0012] Figure 4 This is a schematic diagram of a circuit module of a synchronization unit according to an embodiment of the present disclosure.

[0013] Figure 5 This is a schematic diagram of a programming model according to an embodiment of the present disclosure.

[0014] Figure 6 This is a schematic diagram of a programming model according to another embodiment of the present disclosure.

[0015] Figure 7 This is a schematic diagram of a programming model according to yet another embodiment of the present disclosure.

[0016] Figure 8 is a circuit module schematic diagram of an operand collection unit and a matrix computation data path shown according to an embodiment of the present disclosure.

[0017] Figure 9 is a circuit module schematic diagram of an operand collection unit shown according to another embodiment of the present disclosure.

[0018] Figure 10 is a circuit module schematic diagram of an operand collection unit shown according to yet another embodiment of the present disclosure.

[0019] BRIEF DESCRIPTION OF DRAWINGS

[0020] 100: Artificial intelligence (AI) chip

[0021] 110_1, 110_2, 110_N: Instruction issue unit

[0022] 120: Crossbar

[0023] 130: Memory access queue unit

[0024] 131_1, 131_N: Memory access queue

[0025] 132, 142: Arbiter

[0026] 140: Compute task queue unit

[0027] 141_1, 141_N: Compute task queue

[0028] 150: On-chip memory

[0029] 160: Memory access and compute task scheduling unit

[0030] 161: Synchronization unit

[0031] 162: Memory access scheduling unit

[0032] 170: Operand collection unit

[0033] 171_1, 171_2, 171_N, 173_1, 173_2, 173_N: Register bank

[0034] 172, 174: Shared memory

[0035] 180: Matrix computation data path

[0036] 181_1, 181_2, 181_N: Matrix computation unit

[0037] S210, S220, S230, S240: Step DETAILED DESCRIPTION

[0038] Reference will now be made in detail embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Wherever possible, the same reference numbers are used in the drawings and the description to refer to the same or similar parts.

[0039] The term "coupled" or "connected" used in the specification and claims of the present application can refer to any direct or indirect connection. For example, if a first device is coupled or connected to a second device, it should be interpreted that the first device can be directly connected to the second device, or the first device can be indirectly connected to the second device through other devices or connection means. The terms "first", "second", and the like used in the specification and claims of the present application are used to name components, or to distinguish different embodiments or ranges, and are not intended to limit the upper or lower limit of the number of components, nor to limit the order of components. In addition, wherever possible, the same reference numbers are used in the drawings and embodiments to represent the same or similar parts. Components / elements / steps using the same reference numbers or using the same terms in different embodiments can be referred to each other in the relevant description. It should be understood that the features of the following embodiments can be combined with each other. For example, the features of the second embodiment can be combined with the features of the first embodiment. Those skilled in the art can select the appropriate combination of features according to the actual design requirements.

[0040] Figure 1 is a circuit block diagram of an artificial intelligence (AI) chip 100 according to an embodiment of the present disclosure. Figure 1 The AI chip 100 of includes a plurality of instruction emitting units (e.g. Figure 1The matrix computation accelerator 100 includes instruction issue units 110_1, 110_2, …, 110_N, a crossbar 120, a memory access queue unit 130, a compute task queue unit 140, on-chip storage 150, a memory access and compute task scheduling unit 160, an operand collection unit 170, and a matrix computation data path 180. The number N of instruction issue units 110_1-110_N can be determined according to actual design and application. According to different designs, in some embodiments, the implementation of at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180 can be a hardware circuit. In other embodiments, the implementation of at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180 can be a combination of more than one of hardware, firmware, and software (i.e., a program).

[0041] In hardware form, at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180 can be implemented as logic circuitry on an integrated circuit. For example, the functionality of at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180 can be implemented in various logic blocks, modules, and circuits of one or more hardware controllers, microcontrollers, hardware processors, microprocessors, application-specific integrated circuits (ASICs), digital signal processors (DSPs), field programmable gate arrays (FPGAs), central processing units (CPUs), or other processing units. The functionality of at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180 can be implemented as hardware circuitry, such as various logic blocks, modules, and circuits in an integrated circuit, using hardware description languages (such as Verilog HDL or VHDL) or other suitable programming languages.

[0042] In software or firmware form, the functions of at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180 can be implemented as programming codes. For example, at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180 can be implemented using general programming languages (e.g., C, C++, or assembly language) or other suitable programming languages. The programming codes can be recorded / stored in a "non-transitory machine-readable storage medium." In some embodiments, the non-transitory machine-readable storage medium includes, for example, semiconductor memories and / or storage devices. An electronic device (e.g., a CPU, a hardware controller, a microcontroller, a hardware processor, or a microprocessor) can read and execute the programming codes from the non-transitory machine-readable storage medium to implement the functions of at least one of the instruction issue units 110_1-110_N, the crossbar 120, the memory access queue unit 130, the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the matrix computation data path 180.

[0043] The present embodiments are not limited to the specific implementation of the crossbar 120. For example, based on actual design and application, the crossbar 120 can include a well-known crossbar or other crossbars. The crossbar 120 is coupled to the instruction issue units 110_1-110_N, the memory access queue unit 130, and the compute task queue unit 140. The memory access and compute task scheduling unit 160 is coupled to the memory access queue unit 130, the compute task queue unit 140, the on-chip memory 150, and the operand collection unit 170. The matrix computation data path 180 is coupled to the compute task queue unit 140, the memory access and compute task scheduling unit 160, and the operand collection unit 170.

[0044] Figure 2 is a flowchart of an operation method of an AI chip according to an embodiment of the present disclosure. Please refer to Figure 1 and Figure 2In step S210, the crossbar 120 receives a plurality of instructions issued by the instruction issue units 110_1 to 110_N, where the plurality of instructions include memory access class instructions and compute task class instructions. In step S220, the crossbar 120 stores the memory access class instructions in the memory access queue unit 130 according to instruction type, and the crossbar 120 stores the compute task class instructions in the compute task queue unit 140 according to instruction type. In step S230, the memory access and compute task scheduling unit 160 controls the output of the memory access queue unit 130 and the compute task queue unit 140. For example, the memory access and compute task scheduling unit 160 controls the memory access queue unit 130 and the compute task queue unit 140 such that the memory access class instruction issued by the memory access queue unit 130 is completed (retrieving the corresponding operand) before the compute task class instruction corresponding to the memory access class instruction is output by the compute task queue unit 140.

[0045] According to the control of the memory access and compute task scheduling unit 160, the memory access queue unit 130 and the compute task queue unit 140 output memory access class instructions and compute task class instructions that are cooperatively related. In step S240, the memory access and compute task scheduling unit 160 executes the memory access class instruction output by the memory access queue unit 130 to read the corresponding operand of the memory access class instruction from the on-chip memory 150 and store the corresponding operand in the operand collection unit 170. The matrix computation data path 180 executes the compute task class instruction corresponding to the memory access class instruction. The matrix computation data path 180 fetches the corresponding operand from the operand collection unit 170 to execute the compute task class instruction output by the compute task queue unit 140. After the compute task class instruction is completed, the matrix computation data path 180 stores the result of the compute task class instruction in the on-chip memory 150 or a register file.

[0046] In summary, the AI ​​chip 100 is configured with a crossbar switch 120, a storage access queue unit 130, a computation task queue unit 140, and a storage access and computation task scheduling unit 160. Instruction issuing units 110_1 to 110_N can issue multiple instructions in parallel to the crossbar switch 120 without considering the cooperative relationship between storage access instructions and computation task instructions. Therefore, instruction issuing units 110_1 to 110_N reduce "blocking caused by instruction order," thereby improving efficiency. The crossbar switch 120 stores storage access instructions in the storage access queue unit 130 and computation task instructions in the computation task queue unit 140. The storage access and computation task scheduling unit 160 can synchronously control the storage access queue unit 130 and the computation task queue unit 140, so that the storage access queue unit 130 and the computation task queue unit 140 output cooperative storage access instructions and computation task instructions to the storage access and computation task scheduling unit 160 and the matrix computation data path 180 according to a synchronous timing sequence. For example, the time when the memory access queue unit 130 completes the execution of the memory access instruction (retrieves the corresponding operand) is earlier than the time when the computation task queue unit 140 outputs the corresponding computation task instruction. Therefore, data synchronization between the memory access instruction and the computation task instruction is ensured.

[0047] Figure 3 This is a schematic diagram of the circuit modules of the storage access queue unit 130, the computing task queue unit 140, and the storage access and computing task scheduling unit 160 according to an embodiment of the present disclosure. Figure 3 The storage access queue unit 130, the computation task queue unit 140, and the storage access and computation task scheduling unit 160 shown can be used as... Figure 1 This is one of many implementation examples of the storage access queue unit 130, the computing task queue unit 140, and the storage access and computing task scheduling unit 160 shown. Figure 3 The instruction issuing units 110_1 to 110_N, crossbar switch 120, memory access queue unit 130, computation task queue unit 140, on-chip memory 150, memory access and computation task scheduling unit 160, operand collection unit 170, and matrix computation data path 180 shown can be referenced. Figure 1 and Figure 2 Related explanations.

[0048] exist Figure 3 In the illustrated embodiment, the storage access queue unit 130 includes one or more storage access queues (e.g., Figure 3The storage access queues 131_1, ..., 131_N are shown, along with arbitrator 132. The number N of storage access queues 131_1 to 131_N can be any integer determined according to the actual design and application. Crossbar switch 120 stores each storage access instruction into a corresponding one of the storage access queues 131_1 to 131_N according to the subdivision type of the storage access instruction. Arbitrator 132 is coupled to storage access queues 131_1 to 131_N and synchronization unit 161. Arbitrator 132, under the control of synchronization unit 161, selects to provide the output of one of the storage access queues 131_1 to 131_N to storage access and computation task scheduling unit 160.

[0049] exist Figure 3 In the illustrated embodiment, the computation task queue unit 140 includes one or more computation task queues (e.g., Figure 3 The computation task queues 141_1, ..., 141_N are shown, along with arbitrator 142. The number N of the computation task queues 141_1 to 141_N can be any integer determined according to the actual design and application. Crossbar switch 120 stores each computation task instruction into a corresponding one of the computation task queues 141_1 to 141_N according to the subdivision type of the computation task instruction. Arbitrator 142 is coupled to computation task queues 141_1 to 141_N and synchronization unit 161. Arbitrator 142, under the control of synchronization unit 161, selects to provide the output of one of the computation task queues 141_1 to 141_N to matrix computation data path 180.

[0050] exist Figure 3 In the illustrated embodiment, the storage access and computation task scheduling unit 160 includes a synchronization unit 161 and a storage access scheduling unit 162. The synchronization unit 161 is coupled to the storage access queue unit 130 and the computation task queue unit 140. The synchronization unit 161 controls the storage access queue unit 130 and the computation task queue unit 140 such that the completion time (retrieval of corresponding operands) of a storage access instruction issued by the storage access queue unit 130 is earlier than the time when the computation task queue unit 140 outputs the computation task instruction corresponding to the storage access instruction. The storage access scheduling unit 162 executes the storage access instructions output by the storage access queue unit 130.

[0051] Figure 4 This is a schematic diagram of the circuit module of the synchronization unit 161 according to an embodiment of the present disclosure. Figure 4 The synchronization unit 161 shown can be used as Figure 3 This is one of many implementation examples of the synchronization unit 161 shown. Figure 4In the illustrated embodiment, the synchronization unit 161 includes N synchronization lane state machines. The number N of the synchronization lane state machines can be determined according to actual design and application. The synchronization lane state machines have different synchronization numbers (e.g., 1, 2, …, N). The corresponding one of the memory access class instructions stored in the memory access queue unit 130 and the corresponding one of the compute task class instructions stored in the compute task queue unit 140 have the same one of the different synchronization numbers. For example, there are a first memory access instruction (synchronization number: 1) and a second memory access instruction (synchronization number: 2) in the memory access queue unit 130, and there are a first compute task instruction (synchronization number: 1) and a second compute task instruction (synchronization number: 2) in the compute task queue unit 140. If logically corresponding, i.e., the matrix computation instruction depends on the data retrieved by the memory access instruction, the same matrix computation synchronization lane ID is carried so as to synchronize in the synchronization unit 161. For example, the first memory access instruction and the first compute task instruction having the same synchronization number "1" are in a cooperative relationship (e.g., the first compute task instruction needs to use the operand of the first memory access instruction), and the second memory access instruction and the second compute task instruction having the same synchronization number "2" are in a cooperative relationship.

[0052] Figure 4The plurality of synchronization lane state machines control the memory access queue unit 130 and the compute task queue unit 140 according to different synchronization numbers, such that the time when the memory access queue unit 130 issues a memory access type instruction with the same synchronization number to complete execution (fetch corresponding operands) is earlier than the time when the compute task queue unit 140 outputs a compute task type instruction with the same synchronization number. For example, taking the synchronization lane state machine corresponding to the synchronization number "1" as an example (the other synchronization lane state machines can be similarly deduced), assuming that the memory access queue unit 130 has prepared a first memory access instruction (synchronization number: 1), at this time the arbiter 132 checks the synchronization lane state machine (synchronization number: 1) to confirm whether the space of the operand collection unit 170 is sufficient. If it is confirmed that the space of the operand collection unit 170 is sufficient (state "space released"), the memory access queue unit 130 outputs the first memory access instruction with the synchronization number "1" to the memory access scheduling unit 162, so that the memory access scheduling unit 162 executes the first memory access instruction to load the corresponding operand of the on-chip memory 150 to the operand collection unit 170 (enter the state "operand ready"). If the compute task queue unit 140 has prepared a first compute task instruction (synchronization number: 1), at this time the arbiter 142 checks the synchronization lane state machine (synchronization number: 1) to confirm whether the operand of the operand collection unit 170 is ready. If it is confirmed that the operand of the operand collection unit 170 is ready (state "operand ready"), the compute task queue unit 140 outputs the first compute task instruction (synchronization number: 1) to the matrix computation data path 180. Therefore, the time when the memory access queue unit 130 issues the first memory access instruction with the synchronization number "1" to complete execution (fetch the corresponding operand) is earlier than the time when the compute task queue unit 140 outputs the first compute task instruction with the same synchronization number "1". The operand fetched by the memory access instruction is ready after the operand collection unit 170 is prepared, and the corresponding compute task instruction is issued from the compute task queue unit 140. The data synchronization of the memory access type instruction and the compute task type instruction is ensured.

[0053] After the corresponding operands are loaded into the operand collection unit 170, the synchronization unit 161 instructs the computation task queue unit 140 to output the first computation task instruction with the synchronization number "1" to the matrix computation data path 180, so that the matrix computation data path 180 executes the first computation task instruction and stores the computation result into the on-chip memory 150 or the register bank. (Enter the state "computation completed"). After the computation is completed, the matrix computation data path 180 informs the synchronization unit 161 to release the space of the operand collection unit 170 (enter the state "space released") and informs the synchronization unit 161 to store the next operand. Taking the matrix multiplication computation "D = A x B" as an example, after the computation task instruction is issued, the matrix computation data path 180 reads the matrix operands A and B from the operand collection unit 170. After the matrix computation, the matrix computation data path 180 outputs the computation result to the on-chip memory or the register bank and informs the synchronization unit 161 that the computation is completed.

[0054] Figure 5 is a schematic diagram of a programming model according to an embodiment of the present disclosure. Figure 5 N instruction issue units are illustrated, for example Figure 5 instruction issue unit 0, instruction issue unit 1, instruction issue unit 2, instruction issue unit 3,..., instruction issue unit N are illustrated. Figure 5 instruction issue units 0-N can be used as Figure 1 one of the application examples of instruction issue units 110_1-110_N. Figure 5 instruction issue units 0-N can refer to Figure 1 to Figure 3 the related description of instruction issue units 110_1-110_N. In an embodiment, each instruction issue unit corresponds to a thread bundle (warp), and the thread bundle sending the memory access instruction and the thread bundle sending the matrix computation instruction form a cooperative thread bundle (cooperative warp). In the embodiment illustrated in Figure 5 In the embodiment illustrated in

[0055] The storage access instructions can be issued by one or more instruction issue units, depending on the kind of input data required by the matrix computation instruction. For example, for a matrix multiply-add computation "D = A x B + C", three storage access instructions are required to fetch the matrix operands A, B and C; for a matrix multiply computation "D = A x B", two storage access instructions are required to fetch the matrix operands A and B; for a matrix multiply computation "D = A x constant", only one storage access instruction is required to fetch the matrix operand A. The storage access instructions read the input operands of the matrix computation from the on-chip storage 150 to the operand collection unit 170. In Figure 5 In the illustrated embodiment, the storage access instructions are issued by instruction issue unit 0, instruction issue unit 1 and instruction issue unit 2. For example, instruction issue unit 0 issues the load instructions "Load A0, sc0", "Load A1, sc1", "Load A2, sc2", "Load A3, sc3",... in sequence, where A0, A1, A2, A3 represent different operands. Instruction issue unit 1 issues the load instructions "Load B0, sc0", "Load B1, sc1", "Load B2, sc2", "Load B3, sc3",... in sequence, where B0, B1, B2, B3 represent different operands. Instruction issue unit 2 issues the load instructions "Load C0, sc0", "Load C1, sc1", "Load C2, sc2", "Load C3, sc3",... in sequence, where C0, C1, C2, C3 represent different operands. Instruction issue unit 0, instruction issue unit 1, and instruction issue unit 2 and instruction issue unit 3 do not need to be synchronized.

[0056] For example, for a matrix multiply-add computation "D = A x B + C", after synchronization is completed by synchronization unit 161, the matrix computation data path 180 reads the matrix operands A, B, C from the operand collection unit 170, and then outputs the matrix multiply-add result to the on-chip storage or the register file. If the matrix multiply-add result is output to the on-chip storage 150, the subsequent instructions need to first read the matrix multiply-add result from the on-chip storage 150 to the register file, and then perform the subsequent vector computation. If the matrix multiply-add result is output to the register file, the subsequent instructions can directly read the register file to perform the subsequent vector computation.

[0057] Figure 6 is a schematic diagram of a programming model according to another embodiment of the present disclosure. Figure 6 The instruction issue units 0, 1, 2, 3, 4, 5, 6, 7 can refer to the instruction issue units 0, 1, 2, 3, 4, 5, 6, 7 of the programming model of FIG. 1. Figure 5The description of instruction issue unit 0 - instruction issue unit N and by analogy applies.

[0058] Figure 6 The programming model is that matrix computation is performed in a group of thread warps. In one embodiment, each instruction issue unit corresponds to a thread warp, the thread warp that sends the store access instruction, the thread warp that sends the matrix computation instruction, and the thread warp that provides the matrix operand, form a cooperating thread warp, and together complete a matrix operation. The thread warps that participate in the matrix operation first synchronize among the thread warps, and then one thread warp initiates the matrix operation. For example, to compute the matrix multiplication "D0 = A0 x B0 + C0", "D1 = A1 x B1 + C1", "D2 = A2 x B2 + C2" using the matrix multiply-add instruction "MMA", a group of thread warps corresponding to a group of registers are used as the input of the matrix operands "A0", "A1", "A2" and the matrix operands "C0", "C1", "C2", and at the same time, the matrix operands "B0", "B1", "B2" in the shared memory are used to achieve the effect of sharing the matrix operand B. The store access instruction can be issued by one or more instruction issue units, depending on the type of input data required by the matrix computation instruction. Instruction issue unit 0, instruction issue unit 1, instruction issue unit 2 are programmed to issue the load instructions "Load A", "Load B", "Load C" (an example of a store access type instruction), and instruction issue unit 3 is programmed to issue the matrix multiply-add instruction "MMA" (an example of a computation task type instruction). Instruction issue unit 0 - instruction issue unit 7 first synchronize the thread warps to align the execution progress of the multiple cooperating thread warps. Then, instruction issue unit 0 - instruction issue unit 2 issue the store access type instructions, and instruction issue unit 3 issues the matrix multiply-add instruction "MMA". The store access instruction reads the matrix operands from the on-chip storage 150 to the operand collection unit 170 (a group of registers or shared memory). After the data synchronization is completed, the matrix computation data path 180 reads the matrix operands A, B, C from the operand collection unit 170, and then outputs the matrix multiply-add result to the on-chip storage or the group of registers.

[0059] Figure 7 is a schematic diagram of a programming model according to yet another embodiment of the present disclosure. Figure 7 The instruction issue unit 0, instruction issue unit 1, instruction issue unit 2,... can refer to the description of instruction issue unit 0 - instruction issue unit N and by analogy applies. Figure 5The description of instruction issue unit 0~instruction issue unit N and by analogy applies. sc0, sc1, sc2 represent different synchronization numbers. The same set of memory access instructions and matrix multiply-add instructions can be issued by the same instruction issue unit. For example, in a matrix multiplication computation "D = A x B + C", three memory access instructions are needed to fetch the matrix operands A, B and C. The memory access instructions read the input operands of the matrix computation from on-chip storage 150 to operand collection unit 170. In Figure 7 In the illustrated embodiment, the memory access instructions are issued by instruction issue unit 0, instruction issue unit 1 and instruction issue unit 2 sequentially. For example, instruction issue unit 0 issues the load instructions "Load A0, sc0", "Load B0, sc0", "Load C0, sc0" sequentially, where A0, B0, C0 represent different operands. Instruction issue unit 1 issues the load instructions "Load A1, sc1", "Load B1, sc1", "Load C1, sc1" sequentially, where A1, B1, C1 represent different operands. Instruction issue unit 2 issues the load instructions "Load A2, sc2", "Load B2, sc2", "Load C2, sc2" sequentially, where A2, B2, C2 represent different operands. Next, instruction issue unit 0, instruction issue unit 1 and instruction issue unit 2 issue the matrix multiply-add instructions "MMA, sc0", "MMA, sc1", "MMA, sc2". Instruction issue unit 0, instruction issue unit 1 and instruction issue unit 2 do not need to be synchronized.

[0060] Figure 8 is a circuit module diagram of operand collection unit 170 and matrix computation data path 180 shown in accordance with an embodiment of the present disclosure. Figure 8 The illustrated operand collection unit 170 and matrix computation data path 180 can be used as Figure 3 The illustrated operand collection unit 170 and matrix computation data path 180 are one of many embodiments. Figure 8 The illustrated instruction issue units 110_1~110_N, crossbar 120, memory access queue unit 130, computation task queue unit 140, on-chip storage 150, synchronization unit 161, memory access scheduling unit 162, operand collection unit 170 and matrix computation data path 180 can refer to the description of Figure 3 .

[0061] In Figure 8 In the illustrated embodiment, operand collection unit 170 includes a plurality of register banks (e.g. Figure 8The register banks 171_1-171_N are each coupled to the memory access scheduling unit 162 and the matrix computation data path 180. The memory access scheduling unit 162 reads corresponding operands of the memory access class instruction from the on-chip storage 150 and stores them in one or more corresponding ones of the register banks 171_1-171_N. The shared memory 172 is coupled to the memory access scheduling unit 162 and the matrix computation data path 180. The memory access scheduling unit 162 reads corresponding operands of the memory access class instruction from the on-chip storage 150 and stores them in the shared memory 172. The matrix computation data path 180 fetches the corresponding operands from the register banks 171_1-171_N and the shared memory 172 to execute the computation task class instruction corresponding to the memory access class instruction.

[0062] For example, to compute a matrix multiplication "D0 = A0 x B + C0" with a matrix multiply-add instruction "MMA", the memory access scheduling unit 162 reads corresponding operands "A0" and "C0" of the memory access class instruction from the on-chip storage 150 and stores them in the register bank 171_1, and reads corresponding operand "B" of the memory access class instruction from the on-chip storage 150 and stores it in the shared memory 172. The matrix computation data path 180 fetches the corresponding operands "A0", "B" and "C0" from the register bank 171_1 and the shared memory 172 to execute the matrix multiply-add instruction "MMA" (computation task class instruction) corresponding to the memory access class instruction. Similarly, the memory access scheduling unit 162 reads corresponding operands "AN" and "CN" of the memory access class instruction from the on-chip storage 150 and stores them in the register bank 171_N, and the matrix computation data path 180 fetches the corresponding operands "AN", "B" and "CN" from the register bank 171_N and the shared memory 172 to execute the matrix multiply-add instruction "MMA", i.e., to perform the matrix multiplication computation "DN = AN x B + CN". Matrix computation units belonging to the same group of cooperating thread warps can share the same operand B.

[0063] In the illustrated embodiment, the matrix computation data path 180 includes a plurality of matrix computation units (e.g., matrix computation units 181_1, 181_2,..., 181_N) and a shared memory 182. Each of the matrix computation units 181_1-181_N is coupled to the shared memory 182. The shared memory 182 is coupled to the memory access scheduling unit 162 and the matrix computation data path 180. The memory access scheduling unit 162 reads corresponding operands of the memory access class instruction from the on-chip storage 150 and stores them in the shared memory 182. The matrix computation data path 180 fetches the corresponding operands from the shared memory 182 to execute the computation task class instruction corresponding to the memory access class instruction. Figure 8 In the illustrated embodiment, the matrix computation data path 180 includes a plurality of matrix computation units (e.g., matrix computation units 181_1, 181_2,..., 181_N) and a shared memory 182. Each of the matrix computation units 181_1-181_N is coupled to the shared memory 182. The shared memory 182 is coupled to the memory access scheduling unit 162 and the matrix computation data path 180. The memory access scheduling unit 162 reads corresponding operands of the memory access class instruction from the on-chip storage 150 and stores them in the shared memory 182. The matrix computation data path 180 fetches the corresponding operands from the shared memory 182 to execute the computation task class instruction corresponding to the memory access class instruction. Figure 8The matrix computation units 181_1-181_N are coupled to a corresponding one of the register banks 171_1-171_N and the shared memory 172. One of the matrix computation units 181_1-181_N fetches corresponding operands from the corresponding one of the register banks 171_1-171_N and the shared memory 172 to execute a computation task class instruction corresponding to a memory access class instruction. For example, to execute a matrix multiply-add instruction "MMA" (computation task class instruction) with the matrix computation unit 181_1 to compute a matrix multiplication "D0 = A0 x B + C0", the memory access scheduling unit 162 reads corresponding operands "A0" and "C0" from the on-chip memory 150 and stores them in the register bank 171_1, and the memory access scheduling unit 162 reads a corresponding operand "B" of the memory access class instruction from the on-chip memory 150 and stores it in the shared memory 172. The matrix computation unit 181_1 fetches the corresponding operands "A0", "B", and "C0" from the register bank 171_1 and the shared memory 172 to execute the matrix multiply-add instruction "MMA", i.e., to compute the matrix multiplication "D0 = A0 x B + C0".

[0064] In summary, the above embodiments optimize the sequential execution mode of "the instruction issue unit issues and executes a memory access class instruction first, then checks if data has returned, and then issues a matrix computation class instruction" to "the instruction issue unit issues a memory access class instruction and a matrix computation class instruction in parallel", reducing the blocking caused by the order of instructions and improving efficiency. The memory access class instruction and the matrix computation class instruction belonging to the same group carry the same synchronization number, which can be sent to the memory access queue unit 130 and the computation task queue unit 140 first, and then the synchronization unit 161 synchronizes the memory access queue unit 130 and the computation task queue unit 140. The data synchronization of the memory access class instruction and the matrix computation class instruction is performed after the instruction issue of the instruction issue unit 110_1-110_N, which not only avoids blocking subsequent other instruction issues, but also reduces bubbles on the pipeline from the instruction issue unit 110_1-110_N to the matrix computation data path 180. The instruction issue unit 110_1-110_N, the instruction type, and the instruction queue have no binding relationship, and the programmer can bind any instruction type to any instruction issue unit in the program. The hardware crossbar 120 routes any instruction to the corresponding instruction queue according to the instruction type.

[0065] Taking the matrix operation D=A×B+C as an example, the operands (matrices A and C) are loaded from on-chip storage 150 to register groups 171_1~171_N, increasing the bandwidth of the matrix operation input data (matrices A and C). The operand (matrix B) is loaded from on-chip storage 150 to shared memory 172, allowing for the reuse of matrix B data. The result of the matrix calculation (matrix D) can be directly output to on-chip storage 150 or the register groups without additional caching. Another advantage of the matrix calculation data path 180 being able to read register groups 171_1~171_N and shared memory 172 is that it can support "breaking down a large convolution kernel into smaller convolution kernels corresponding to each register group 171_1~171_N," avoiding the performance waste of large convolution kernels processing small matrices, and also avoiding the chip yield reduction problem caused by excessively large convolution kernel areas.

[0066] Figure 9 This is a schematic diagram of the circuit module of the operand collection unit 170 according to another embodiment of the present disclosure. Figure 9 The operand collection unit 170 shown can be used as Figure 3 This is one of many implementation examples of the operand collection unit 170 shown. Figure 9 The instruction issuing units 110_1 to 110_N, crossbar switch 120, memory access queue unit 130, computation task queue unit 140, on-chip memory 150, synchronization unit 161, memory access scheduling unit 162, operand collection unit 170, and matrix computation data path 180 shown can be referenced. Figure 3 Related explanations.

[0067] exist Figure 9 In the illustrated embodiment, the operand collection unit 170 includes multiple register groups (e.g., ... Figure 9The register banks 173_1-173_N are coupled to the storage access scheduling unit 162 and the matrix computation data path 180. The storage access scheduling unit 162 reads corresponding operands of a storage access class instruction from the on-chip storage 150 and stores into the register banks 173_1-173_N. The matrix computation data path 180 fetches the corresponding operands from the register banks 173_1-173_N to execute the computation task class instruction corresponding to the storage access class instruction. For example, to execute a matrix multiply-add instruction "MMA" (computation task class instruction) to compute a matrix multiplication "D = A x B + C" with the matrix computation data path 180, the storage access scheduling unit 162 reads the corresponding operands "A", "B", and "C" from the on-chip storage 150 and stores into the same (or different) one of the register banks 173_1-173_N. The matrix computation data path 180 fetches the corresponding operands "A", "B", and "C" from the same (or different) one of the register banks 173_1-173_N to execute the matrix multiply-add instruction "MMA", i.e., to compute the matrix multiplication "D = A x B + C".

[0068] Figure 10 Figure 17B shows a circuit module diagram of the operand collection unit 170 according to yet another embodiment of the disclosure. Figure 10 The operand collection unit 170 can be implemented as Figure 3 The operand collection unit 170 can be implemented as Figure 10 The instruction issue units 110_1-110_N, the crossbar 120, the storage access queue unit 130, the computation task queue unit 140, the on-chip storage 150, the synchronization unit 161, the storage access scheduling unit 162, the operand collection unit 170, and the matrix computation data path 180 can be implemented as described with reference to Figure 3 The instruction issue units 110_1-110_N, the crossbar 120, the storage access queue unit 130, the computation task queue unit 140, the on-chip storage 150, the synchronization unit 161, the storage access scheduling unit 162, the operand collection unit 170, and the matrix computation data path 180 can be implemented as described with reference to Figure 10 The operand collection unit 170 can be implemented as

[0069] For example, taking the example of the matrix computation data path 180 executing a matrix multiply-add instruction "MMA" (a compute task class instruction) to compute a matrix multiplication "D = A x B + C", the memory access scheduling unit 162 reads the corresponding operands "A", "B", and "C" from the on-chip storage 150 and stores them in the shared memory 174. The matrix computation data path 180 takes the corresponding operands "A", "B", and "C" from the shared memory 174 to execute the matrix multiply-add instruction "MMA", i.e., to compute the matrix multiplication "D = A x B + C".

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than limit them; although the present disclosure has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. An artificial intelligence chip, characterized in that, The artificial intelligence chip includes: Multiple command issuing units; Storage access queue unit; Calculate the task queue unit; A cross switch is coupled to the plurality of instruction issuing units, the storage access queue unit, and the computing task queue unit, wherein the cross switch receives a plurality of instructions issued in parallel by the plurality of instruction issuing units, the plurality of instructions including storage access instructions and computing task instructions, the cross switch stores the storage access instructions into the storage access queue unit according to the instruction type, and the cross switch stores the computing task instructions into the computing task queue unit according to the instruction type; On-chip storage; Operand collection unit; and A storage access and computation task scheduling unit is coupled to the on-chip storage, the operand collection unit, the storage access queue unit, and the computation task queue unit. The storage access and computation task scheduling unit controls the outputs of the storage access queue unit and the computation task queue unit, and executes the storage access instructions output by the storage access queue unit to read data from the on-chip storage to the operand collection unit.

2. The artificial intelligence chip according to claim 1, characterized in that, The artificial intelligence chip also includes: A matrix calculation data path is coupled to the calculation task queue unit, wherein the matrix calculation data path executes the calculation task class instructions output by the calculation task queue unit.

3. The artificial intelligence chip according to claim 2, characterized in that, The operand collection unit is coupled to the matrix computation data path. The storage access and computation task scheduling unit reads the corresponding operand of the storage access instruction from the on-chip storage and stores it in the operand collection unit. The matrix computation data path retrieves the corresponding operand from the operand collection unit to execute the computation task instruction corresponding to the storage access instruction.

4. The artificial intelligence chip according to claim 3, characterized in that, The operand collection unit includes: Multiple register groups, each of which is coupled to the memory access and computation task scheduling unit and the matrix computation data path, and the memory access and computation task scheduling unit reads the first corresponding operand of the memory access instruction from the on-chip memory and stores it into at least one corresponding register group; and Shared memory is coupled to the storage access and computation task scheduling unit and the matrix computation data path, wherein the storage access and computation task scheduling unit reads the second corresponding operand of the storage access instruction from the on-chip storage and stores it in the shared memory, and the matrix computation data path retrieves the first corresponding operand and the second corresponding operand from the plurality of register groups and the shared memory to execute the computation task instruction corresponding to the storage access instruction.

5. The artificial intelligence chip according to claim 4, characterized in that, The matrix calculation data path includes: A plurality of matrix computation units, wherein each of the plurality of matrix computation units is coupled to one of the corresponding register groups and the shared memory, and one of the plurality of matrix computation units retrieves a first corresponding operand and a second corresponding operand from one of the corresponding register groups and the shared memory to execute the computation task instruction corresponding to the storage access instruction.

6. The artificial intelligence chip according to claim 3, characterized in that, The operand collection unit includes: Multiple register groups, each of which is coupled to the memory access and computation task scheduling unit and the matrix computation data path, wherein the memory access and computation task scheduling unit reads the first corresponding operand and the second corresponding operand of the memory access instruction from the on-chip memory and stores them into the multiple register groups, and the matrix computation data path retrieves the first corresponding operand and the second corresponding operand from the multiple register groups to execute the computation task instruction corresponding to the memory access instruction.

7. The artificial intelligence chip according to claim 3, characterized in that, The operand collection unit includes: Shared memory is coupled to the storage access and computation task scheduling unit and the matrix computation data path, wherein the storage access and computation task scheduling unit reads the first corresponding operand and the second corresponding operand of the storage access class instruction from the on-chip storage and stores them into the shared memory, and the matrix computation data path retrieves the first corresponding operand and the second corresponding operand from the shared memory to execute the computation task class instruction corresponding to the storage access class instruction.

8. The artificial intelligence chip according to claim 1, characterized in that, The storage access and computation task scheduling unit controls the storage access queue unit and the computation task queue unit so that the storage access class instruction issued by the storage access queue unit completes execution earlier than the time when the computation task queue unit outputs the computation task class instruction corresponding to the storage access class instruction.

9. The artificial intelligence chip according to claim 1, characterized in that, The storage access and computing task scheduling unit includes: A synchronization unit, coupled to the storage access queue unit and the computation task queue unit, controls the storage access queue unit and the computation task queue unit to ensure that the completion time of the storage access instruction issued by the storage access queue unit is earlier than the time when the computation task queue unit outputs the corresponding computation task instruction; and A storage access scheduling unit is coupled to the storage access queue unit and the synchronization unit, wherein the storage access scheduling unit executes the storage access instructions output by the storage access queue unit.

10. The artificial intelligence chip according to claim 9, characterized in that, The synchronization unit includes: Multiple synchronization channel state machines, wherein the multiple synchronization channel state machines have different synchronization numbers, one of the storage access class instructions in the storage access queue unit and one of the corresponding computing task class instructions in the computing task queue unit both carry the same synchronization number from the different synchronization numbers, and the multiple synchronization channel state machines control the storage access queue unit and the computing task queue unit according to the different synchronization numbers, so that the execution time of the storage access class instruction with the same synchronization number issued by the storage access queue unit is earlier than the time of the computing task queue unit outputting the computing task class instruction with the same synchronization number.

11. The artificial intelligence chip according to claim 1, characterized in that, The storage access queue unit includes: At least one storage access queue, wherein the cross switch stores each of the storage access class instructions into a corresponding one of the at least one storage access queue according to the subdivision type of the storage access class instructions; and An arbitrator, coupled to the at least one storage access queue and the storage access and compute task scheduling unit, wherein the arbitrator selects the output of one of the at least one storage access queues according to the control of the storage access and compute task scheduling unit.

12. The artificial intelligence chip according to claim 1, characterized in that, The computing task queue unit includes: At least one computation task queue, wherein the cross switch stores each instruction of the computation task class into a corresponding one of the at least one computation task queue according to the subdivision type of the computation task class instruction; and An arbitrator, coupled to the at least one compute task queue and the storage access and compute task scheduling unit, wherein the arbitrator selects the output of one of the at least one compute task queues according to the control of the storage access and compute task scheduling unit.

13. A method for operating an artificial intelligence chip, characterized in that, The operation method includes: The cross switch of the artificial intelligence chip receives multiple instructions that are sent in parallel by multiple instruction sending units of the artificial intelligence chip. The multiple instructions include storage access instructions and computing task instructions. The cross switch is coupled to the multiple instruction sending units, the storage access queue unit of the artificial intelligence chip, and the computing task queue unit of the artificial intelligence chip. The cross switch of the artificial intelligence chip stores the storage access instructions into the storage access queue unit according to the instruction type; The cross switch stores the computation task type instruction into the computation task queue unit according to the instruction type; The storage access and computation task scheduling unit of the artificial intelligence chip controls the output of the storage access queue unit and the computation task queue unit, wherein the storage access and computation task scheduling unit is coupled to the on-chip storage of the artificial intelligence chip, the operand collection unit of the artificial intelligence chip, the storage access queue unit, and the computation task queue unit; and The storage access and computing task scheduling unit executes the storage access queue unit's output storage access instructions to read data from the on-chip storage into the operand collection unit.

14. The operating method according to claim 13, characterized in that, The operation method further includes: The matrix calculation data path of the artificial intelligence chip executes the calculation task class instructions output by the calculation task queue unit, wherein the matrix calculation data path is coupled to the calculation task queue unit.

15. The operating method according to claim 14, characterized in that, The operation method further includes: The storage access and computation task scheduling unit reads the corresponding operands of the storage access instructions from the on-chip storage of the AI ​​chip and stores them in the operand collection unit of the AI ​​chip, wherein the on-chip storage is coupled to the storage access and computation task scheduling unit, and the operand collection unit is coupled to the storage access and computation task scheduling unit and the matrix computation data path; and The matrix calculation data path retrieves the corresponding operand from the operand collection unit to execute the computation task instruction corresponding to the storage access instruction.

16. The operating method according to claim 15, characterized in that, The operation method further includes: The storage access and computation task scheduling unit reads the first corresponding operand of the storage access instruction from the on-chip storage and stores it into at least one of the corresponding register groups of the operand collection unit, wherein each of the register groups is coupled to the storage access and computation task scheduling unit and the matrix computation data path; and The storage access and computation task scheduling unit reads the second corresponding operand of the storage access instruction from the on-chip storage and stores it in the shared memory of the operand collection unit, wherein the shared memory is coupled to the storage access and computation task scheduling unit and the matrix computation data path; and The matrix calculation data path retrieves the first corresponding operand and the second corresponding operand from the plurality of register groups and the shared memory to execute the computation task class instruction corresponding to the storage access class instruction.

17. The operating method according to claim 16, characterized in that, The operation method further includes: One of the plurality of matrix computation units of the matrix computation data path retrieves the first corresponding operand and the second corresponding operand from one of the corresponding register groups and the shared memory to execute the computation task class instruction corresponding to the storage access class instruction, wherein each of the plurality of matrix computation units is coupled to one of the corresponding register groups and the shared memory.

18. The operating method according to claim 15, characterized in that, The operation method further includes: The storage access and computation task scheduling unit reads the first and second corresponding operands of the storage access instruction from the on-chip storage and stores them into multiple register groups of the operand collection unit, wherein each of the multiple register groups is coupled to the storage access and computation task scheduling unit and the matrix computation data path; and The matrix calculation data path retrieves the first corresponding operand and the second corresponding operand from the plurality of register groups to execute the computation task instruction corresponding to the storage access instruction.

19. The operating method according to claim 15, characterized in that, The operation method further includes: The storage access and computation task scheduling unit reads the first and second corresponding operands of the storage access instruction from the on-chip storage and stores them in the shared memory of the operand collection unit, wherein the shared memory is coupled to the storage access and computation task scheduling unit and the matrix computation data path; and The matrix calculation data path retrieves the first corresponding operand and the second corresponding operand from the shared memory to execute the computation task instruction corresponding to the storage access instruction.

20. The operating method according to claim 13, characterized in that, The operation method further includes: The storage access and computation task scheduling unit controls the storage access queue unit and the computation task queue unit so that the storage access class instruction issued by the storage access queue unit completes execution earlier than the time when the computation task queue unit outputs the computation task class instruction corresponding to the storage access class instruction.

21. The operating method according to claim 13, characterized in that, The operation method further includes: The storage access queue unit and the computing task queue unit are controlled by a synchronization unit of the storage access and computing task scheduling unit, such that the completion time of the storage access instruction issued by the storage access queue unit is earlier than the completion time of the computing task instruction corresponding to the storage access instruction issued by the computing task queue unit, wherein the synchronization unit is coupled to the storage access queue unit and the computing task unit; and The storage access scheduling unit of the storage access and computing task scheduling unit executes the storage access class instructions output by the storage access queue unit, wherein the storage access scheduling unit is coupled to the storage access queue unit and the synchronization unit.

22. The operating method according to claim 21, characterized in that, The synchronization unit includes multiple synchronization channel state machines, each with a different synchronization number. One of the storage access instructions in the storage access queue unit and one of the corresponding computation task instructions in the computation task queue unit both carry the same synchronization number from among the different synchronization numbers. The operation method further includes: The multiple synchronization channel state machines control the storage access queue unit and the computing task queue unit according to the different synchronization numbers, so that the storage access class instruction issued by the storage access queue unit with the same synchronization number completes earlier than the computing task queue unit outputs the computing task class instruction with the same synchronization number.

23. The operating method according to claim 13, characterized in that, The operation method further includes: The cross switch stores each of the memory access instructions into a corresponding one of at least one memory access queue of the memory access queue unit according to the subdivision type of the memory access instruction; and The arbitrator of the storage access queue unit selects the output of one of the at least one storage access queues according to the control of the storage access and computing task scheduling unit, wherein the arbitrator is coupled to the at least one storage access queue and the storage access and computing task scheduling unit.

24. The operating method according to claim 13, characterized in that, The operation method further includes: The cross switch stores each of the computation task class instructions into a corresponding one in at least one computation task queue of the computation task queue unit according to the subdivision type of the computation task class instructions; and The arbitrator of the computing task queue unit selects the output of one of the at least one computing task queues according to the control of the storage access and computing task scheduling unit, wherein the arbitrator is coupled to the at least one computing task queue and the storage access and computing task scheduling unit.

Images