Instruction flow control system applied to neural network accelerator

Abstract

The invention relates to a neural network accelerator, in particular to an instruction flow control system applied to the neural network accelerator, which comprises a first-stage instruction flow control unit, a second-stage instruction flow control unit, a third-stage instruction flow control unit, a fourth-stage instruction flow control unit and a fourth-stage instruction flow control unit, and is characterized in that the first-stage instruction flow control unit is responsible for prefetching, decoding, executing and distributing instructions of the neural network accelerator and responding to pause influence of debugging events and synchronization events on instruction distribution; the second-level instruction flow control unit is located in each execution unit, extracts execution parameters from the second-level execution instructions distributed by the first-level instruction flow control unit, is responsible for translating and maintaining the instruction execution dependency relationship among the execution units, and ensures that the execution units execute the instructions in sequence according to the instruction execution dependency relationship among the execution units; according to the technical scheme, the defect that in the prior art, it is difficult to efficiently execute instructions with the execution dependency relationship between the execution units can be effectively overcome.

Description

Technical Field

[0001] This invention relates to neural network accelerators, and more specifically to an instruction flow control system applied to neural network accelerators. Background Technology

[0002] Neural network accelerators are high-performance hardware circuits specifically designed for deep learning applications. Common neural network accelerators often employ complex instruction sets, utilizing a unified instruction management unit to handle instruction interface management, instruction parsing and generation, and instruction control. Ultimately, the instruction parsing results are sent directly to each execution unit in the form of control parameters, and each execution unit then performs independent computation based on these control parameters.

[0003] When the instructions handled by different execution units have execution dependencies on each other, the instruction distribution by the instruction management unit will generate blocking events, causing instruction stream transmission to be blocked. Furthermore, when reordering is performed based on the internal dependencies of the instruction management unit, the functional complexity of the instruction management unit itself will increase significantly, severely impacting system performance. At the same time, the centralized instruction stream control unit approach is also detrimental to the modular expansion of system components. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] In view of the above-mentioned shortcomings of the prior art, the present invention provides an instruction flow control system for neural network accelerators, which can effectively overcome the shortcomings of the prior art in efficiently executing instructions with execution dependencies between execution units.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] An instruction flow control system for neural network accelerators includes:

[0009] The Level 1 instruction flow control unit is responsible for prefetching, decoding, executing, and distributing instructions to the neural network accelerator, and responding to the impact of debugging events and synchronization events on instruction distribution.

[0010] The secondary instruction flow control unit, located within each execution unit, extracts execution parameters from the secondary execution instructions distributed by the primary instruction flow control unit, and is responsible for translating and maintaining the instruction execution dependencies between execution units, ensuring that each execution unit executes instructions in the order of their respective instruction execution dependencies.

[0011] Preferably, the first-level instruction flow control unit initiates a DMA instruction fetch operation by configuring the DMA instruction fetch channel, uses the front-end DMA controller to receive the original instructions and store them in the local instruction memory queue, and reads the pre-stored instructions from the local instruction memory queue. According to the instruction type, the instructions are classified into first-level execution instructions executed in the first-level instruction flow control unit and second-level execution instructions executed in the second-level instruction flow control unit.

[0012] The first-level execution instructions are initially parsed by the first-level instruction flow control unit, and then packaged with the second-level execution instructions and distributed to the second-level instruction flow control unit for execution after being controlled by Debug, Event, and Sync events.

[0013] Preferably, the first-level instruction flow control unit includes an instruction fetch front-end, which is responsible for establishing and configuring the DMA instruction fetch channel, and receiving raw instructions and storing them in the local instruction memory queue. The instruction fetch front-end supports dynamic configuration of the first instruction address dmaStartAddr and the single instruction fetch length dmaBlockSize of the DMA instruction fetch channel.

[0014] Preferably, when the system receives the global execution enable control signal GlobalRunStart and the local instruction memory queue is not full, it enters the DMA_INIT state to establish a DMA instruction fetch channel. After the current-level DMA controller sends the DMA establishment success flag dma_accept, it enters the DMA_PICK state. The system dynamically controls the read and write process according to the current local instruction cache state, and terminates the DMA instruction fetch operation when the number of successfully received packets dmaCnt reaches the preset single instruction fetch length dmaBlockSize, that is, it enters the DMA_FINISH state. When the system receives the transfer success completion flag dma_rd_resp from the previous-level DMA controller, it returns to the next DMA initialization state.

[0015] Preferably, the primary instruction flow control unit further includes an instruction post-processing section, which is responsible for executing primary execution instructions, responding to the impact of debugging events and synchronization events on instruction distribution, and maintaining the execution status.

[0016] Preferably, the instruction issuing stage selects to enter the normal execution state or the instruction multi-step execution state based on the current local instruction cache state and the global execution enable control signal GlobalRunStart;

[0017] In the multi-step execution state of instructions, the system monitors and compares the control word information and the current execution state of the instructions to enter the debug state and pause the instruction distribution. It supports entering the Debug Halt state at a specified instruction execution position. The system only supports the interaction of debugging information with the external debugging unit in the Debug Halt state.

[0018] The instruction sending stage responds to the Event event and enters the Event Halt state. The system pauses and releases according to the debugging event, ensuring that the instruction sending is controlled by the execution unit's execution event state.

[0019] After issuing the command, the system responds to the control signal of the multi-core synchronous control unit and enters the Sync Halt state. Based on the synchronous event, the system pauses and releases, ensuring coordinated control under multi-core execution conditions.

[0020] Preferably, the primary instruction flow control unit maintains a series of debugging event ports connected to an external debugging unit for status interaction and debugging control with the debugging unit;

[0021] The primary instruction flow control unit maintains a series of synchronization event ports connected to external synchronization units for status interaction with the synchronization units and enters a pause waiting state based on synchronization control information.

[0022] Preferably, the secondary instruction flow control unit independently maintains a list of instruction execution dependency information. Each execution unit queries the list of instruction execution dependency information stored inside the target dependent execution unit through routing, and dynamically adjusts its own execution and pause status according to the return flag.

[0023] Preferably, the secondary instruction flow control unit adopts Depend / Inform state control, which dynamically controls the pause and release of each execution unit according to the instruction execution dependency information list stored inside each execution unit.

[0024] Preferably, the first-level execution instructions include various debug IDLE instructions, FINISH instructions, and dependency register refresh instructions;

[0025] The secondary execution instructions are recoded subsets of the original instructions, including the encoding of instruction execution dependencies between execution units and the execution parameter information of each execution unit for the current instruction. The secondary execution instructions are distributed to the secondary instruction flow control unit for execution through the instruction distribution custom internal bus Master interface.

[0026] (III) Beneficial Effects

[0027] Compared with existing technologies, the instruction flow control system for neural network accelerators provided by this invention decomposes the instruction flow control system into a centralized first-level instruction flow control unit and a distributed second-level instruction flow control unit. This isolates instruction prefetching, decoding, execution, pause, and distribution from the instruction execution sequence control between execution units, simplifies the system structure of multiple control paths, and improves the scalability of system execution units. This instruction flow control system features a small overall area and high timing performance, effectively improving the system's execution efficiency for instructions with execution dependencies between execution units, and enhancing the system performance of neural network processing. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0029] Figure 1 This is a schematic diagram of the first-level instruction flow control unit in this invention;

[0030] Figure 2 This is the state machine of the instruction fetching front end in the first-level instruction flow control unit of this invention;

[0031] Figure 3 This is the state machine of the instruction-issuing stage in the first-level instruction flow control unit of this invention;

[0032] Figure 4 This is a schematic diagram of the two-level instruction flow control unit in this invention. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0034] The following describes the instruction flow control system and its technical effects applied to neural network accelerators provided by this invention, using specific examples. The system functional modules include:

[0035] Level 1 instruction flow control unit (e.g.) Figure 1As shown, it is responsible for prefetching, decoding, executing, and distributing instructions to the neural network accelerator, and responding to the impact of debugging events and synchronization events on instruction distribution.

[0036] Secondary instruction flow control unit (e.g.) Figure 4 As shown, it is located inside each execution unit, extracts execution parameters from the secondary execution instructions distributed by the primary instruction flow control unit, and is responsible for translating and maintaining the instruction execution dependencies between execution units, ensuring that each execution unit executes instructions in the order of their respective instruction execution dependencies.

[0037] Level 1 instruction flow control unit

[0038] 1) The first-level instruction flow control unit initiates DMA instruction fetching operation by configuring the DMA instruction fetching channel. It uses the front-end DMA controller to receive the original instructions and store them in the local instruction memory queue. It also reads the pre-stored instructions from the local instruction memory queue and classifies the instructions into first-level execution instructions executed in the first-level instruction flow control unit and second-level execution instructions executed in the second-level instruction flow control unit according to the instruction type.

[0039] The first-level execution instructions are initially parsed by the first-level instruction flow control unit, and then packaged with the second-level execution instructions and distributed to the second-level instruction flow control unit for execution after being controlled by Debug, Event, and Sync events.

[0040] Specifically, the first-level execution instructions include various debug IDLE instructions, FINISH instructions, and dependency register refresh instructions;

[0041] The secondary execution instructions are recoded subsets of the original instructions, including the encoding of instruction execution dependencies between execution units and the execution parameter information of each execution unit for the current instruction. The secondary execution instructions are distributed to the secondary instruction flow control unit for execution through the instruction dispatch custom internal bus Master interface.

[0042] 2) The first-level instruction flow control unit includes the instruction fetch front-end, which is responsible for establishing and configuring the DMA instruction fetch channel, and receiving raw instructions and storing them in the local instruction memory queue. The instruction fetch front-end supports dynamic configuration of the first instruction address dmaStartAddr and the single instruction fetch length dmaBlockSize of the DMA instruction fetch channel.

[0043] like Figure 2As shown, when the system receives the global execution enable control signal GlobalRunStart and the local instruction memory queue is not full, it enters the DMA_INIT state and establishes a DMA instruction fetch channel. After the current-level DMA controller sends the DMA establishment success flag dma_accept, it enters the DMA_PICK state. The system dynamically controls the read and write process according to the current local instruction cache state, and terminates the DMA instruction fetch operation when the number of successfully received packets dmaCnt reaches the preset single instruction fetch length dmaBlockSize, that is, it enters the DMA_FINISH state. When the system receives the transfer success completion flag dma_rd_resp from the previous-level DMA controller, it returns to the next DMA initialization state.

[0044] 3) The first-level instruction flow control unit also includes an instruction post-processing section, which is responsible for executing first-level execution instructions, responding to debugging events and the impact of synchronization events on instruction distribution, and maintaining the execution status.

[0045] like Figure 3 As shown, after issuing the instruction, the process selects to enter the normal execution state or the instruction multi-step execution state based on the current local instruction cache state and the global execution enable control signal GlobalRunStart.

[0046] In the multi-step execution state of instructions, the system monitors and compares the control word information and the current execution state of the instructions to enter the debug state and pause the instruction distribution. It supports entering the Debug Halt state at a specified instruction execution position. The system only supports the interaction of debugging information with the external debugging unit in the Debug Halt state.

[0047] After issuing the instruction, the system responds to the Event event and enters the Event Halt state. Based on the debugging event, the system pauses and releases, ensuring that the instruction sending is controlled by the execution unit's execution event state.

[0048] After issuing the command, the system responds to the control signal of the multi-core synchronous control unit and enters the Sync Halt state. Based on the synchronous event, the system pauses and releases, ensuring coordinated control under multi-core execution conditions.

[0049] 4) The primary instruction flow control unit maintains a series of debugging event ports connected to the external debugging unit for status interaction and debugging control with the debugging unit;

[0050] The primary instruction flow control unit maintains a series of synchronization event ports connected to external synchronization units for status interaction with the synchronization units and enters a pause waiting state based on synchronization control information.

[0051] Second-level command flow control unit

[0052] The secondary instruction flow control unit independently maintains a list of instruction execution dependency information. Each execution unit queries the list of instruction execution dependency information stored in the target dependent execution unit through routing, and dynamically adjusts its own execution and pause status according to the return flag.

[0053] like Figure 4 As shown, the secondary instruction flow control unit adopts Depend / Inform state control, which dynamically controls the pause and release of each execution unit based on the instruction execution dependency information list stored inside each execution unit.

[0054] In this technical solution, the instruction flow control system is decomposed into a centralized first-level instruction flow control unit and a distributed second-level instruction flow control unit. This isolates the instruction prefetching, decoding, execution, pause, and distribution from the instruction execution sequence control between the execution units, simplifies the system structure with multiple control paths, and improves the scalability of the system execution units. This instruction flow control system features a small overall area and high timing performance, effectively improving the system's execution efficiency for instructions with execution dependencies between execution units and enhancing the system performance of neural network processing.

[0055] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An instruction flow control system for neural network accelerators, characterized in that: include: The Level 1 instruction flow control unit is responsible for prefetching, decoding, executing, and distributing instructions to the neural network accelerator, and responding to the impact of debugging events and synchronization events on instruction distribution. The secondary instruction flow control unit, located within each execution unit, extracts execution parameters from the secondary execution instructions distributed by the primary instruction flow control unit, and is responsible for translating and maintaining the instruction execution dependencies between execution units, ensuring that each execution unit executes instructions in the order of their respective instruction execution dependencies.

2. The instruction flow control system for neural network accelerators according to claim 1, characterized in that: The first-level instruction flow control unit initiates a DMA instruction fetch operation by configuring the DMA instruction fetch channel. It uses the front-end DMA controller to receive the original instructions and store them in the local instruction memory queue. It also reads the pre-stored instructions from the local instruction memory queue and classifies the instructions into first-level execution instructions executed within the first-level instruction flow control unit and second-level execution instructions executed within the second-level instruction flow control unit according to the instruction type. The first-level execution instructions are initially parsed by the first-level instruction flow control unit, and then packaged with the second-level execution instructions and distributed to the second-level instruction flow control unit for execution after being controlled by Debug, Event, and Sync events.

3. The instruction flow control system for a neural network accelerator according to claim 2, characterized in that: The first-level instruction flow control unit includes an instruction fetch front-end, which is responsible for establishing and configuring the DMA instruction fetch channel, and receiving raw instructions and storing them in the local instruction memory queue. The instruction fetch front-end supports dynamic configuration of the first instruction address dmaStartAddr and the single instruction fetch length dmaBlockSize of the DMA instruction fetch channel.

4. The instruction flow control system for a neural network accelerator according to claim 3, characterized in that: When the system receives the global execution enable control signal GlobalRunStart and the local instruction memory queue is not full, it enters the DMA_INIT state to establish a DMA instruction fetch channel. After the current-level DMA controller sends the DMA establishment success flag dma_accept, it enters the DMA_PICK state. The system dynamically controls the read and write process according to the current local instruction cache state, and terminates the DMA instruction fetch operation when the number of successfully received packets dmaCnt reaches the preset single instruction fetch length dmaBlockSize, that is, it enters the DMA_FINISH state. When the system receives the transfer success completion flag dma_rd_resp from the previous-level DMA controller, it returns to the next DMA initialization state.

5. The instruction flow control system for a neural network accelerator according to claim 3, characterized in that: The primary instruction flow control unit also includes an instruction post-processing section, which is responsible for executing primary execution instructions, responding to the impact of debugging events and synchronization events on instruction distribution, and maintaining the execution status.

6. The instruction flow control system for a neural network accelerator according to claim 5, characterized in that: The instruction post-processing stage selects to enter the normal execution state or the instruction multi-step execution state based on the current local instruction cache state and the global execution enable control signal GlobalRunStart. In the multi-step execution state of instructions, the system monitors and compares the control word information and the current execution state of the instructions to enter the debug state and pause the instruction distribution. It supports entering the Debug Halt state at a specified instruction execution position. The system only supports the interaction of debugging information with the external debugging unit in the Debug Halt state. The instruction sending stage responds to the Event event and enters the Event Halt state. The system pauses and releases according to the debugging event, ensuring that the instruction sending is controlled by the execution unit's execution event state. After issuing the command, the system responds to the control signal of the multi-core synchronous control unit and enters the Sync Halt state. Based on the synchronous event, the system pauses and releases, ensuring coordinated control under multi-core execution conditions.

7. The instruction flow control system for a neural network accelerator according to claim 5, characterized in that: The primary instruction flow control unit maintains a series of debugging event ports connected to the external debugging unit for status interaction and debugging control with the debugging unit. The primary instruction flow control unit maintains a series of synchronization event ports connected to external synchronization units for status interaction with the synchronization units and enters a pause waiting state based on synchronization control information.

8. The instruction flow control system for a neural network accelerator according to claim 1, characterized in that: The secondary instruction flow control unit independently maintains a list of instruction execution dependency information. Each execution unit queries the list of instruction execution dependency information stored in the target dependent execution unit through routing, and dynamically adjusts its own execution and pause status according to the return flag.

9. The instruction flow control system for a neural network accelerator according to claim 8, characterized in that: The secondary instruction flow control unit adopts Depend / Inform state control, which dynamically controls the pause and release of each execution unit based on the instruction execution dependency information list stored inside each execution unit.

10. The instruction flow control system for a neural network accelerator according to any one of claims 1-9, characterized in that: The first-level execution instructions include various debug IDLE instructions, FINISH instructions, and dependency register refresh instructions; The secondary execution instructions are recoded subsets of the original instructions, including the encoding of instruction execution dependencies between execution units and the execution parameter information of each execution unit for the current instruction. The secondary execution instructions are distributed to the secondary instruction flow control unit for execution through the instruction distribution custom internal bus Master interface.

Images