An instruction set apparatus for a reconfigurable deep neural network accelerator

Abstract

The application discloses an instruction set device for a reconfigurable deep neural network accelerator, the reconfigurable deep neural network accelerator comprising an instruction controller and a plurality of hardware modules, the hardware modules comprising input / output modules, matrix calculation modules and vector calculation modules; a microkernel programming paradigm is used to provide multiple levels of hardware configuration; the calculation task of the deep neural network accelerator is compiled into a plurality of microkernels, each microkernel being coded as a plurality of hardware instructions; each hardware instruction is used for module hardware configuration control and time level configuration control of a specific calculation or data movement operation; each hardware instruction comprises the following fields: instruction type, module type, configuration address, dependency flag and module configuration content. The application uses hardware instructions to represent data flow reconfiguration and function reconfiguration of the reconfigurable deep neural network accelerator, and realizes efficient programming of a plurality of complex reconfigurable function neural network hardware accelerators.

Description

Technical Field

[0001] This invention relates to hardware instruction set architecture technology, specifically to an instruction set interaction interface device for a reconfigurable neural network accelerator. Background Technology

[0002] Instruction Set Architecture (ISA), also known as instruction set or instruction set architecture, is the programming-related part of computer architecture. It includes basic data types, instruction sets, registers, addressing modes, memory architecture, interrupts, exception handling, and external I / O. The instruction set architecture contains a series of opcodes (machine language) and the basic commands executed by a specific processor.

[0003] Deep Neural Network (DNN) accelerators are a new type of computer hardware architecture designed for efficient processing of various neural network applications. Compared to traditional computers, DNN accelerators have the following characteristics: (1) High parallelism: The accelerator contains thousands of processing elements (PEs), which form rectangular or tree-like interconnected arrays. Data is transferred between different PEs through hardware data flow. (2) Limited algorithm support: DNN accelerators typically only need to support various DNN algorithms, such as matrix multiplication, convolution, activation functions, etc., without needing to support programming of general-purpose algorithms. (3) Simple control logic: DNN accelerators are usually a subsystem of a complete computer, so they do not need to support all the functions of a complete computer, such as complex branch control, interrupts, etc. (4) Explicit memory access: Unlike traditional computers that cache data through multiple layers of cache, DNN accelerators use an explicit memory access mechanism. Users need to specify the specific location of data in each layer of memory and the access sequence of each layer of memory in each cycle through instructions.

[0004] A reconfigurable DNN accelerator is a novel hardware architecture for DNN accelerators. This architecture implements one or more reconfigurable features: dataflow reconfiguration, function reconfiguration, and multi-module reconfiguration. Dataflow reconfiguration can dynamically adjust the way data is transmitted in the PE array. Function reconfiguration can dynamically adjust the algorithms implemented by the ALU units. Multi-module reconfiguration allows different DNN computation tasks to be run in multiple sub-modules.

[0005] In recent years, some work has been done to design instruction set architectures for DNN accelerators. These include Cambricon[1], VTA[2], Gemmini[3], etc. These instruction set architectures are all adapted to specific DNN accelerator hardware, but the existing instruction sets have limited support for reconfigurable DNN accelerators, and usually can only support one reconfigurable feature. Faced with accelerators with multiple reconfigurable features, there is an urgent need for a new instruction set architecture that can support multiple reconfigurable features of multiple DNN accelerators, thereby improving the operating efficiency of the accelerator.

[0006] References

[0007] [1] Liu, Shaoli, Zidong Du, Jinhua Tao, Dong Han, Tao Luo, Yuan Xie, Yunji Chen, and Tianshi Chen. "Cambricon: An instruction set architecture for neural networks." ACM SIGARCH Computer Architecture News 44, no.3 (2016): 393-405.

[0008] [2]Moreau, Thierry, Tianqi Chen, Luis Vega, Jared Roesch, Eddie Yan, Lianmin Zheng, JoshFromm et al. "A hardware–software blueprint for flexibledeep learning specialization." IEEEMicro 39, no.5 (2019): 8-16.

[0009] [3] Genc, ​​Hasan, et al. "Gemmini: Enabling systematic deep-learning architecture evaluation via full-stack integration." 2021 58th ACM / IEEE DesignAutomation Conference (DAC). IEEE, 2021. Summary of the Invention

[0010] To overcome the shortcomings of the prior art, this invention proposes an instruction set device for reconfigurable deep neural network accelerators. It is an instruction set architecture that supports complex reconfigurable neural network accelerators. By using hardware instructions to represent the data flow reconstruction and functional reconstruction characteristics of neural network accelerators, it achieves efficient programming of complex neural network hardware accelerators, reduces the length of programming code, and supports higher hardware operating efficiency.

[0011] For convenience, the following terms are used in this invention:

[0012] PE (Processing Element) computing unit

[0013] DMA (Direct Memory Access)

[0014] FSM (Finite State Machine)

[0015] SRAM (Static Random-Access Memory)

[0016] DRAM (Dynamic Random Access Memory)

[0017] The technical solution of the present invention:

[0018] An instruction set device for a reconfigurable deep neural network accelerator.

[0019] Compared with the prior art, the beneficial effects of the present invention are:

[0020] Existing instruction set architectures typically only support one type of reconfigurable DNN accelerator hardware. The instruction set architecture proposed in this invention can support multiple hardware reconfiguration features, thereby improving the programming efficiency of reconfigurable hardware accelerators. Detailed Implementation

[0021] The present invention is further described below through embodiments, but these embodiments are not intended to limit the scope of the invention in any way.

[0022] This invention provides an instruction set device for a reconfigurable deep neural network accelerator. By using instructions to represent the data flow reconstruction and functional reconstruction characteristics of the accelerator, it can support a variety of complex reconfigurable neural network accelerators.

[0023] The instruction set apparatus for a reconfigurable deep neural network accelerator provided by the present invention includes an instruction set format, comprising: instruction type, module type, configuration address, dependency flag, and module configuration content.

[0024] This invention relates to an instruction set architecture for a reconfigurable deep neural network accelerator. The accelerator includes an instruction controller and multiple hardware modules, including input / output modules, matrix computation modules, and vector computation modules, and can be used to handle various deep neural network computation tasks.

[0025] The overall format of the instruction set architecture:

[0026] The instruction set architecture designed in this invention employs a microkernel programming paradigm to provide multi-level hardware configuration. The entire deep neural network (DNN) computation task is compiled into multiple microkernels, each encoded as multiple hardware instructions. Each hardware instruction controls the module hardware configuration and time-level configuration for a specific computation or data movement operation. The module hardware configuration determines the data flow and functionality of each hardware module in the reconfigurable deep neural network accelerator. Time-level configuration is achieved by programming the finite state machines (FSMs) within the accelerator's computation modules and DMA modules, enabling time-level control of the multi-layered nested loop algorithms and data transfer tasks within the accelerator's deep neural network.

[0027] Table 1 Instruction Set Architecture Format

[0028] data Instruction type Module type Dependency flags Configuration address Configuration content

[0029] Table 1 shows the format of the 128-bit Instruction Set Architecture (ISA) used in this invention. Each hardware instruction contains 5 fields: Inst Type (2 bits), Module Type (2 bits), Dep.Flags (8 bits), Config Addr (4 bits), and Config Payload (112 bits). The meaning and purpose of each field are described below.

[0030] 1) Instruction type, module type, configuration address

[0031] The first two digits (Inst Type) determine the type of hardware instruction, which can be one of the following two: a) pure configuration or b) configuration-execution. If the type is configuration-execution, the instruction controller sends a start signal to the corresponding hardware module (input / output module, matrix calculation module, or vector calculation module) after writing the corresponding configuration to invoke the microkernel and waits for it to complete. Otherwise, it means that the instruction configuration of the microkernel has not yet been completed, and the next instruction is retrieved from the user-written program to continue the configuration. Next, we designed a two-dimensional table to configure different contents for each hardware module, as shown in Table 2.

[0032] In each instruction, the module type field defines the column index of Table 2, indicating the hardware module number that the reconfigurable deep neural network accelerator needs to configure. The hardware module type can be one of the following four: a) input module; b) matrix computation module; c) vector computation module; d) output module, with module numbers 0, 1, 2, and 3 respectively. The configuration address field defines the row index of Table 2, indicating the configuration type within the hardware module. When translating (encoding) the microkernel computation task into hardware instructions, all configuration addresses in the hardware module configured by that hardware instruction require one hardware instruction for configuration. Specifically, addresses 3 through 7 of the input and output modules are invalid, therefore addresses 0 through 2 need to be configured, requiring 3 instructions; the matrix and vector computation modules require configuration of all addresses 0 through 7, a total of 8 instructions. For example, when programming a microkernel containing K instructions, the first K-1 instructions need to use a pure configuration type, and the last instruction uses a configuration-execution type to initiate hardware execution.

[0033] 2) Dependency markers

[0034] We use an 8-bit dependency flag field to encode the dependencies between different microkernels. When the x-th bit (from low to high) of the lower 4 bits is 1, the instruction requires a ready signal sent by module x before execution can begin. When the x-th bit (from low to high) of the higher 4 bits is 1, it indicates that a ready signal is sent to module x after the instruction completes. For example, if a vector computation instruction depends on a read instruction, the instruction dependency encoding for the vector computation module is 00000001, with the 0th bit of the lower 4 bits being 1, indicating that the vector computation instruction depends on a ready signal sent by module 0 (the read module). Simultaneously, the dependency flag for the read module is set to 01000000, with the 2nd bit of the higher 4 bits being 1, indicating that a ready signal is sent to module 2 (the vector computation module) after the read instruction completes.

[0035] Table 2. Configuration details of each module at different addresses.

[0036]

[0037] 3) Configuration Content

[0038] Table 2 shows the configuration details for each module type at different addresses. The specific details of each configuration are described below.

[0039] 3.1 Global Interconnection Network Configuration

[0040] All modules share the same global interconnect configuration register at address 1. This register determines which module reads or writes to each memory location. Each memory location uses 4 bits of interconnect configuration information. The first two bits represent write information, indicating the module type writing to the memory; the last two bits represent read information, indicating the module type reading the memory. The relationship between module type and corresponding value is shown in Table 3. Each instruction only changes the configuration information of one part of the memory; the configuration information of the other part remains unchanged and needs to be filled with 0 in the corresponding positions. Since this register is shared by all modules, writes to this register should be masked (the mask bit is at address 0) to avoid conflicts. Therefore, for all locations where a non-zero value is filled, the corresponding mask bit needs to be filled with 3.

[0041] Table 3 shows the global interconnection network configuration values ​​corresponding to the module types.

[0042] Matrix calculation module 1 Vector computation module 2 Input, Output 3

[0043] 3.2 Module-Specific Configuration

[0044] At address 2, each module is configured with its specific runtime configuration information. For input and output modules, the configuration includes the DRAM base address (32 bits), SRAM base address (16 bits), and read / write length (16 bits). For the matrix calculation module, the configuration includes a reset bit (determining whether the output matrix is ​​reset; 1 for reset, 0 for no reset) and a data flow (determining how data is transmitted in the matrix calculation module; 1 for output retention, 0 for weight retention). For the vector calculation module, both its operation configuration and data flow configuration contain multiple entries, which will be discussed separately below.

[0045] 3.3 Operation Configuration of Vector Calculation Module

[0046] The accelerator may contain multiple vector computation modules. For computations using immediate values, "Use Immediate Values" should be configured to 1, and "Immediate Value Value" should be configured to the immediate value required for the computation. Otherwise, "Use Immediate Values" should be configured to 0. Additionally, the operands, the source ports for each input operand, and the output destination ports must be configured. The operation configuration for each vector computation module is shown in Table 4.

[0047] Table 4. Operation configuration contents and bit width occupied by each configuration item

[0048] Use immediate numbers? 1 Immediate value 16 operand 4 Input operand A source port 2 Input operand B source port 2 Input operand C source port 2 Output destination port 2

[0049] The configuration values ​​for each operand are shown in Table 5. For example, when setting the operand of the vector computation module to Mul, the corresponding operand position of the vector computation module needs to be set to 7.

[0050] Table 5. Operand definitions and corresponding configuration values

[0051] Min 1 Max 2 Add 3 Sub 4 Shl 5 Shr 6 Mul 7 Mac 8 exp 9 log 10 sigmoid 11 tanh 12 nop 13

[0052] For the source of input operand A, the source of input operand B, the source of input operand C, and the destination of output, their configuration values ​​represent different port numbers, as shown in Table 6.

[0053] Table 6. Configuration methods for input operand sources and output destinations.

[0054] No input / no output 0 Port 1 1 Port 2 2 Port 3 3

[0055] 3.4 Data Flow Configuration of the Vector Operation Module

[0056] The data flow configuration of the vector computation module defines the data flow used by each operand of the vector computation module. An accelerator may have multiple vector computation modules, but these modules share the same data flow configuration. The bit width occupied by each configuration is shown in Table 7.

[0057] Table 7 Data Stream Configuration Contents and Bit Width Occupied by Each Configuration Item

[0058] Input operand A data stream 2 Input operand B data stream 2 Input operands C data stream 2 Output operand data stream 2

[0059] The correspondence between the data stream of operand A and the configuration value is shown in Table 8.

[0060] Table 8 shows the configuration values ​​corresponding to the data stream of input operand A.

[0061] Horizontal multicast 0 Horizontal pulsation 1 Horizontal pulsation, vertical multicast 2 Horizontal unicast, vertical multicast 3

[0062] The correspondence between the data streams of operands B, C, and the output operands and the configuration values ​​is shown in Table 9.

[0063] Table 9 shows the configuration values ​​for the data streams of input operand B or C and output operands.

[0064]

[0065] 3.5 Loop and Memory Access Configuration

[0066] The ISA designed in this invention uses the remaining configuration registers (addresses 3-7) to handle the control and data access of any four perfectly nested loops in the vector calculation module and matrix calculation module. This includes the range of each loop, the on-chip cache base address, and the data access step size of each loop. Specifically, the range of each loop is configured at address 3, with a total of four loops, each loop occupying 16 bits, for a total of 64 bits.

[0067] Addresses 4 through 7 are configured with the base address of the memory accessed by each operand, as well as the access step size for each loop. The addresses used by each operand are shown in Table 10.

[0068] Table 10 shows the configuration address used for each operand.

[0069]

[0070]

[0071] For each operand, the memory address Addr it accesses can be represented by the following formula:

[0072] Addr=Base+St1×Idx1+St2×Idx2+St3×Idx3+St4×Idx4

[0073] Here, Idx1 to Idx4 are the indices of these four loops, dynamically generated during accelerator runtime. For each operand, Base and St1 to St4 are configured at their respective configuration addresses. Each number occupies 16 bits, for a total of 80 bits.

[0074] The instruction set architecture proposed in this invention provides programming interfaces for both software and hardware, offering the necessary controllers, data flows, and interconnection support for the hardware. On the software side, the deep neural network runtime framework should include a compiler for reconfigurable accelerators, used to compile and generate configuration code conforming to the definitions of this instruction set. On the hardware side, the reconfigurable deep neural network accelerator should implement the corresponding functions according to the configuration described in this invention, including instruction controllers, data flows, operands, and interconnection network configurations. These configurations are used to implement the corresponding functions, thereby enabling efficient operation of various deep neural network applications.

[0075] It should be noted that the purpose of disclosing the embodiments is to help further understand the present invention. However, those skilled in the art will understand that various substitutions and modifications are possible without departing from the scope of the present invention and the appended claims. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the scope of protection of the present invention is defined by the scope of the claims.

Claims

1. An instruction set device for a reconfigurable deep neural network accelerator, characterized in that, The reconfigurable deep neural network accelerator includes an instruction controller and multiple hardware modules, including input / output modules, matrix computation modules, and vector computation modules. By using hardware instructions to represent the data flow reconstruction and functional reconstruction of the reconfigurable deep neural network accelerator, efficient programming of various complex reconfigurable neural network hardware accelerators can be achieved. The instruction set device employs a microkernel programming paradigm to provide multi-level hardware configuration; it compiles the computational tasks of the deep neural network accelerator into multiple microkernels, each of which is encoded into multiple hardware instructions; each hardware instruction is used for module hardware configuration control and time-level configuration control for specific computation or data movement operations. The instruction set adopts the 128-bit instruction set architecture (ISA) format; each hardware instruction includes the following fields: instruction type, module type, configuration address, dependency flags, and module configuration content. The instruction types for hardware instructions include pure configuration or configuration-execution; if the type is configuration-execution, the instruction controller sends a start signal to the corresponding hardware module to call the microkernel after writing the corresponding configuration, and waits for it to complete. Otherwise, retrieve the next instruction and continue with the configuration; The module type field of each instruction indicates the module number corresponding to the type of hardware module that the reconfigurable deep neural network accelerator needs to be configured with; the hardware module types include: input module; matrix calculation module; vector calculation module; output module; The configuration address field is used to indicate the configuration type inside the hardware module; all configuration addresses in the hardware module configured by the hardware instruction are configured using a single hardware instruction. The dependency flag field is used to encode the dependencies between different microkernels; it uses 8 bits, where when the xth bit from the lowest to the highest of the lower 4 bits is 1, it means that the hardware instruction can only start execution after the ready signal sent by the module with number x is completed; when the xth bit from the lowest to the highest of the higher 4 bits is 1, it means that the ready signal is sent to the xth module after the hardware instruction is completed. The module configuration includes global interconnection network configuration data, module-specific configuration, operation configuration of the vector computation module, data flow configuration of the vector operation module, and loop and memory access configuration. All modules share the same global interconnection network configuration data, which determines which module reads or writes memory. The module-specific configuration includes module-specific runtime configuration information. The operation configuration of the vector computation module is used to configure the corresponding operations of the vector computation module. The data flow configuration of the vector operation module is used to define the data flow used by each operand of the vector operation module, and multiple vector operation modules share the same data flow configuration. The loop and memory access configuration is used to handle the control and data access of any four levels of perfectly nested loops in the vector computation module and matrix computation module, including the range of each loop, the on-chip cache base address, and the data access step size of each loop.

2. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 1, characterized in that, The module hardware configuration determines the data flow and function of each hardware module in the reconfigurable deep neural network accelerator; the time-level configuration achieves time-level control of the multi-layer nested loop algorithms and data transmission tasks within the deep neural network of the accelerator by programming the finite state machines inside the accelerator's computation modules and input / output modules.

3. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 1, characterized in that, Each hardware instruction includes the following fields: instruction type (2 bits), module type (2 bits), dependency flag (8 bits), configuration address (4 bits), and configuration content (112 bits).

4. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 3, characterized in that, The module numbers for the hardware module types are 0, 1, 2, and 3, respectively.

5. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 4, characterized in that, When translating microkernel computing tasks into hardware instructions, all configuration addresses in the hardware module configured by the hardware instruction are configured using a single hardware instruction. Specifically, addresses 3 to 7 of the input and output modules are invalid, requiring configuration of addresses 0 to 2 (a total of 3 instructions). The matrix and vector computing modules require configuration of all addresses 0 to 7, totaling 8 instructions.

6. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 5, characterized in that, When programming a microkernel containing K instructions, the first K-1 instructions should use the pure configuration type, and the last instruction should use the configuration-execution type to initiate hardware execution.

7. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 5, characterized in that, Each memory uses 4 bits of global interconnect network configuration information; the first two bits are write information, i.e., the module type being written to the memory; the last two bits are read information, i.e., the module type being read from the memory.

8. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 7, characterized in that, In the module-specific configuration, the configuration content for input and output modules includes DRAM base address, SRAM base address, and read / write length; for matrix calculation module, the configuration content includes reset bit and data stream.

9. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 7, characterized in that, In the module-specific configuration, for the vector computation module, both the operation configuration and the data flow configuration contain multiple items, including: configuring the computation using immediate values, operands, the source port of each input operand, and the output destination port.

10. The instruction set apparatus for a reconfigurable deep neural network accelerator as described in claim 9, characterized in that, The loop and memory access configuration specifically uses the configuration registers at addresses 3-7 to handle the control and data access of any four perfectly nested loops in the vector calculation module and matrix calculation module; where address 3 configures the range of each loop, there are a total of four loops, each loop occupies 16 bits, and a total of 64 bits.

Images