A storage-computing integrated hardware accelerator and its instruction set architecture

Abstract

The present invention discloses a storage-computation integrated hardware accelerator and its instruction set architecture, the hardware accelerator includes a control unit, an operation execution unit and a cache unit; the operation execution unit includes a weight data cache module, a feature data cache module and a data processing module, the data processing module performs convolution, activation, pooling and full connection operations on weight data and feature data, and the operation result is sent to the cache unit; the cache unit judges the data transmission direction according to the arbitrator, transmits the operation result back to the CPU, or temporarily stores the processing result to the on-chip cache module, and sends the operation result to the feature data cache module during the next operation. The present invention can realize the online configuration of four operators of convolution, activation, pooling and full connection in the neural network algorithm through the data flow configuration controlled by the instruction, and can realize the reuse of the intermediate operation results through the instruction configuration arbitrator, reducing the data interaction with the upper host during the data operation process.

Description

Technical Field

[0001] The present invention relates to the field of storage-computing integrated technology, and in particular to a storage-computing integrated hardware accelerator and an instruction set architecture thereof. Background Art

[0002] With the rapid development of artificial intelligence and deep learning technologies, traditional hardware accelerators face storage bottlenecks and power consumption bottlenecks when processing massive amounts of data. The introduction of the storage-computing integrated architecture provides a new idea for solving these problems. The storage-computing integrated architecture reduces the number of data transfers, improves computing efficiency and reduces power consumption by performing calculations directly in the storage unit. However, the existing instruction set architecture cannot give full play to the advantages of the storage-computing integrated architecture. Due to the unique calculation method of in-memory calculation, its instruction design and data flow scheduling method are quite different from those of the traditional storage-computing separation computing unit. The application of traditional computing instructions cannot give full play to the advantages of the storage-computing computing unit, and will make its work efficiency lower than that of traditional hardware accelerators. An embedded processor, instruction set and data processing method for a storage-computing integrated chip (publication number CN110990060A) discloses matrix multiplication and addition operations through a storage-computing integrated array, and other operations are performed by the CPU, resulting in the need for data to be repeatedly transferred between the storage-computing integrated core and the CPU, resulting in low computing efficiency and the inability to truly achieve storage-computing integration. Therefore, a storage-computing integrated architecture and its instruction set are needed to solve the above problems. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a storage-computing integrated hardware accelerator with strong data reusability and high computing efficiency. The second purpose of the present invention is to provide an instruction set architecture for storage-computing integrated with standardization and ease of use.

[0004] Technical solution: The storage-computing integrated hardware accelerator of the present invention includes a control unit, a calculation execution unit and a cache unit;

[0005] The control unit is used to receive instructions from the CPU, decode the instructions and send them to the operation execution unit and the cache unit; and is also used to notify the CPU by setting an interrupt after the task is completed;

[0006] The operation execution unit includes a weight data cache module, a feature data cache module and a data processing module. The weight data cache module stores weight data required for the operation, the feature data cache module stores feature data required for the operation, and the data processing module performs operations on the weight data and feature data, and sends the operation results to the cache unit.

[0007] The cache unit transmits the operation result back to the CPU, or temporarily stores the processing result in the on-chip cache module, and sends the operation result to the feature data cache module during the next operation.

[0008] Furthermore, the data processing module in the operation execution unit includes an in-memory computing module, an addition tree module, a data activation module and a data pooling module, and the in-memory computing module is used to perform convolution and / or full connection operations.

[0009] Furthermore, the in-memory computing module, addition tree module, data activation module, data pooling module and the on-chip cache module are connected in sequence, and whether the in-memory computing module, addition tree module, data activation module and data pooling module are enabled is configured through instructions.

[0010] Furthermore, the cache unit uses an arbitrator to determine whether the operation result is transmitted back to the CPU or temporarily stored in an on-chip cache module.

[0011] The data processing method of the hardware accelerator of the present invention comprises the following steps:

[0012] The control unit receives instructions from the CPU, decodes the instructions and sends them to the operation execution unit and cache unit;

[0013] The CPU sends the weight data to the weight data cache module of the operation execution unit, and sends the feature data to the feature data cache module. The data processing module performs operations on the weight data and the feature data, and sends the operation results to the cache unit;

[0014] The cache unit transmits the calculation result back to the CPU or temporarily stores the processing result in the on-chip cache module, and sends the calculation result to the feature data cache module during the next calculation;

[0015] Notify the CPU by setting an interrupt.

[0016] The instruction set architecture of the hardware accelerator described in the present invention includes instructions for each module in the operation execution unit and the cache unit, and each instruction includes the activation, scale, function and working status of the corresponding module.

[0017] Furthermore, it also includes an arbitrator control instruction for determining the output direction of the internal data of the on-chip cache module. If the data is the final result, it is output to the CPU. If the data is used for the next calculation, it is stored in the on-chip cache module and sent to the feature data cache module during the next calculation.

[0018] Furthermore, the instructions of the in-memory computing module in the operation execution unit include the working status of the in-memory computing module, the number of enabled operation units, functions and reserved bits; the working status of the in-memory computing module includes the completed configuration standby state, the working state and the completed operation state; the functions of the in-memory computing module include convolution and full connection.

[0019] Furthermore, the instructions of the addition tree module, data activation module and data pooling module in the operation execution unit include their working status, the number of enabled operation units and reserved bits; the working status of the addition tree module, data activation module and data pooling module all include the completed configuration standby status, the working status and the completed operation status.

[0020] Furthermore, the arbiter control instruction includes a data transmission direction and a reserved bit; the data transmission direction includes transmission to an off-chip data bus and transmission to a feature data cache module.

[0021] Beneficial effects: Compared with the prior art, the advantages of the present invention are:

[0022] (1) The hardware accelerator of the present invention has good algorithm compatibility. The four operators of in-memory calculation (multiplication and accumulation), activation, pooling, and full connection are arranged in sequence, which conforms to the front-to-back arrangement relationship of operators in most neural network algorithms, abandons the complex data path network design on the chip, and reduces the hardware complexity and power consumption of the hardware accelerator. Through the data flow configuration controlled by instructions, the online configuration of the four operators of convolution, activation, pooling, and full connection in the neural network algorithm can be realized, thereby realizing a wider range of neural network algorithm support; by configuring whether each module is enabled or not, the functional modules not needed in a single round of neural network operation can be put into sleep mode to save power consumption, and at the same time, it can also meet the requirements of different calculation layers of the neural network algorithm for different operator arrangements and combinations.

[0023] (2) The hardware accelerator of the present invention improves data reusability. By configuring the arbitrator through instructions, it can realize the reuse of intermediate calculation results, reduce the data interaction with the upper host during the data calculation process, improve the calculation efficiency, and reduce power consumption.

[0024] (3) The instruction set architecture of the present invention realizes format standardization. Through the unified allocation of instruction addresses and the provisions of instruction set length and format, sufficient address space is allocated to the overall register and each sub-module register respectively, and each instruction has a unified instruction format to ensure that each module can have sufficient instruction resources and the driver layer can easily issue specific instructions. The flexibility and scalability of the system are improved, and the design complexity of the driver layer is reduced. As computing tasks continue to develop and change, users can adapt to new computing needs by adding new instructions and expanding the functions of registers. The provision of a unique instruction format improves the standardization and ease of use of instruction reading and use, and can be easily integrated into various computing systems.

[0025] (4) The instruction set architecture of the present invention has dedicated high-efficiency instructions for storage and computing. Through the unique instructions, the storage characteristics of the input weight data of the computing module in the memory are called, so that the input weight data is stored once and participates in the calculation multiple times, which reduces data transportation, breaks the shackles of the memory wall in traditional hardware accelerators, and improves computing performance and computing speed.

[0026] (5) The instruction set architecture of the present invention is deeply integrated with the in-memory computing technology. By setting the instruction set format specification, precise control of sub-modules, data flow scheduling and optimizing the design of the integrated storage and computing architecture, the advantages of the in-memory computing architecture are fully utilized, and the performance and efficiency of the system are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 It is a schematic diagram of the hardware accelerator structure of the present invention.

[0028] Figure 2 This is a flowchart of data processing of the hardware accelerator of the present invention.

[0029] Figure 3 Schematic diagram of instruction address allocation according to an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The technical solution of the present invention is further described below in conjunction with the accompanying drawings.

[0031] Example 1

[0032] like Figure 1 As shown, the storage-computing integrated hardware accelerator of the present invention includes a control unit, an operation execution unit and a cache unit.

[0033] The control unit designs the instruction control module and the interrupt control module according to the storage and computing integrated characteristics to realize the reception, parsing, issuance and feedback of the instruction set. By issuing different instructions, the execution scale, working status and data flow direction of each functional unit of the hardware accelerator can be configured.

[0034] The operation execution unit includes a weight data cache module, a feature data cache module, an in-memory calculation module, an addition tree module, a data activation module and a data pooling module, which are connected in sequence. The weight data cache module and the feature data cache module are connected to the CPU through a data bus, and the weight data and feature data are obtained from the main memory. The weight data can be reused multiple times; the in-memory calculation module reuses the convolution calculation module or the fully connected calculation module through configuration.

[0035] The operation execution unit configures the data processing module and the data path switch by the instruction set to achieve reconfiguration of the data flow, thereby improving the multi-algorithm compatibility of the hardware accelerator without occupying too many resources; taking advantage of the integrated storage and computing architecture, part of the input and output data is stored in the computing array under instruction control, so that the overall calculation process reduces the update of weight data, reuses the weight data multiple times, and can also reuse the results of the current operation, reducing the use of cache resources and register resources inside the hardware accelerator, and reducing the frequency of moving intermediate data back and forth during calculations inside the accelerator, thereby improving performance and reducing power consumption.

[0036] The cache unit includes an on-chip cache module and an arbitrator. The on-chip cache module is connected to the data pooling module of the operation execution unit. The operation result is stored in the on-chip cache module. The output direction of the operation result is determined by the arbitrator. If the operation result is final data, it is transmitted from the on-chip cache module to the main memory of the CPU through the data bus. If the operation result participates in the next operation, it is temporarily stored in the on-chip cache module and sent to the feature data cache module of the operation execution unit during the next operation to reduce the transmission of feature data.

[0037] The functional modules in the hardware accelerator that can be configured through the instruction set include the in-memory computing module, the addition tree module, the data activation module, the data pooling module and the arbitrator module. The configurable options mainly include: whether the module is enabled, the module enabling scale, the data flow direction, and the task completion status of each module can also be read.

[0038] Example 2

[0039] like Figure 2 As shown, the data processing flow of the storage-computing integrated hardware accelerator includes a five-stage pipeline, and the specific steps are as follows.

[0040] (1) Initialization and configuration.

[0041] (1.1) Loading configuration: When the system starts, the CPU initializes the hardware accelerator. For the hardware accelerator, during this process, the instruction control module completes the interception and decoding of instructions and sends the configuration information to the in-memory computing module, the addition tree module, the data activation module, the data pooling module and the arbitrator module.

[0042] (1.2) Interrupt configuration: Configure the interrupt of the hardware accelerator so that it can notify the CPU when the task is completed.

[0043] (2) Data transmission.

[0044] Data preparation: The CPU (central processing unit) transfers the data to be processed from the main memory to the weight data cache module and feature data cache module of the hardware accelerator.

[0045] (3) Data processing.

[0046] (3.1) Execute acceleration task: The hardware accelerator starts processing data according to the configuration and control commands.

[0047] (3.2) Status monitoring: During data processing, the hardware accelerator will update the internal status register to record the processing progress and status.

[0048] (4) Data is transmitted back.

[0049] (4.1) Data output: After processing is completed, the hardware accelerator stores the calculation results in the on-chip cache module.

[0050] (4.2) Data return: The calculation results are transferred from the on-chip cache of the hardware accelerator back to the main memory through the data bus; or the calculation results are temporarily stored in the on-chip cache and wait to be input into the feature data cache module in the next round of calculation.

[0051] (5) Notify the CPU.

[0052] (5.1) Triggering an interrupt: After the hardware accelerator completes the task, it notifies the CPU through an interrupt signal.

[0053] (5.2) Interrupt processing: The CPU responds to an interrupt, executes the corresponding interrupt service routine, reads the processing result, and performs subsequent processing.

[0054] Example 3

[0055] In order to give full play to the advantages of in-memory computing architecture, the instruction set architecture described in the present invention includes the following contents.

[0056] (1) Instruction addresses are uniformly allocated.

[0057] The instruction register includes three types of content, namely the overall register, the module register and the reserved bit.

[0058] a) Overall register: represents the hardware accelerator version information, summarizes the status of the in-memory computing module, addition tree module, data activation module, data pooling module and arbiter module, and implements the reset operation of the above modules.

[0059] b) Sub-module registers: specify the activation, scale, and function of the corresponding modules (including in-memory computing module, addition tree module, data activation module, data pooling module, and arbitrator module), and feedback the working status of the module.

[0060] c) Reserved bit: used for subsequent function expansion.

[0061] By reasonably allocating and configuring instruction addresses, the register instruction controller of the present invention can support up to 128 instruction encodings. The registers of each module are strictly divided according to the address, and the corresponding register group can be directly read and written through the configuration bus, which is convenient for driver side development. Figure 3 The figure shows a schematic diagram of instruction address allocation in the instruction register in this embodiment, which can be adjusted according to actual needs during application.

[0062] (2) Instruction set length and format regulations.

[0063] In this embodiment, the length of a single instruction is uniformly specified as 16 bits, which can be adjusted according to actual needs during application. A reserved bit is designed within a single instruction, and the internal composition format is uniform.

[0064] a) Example of in-memory computing module instruction format

[0065] The 0x32 position stores an in-memory computing module instruction, which functions to store the in-memory computing module status. The instruction is shown in Table 1.

[0066] Table 1

[0067]

[0068] The meaning of this instruction is as follows: [1:0] indicates the working status of the in-memory computing module, which are Ready (configuration completed standby status), Busy (working), and Done (computation completed status), corresponding to status values ​​00, 01, and 11, respectively, and are updated by the in-memory computing module; [8:2] indicates the number of enabled computing units, a total of 7 bits, which can represent up to 128 computing units; [10:9] indicates module function options, which can be used for convolution or full connection; [15:11] are reserved bits.

[0069] In this instruction, by configuring the different functions of the storage and computing modules, the in-memory computing modules can be reused as multiplication-accumulation convolution or fully connected functions, that is, there is no need to design additional fully connected layer computing modules; by configuring the number of in-memory computing modules, the number of storage and computing units can be flexibly configured according to the amount of calculation in the algorithm and the size of the weight data (the compiler decides the specific configuration, and the hardware accelerator is responsible for execution), and the weight data can be stored in the weight data cache module on demand, giving full play to the advantages of the integrated storage and computing architecture.

[0070] b) Example of instruction format for addition tree module

[0071] The position 0x42 stores an addition tree module instruction, which functions as the addition tree module status. The instruction is shown in Table 2.

[0072] Table 2

[0073]

[0074] The meaning of this instruction is as follows: [1:0] indicates the working status of the addition tree module, which are Ready (configuration completed standby status), Busy (working), and Done (computation completed status), and the corresponding status values ​​are 00, 01, and 11, respectively, and are updated by the in-memory calculation module; [8:2] indicates the operation scale; [15:9] are reserved bits.

[0075] c) Example of instruction format for addition tree module

[0076] The position 0x40 stores an addition tree module instruction, which is used to determine whether the addition tree is enabled. The instruction is shown in Table 3.

[0077] Table 3

[0078]

[0079] The meaning of this instruction is as follows: only the lowest bit is valid, 0 means disabled, and 1 means enabled. By designing whether to enable or not, it can meet the requirements of whether specific operators are required to participate in the calculation process of different algorithms, thereby improving the algorithm compatibility of the hardware accelerator.

[0080] d) Data activation module instruction example

[0081] The position 0x52 stores a data activation module instruction, the function of which is the data activation module status. The instruction is shown in Table 4.

[0082] Table 4

[0083]

[0084] The meaning of this instruction is as follows: [1:0] indicates the working status of the data activation module, which are Ready (configuration standby status), Busy (working), and Done (computation completed status), and the corresponding status values ​​are 00, 01, and 11, respectively, and are updated by the in-memory calculation module; [8:2] indicates the operation scale; [15:9] are reserved bits.

[0085] e) Data pooling module instruction example

[0086] The position 0x62 stores a data pooling module instruction, whose function is the data pooling module status. The instruction is shown in Table 5.

[0087] Table 5

[0088]

[0089] The meaning of this instruction is as follows: [1:0] indicates the working status of the data pooling module, which are Ready (configuration standby status), Busy (working), and Done (computation completed status), and the corresponding status values ​​are 00, 01, and 11, respectively, and are updated by the addition tree module; [8:2] indicates the operation activation scale; [15:9] are reserved bits.

[0090] f) Arbitrator Instruction Example

[0091] The 0x92 position stores an arbiter module instruction, whose function is the arbiter module status. The instruction is shown in Table 6.

[0092] Table 6

[0093]

[0094] The meaning of this instruction is as follows: [1:0] indicates the data transmission direction determined by the arbitrator, which is to the off-chip data bus and to the feature data cache, respectively, with corresponding values ​​of 00 and 11, which are updated by the arbitrator module; [15:2] is a reserved bit. The existence of the arbitrator module and this proprietary instruction makes it possible to reuse the calculation results, further reducing the interaction of on-chip and off-chip data, reducing power consumption and improving operating efficiency.

[0095] (3) Instruction set scheduling of data flow.

[0096] In the data processing stage of the hardware accelerator's five-stage pipeline, the in-memory computing module, addition tree module, data activation module, data pooling module and on-chip cache module in the hardware accelerator are connected in sequence. The order conforms to the logic of operator arrangement in most algorithms. In a single-round configuration, it is set whether each module is enabled and whether the data flow between modules is bypassed. If not enabled, the data flow can bypass the current module and enter the next computing module.

[0097] (4) Design of instructions adapted to storage and computing architecture.

[0098] The advantage of the integrated storage and computing architecture is that it has both storage and computing functions. In the data processing stage of the five-level pipeline of the hardware accelerator, the number of computing units in the in-memory computing module is configured through the in-memory computing unit configuration instructions, so as to pre-store weight data to the greatest extent. Multiple sets of feature data matching the same set of weights enter the in-memory computing module and directly participate in the calculation, avoiding the disadvantage of the traditional architecture that weight data and feature data need to be continuously moved at the same time during calculation, breaking the shackles of the memory wall in the traditional hardware accelerator, and improving computing performance and speed.

[0099] (5) Optimization of computing process by instruction set.

[0100] In the data processing stage and data return stage of the hardware accelerator's five-stage pipeline, the arbitrator control instruction is used to configure the arbitrator to determine the output direction of the internal data of the on-chip cache module. If the internal data of the on-chip cache module can be used as input data for the next calculation, the data will be directly passed into the feature data cache during the next calculation; if the data is already the final result, the internal data of the on-chip cache will be output to the outside through the data bus.

Claims

1. A storage and computing integrated hardware accelerator, characterized in that: It includes a control unit, an operation execution unit and a cache unit; The control unit is used to receive instructions from the CPU, decode the instructions and send them to the operation execution unit and the cache unit; and is also used to notify the CPU by setting an interrupt after the task is completed; The operation execution unit includes a weight data cache module, a feature data cache module and a data processing module. The weight data cache module stores the weight data required for the operation, the feature data cache module stores the feature data required for the operation, and the data processing module performs operations on the weight data and the feature data, and the operation results are sent to the cache unit; the data processing module in the operation execution unit includes an in-memory calculation module, an addition tree module, a data activation module and a data pooling module, and the in-memory calculation module is used to perform convolution and / or full connection operations; the in-memory calculation module, the addition tree module, the data activation module, and the data pooling module are connected to the on-chip cache module in sequence, and whether the in-memory calculation module, the addition tree module, the data activation module, and the data pooling module are enabled or not is configured through instructions; The cache unit transmits the operation result back to the CPU, or temporarily stores the processing result in the on-chip cache module, and sends the operation result to the feature data cache module during the next operation.

2. The storage-computing integrated hardware accelerator according to claim 1, characterized in that: The cache unit uses an arbiter to determine whether the operation result is transmitted back to the CPU or temporarily stored in an on-chip cache module.

3. A data processing method of the hardware accelerator according to any one of claims 1 to 2, characterized in that: The steps include: The control unit receives instructions from the CPU, decodes the instructions and sends them to the operation execution unit and cache unit; The CPU sends the weight data to the weight data cache module of the operation execution unit, and sends the feature data to the feature data cache module. The data processing module performs operations on the weight data and the feature data, and sends the operation results to the cache unit; The cache unit transmits the calculation result back to the CPU or temporarily stores the processing result in the on-chip cache module, and sends the calculation result to the feature data cache module during the next calculation; Notify the CPU by setting an interrupt.

4. An instruction set architecture of the hardware accelerator according to any one of claims 1 to 2, characterized in that: It includes instructions for each module in the operation execution unit and the cache unit, and each instruction includes the activation, scale, function and working status of the corresponding module.

5. The instruction set architecture according to claim 4, characterized in that: It also includes an arbiter control instruction for determining the output direction of the internal data of the on-chip cache module. If the data is the final result, it is output to the CPU. If the data is used for the next calculation, it is stored in the on-chip cache module and sent to the feature data cache module during the next calculation.

6. The instruction set architecture according to claim 4, characterized in that: The instructions of the in-memory computing module in the operation execution unit include the working state of the in-memory computing module, the number of enabled operation units, functions and reserved bits; The working states of the in-memory computing module include the configuration completion standby state, the working state and the operation completion state; the functions of the in-memory computing module include convolution and full connection.

7. The instruction set architecture according to claim 4, characterized in that: The instructions of the addition tree module, the data activation module, and the data pooling module in the operation execution unit include their working status, the number of enabled operation units, and the reserved bits; The working states of the addition tree module, the data activation module and the data pooling module include the completed configuration standby state, the working state and the completed operation state.

8. The instruction set architecture according to claim 5, characterized in that: Arbiter control instructions include data transmission direction and reserved bits; The data transmission direction includes transmission to an off-chip data bus and transmission to a feature data cache module.

Images