An acceleration computing device and method based on RISC-V instruction set extension

Abstract

The application provides an acceleration computing device and method based on RISC-V instruction set expansion, which comprises a RISC-V microprocessor core module configured to control reading and writing of a quantized neural network, store instructions and small-scale data, perform instruction fetching operation, and generate a customized instruction; a coprocessor core module configured to execute the customized instruction sent from the RISC-V microprocessor core module; and a storage module connected with an AXI interconnection device through an AXI bus, wherein the storage module comprises a DDR memory configured to store a large batch of quantized neural network parameters used for generating the customized instruction, and a DDR controller configured to control reading and writing of the DDR memory, so as to solve the problems that the calculation amount and parameter amount of the current CNN model are very large, and the model needs to occupy a large amount of memory and consume super-high computing power during running.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to an accelerated computing device and method based on RISC-V instruction set extensions. Background Technology

[0002] Deep learning algorithms, represented by CNNs, are one of the emerging research directions in the field of artificial intelligence. In recent years, they have made continuous breakthroughs and have been widely used in fields such as image recognition and speech recognition.

[0003] Currently, many terminal devices have a demand for intelligence, which has led to the widespread deployment of CNNs in embedded and IoT devices. However, the mainstream CNN network models are currently large and computationally intensive, making them difficult to deploy directly on embedded and IoT devices with extremely limited resources.

[0004] Currently, mainstream CNN models have a very large amount of computation and parameters, which means that the models need to consume a lot of memory and consume extremely high computing power during operation. Embedded devices have extremely limited resources and cannot meet these two requirements, thus limiting the deployment of mainstream CNN models on embedded terminal devices. Summary of the Invention

[0005] This application provides an accelerated computing device and method based on RISC-V instruction set extensions to solve the problem that existing CNN models have a very large amount of computation and parameters, which leads to the model consuming a lot of memory and high computing power during operation.

[0006] The first aspect of this application provides an accelerated computing device based on the RISC-V instruction set extension, comprising:

[0007] The RISC-V microprocessor core module is connected to the AXI interconnect device via the AXI bus. It is configured to control the reading and writing of the quantized neural network, store instructions and small-scale data, perform instruction fetching operations, and generate custom instructions.

[0008] The coprocessor core module is communicatively connected to the RISC-V microprocessor core module and connected to the AXI interconnect device via the AXI bus. It is configured to execute custom instructions sent from the RISC-V microprocessor core module.

[0009] A storage module, connected to the AXI interconnect device via the AXI bus, wherein the storage module includes:

[0010] DDR memory is configured to store a large number of quantized neural network parameters used to generate custom instructions;

[0011] The DDR controller is configured to control the reading and writing of DDR memory.

[0012] In some embodiments, the coprocessor core module includes:

[0013] The decoding unit is configured to decode custom instructions sent by the RISC-V microprocessor core module. If the input custom instruction is supported by the coprocessor core module, the custom instruction is sent to the status unit.

[0014] The status unit, which is communicatively connected to the decoding unit, is configured to enter an instruction status corresponding to the decoded instruction ID.

[0015] The execution unit, which is communicatively connected to the state unit, is configured to generate and output the calculation results corresponding to the instruction state.

[0016] In some embodiments, the state unit includes:

[0017] The first state unit is configured to, when the custom instruction is the labuf instruction, read data from the memory module through the AXI bus and write the continuous data corresponding to the starting address indicated in rs1 into the on-chip allocated space buffer_a; the lower 3 bits stored in buffer_a are defined as the quantization bit width selection signal, and the labuf instruction reads the quantization bit width selection signal through the starting address indicated in rs2.

[0018] The second state unit, which is communicatively connected to the first state unit, is configured to read data from the storage module through the AXI bus when the custom instruction is the lbbuf instruction, and write the continuous data corresponding to the first address indicated in rs1 into the buffer_b of the on-chip allocated space.

[0019] The third state unit, which is communicatively connected to the second state unit, is configured to, when the custom instruction is a matrix instruction, read the quantization bit width of the matrix instruction from buffer_a, fill the data of buffer_a and buffer_b with zeros through MUX, input the data into the systolic array to calculate matrix multiplication, and store the accumulated data into the output buffer.

[0020] The fourth state unit is configured to, when the custom instruction is the sbuf instruction, write the data in the output buffer into the contiguous DDR memory corresponding to the starting address indicated in rs1 via the AXI bus.

[0021] In some embodiments, the execution unit includes:

[0022] The accelerator is configured to perform multi-precision quantized neural network computations;

[0023] The accelerator includes: a parameterized configurable systolic array, two on-chip storage spaces, buffer_a and buffer_b, for temporarily storing input data, one on-chip storage space, outputbuffer, for temporarily storing output data, one accumulator, one counter for controlling data input, and two MUXs for data arrangement and zero padding before data enters the systolic array.

[0024] In some embodiments, the pulsating array consists of a plurality of isomorphic PE units.

[0025] In some embodiments, the PE unit consists of a multiply-accumulate unit and several gating and timing logics, wherein the multiply-accumulate unit consists of a multi-precision multiplication tree and an accumulator, and is configured to compute multi-precision multiplication and addition.

[0026] A second aspect of this application provides an accelerated computing method based on RISC-V instruction set extensions, applied to an accelerated computing device based on RISC-V instruction set extensions as described in any of the preceding claims, comprising:

[0027] Based on the RISC-V microprocessor core module, it controls the reading and writing of quantized neural networks, stores instructions and small-scale data, performs instruction fetching operations, and generates custom instructions;

[0028] The coprocessor core module executes custom instructions sent by the RISC-V microprocessor core module.

[0029] The DDR memory based on the storage module stores a large number of quantized neural network parameters used to generate custom instructions, and controls the reading and writing of the DDR memory through the DDR controller.

[0030] In some embodiments, the accelerated computing method based on RISC-V instruction set extensions further includes:

[0031] After being recognized as a custom instruction in the RISC-V microprocessor core module, the extended instruction enters the decoding unit of the coprocessor core module for decoding. If the input instruction is supported by the coprocessor core module, processing of the instruction begins. The status unit enters the instruction state corresponding to the decoded instruction ID. When entering each instruction state, the status unit sends an instruction start signal to the execution unit. After each instruction ends, the execution unit sends an instruction end signal to the status unit. Upon receiving the end signal, the status unit transitions to the idle state (IDLE) and waits for the next instruction input.

[0032] In some embodiments, the accelerated computing method based on RISC-V instruction set extensions further includes:

[0033] Based on the execution unit accelerator, the system is a parameterized configurable systolic array composed of multiple PE units, two on-chip storage spaces (buffer_a and buffer_b) for temporarily storing input data, one on-chip storage space (output buffer) for temporarily storing output data, one accumulator, one counter for controlling data input, and two MUX generators for arranging and padding data before it enters the systolic array and outputting the calculation results corresponding to the instruction state.

[0034] In some embodiments, the core of the PE unit is a multi-precision multiplication tree.

[0035] This application provides an accelerated computing device and method based on RISC-V instruction set extensions, comprising: a RISC-V microprocessor core module connected to an AXI interconnect device via an AXI bus, configured to control the reading and writing of a quantized neural network, store instructions and small-scale data, perform instruction fetching operations, and generate custom instructions; a coprocessor core module communicatively connected to the RISC-V microprocessor core module and connected to the AXI interconnect device via the AXI bus, configured to execute custom instructions sent from the RISC-V microprocessor core module; and a storage module connected to the AXI interconnect device via the AXI bus, wherein the storage module includes: a DDR memory configured to store a large number of quantized neural network parameters used to generate custom instructions; and a DDR controller configured to control the reading and writing of the DDR memory to fully adapt to multi-precision quantization calculations, improve the parallelism of the calculation, and thus improve computing power and energy efficiency; maximize the reuse of on-chip resources, resulting in a significant improvement in resource utilization; and employ a tightly coupled architecture to construct the coprocessor, thereby greatly increasing the flexibility of the algorithm. Attached Figure Description

[0036] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0037] Figure 1 This is a schematic diagram of the structure of the accelerated computing device based on the RISC-V instruction set extension in this application;

[0038] Figure 2 This is a schematic diagram of the structure of the state unit in this application;

[0039] Figure 3This is a schematic diagram of the structure of the execution unit in this application;

[0040] Figure 4 This is a schematic diagram of the operating mode of the accelerated computing device based on the RISC-V instruction set extension in this application;

[0041] Figure 5 This is a schematic diagram of the working mode of the coprocessor core module in this application;

[0042] Figure 6 This is a schematic diagram of the operating mode of the accelerator in this application;

[0043] Figure 7 This is a schematic diagram of the working mode of the PE unit in this application;

[0044] Figure 8 A flowchart illustrating the process of preparing the zero-padding data for the 0th cycle of this application for input into the pulsating array;

[0045] Figure 9 This is a schematic diagram of the first column of matrix A and the first row of matrix B after the rearrangement of the first cycle in this application, which is then used as the input pulsation array.

[0046] Figure 10 This is a schematic diagram of the input pulsation array in the second column of matrix A and the second row of matrix B after the rearrangement of the second cycle in this application;

[0047] Figure 11 This is a schematic diagram of the output mode of the pulsating array in this application;

[0048] Figure 12 This is a schematic diagram of the 16-bit multiplication number in this application.

[0049] Explanation of reference numerals in the attached figures:

[0050] 1-RISC-V microprocessor core module; 2-coprocessor core module; 21-decoding unit; 22-state unit; 221-first state unit; 222-second state unit; 223-third state unit; 224-fourth state unit; 23-execution unit; 231-accelerator; 3-storage module; 31-DDR memory; 32-DDR controller. Detailed Implementation

[0051] To enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of this application.

[0052] Because CNN models in some technologies involve a huge amount of computation and parameters, they require a large amount of memory and consume extremely high computing power during operation. To solve this problem, this application provides an accelerated computing device and method based on RISC-V instruction set extensions. The accelerated computing device and method based on RISC-V instruction set extensions are described below:

[0053] Depend on Figure 1 and Figure 4 As can be seen, the first aspect of this application provides an accelerated computing device based on the RISC-V instruction set extension, comprising: a RISC-V microprocessor core module 1, connected to an AXI interconnect device via an AXI bus, configured to control the reading and writing of a quantized neural network, store instructions and small-scale data, perform instruction fetching operations, and generate custom instructions; a coprocessor core module 2, communicatively connected to the RISC-V microprocessor core module 1, connected to the AXI interconnect device via the AXI bus, configured to execute custom instructions sent from the RISC-V microprocessor core module 1; and a storage module 3, connected to the AXI interconnect device via the AXI bus, wherein the storage module 3 includes: a DDR memory 31, configured to store a large number of quantized neural network parameters used to generate custom instructions; and a DDR controller 32, configured to control the reading and writing of the DDR memory 31.

[0054] For example, CNN models are currently widely used in various tasks, such as natural language processing and image processing. However, the computational and parameter counts of mainstream CNN models are extremely large, resulting in the models requiring a large amount of memory and consuming extremely high computing power during operation. Embedded devices have extremely limited resources, making it difficult to meet these two requirements, thus limiting the deployment of mainstream CNN models on embedded terminal devices.

[0055] Model training typically occurs on the server side, while embedded terminals only need to deploy the model inference portion. Compared to training, CNN inference significantly reduces computational load; however, deploying larger network models on embedded terminals is still limited by factors such as storage space and power consumption. To address the challenges of deploying neural networks on embedded terminals, optimizations can be made at both the algorithm and hardware levels.

[0056] Algorithm-level optimization typically employs model compression, which reduces computational load, storage space requirements, and power consumption, making neural networks easier to deploy on embedded devices. Among various model compression schemes, the mature quantization scheme has garnered significant attention due to its convenient and efficient compression capabilities. In quantization schemes, single-precision floating-point data in CNNs can be represented using lower-bit-width floating-point numbers or fixed-point numbers. However, a single quantization precision can lead to insufficient flexibility and adaptability. To better tailor quantization to different neural network models and specific needs, multi-precision quantization schemes are a suitable choice, maintaining the advantages of quantization while exhibiting broader adaptability. When the hardware supports quantization bit-width data computation, quantization demonstrates higher computational efficiency.

[0057] Hardware-level optimizations are generally achieved through GPUs or by building hardware accelerators. GPUs are the fastest at calculating neural network parameters, but their power consumption and cost are too high, making them difficult to deploy in embedded systems; they are often used for training server-side models. Hardware accelerators mainly have two implementation approaches: ASICs and FPGAs. ASIC-based accelerators offer high performance and energy efficiency, but lack flexibility and require tape-out, resulting in long development cycles and high costs. FPGAs, on the other hand, have a large number of configurable logic resources, making FPGA-based accelerators highly flexible, low-cost, and with short development cycles. They are suitable for configuring different CNN hardware acceleration units and are also suitable as a hardware-level optimization solution.

[0058] Embedded and IoT devices all require microcontroller units, which can also control the deployment and operation of neural networks. Embedded CPUs have relatively limited computing performance; directly running CNN inference processes requiring extensive parallel computation would result in slow speeds and low energy efficiency. However, combining a CPU with custom acceleration cores to build a domain-specific architecture can significantly improve computing power and energy efficiency. Currently, mainstream CPU instruction set architectures such as x86 and ARM are difficult to develop custom instruction set extensions due to commercial limitations. RISC-V, on the other hand, is an open-source instruction set architecture based on the principles of reduced instruction set computing. Compared to x86 and ARM architectures, RISC-V allows for easier instruction set extensions and modifications, enabling customization and optimization for different application domains. Furthermore, the feasibility of customizing RISC-V-based DSA extensions is even higher. Therefore, RISC-V is suitable as the microprocessor component of embedded acceleration systems. Using a RISC-V open-source instruction set architecture combined with DSA extensions to build an inference computing device, and then constructing a hardware system on an FPGA capable of setting multiple precision selection modes to run quantized neural networks, can effectively solve existing problems and challenges.

[0059] In this embodiment, the RISC-V microprocessor core module 1 is mainly used to control the reading and writing of the quantized neural network, store instructions and small-scale data, perform instruction fetching and other operations, and mount a coprocessor core in the pipeline stage; the coprocessor core module 2 is used to compute the quantized neural network, and the custom instructions for computing the neural network are executed in the coprocessor after being fetched and decoded by the main processor; the storage module 3 mainly includes the DDR memory 31 and the DDR controller 32, wherein the DDR memory 31 is used to store a large number of quantized neural network parameters, and the DDR controller 32 is used to control the reading and writing of the DDR; the instruction and data transmission interface between the main processor and the coprocessor supports multiple protocols, such as AXI, ICB, ROCC, etc.

[0060] Depend on Figure 1 It is known that the coprocessor core module 2 includes: a decoding unit 21, configured to decode custom instructions sent by the RISC-V microprocessor core module 1, and if the input custom instruction is supported by the coprocessor core module 2, then send the custom instruction to the status unit 22; the status unit 22, communicatively connected to the decoding unit 21, configured to enter an instruction state corresponding to the decoded instruction ID; and an execution unit 23, communicatively connected to the status unit 22, configured to generate and output the calculation result corresponding to the instruction state.

[0061] The decoding unit 21 is a program element on the coprocessor core module 2 used to decode custom instructions sent by the RISC-V microprocessor core module 1; the state unit 22 is a program element on the coprocessor core module 2 used to enter an instruction state corresponding to the decoded instruction ID; and the execution unit 23 is a program element on the coprocessor core module 2 used to generate and output the calculation results corresponding to the instruction state.

[0062] Depend on Figure 2 and Figure 5It can be seen that the state unit 22 includes: a first state unit 221, configured to, when the custom instruction is a labuf instruction, read data from the storage module 3 through the AXI bus and write the continuous data corresponding to the starting address indicated in rs1 into the on-chip allocated space buffer_a; the lower 3 bits stored in buffer_a are defined as quantization bit width selection signals, and the labuf instruction reads the quantization bit width selection signals through the starting address indicated in rs2; a second state unit 222, communicatively connected to the first state unit 221, configured to, when the custom instruction is an lbbuf instruction, read data from the storage module 3 through the AXI bus and write the continuous data corresponding to the starting address indicated in rs1 into the on-chip allocated space buffer_a; the lower 3 bits stored in buffer_a are defined as quantization bit width selection signals, and the labuf instruction reads the quantization bit width selection signals through the starting address indicated in rs2; and ... configured to, when the custom instruction is a lbbuf instruction, read data from the storage module 3 through the AXI bus and write the continuous data corresponding to the starting address indicated in rs1 into the on-chip allocated space buffer_a. The first address indicated is used to write the continuous data corresponding to the first address to the on-chip allocated space buffer_b; the third state unit 223, which is communicatively connected to the second state unit 222, is configured to, when the custom instruction is a matrix instruction, read the quantization bit width from buffer_a, fill the data of buffer_a and buffer_b with zeros through MUX, input the data into the systolic array to calculate matrix multiplication, and store the accumulated data into the output buffer; the fourth state unit 224 is configured to, when the custom instruction is an sbuf instruction, write the data in the output buffer to the continuous DDR memory 31 corresponding to the first address indicated in rs1 through the AXI bus.

[0063] In this embodiment, this application defines four extended instructions for reading, writing, and controlling the computational core, respectively. All four custom instructions are standard R-type instructions used to operate on the target and source registers, and all use the same opcode encoding space: custom3 (opcode = 7'b1111011). The specific instructions are as follows.

[0064] 1. When the instruction is labufx0,rs1,rs2; the labuf instruction reads data from DDR through the AXI bus and writes the continuous data corresponding to the starting address indicated in rs1 into the on-chip allocated space buffer_a; the lower 3 bits stored in buffer_a are defined as the quantization width selection signal, and the labuf instruction reads the quantization width selection signal through the starting address indicated in rs2; funct3 = 011, funct7 = 0000001.

[0065] 2. When the instruction is lbbufx0,rs1,x0; the lbbuf instruction reads data from DDR through the AXI bus and writes the continuous data corresponding to the starting address indicated in rs1 into the buffer_b of the on-chip allocated space; funct3 = 010, funct7 = 0000010.

[0066] 3. When the instruction is matrixxx0,x0,x0; after the matrix instruction reads the quantization bit width from buffer_a, it inputs the data of buffer_a and buffer_b into the systolic array after zero padding through the MUX to calculate matrix multiplication and stores the accumulated data into the output buffer; funct3 = 000, funct7 = 0000110.

[0067] 4. When the instruction is sbufx0,rs1,x0; the sbuf instruction writes the data in the output buffer into the contiguous DDR memory space corresponding to the starting address indicated in rs1 through the AXI bus; funct3 = 010, funct7 = 0000111.

[0068] Depend on Figure 3 and Figure 6 It is known that the execution unit 23 includes: an accelerator 231, configured to perform multi-precision quantization neural network computation; wherein the accelerator 231 includes: a parameterizable configurable systolic array, two on-chip storage spaces buffer_a and buffer_b for temporarily storing input data, one on-chip storage space outputbuffer for temporarily storing output data, an accumulator, a counter for controlling data input, and two MUXs for data arrangement and zero padding before data enters the systolic array.

[0069] In this embodiment, after the extended instruction is recognized as a custom instruction by the RISC-V microprocessor core module 1, it enters the decoding unit 21 of the coprocessor core module 2 for decoding. If the input instruction is supported by the coprocessor core module 2, processing of the instruction begins. The state unit 22 enters the instruction state corresponding to the decoded instruction ID. To avoid potential conflicts, the extended instructions have a fixed priority, in the order of labuf > lbbuf > matrix > sbuf. Upon entering each instruction state, the state unit 22 sends an instruction start signal to the execution unit 23; after each instruction ends, the execution unit 23 sends an instruction end signal to the state unit 22. Upon receiving the end signal, the state unit 22 transitions to the idle state (IDLE) and waits for the next instruction input. The operating mode of the accelerator 231 is as follows: Figure 6 As shown.

[0070] For example, the sizes of the three on-chip storage spaces are parameterized and configurable. Each storage space has three configurable dimensions, and a specific dimension of the storage space matches the corresponding dimension of the systolic array. The size of buffer_b is , where L is the data length, N matches the systolic array length, and T matches the number of systolic array tiles. The bit width is , which is the maximum bit width supported for quantized data. The size of buffer_a is , where L is the data length (this dimension must be the same for both input buffers), M matches the systolic array width, and T matches the number of systolic array tiles. The extra storage space in buffer_a is set to store the quantization bit width selection signal. The size of the output buffer is , where M matches the systolic array width, N matches the systolic array length, and T matches the number of systolic array tiles, and it is used to store the data output from the systolic array. Based on the counter's count and the quantization bit width selection signal, the MUX selects the data to be sent to the systolic array for calculation from buffer_a and buffer_b and sends it to the systolic array for calculation. After the calculation is completed, the systolic array automatically starts to output data to the accumulator in a systolic manner. The accumulator adds the data input each time to the data at the corresponding position in the output buffer and stores it back into the output buffer.

[0071] The instruction execution flow is as follows:

[0072] In process 1, after the labuf and lbbuf instructions begin, data is read from DDR into two on-chip memory spaces, buffer_a and buffer_b, and the quantization bit width selection signal is stored in buffer_a; after the data of a specific size is read, the labuf and lbbuf instructions end respectively.

[0073] In process 2, after the matrix instruction starts, the MUX selects the data to be sent to the systolic array for calculation based on the counter count and the quantization bit width selection signal (data from the buffer or data padded with zeros).

[0074] In process 3, the pulsating array is activated by the matrix instruction start signal, receives the data selected by the MUX and performs calculations, and some results are temporarily stored in each PE of the pulsating array.

[0075] Step 4: After the calculation is completed, the systolic array automatically starts to output data to the accumulator in a systolic manner. The accumulator will add the data input each time to the data at the corresponding position in the output buffer and store it back into the output buffer. When the systolic array is empty, the matrix instruction ends.

[0076] In step 5, after the sbuf instruction starts, data is written from the output buffer to DDR. Once the data of a specific size has been written, the sbuf instruction ends.

[0077] Depend on Figure 6 It is understood that the pulsating array is composed of multiple isomorphic PE units, and the pulsating array is the core of the accelerator 231, which is parameterized and configurable in three dimensions. The pulsating array consists of multiple isomorphic PE units, and the entire array has PEs, where M is the array width, N is the array length, and T is the number of array tiles. Each tile is computed simultaneously in parallel, and the data does not interfere with each other. The PE units support... N positions in total max Fixed-point multiplication and accumulation operations with different precisions.

[0078] Depend on Figure 7 It can be seen that the PE unit consists of a multiply-accumulate unit and several gating and timing logics. The multiply-accumulate unit consists of a multi-precision multiplication tree and an accumulator, and is configured to calculate multi-precision multiplication and addition.

[0079] In this embodiment, the PE unit operates by inputting data from the W side of buffer_a and data from the N side of buffer_b.

[0080] The PE has two control signals, EnableMul and EnableShift, which control three states: Mul, Shift, and Freeze.

[0081] 1. When both control signals are low, it is in Freeze state. At this time, the data is frozen in the PE and not transmitted. MUX is strobed 0.

[0082] 2. When the EnableMul signal is high, it is in Mul calculation state. At this time, aOut broadcasts the data of aIn to E, bOut broadcasts the data of bIn to S, the multiply-accumulate unit performs multiply-accumulate calculation, and cOut outputs the accumulated data to cIn of the same PE.

[0083] 3. With the EnableMul signal low, if the EnableShift signal is high, the PE enters the Shift shift output state. At this time, the MUX of both aOut and bOut outputs is selected to 0, and the MUX of cOut selects the accumulator signal. Simultaneously, the output direction of cOut is cIn of the adjacent PE to the S-axis. This allows the calculation results temporarily stored in the PE to be output sequentially to the S-axis.

[0084] Among them, the multiplier of the multiply-accumulator is a multi-precision multiplication tree that supports bit-fixed point multiplication calculation.

[0085] A second aspect of this application provides an accelerated computing method based on RISC-V instruction set extensions, applied to an accelerated computing device based on RISC-V instruction set extensions as described in any of the preceding claims, comprising:

[0086] Based on the RISC-V microprocessor core module, it controls the reading and writing of quantized neural networks, stores instructions and small-scale data, performs instruction fetching operations, and generates custom instructions;

[0087] The coprocessor core module executes custom instructions sent by the RISC-V microprocessor core module.

[0088] The DDR memory based on the storage module stores a large number of quantized neural network parameters used to generate custom instructions, and controls the reading and writing of the DDR memory through the DDR controller.

[0089] In this embodiment, the accelerated computing method based on RISC-V instruction set extensions further includes:

[0090] After being recognized as a custom instruction in the RISC-V microprocessor core module, the extended instruction enters the decoding unit of the coprocessor core module for decoding. If the input instruction is supported by the coprocessor core module, processing of the instruction begins. The status unit enters the instruction state corresponding to the decoded instruction ID. When entering each instruction state, the status unit sends an instruction start signal to the execution unit. After each instruction ends, the execution unit sends an instruction end signal to the status unit. Upon receiving the end signal, the status unit transitions to the idle state (IDLE) and waits for the next instruction input.

[0091] In this embodiment, the accelerated computing method based on RISC-V instruction set extensions further includes:

[0092] Based on the execution unit accelerator, the system is a parameterized configurable systolic array composed of multiple PE units, two on-chip storage spaces (buffer_a and buffer_b) for temporarily storing input data, one on-chip storage space (output buffer) for temporarily storing output data, one accumulator, one counter for controlling data input, and two MUX generators for arranging and padding data before it enters the systolic array and outputting the calculation results corresponding to the instruction state.

[0093] The core of the execution unit (the part used for matrix multiplication calculation) is a systolic array, and the matrix multiplication calculation process is as follows.

[0094] Since systolic arrays are adapted for matrix multiplication, the data input to the systolic array via the buffer should first be rearranged. The MUX and counter perform the data rearrangement, while the matrix multiplication is performed by the systolic array.

[0095] Since the (1,1)th element of buffer_a and the (1,2)th element of buffer_b should arrive at the (1,2)th PE cell of the systolic array simultaneously for multiplication and accumulation within that cell, if the data is not rearranged, the (1,1)th element of buffer_a will arrive at the (1,2)th PE of the systolic array in the second clock cycle, while the (1,2)th element of buffer_b will arrive at the same PE in the first clock cycle. This timing mismatch prevents the calculation from proceeding correctly. Therefore, for the i-th row (column) of buffer_a (buffer_b), i-1 zeros need to be added before the first element of that row (column). This means that each row (column) is shifted one element to the right compared to the previous row (column), and the previous row (column) must correspondingly add a zero to the end of its row (column) to match the shift length. Therefore, the end of the first row of buffer_a should be padded with M-1 zeros, and the end of the first column of buffer_b should be padded with N-1 zeros.

[0096] Each element of buffer_a (or buffer_b) needs to be calculated once in each PE (Process Entity) of its row (or column), including the last input element (i.e., the element in the last row and last column, which is not padded with zeros). To ensure that this element is also calculated once in each PE of its row (or column), N-1 zeros need to be padded after the last element of each row of buffer_a, and M-1 zeros need to be padded after the last element of each column of buffer_b.

[0097] The process of inputting data into the pulsating array for processing after zero-padding and rearrangement is as follows.

[0098] In the 0th cycle, the process of rearranging and zero-padding data to prepare for input into the pulsating array is as follows: Figure 8 As shown;

[0099] In the first cycle, the first column of matrix A and the first row of matrix B are rearranged and input into the pulsating array; the result of the multiplication and accumulation is temporarily stored in PE, and the input data is prepared to be broadcast to the adjacent PE in the next cycle (the broadcast diagram of 0 is omitted), such as Figure 9 As shown;

[0100] In the second cycle, the second column of A and the second row of B are input to the pulsating array; the data broadcast in the previous cycle is multiplied and accumulated with the new input data in the adjacent PE and temporarily stored. The new input data is also prepared to be broadcast to the adjacent PE in the next cycle, such as... Figure 10 As shown;

[0101] The output method of the systolic array is that the data temporarily stored in PE is output one line at a time from N to S from the bottom row of the systolic array, such as... Figure 11 As shown;

[0102] In this embodiment, the core of the PE unit is a multi-precision multiplication tree. The multiplier of the multiply-accumulator is implemented using a 1-bit multiplication tree, supporting bit-wise fixed-point multiplication and achieving complete hardware reuse for different precisions. The multi-precision multiplication tree is written recursively. A 2-bit multiplier forms the basis of the multiplication tree; four 2-bit multipliers generate a 4-bit multiplication tree, four 4-bit multiplication trees generate an 8-bit multiplication tree, and so on.

[0103] Taking a 16-bit multiplication tree as an example, the specific implementation is as follows: Figure 12 As shown.

[0104] As can be seen, since every 16-bit number can be decomposed into high 8 bits and low 8 bits, a 16-bit multiplication can be decomposed into the sum of the results of four 8-bit multiplications. Multiplying low 8 bits by low 8 bits requires no shifting, adding the results of two low 8 bits multiplied by high 8 bits requires left shifting by 8 bits, and multiplying high 8 bits requires left shifting by 16 bits. Therefore, using four 8-bit multipliers and one adder, a 16-bit multiplication operation can be completed in one step. The same structure can obviously be used to calculate the sum of four 8-bit multiplications, simply by disabling the shifting function after multiplication. Therefore, this structure can calculate either a 16-bit multiplication or the sum of four 8-bit multiplications in one operation.

[0105] Similarly, an 8-bit multiplication tree can be implemented using four 4-bit multiplication trees, and a 4-bit multiplication tree can be implemented using four 2-bit multiplication trees. Therefore, a 16-bit multiplication tree can perform one 16-bit multiplication, four 8-bit multiplications, sixteen 4-bit multiplications, and sixty-four 2-bit multiplications at a time, supporting four quantization bit widths while achieving complete hardware reuse. For bit-level multiprecision multiplication trees, maximum hardware reuse can be achieved while supporting bit-level fixed-point multiplication.

[0106] Taking a 16-bit multiplication tree as an example, the throughput of a multi-precision multiplication tree varies when calculating data with different quantization bit widths. When calculating a 16-bit multiplication, two 16-bit data bits are input at a time. However, when calculating an 8-bit multiplication, since four multiplications can be performed at once, eight 8-bit data bits (two 32-bit data bits) are required, doubling the bandwidth. Therefore, by analogy, 64 2-bit multiplications require two 128-bit data bits. Because different quantization bit widths require different bandwidths, this work implements a simple mechanism to match the throughput of different precisions. This mechanism is achieved through a MUX used for data rearrangement and zero-padding. Two MUXs read signals from the storage space of buffer_a that stores the quantization bit width selection signal and select the amount of data to enter the systolic array based on these signals. The lower the quantization bit width, the higher the bandwidth controlled by the MUX. Taking a 16-bit multiplication tree as an example, transmitting 8-bit data at once is twice the amount of data transmitted with 16-bit data, equivalent to transmitting two 16-bit data blocks to the same PE. Similarly, transmitting 4-bit data is four times the amount of data transmitted with 16-bit data, and transmitting 2-bit data is eight times the amount of data transmitted with 16-bit data. If the number of 16-bit data blocks in the buffer cannot be divided into appropriate sizes (i.e., divisible by 2 / 4 / 8), the MUX will automatically pad the last transmitted data with zeros to meet the requirements; the same applies to multi-precision multiplication.

[0107] The above detailed embodiments further illustrate the purpose, technical solution, and beneficial effects of the embodiments of this application. It should be understood that the above are merely specific embodiments of the embodiments of this application and are not intended to limit the protection scope of the embodiments of this application. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solutions of the embodiments of this application should be included within the protection scope of the embodiments of this application.

Claims

1. An acceleration computing device based on RISC-V instruction set extension, characterized in that, The application relates to a RISC-V microprocessor core module (1) connected with an AXI interconnection device through an AXI bus and configured to control reading and writing of a quantized neural network, store instructions and small-scale data, perform a fetch operation and generate a custom instruction; a coprocessor core module (2) in communication connection with the RISC-V microprocessor core module (1) and connected with the AXI interconnection device through the AXI bus and configured to execute the custom instruction sent from the RISC-V microprocessor core module (1); and a storage module (3) connected with the AXI interconnection device through the AXI bus, wherein the storage module (3) comprises: a DDR memory (31) configured to store a large number of quantized neural network parameters for generating the custom instruction; and a DDR controller (32) configured to control reading and writing of the DDR memory (31); the coprocessor core module (2) comprises a decoding unit (21), a state unit (22) and an execution unit (23), the state unit (22) is in communication connection with the decoding unit (21), and the state unit (22) comprises: a first state unit (221) configured to read data of a labuf instruction from the storage module (3) through the AXI bus and write continuous data corresponding to a first address indicated in rs1 into an on-chip allocated space buffer_a when the custom instruction is the labuf instruction, and the low 3 bits stored in buffer_a are defined as quantization bit width selection signals, and the labuf instruction reads the quantization bit width selection signals through a first address indicated in rs2; a second state unit (222) in communication connection with the first state unit (221) and configured to read data of a lbbuf instruction from the storage module (3) through the AXI bus and write continuous data corresponding to a first address indicated in rs1 into an on-chip allocated space buffer_b when the custom instruction is the lbbuf instruction; a third state unit (223) in communication connection with the second state unit (222) and configured to read quantization bit width of a matrix instruction from buffer_a, input data of buffer_a and buffer_b into a systolic array after zero padding through a MUX, calculate matrix multiplication and store the data in an output buffer after accumulation when the custom instruction is the matrix instruction; and a fourth state unit (224) configured to write data in the output buffer into the DDR memory (31) corresponding to the first address indicated in rs1 through the AXI bus when the custom instruction is an sbuf instruction; and the execution unit (23) comprises: an accelerator (231) configured to perform multi-precision quantized neural network calculation. ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ The accelerator (231) comprises: a parameterized configurable systolic array, 2 on-chip storage spaces buffer_a and buffer_b for temporarily storing input data, 1 on-chip storage space output buffer for temporarily storing output data, 1 accumulator, 1 counter for controlling data input, and 2 MUXs for data arrangement and zero padding before data enters the systolic array.

2. The acceleration device based on the RISC-V instruction set extension according to claim 1, characterized in that, The decoding unit (21) is configured to decode the custom instruction sent by the RISC-V microprocessor core module (1), and send the custom instruction to the state unit (22) if the input custom instruction is supported by the coprocessor core module (2); The state unit (22) is configured to enter a corresponding instruction state according to the decoded instruction ID; The execution unit (23) is configured to generate and output the calculation result corresponding to the instruction state.

3. The accelerator computing device based on RISC-V instruction set extension according to claim 1, wherein, The systolic array is composed of a plurality of isomorphic PE units.

4. The accelerator computing device based on RISC-V instruction set extension according to claim 3, wherein, The PE unit is composed of a multiply-accumulator and a plurality of gating and timing logic, wherein the multiply-accumulator is composed of a multi-precision multiplication tree and an accumulator, and is configured to calculate multi-precision multiplication and addition.

5. An acceleration method based on RISC-V instruction set extension, applied to the acceleration device based on RISC-V instruction set extension in any one of claims 1 to 4, characterized in that, It includes: The RISC-V microprocessor core module controls the reading and writing of the quantized neural network, stores instructions and small-scale data, performs instruction fetching, and generates custom instructions; The coprocessor core module executes the custom instructions sent by the RISC-V microprocessor core module; The storage module is based on the DDR memory to store a large amount of quantized neural network parameters for generating custom instructions and control the reading and writing of the DDR memory through the DDR controller.

6. The method of claim 5, wherein the RISC-V instruction set extension is based on a vector extension (V) and a floating-point extension (F) of the RISC-V instruction set. It also includes: After the extension instruction is recognized as a custom instruction in the RISC-V microprocessor core module, it is decoded in the decoding unit of the coprocessor core module. If the input instruction is supported by the coprocessor core module, the instruction starts to be processed. The state unit enters a corresponding instruction state according to the decoded instruction ID. When entering each instruction state, the state unit sends an instruction start signal to the execution unit. After each instruction ends, the execution unit sends an instruction end signal to the state unit. After receiving the end signal, the state unit enters an idle state IDLE. The state unit waits for the next instruction input.

7. The method of claim 5, wherein the method is based on an extension of the RISC-V instruction set. It also includes: The parameterized configurable systolic array, 2 on-chip storage spaces buffer_a and buffer_b for temporarily storing input data, 1 on-chip storage space output buffer for temporarily storing output data, 1 accumulator, 1 counter for controlling data input, and 2 MUXs for data arrangement and zero padding before data enters the systolic array generate and output the calculation result corresponding to the instruction state; the systolic array is composed of a plurality of isomorphic PE units, and the systolic array is the core of the accelerator in the execution unit.

8. The method of claim 7, wherein the RISC-V instruction set extension is based on a vector extension (V) and a floating-point extension (F) of the RISC-V instruction set. The core of the PE unit is a multi-precision multiplication tree.

Images