Data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions

Abstract

This invention discloses a data quantization method for a convolutional neural network coprocessor based on custom RISC-V instructions, comprising the following steps: Step 1, fetching instructions; Step 2, parsing instructions; Step 3, parsing and executing the LRELU instruction; Step 4, parsing and executing the LBIAS instruction; Step 5, parsing and executing the quantization instruction REPO; Step 6, parsing and executing the memory write instruction SP. In Step 1, if it is necessary to read RS1, RS2, or RD registers, the read values ​​are sent to the nice core. The nice core is the coprocessor, which establishes a data connection with the E203 core through the nice interface. The nice interface includes a nice controller and a data extractor. This invention is developed based on the RISC-V open-source soft core, and designs the quantization unit in the neural network coprocessor through custom instructions, saving a large amount of storage space. It requires fewer instructions to perform the same quantization operation, and has the advantages of low consumption of multiply-adder resources and high flexibility.

Description

Technical Field

[0001] This invention relates to the field of computer technology, specifically to a data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions. Background Technology

[0002] With the development of technology, artificial intelligence is gradually emerging in production and daily life. The research and application of neural networks have always been a core aspect of AI, providing high-precision inference for a wide range of applications. As neural networks become more complex and sophisticated, the resources and memory they require are increasing daily. To enable more efficient networks to be deployed in low-power SoCs, neural network compression algorithms are essential. Quantization is one of the main methods for neural network compression, which involves storing 32-bit or 64-bit floating-point numbers in a form that uses less memory, such as 8 bits, to achieve neural network compression. In current technologies, neural network compression methods are often deployed on FPGAs or ASICs, which suffer from insufficient flexibility or excessively high development costs, often requiring reconfiguration for different neural networks. Summary of the Invention

[0003] The purpose of this invention is to provide a data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions, so as to solve the problems mentioned in the background art.

[0004] To achieve the above objectives, the present invention provides the following technical solution: a data quantization method for a convolutional neural network coprocessor based on custom RISC-V instructions, comprising the following steps: Step 1, fetching instructions; Step 2, parsing instructions; Step 3, parsing and executing the LRELU instruction; Step 4, parsing and executing the LBIAS instruction; Step 5, parsing and executing the quantization instruction REPO; Step 6, parsing and executing the memory write instruction SP.

[0005] In step one above, the instruction stored in the E203 core ITCM is retrieved. After decoding, it is found to be a custom instruction, and the instruction is sent to the nice core through the nice interface.

[0006] In step two above, the nice core decoding unit decodes the instructions and further analyzes the function of the instructions. If the analysis reveals that the instructions are reading feature maps, BIAS, convolution kernels, or other instructions, then the unit requests to read the data in the DTCM through the ICB handshake protocol and writes it into the nice core's cache.

[0007] In step three above, assuming that the quantization data has been read in step two, if the instruction LRELU is parsed next, the zero_point, scale, shift, maxdata, and mindata data in DTCM are requested to be read through the ICB handshake protocol and written into the quantization activation unit of the nice core.

[0008] In step four above, assuming that the quantization data has been read in step two, if the instruction LBIAS is parsed next, then the bias data in DTCM is requested to be read through the ICB handshake protocol and written into the bias accumulation unit of the nice core.

[0009] In step five above, assuming that steps two, three and four are completed, if the quantization instruction REPO is parsed next, then the bias accumulation unit, quantization activation unit and pooling unit are enabled, and the processed data is written into the output buffer.

[0010] In step six above, assuming the first five steps are completed, if the write memory instruction SP is parsed next, the nice core cache is written into the E203 core peripheral SRAM through the ICB handshake protocol.

[0011] Preferably, in step one, the E203 core is a RISC-V soft-core Hummingbird E203 core.

[0012] Preferably, in step one, if it is necessary to read RS1, RS2 or RD registers, the read values ​​are sent to the nice core together.

[0013] Preferably, the nice core is a coprocessor that establishes a data connection with the E203 core through the nice interface.

[0014] Preferably, the nice interface includes a nice controller and a data extractor.

[0015] Preferably, in step five, executing the quantization instruction REPO specifically involves: the control unit activating the quantization unit, inputting feature map data, serially outputting it from the buffer, and after biasing, quantizing, activating, and pooling, finally outputting the result to the output buffer until the SP instruction arrives, at which point the result is written into the E203 core peripheral SRAM.

[0016] Preferably, in step five, the quantization activation unit adopts asymmetric quantization, defining q as the quantized number, qx as the quantized scale value, and minxf as the smallest number before quantization. Then q = round(qx*xf-minxf*qx), and then shifting q to the left by 128, we get the final value of asymmetric quantization; zeropoint = round(-minxf*qx), and subtracting zeropoint from 128, we get the final quantization zero point; qx = 2*8-1 / Xmax-Xmin.

[0017] Preferably, the quantization activation unit uses the ReLU activation function, that is, if the input is greater than 0, it directly returns the value provided as the input; if the input is 0 or less, it returns the value 0. Because the quantization activation unit uses asymmetric quantization, the final output quantization zero point should be zeropoint = round(-minxf*qx). Subtracting 128 from zeropoint, we get the quantized zero point zero_line. If the input is greater than zero_line, it directly returns the value provided as the input; if the input is zero_line or less, it returns the value zero_line.

[0018] Compared with the prior art, the beneficial effects of the present invention are: the present invention is developed based on the RISC-V open source soft core, and the quantization unit in the neural network coprocessor is designed by custom instructions, which saves a lot of storage space, requires fewer instructions to perform the same quantization operation, and has the advantages of low consumption of multiply-adder resources and high flexibility. Attached Figure Description

[0019] Figure 1 This is a flowchart of the method of the present invention;

[0020] Figure 2 This is a block diagram of the coprocessor system of the present invention;

[0021] Figure 3 The figure shows the simulation results in an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Please see Figure 1-3The present invention provides an embodiment of a data quantization method for a convolutional neural network coprocessor based on custom RISC-V instructions, comprising the following steps: Step 1, fetching instructions; Step 2, parsing instructions; Step 3, parsing and executing the LRELU instruction; Step 4, parsing and executing the LBIAS instruction; Step 5, parsing and executing the quantization instruction REPO; Step 6, parsing and executing the memory write instruction SP.

[0024] In step one above, the instruction stored in the ITCM of the RISC-V soft core Hummingbird E203 is retrieved. After decoding and finding that it is a user-defined instruction, the instruction is sent to the nice core through the nice interface. If it is necessary to read RS1, RS2, or RD registers, the read values ​​are also sent to the nice core. The nice core is a coprocessor that establishes a data connection with the E203 core through the nice interface, which includes a nice controller and a data extractor.

[0025] In step two above, the nice core decoding unit decodes the instructions and further analyzes the function of the instructions. If the analysis reveals that the instructions are reading feature maps, BIAS, convolution kernels, or other instructions, then the unit requests to read the data in the DTCM through the ICB handshake protocol and writes it into the nice core's cache.

[0026] In step three above, assuming that the quantization data has been read in step two, if the instruction LRELU is parsed next, the zero_point, scale, shift, maxdata, and mindata data in DTCM are requested to be read through the ICB handshake protocol and written into the quantization activation unit of the nice core.

[0027] In step four above, assuming that the quantization data has been read in step two, if the instruction LBIAS is parsed next, then the bias data in DTCM is requested to be read through the ICB handshake protocol and written into the bias accumulation unit of the nice core.

[0028] In step five above, assuming steps two, three, and four are completed, if the quantization instruction REPO is subsequently parsed, the bias accumulation unit, quantization activation unit, and pooling unit are activated, and the processed data is written to the output buffer. Specifically, the control unit activates the quantization unit, inputs feature map data, and outputs it serially from the buffer. After biasing, quantizing, activation, and pooling, the final output result is written to the output buffer until the SP instruction arrives, at which point the result is written to the E203 core peripheral SRAM. The quantization activation unit uses asymmetric quantization, defining q as the quantized number, qx as the quantized scale value, and minxf as the smallest number before quantization. Then, q = round(qx*xf - minxf*qx), and shifting q left by 128 yields the final asymmetric quantized value. `point = round(-minxf * qx)`, subtracting 128 from `zeropoint`, yields the final quantization zero point; `qx = 2 * 8 - 1 / Xmax - Xmin`; the quantization activation unit uses the ReLU activation function, meaning if the input is greater than 0, it directly returns the value provided as input; if the input is 0 or less, it returns 0; because the quantization activation unit uses asymmetric quantization, the final output quantization zero point should be `zeropoint = round(-minxf * qx)`, subtracting 128 from `zeropoint`, yields the quantized zero point `zero_line`. If the input is greater than `zero_line`, it directly returns the value provided as input; if the input is `zero_line` or less, it returns `zero_line`; for example... Figure 3 The input feature map is int32, the output result is int8, the input feature map maxdata is 400, and mindata is -110. The simulation result calculated in the quantization calculation unit according to the above formula is the same as the correct result.

[0029] In step six above, assuming the first five steps are completed, if the write memory instruction SP is parsed next, the nice core cache is written into the E203 core peripheral SRAM through the ICB handshake protocol.

[0030] In the above embodiments, the custom quantization instructions are shown in the table below:

[0031]

[0032]

[0033] Based on the above, the advantages of the present invention are that the neural network coprocessor designed based on the RISC-V open-source soft core can customize RISC-V instructions, and has the advantages of flexible operation, low development difficulty and strong configurability, which solves the problems of insufficient flexibility and high development cost in existing neural network compression technologies.

[0034] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions, comprising the following steps: Step 1: Fetch instruction; Step 2: Parse instruction; Step 3: Parse and execute instruction LRELU; Step 4: Parse and execute instruction LBIAS; Step 5: Parse and execute quantization instruction REPO; Step 6: Parse and execute memory write instruction SP; The characteristic is that: In step one above, the instruction stored in the RISC-V soft core Hummingbird E203 core ITCM is retrieved. After decoding, it is found to be a custom instruction, and the instruction is sent to the nice core through the nice interface. In step two above, the nice core decoding unit decodes the instructions and further analyzes the function of the instructions. If the analysis reveals that the instructions are reading feature maps, BIAS, convolution kernels, or other instructions, then the unit requests to read the data in the DTCM through the ICB handshake protocol and writes it into the nice core's cache. In step three above, after the quantization data is read in step two, if the instruction LRELU is parsed next, the zero_point, scale, shift, maxdata, and mindata data related to quantization in DTCM are requested to be read through the ICB handshake protocol and written into the quantization activation unit of the nice core. In step four above, after the quantization data is read in step two, if the instruction LBIAS is parsed next, the bias data in DTCM is requested to be read through the ICB handshake protocol and written into the bias accumulation unit of the nice core. In step five above, assuming that steps two, three and four are completed, if the quantization instruction REPO is parsed next, then the bias accumulation unit, quantization activation unit and pooling unit are enabled, and the processed data is written into the output buffer. In step six above, assuming the first five steps are completed, if the write memory instruction SP is parsed next, the nice core cache is written into the E203 core peripheral SRAM through the ICB handshake protocol.

2. The data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions according to claim 1, characterized in that: In step one, if it is necessary to read RS1, RS2 or RD registers, the read values ​​are sent to the nice core together.

3. The data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions according to claim 2, characterized in that: The nice core, or coprocessor, establishes a data connection with the E203 core through the nice interface.

4. The data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions according to claim 3, characterized in that: The nice interface includes a nice controller and a data extractor.

5. The data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions according to claim 1, characterized in that: In step five, executing the quantization instruction REPO specifically involves: the control unit activating the quantization unit, inputting feature map data, serially outputting it from the buffer, and after biasing, quantizing, activating, and pooling, finally outputting the result to the output buffer until the SP instruction arrives, at which point the result is written into the E203 core peripheral SRAM.

6. The data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions according to claim 1, characterized in that: In step five, the quantization activation unit uses asymmetric quantization, where q is the quantized number, qx is the quantized scale value, and minxf is the smallest number before quantization. Therefore, q = round(qx × xf - minxf × qx), and then q is left-shifted by 128 to obtain the final asymmetric quantization value; zeropoint = round(-minxf × qx), and zeropoint - 128 is used to obtain the final quantization zero point; qx = (2 8 −1) / (Xmax−Xmin).

7. The data quantization method for convolutional neural network coprocessors based on custom RISC-V instructions according to claim 6, characterized in that: The quantization activation unit uses the ReLU activation function, which means that if the input is greater than 0, it directly returns the value provided as the input. If the input is 0 or less, the return value is 0; because the quantization activation unit uses asymmetric quantization, the final output quantization zero point should be zeropoint=round(-minxf*qx). Subtracting 128 from zeropoint, we get the quantized zero point zero_line. If the input is greater than zero_line, the value provided as the input is returned directly; if the input is zero_line or less, the return value is zero_line.

Images