Lightweight neural network coprocessor based on ROCC interface

Abstract

The invention discloses a lightweight neural network coprocessor based on an ROCC interface. The lightweight neural network coprocessor comprises a control sub-module, a memory access sub-module and an operation accelerator sub-module, and the operation accelerator sub-module comprises a linear operation accelerator sub-module and a nonlinear operation accelerator sub-module; the control and memory access sub-module provides instruction decoding control and memory access support for the operation accelerator sub-module; acceleration of a linear part in neural network operation is carried out through a linear operation accelerator sub-module, and linear operations such as addition and subtraction, multiplication, multiplication and addition mixing, accumulation and summation are carried out on a vector operation level; acceleration of a nonlinear part in neural network operation is carried out through a nonlinear operation accelerator sub-module, and 16 types of nonlinear function operation are executed in batches; compared with an existing single-function accelerator, the coprocessor has more functions; compared with the existing multifunctional accelerator, the coprocessor provided by the invention realizes acceleration of various linear operations and non-linear operations by lower integral area power consumption integration.

Description

Technical Field

[0001] The present invention belongs to the technical field of neural network coprocessors, and in particular relates to a lightweight neural network coprocessor based on an ROCC interface. Background Art

[0002] With the increasingly widespread application of neural networks, platforms with highly limited area and power, including embedded devices and wearable devices, etc., have gradually started to use increasingly complex neural network algorithms. However, the computing resources (area and power consumption) on these platforms are highly limited, and it takes a long time to compute the linear operations (mainly vector and matrix operations) or non-linear operations (mainly activation functions and signal processing) of neural networks.

[0003] In terms of linear operations, the out-of-order parallelism and SIMD instructions widely used in high-performance processors can effectively accelerate general linear vector and matrix operation operations, usually sufficient to reach the memory bandwidth bottleneck. However, the area and power consumption costs of these methods are huge and cannot be applied to application environments with limited computing resources, and targeted accelerators need to be designed. Y.-H. Chen (Y.-H. Chen, T. Krishna, J. S. Emer and V. Sze, "Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks," in IEEE Journal of Solid-State Circuits, vol. 52, no. 1, pp. 127-138, Jan. 2017, doi: 10.1109 / JSSC.2016.2616357.) et al. and S. Yin (S. Yin et al., "A High Energy Efficient Reconfigurable Hybrid Neural Network Processor for Deep Learning Applications," in IEEE Journal of Solid-State Circuits, vol. 53, no. 4, pp.

[0004] 968-982, April 2018, doi: 10.1109 / JSSC.2017.2778281.) et al. proposed accelerators to accelerate highly parallelized multiply-accumulate operations through a PE array. The hardware accelerator scheme based on the PE array has extremely high computing speed, but extremely large area and power consumption.

[0005] In terms of non - linear operations, in recent years, multiple accelerators designed for a single function have been proposed. For example, Y. Luo et al. (Y. Luo, Y. Wang, H. Sun, Y. Zha, Z. Wang, and H. Pan, “Cordic - based architecture for computing nth root and its implementation,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 65, no. 12, pp. 4183–4195, 2018) proposed a single - function accelerator for x 1 / N computation; S. Mopuri et al. (S. Mopuri and A. Acharyya, “Low complexity generic vlsi architecture design methodology for nth root and nth power computations,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 66, no. 12, pp. 4673–4686, 2019) proposed a single - function accelerator for x 1 / N or x N computation. These single - function accelerators can only accelerate specific functions and cannot independently solve multiple functions. However, neural networks and signal processing algorithms usually require the use of multiple non - linear functions. This means that a complete project needs to use multiple different accelerators, resulting in a significant increase in the total area and power consumption. To solve this problem, H. Chen (H. Chen, Z. Yu, J. Xu, L. Jiang, Z. Lu, Y. Fu, and L. Li, “Huicore: A generalized hardware accelerator for complicated functions,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 69, no. 6, pp.

[0006] (2022, 2463–2476) et al. proposed a multi-functional accelerator that contains eight CORDIC (Coordinate Rotation Digital Computer) sub-modules with different modes and can solve 25 non-linear functions, including common trigonometric functions, hyperbolic functions, exponential functions, and logarithmic functions. However, there are many highly similar repetitive modules in this accelerator, and effective computing resource reuse is not carried out. Its area and power reach 132000 μm 2 and 61.3 mW, which is approximately the sum of three single-functional accelerators, and it is difficult to be applied to application environments with highly limited resources. Summary of the Invention

[0007] In order to overcome the above-mentioned disadvantages in the prior art, the purpose of the present invention is to provide a lightweight neural network coprocessor based on the ROCC interface, which can integrate and implement the acceleration of various linear operations and non-linear operations with lower overall area power consumption. By flexibly selecting the operands of the multiply-accumulate operation, multiple vector multiply-accumulate operations are integrated and implemented in a single linear operation accelerator sub-module. By flexibly configuring the CORDIC calculation mode, 16 commonly used non-linear functions are integrated and implemented in a single non-linear operation accelerator sub-module. The coprocessor of the present invention can accelerate the linear operation by 2.24 to 20.65 times compared with directly using a RISC-V processor for calculation, accelerate the non-linear operation by about 14 times compared with a RISC-V processor using a piecewise linear fitting algorithm with lower precision, and accelerate several lightweight neural networks used as test cases by 4.15 to 10.46 times. The overall area power consumption of the coprocessor of the present invention is only 37.2% and 10.4% higher than the average of the processor core and the floating-point unit of the RISC-V processor used in the test.

[0008] In order to achieve the above purpose, the technical solution adopted by the present invention is as follows:

[0009] A lightweight neural network coprocessor based on the ROCC interface includes a control sub-module, a memory access sub-module, and an accelerator sub-module; the accelerator sub-module includes a linear operation accelerator sub-module and a non-linear operation accelerator sub-module;

[0010] The control sub-module and the memory access sub-module provide instruction decoding control and memory access support for the accelerator sub-module. The control sub-module is connected to the processor core through the ROCC instruction interface, receives the ROCC instructions from the processor core and the data of the source register RS1 and the source register RS2, and outputs the operation result of the accelerator sub-module back to the destination register of the processor core; at the same time, the memory access sub-module is connected to the L1 data cache through the ROCC memory interface to read and write the memory data in the L1 data cache;

[0011] Linear operation accelerator sub-module: It performs multiplication and addition linear operations at the vector level. This module includes an internal data cache and multiple parallel multiplication and addition units; the internal data cache is used to pre-cache the data of the read data stream from the memory access sub-module and read it multiple times, write the data of the read data stream and scalar values into multiple multiplication and addition units, and perform vector multiplication and addition summation operations for the linear part in the neural network according to the function selection signal from the control sub-module; accumulate and sum the operation results of all multiplication and addition units, and output the multiplication and addition operation results to the destination register of the processor core or output to the write data stream of the access sub-module, and the access sub-module writes back to the memory of the L1 data cache; during the operation, use the SRC register and DST register from the control sub-module as the starting addresses for memory read / write, use the source register RS1 as the scalar operand, use the source register RS2 as the read / write sequence length, and transfer the read / write starting address and read / write sequence length to the memory access sub-module;

[0012] Non-linear operation accelerator sub-module: Based on the CORDIC algorithm, it batch-executes 16 non-linear function operations and performs CORDIC calculations for the non-linear part in neural network operations; during the operation, use the SRC register and DST register from the control sub-module as the starting addresses for memory read / write, use the RS1 register as the read / write sequence length, and transfer the read / write starting address and read / write sequence length to the memory access sub-module; this module includes a controller, an input unit, an output unit, and multiple cascaded CORDIC units;

[0013] Controller: Receive the function selection signal from the control sub-module, decode it, and provide the corresponding mode selection signal to each unit;

[0014] Input unit: Calculate the initial values of the 4 variables XYZT required for CORDIC unit iteration according to the input value x of the data of the read data stream from the memory access sub-module, and perform 6 input mode switches;

[0015] Output unit: Calculate the required function value according to the XYZ values generated by CORDIC unit iteration and output it to the write data stream of the memory access sub-module, and perform 8 output mode switches.

[0016] The CORDIC unit integrates 16 non - linear functions: sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, asinh, acosh, atanh, exp, log, sqrtx, sigmoid; these 16 non - linear functions are divided into 8 CORDIC calculation modes: circular rotation mode, circular vector mode, hyperbolic rotation mode, hyperbolic vector, asin mode, acos mode, asinh mode, acosh mode, and CORDIC iterative calculation is performed by switching these 8 CORDIC calculation modes.

[0017] The types of vector multiply - add and sum operations in the linear operation accelerator sub - module for the linear part of the neural network include vector ± vector, vector ⊙ vector, vector ± scalar, vector · scalar, vector inner product, vector cumulative sum, scalar · vector ± vector, vector ⊙ vector ± scalar;

[0018] The multiply - add unit: is used to perform the multiply - add operation of three operands A×B±C, determines the ± operator according to the function selection signal from the control sub - module, and selects the three operands A, B, and C from four sources:

[0019] 1. No - effect operation: that is, multiply by 1.0 and add 0.0 operations;

[0020] 2. Read data stream: data from the memory read data stream in the memory access sub - module;

[0021] 3. Scalar value: scalar data from the source register RS1 in the ROCC instruction;

[0022] 4. Cache data: data from the read data stream pre - input to the internal data cache. [[ID=2)]]

[0023] The switching relationships between the 6 input modes of the input unit, the 8 output modes of the output unit, the 8 CORDIC calculation modes of the CORDIC unit, and the 16 non - linear functions are shown in the following table:

[0024]

[0025] The formulas for calculating the 8 CORDIC modes in the CORDIC unit are as follows:

[0026]

[0027] In the formula, the iteration direction σ and the 5 plus - minus operators ± 1～5 take the values as shown in the following table:

[0028]

[0029]

[0030] The control sub-module: receives ROCC instructions issued by the processor core through the ROCC instruction interface, decodes the FUNCT field of the ROCC instruction, selects and enables one of the two accelerator sub-modules to execute the current computing task according to the sub-module function selection signal generated after the instruction decoding, and transfers the source register RS1 and source register RS2 carried by the ROCC instruction, as well as the SRC register and DST register, a total of four operands, to the accelerator sub-module for operation; after the operation of the accelerator sub-module is completed, the operation result of the accelerator sub-module is sent back to the RD register of the processor core through the ROCC instruction interface.

[0031] When the accelerator sub-module performs an operation with more than two operands, first write the addresses required for the operation into the source address register and destination address register inside the control sub-module through a write instruction of the source register RS1 and source register RS2, and then start the corresponding accelerator sub-module for operation through the subsequent received ROCC instruction.

[0032] The memory access sub-module: is used to process memory read and write operations through the ROCC memory interface; when reading memory data, send a read request containing the address of the data to be retrieved to the request end of the ROCC memory interface. After the ROCC memory interface obtains the data to be retrieved, the response end of the ROCC memory interface receives a response signal containing the previously sent read request and the read data; when writing memory data, a write request containing the data and the data address needs to be sent to the request end of the ROCC memory interface. After the ROCC memory interface completes the write operation, the response end of the ROCC memory interface will present a response signal identical to the request to confirm the completion of the write;

[0033] The memory access sub-module internally includes a read FIFO and a write FIFO, as well as a read address counter and a write address counter for recording the memory request addresses; when the accelerator sub-module is enabled and needs to perform memory read and write operations, it will obtain the required memory read data stream through the read FIFO, write the memory write data stream into the write FIFO, and the read address counter and the write address counter will receive the read / write start address and the read / write sequence length from the accelerator sub-module, initialize both the read address counter and the write address counter to the read / write start address, and send a memory operation request to the ROCC memory interface.

[0034] The logic for the memory access sub-module to send memory read and write operation requests to the ROCC memory interface is as follows:

[0035] If the write FIFO is not empty, first read a data from the write FIFO preferentially and send a write request to increment the write address counter;

[0036] If the write FIFO is empty, and the total amount of data already in the read FIFO and the in - transit data for which a read request has been sent but no response has been received is less than the maximum capacity of the read FIFO, a read request is sent to the ROCC memory interface, and the read address counter is incremented;

[0037] If the read address counter reaches the read start address and the read sequence length, and at the same time the write address counter reaches the write start address and the write sequence length, the memory read / write operation ends; meanwhile, the read data from the response end of the ROCC memory interface is monitored, and the read data is filled into the read FIFO.

[0038] The processor core is a RISC - V architecture processor.

[0039] Compared with the prior art, the beneficial effects of the present invention are:

[0040] The lightweight neural network co - processor designed by the present invention based on the ROCC interface is mainly for low - power application environments with limited computing resources, focusing on accelerating common neural networks with lower area power consumption. At the same time, using the ROCC co - processor interface provided by the RISC - V processor RocketChip for control and data access, compared with common memory - bus - based accelerators, it can improve the instruction interaction speed and optimize the acceleration effect of small - scale neural networks.

[0041] In terms of linear operations, the co - processor mainly accelerates the multiply - add operations at the vector level through the linear operation accelerator sub - module. Specifically, for the multiply - add operation S = A×B±C, except for the ± operator controlled by the instruction, its three operands are all controlled by the instruction and selected from four sources: no - effect operation, read data stream, scalar value, and cache data. These three operands can be independently configured to generate common linear operations in neural network operations such as vector±vector, vector±scalar, vector*vector, vector*scalar, and vector dot product.

[0042] In terms of non - linear operations, a non - linear operation accelerator sub - module is designed based on the CORDIC algorithm, which can calculate 16 common non - linear functions, focusing on application scenarios with limited area and power, such as embedded devices and wearable devices. It performs computational resource reuse and focuses on reducing area and power. Compared with existing single - function accelerators, it realizes more functions and can replace multiple accelerators in the overall project. Compared with existing multi - function accelerators, the present invention reduces the area and power to 28.0% and 22.5% at the cost of reducing the computing speed to 37.5%, and and respectively improve by 33.9% and 66.7%. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1Schematic diagram of the lightweight neural network coprocessor based on the ROCC interface of the present invention.

[0044] Figure 2 Schematic diagram of the control sub-module provided by the present invention.

[0045] Figure 3 Schematic diagram of the memory access sub-module provided by the present invention.

[0046] Figure 4 Schematic diagram of the linear operation accelerator sub-module provided by the present invention.

[0047] Figure 5 Schematic diagram of the multiply-accumulate unit in the linear operation accelerator sub-module provided by the present invention.

[0048] Figure 6 Schematic diagram of the non-linear operation accelerator sub-module provided by the present invention.

[0049] Figure 7 Schematic diagram of the CORDIC unit in the non-linear operation accelerator sub-module provided by the present invention.

[0050] Figure 8 Schematic diagrams of 16 non-linear functions and their corresponding errors in the non-linear operation accelerator sub-module provided by the present invention, where Figure 8 a is the sin and cos functions, Figure 8 b is the asin and acos functions, Figure 8 c is the tan and atan functions, Figure 8 d is the exp and dog functions, Figure 8 e is the sinh and cosh functions, Figure 8 f is the asinh and acosh functions, Figure 8 g is the tanh and atanh functions, Figure 8 h is the sqrt and sigmoid functions. Detailed implementation manners

[0051] The technical solutions of the present invention will be further introduced below in conjunction with the accompanying drawings and embodiments.

[0052] As Figure 1As shown in the figure, the lightweight neural network co-processor based on the ROCC interface of the present invention is based on the open-source RISC-V project RocketChip and the ROCC co-processor interface provided by it. The ROCC co-processor interface includes a ROCC instruction interface and a ROCC memory interface. In the overall architecture of the processor, the co-processor of the present invention includes: a control sub-module, a memory access sub-module, and an accelerator sub-module; the control sub-module and the memory access sub-module provide instruction decoding control and memory access support for the accelerator sub-module. The control sub-module is connected to the processor core through the ROCC instruction interface, receives ROCC instructions and data of two source registers (RS1 and RS2) from the processor core through this ROCC instruction interface, and outputs the operation result of the accelerator sub-module back to the destination register (RD) of the processor core. At the same time, the memory access sub-module is connected to the L1 data cache through the ROCC memory interface, and reads and writes the memory data in the L1 data cache through this ROCC memory interface; the processor core is a RISC-V architecture processor; the ROCC instruction interface and the ROCC memory interface are shown in Table 1 and Table 2 respectively.

[0053] Table 1 ROCC Instruction Interface

[0054] Category Name Direction Function Control busy Out Indicates that the coprocessor is busy Control interrupt Out Related to interrupt control Control exception In Related to exception control Instruction cmd_ready Out Indicates that the coprocessor can accept an instruction Instruction cmd_valid In Indicates that the instruction is valid Instruction cmd_bits_inst_opcode In Opcode of the coprocessor operation Instruction cmd_bits_inst_funct In Used to control the behavior of the coprocessor Instruction cmd_bits_inst_xd In Indicates that a return value is required Instruction cmd_bits_inst_rd In Destination register number Instruction cmd_bits_inst_xs1 In Indicates the existence of source register RS1 Instruction cmd_bits_inst_rs1 In Source register RS1 number Instruction cmd_bits_inst_xs2 In Indicates the existence of source register RS2 Instruction cmd_bits_inst_rs2 In Source register RS2 number Instruction cmd_bits_rs1 In Source register RS1 data Instruction cmd_bits_rs2 In Source register RS2 data Return resp_ready In Indicates that the processor can return Return resp_valid Out Indicates that the return data is valid Return resp_bits_rd Out Destination register number Return resp_bits_data Out Destination register data

[0055] Table 2 ROCC Memory Interface

[0056] Category Name Direction Function Request req_ready In Indicates that a request can be accepted Request req_valid Out Indicates that the request is valid Request req_bits_cmd Out Request command, used to determine read / write functions Request req_bits_tag Out Request identifier, used to distinguish multiple returns Request req_bits_addr Out Memory operation address Request req_bits_data Out Write operation data Request req_bits_mask Out Identifies valid bytes Request req_bits_size Out Number of bytes Return resp_valid In Indicates that the return is valid Return resp_bits_has_data ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​ ​

[0057] As ​ shown, the control sub-module: receives the ROCC instruction sent by the processor core through the ROCC instruction interface, decodes the FUNCT field of the ROCC instruction, and according to the instruction content, selects and activates an accelerator sub-module that executes the current calculation task from the two accelerator sub-modules; the FUNCT field is the cmd_bits_inst_funct in Table 1.

[0058] Since the ROCC instruction can only carry two source registers (RS1 and RS2), and the accelerator sub-module needs more than two operands for some operations, when performing operations that require more than two operands, first write the addresses required for the operation into the source address register (SRC) and the destination address register (DST) inside the control sub-module through a write instruction with two source registers (RS1 and RS2), and then activate the corresponding accelerator sub-module through the ROCC instruction received in the subsequent control sub-module for operation.

[0059] When performing operations, the sub-module function selection signal generated after instruction decoding is passed to the accelerator sub-module to select and activate an accelerator sub-module for executing the current computing task. The source register RS1 and source register RS2 carried by the ROCC instruction, along with the SRC register and DST register inside the control sub-module, a total of four operands, are passed to the accelerator sub-module for operation. After the operation in the accelerator sub-module is completed, the operation result of the accelerator sub-module is sent back to the RD register in the processor core through the ROCC instruction interface.

[0060] As ​ shown, the memory access sub-module is used to handle memory read and write operations through the ROCC memory interface. In the ROCC memory interface, memory operations are carried out through a request / response mechanism, that is, the memory access sub-module sends memory operation requests to the ROCC memory interface without blocking, and the ROCC memory interface asynchronously returns responses after completing the corresponding requests. When reading memory data, a read request containing the address of the data to be retrieved needs to be sent to the request end of the ROCC memory interface. After the ROCC memory interface obtains the data to be retrieved, the response end of the ROCC memory interface receives a response signal containing the previously sent read request and the retrieved data. When writing memory data, a write request containing the data and the data address needs to be sent to the request end of the ROCC memory interface. After the ROCC memory interface completes the write operation, the response end of the ROCC memory interface will present a response signal identical to the request to confirm the completion of the write.

[0061] The memory access sub-module internally contains two FIFOs (First Input First Output, first-in-first-out memories) for reading and writing, as well as a read address counter and a write address counter for recording memory request addresses. When the accelerator sub-module is activated and needs to perform memory read and write operations, it will obtain the required memory read data stream through the read FIFO, write the memory write data stream into the write FIFO, and the read address counter and write address counter will receive the read / write starting address and the read / write sequence length from the accelerator sub-module, initialize both the read address counter and the write address counter to the read / write starting address, and send memory read and write operation requests to the ROCC memory interface according to the following logic:

[0062] If the write FIFO is not empty, preferentially read a data from the write FIFO and send a write request to increment the write address counter;

[0063] If the write FIFO is empty and the total amount of data in the read FIFO and the in-transit data that has been sent a read request but has not received a response is less than the maximum capacity of the read FIFO, send a read request to the ROCC memory interface to increment the read address counter;

[0064] If the read address counter reaches the read start address and the read sequence length position, and at the same time the write address counter reaches the write start address and the write sequence length position, the memory read / write operation ends. Meanwhile, listen for the read data from the response end of the ROCC memory interface, and fill the read data into the read FIFO.

[0065] The accelerator sub-module includes a linear operation accelerator sub-module and a non-linear operation accelerator sub-module.

[0066] As ​ shown, the linear operation accelerator sub-module: performs multiplication-addition linear operations at the vector level, and performs linear operations such as addition, subtraction, multiplication, multiplication-addition mixture, and cumulative summation at the vector operation level; this module contains a 256-byte internal data cache and multiple parallel-running multiplication-addition units; the internal data cache can pre-cache the data of the read data stream from the memory access sub-module and read it multiple times to save memory bandwidth; write the data of the read data stream and scalar values into multiple multiplication-addition units, and perform vector multiplication-addition summation operations in the linear part of the neural network according to the function selection signal from the control sub-module, that is, perform vector ± vector, vector ⊙ vector, vector ± scalar, vector · scalar, vector inner product, vector cumulative summation, scalar · vector ± vector, vector ⊙ vector ± scalar operations; accumulate and sum the operation results of all multiplication-addition units, and output the multiplication-addition operation results to the destination register (RD) of the processor core through the linear operation accelerator sub-module, or output the multiplication-addition operation results to the write data stream of the access sub-module through the linear operation accelerator sub-module, and the access sub-module writes them back to the memory of the L1 data cache. When performing operations, use the SRC register and DST register from the control sub-module as the start addresses for memory read / write, use the source register RS1 from the processor core as the scalar operand, use the source register RS2 from the processor core as the read / write sequence length, and pass the read / write start address and the read / write sequence length to the memory access sub-module; the scalar value is the scalar data from the source register RS1 in the ROCC instruction;

[0067] Taking the example of using a RV64-bit processor and performing calculations with 16-bit data, at this time, the linear operation accelerator sub-module contains 4 parallel multiplication-addition units, and can perform 4 multiplication-addition mixture operations per clock cycle.

[0068] As ​ shown, the multiplication-addition unit: is used to perform the multiplication-addition operation of three operands A×B±C, determine the specific ± operator according to the function selection signal from the control sub-module, and independently select the three operands A, B, and C from four sources:

[0069] 1. No-effect operation: that is, multiply by 1.0 and add 0.0 operations.

[0070] 2. Read data stream: The data from the memory read data stream in the memory access sub-module, which is used for general vector input.

[0071] 3. Scalar value: The scalar data from the source register RS1 in the ROCC instruction, which is used for vector-scalar mixed operations.

[0072] 4. Cached data: The data from the read data stream pre-input to the internal data cache, which can be used multiple times. It is used to cache the vector operands that need to be used multiple times in multi-operand operations, avoiding repeated reads, thereby saving memory bandwidth. For example, in the common matrix-vector process, when calculating the vector inner product multiple times, the vector operand needs to be read multiple times and perform dot product operations with each row of the matrix. At this time, pre-reading this vector operand into the internal cache can avoid repeated reads.

[0073] Specifically, for vector ± vector, vector ⊙ vector, vector ± scalar, vector · scalar, vector inner product, vector accumulation summation, scalar · vector ± vector, and vector ⊙ vector ± scalar operations, the following operand selections are made respectively. The operands not mentioned are ineffective operations:

[0074] Vector ± vector: Cache one vector in advance. A and C are respectively selected as the read data stream and the cached data, and the resulting addition and subtraction results are output to the write data stream.

[0075] Vector ⊙ vector: Cache one vector in advance. A and B are respectively selected as the read data stream and the cached data, and the resulting multiplication results are output to the write data stream.

[0076] Vector ± scalar: A and C are respectively selected as the read data stream and the scalar value, and the resulting addition and subtraction results are output to the write data stream.

[0077] Vector · scalar: A and B are respectively selected as the read data stream and the scalar value, and the resulting multiplication results are output to the write data stream.

[0078] Vector inner product: Cache one vector in advance. A and C are respectively selected as the read data stream and the cached data, close the write data stream, and output the accumulated result as the operation result to the destination register (RD) of the processor core through the control sub-module.

[0079] Vector accumulation summation: Only select A as the read data stream, close the write data stream, and output the accumulated result as the operation result to the destination register (RD) of the processor core through the control sub-module.

[0080] Scalar · vector ± vector mixed operation: Cache one vector in advance. A, B, and C are respectively selected as the scalar value, the read data stream, and the cached data, and the resulting multiplication and addition results are output to the write data stream.

[0081] Vector ⊙ Vector ± Scalar Hybrid Operation: Cache a vector in advance. A, B, and C are respectively selected as the read data stream, cached data, and scalar value. The resulting multiply-accumulate result is output to the write data stream.

[0082] As ​ shown, the Nonlinear Operation Accelerator Sub-module: Based on the CORDIC algorithm, it batch-executes 16 common nonlinear function operations, performs CORDIC calculations for the nonlinear part in neural network operations, and is applicable to activation functions or signal processing algorithms; this module includes a controller, an input unit, an output unit, and multiple cascaded CORDIC units;

[0083] Controller: Receives the function selection signal from the control sub-module, decodes and provides the specific mode selection signals required for each unit, and provides control for multi-cycle iterative calculations;

[0084] Input Unit: Calculates the initial values of the 4 variables XYZT required for CORDIC unit iteration based on the input value x of the data in the read data stream of the memory access sub-module, and performs 6 input mode switches. This unit only involves a small number of simple addition, subtraction, and shift operations and can be directly implemented in hardware through a 6-to-1 selector; The 6 input modes of the input unit are as follows:

[0085] 1. For atan and atanh functions, do not use T, XYZ = (1, x, 0).

[0086] 2. For sin, cos, tan, sinh, cosh, tanh, and exp functions, do not use T, XYZ = (1 / K, 0, x), where 1 / K is a constant.

[0087] 3. For sigmoid function, do not use T, XYZ = (1 / K, 0, x / 2), where 1 / K is a constant.

[0088] 4. For log function, do not use T, XYZ = (x + 1, x - 1, 0).

[0089] 5. For sqrt function, do not use T, XYZ = (x + 1 / 4, x - 1 / 4, 0).

[0090] 6. For asin, acos, asinh, and acosh functions, XYZT = (1, 0, 0, x).

[0091] The output unit calculates the desired function value based on the X, Y, and Z values iteratively generated by the CORDIC unit (T is used only in the iteration process and does not generate output), outputs it to the write data stream of the memory access submodule, and switches between eight output modes. In addition to hardware-implemented addition, subtraction, and shift operations directly through an 8-to-1 selector, this unit also includes a hardware divider for calculating X / K and Y / X. The eight output modes of the output unit are as follows:

[0092] 1. For the cosine and cosh functions, output X.

[0093] 2. For the sin and sinh functions, output Y.

[0094] 3. For the asin, acos, atan, asinh, acosh, and atanh functions, output Z.

[0095] 4. For the exp function, output X+Y.

[0096] 5. For the log function, output 2Z.

[0097] 6. For the sqrt function, output X / K.

[0098] 7. For the tan and tanh functions, output Y / X.

[0099] 8. For the sigmoid function, the output is (Y / X) / 2+1 / 2.

[0100] CORDIC unit: Integrates 16 commonly used nonlinear functions: sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, asinh, acosh, atanh, exp, log, sqrtx, and sigmoid. These 16 commonly used nonlinear functions are divided into eight CORDIC calculation modes: circular rotation mode, circular vector mode, hyperbolic rotation mode, hyperbolic vector mode, asin mode, acos mode, asinh mode, and acosh mode. CORDIC iterative calculations are performed by switching between these eight CORDIC calculation modes, as shown in Table 3.

[0101] Table 3 16 nonlinear functions supported by the nonlinear operation accelerator submodule

[0102]

[0103] The present invention calculates the above eight CORDIC calculation modes using a universal CORDIC formula, which is as follows:

[0104]

[0105] Based on this general CORDIC calculation formula, the CORDIC unit uses the addition, subtraction, and shift operations common to 8 CORDIC calculation modes as a shared module, and the part related to the mode as a switchable module, thus realizing the integration of multiple CORDIC calculation modes on a single CORDIC unit. For example, ​ As shown, the addition, subtraction, and shift operations common to 8 CORDIC calculation modes mainly include a small memory (ROM) that provides constant coefficients, 5 right-shift operators, 2 three-input addition / subtraction operators, and 2 two-input addition / subtraction operators; when switching the calculation mode, the CORDIC unit needs to handle the following differences:

[0106] 1. The constant coefficient ROM has two modes. It generates 2atan(2 -i ) when calculating trigonometric series functions and 2atanh(2[[ID=*]] -i ) when calculating hyperbolic series functions;

[0107] 2. The selection of the CORDIC calculation mode is carried out in two steps:

[0108] The first step: Calculate the iteration direction σ, and there are 6 modes:

[0109] 1. For the sin, cos, tan, sinh, cosh, tanh, exp, and sigmoid functions, σ = sign(Z).

[0110] 2. For the atan, atanh, log, and sqrt functions, σ = sign(Y).

[0111] 3. For the acosh function, σ = sign(T - X).

[0112] 4. For the asinh function, σ = sign(T - Y).

[0113] 5. For the acos function, σ = sign(Y(X - T)).

[0114] 6. For the asin function, σ = sign(X(T - Y)).

[0115] The second step: Calculate the values of the 5 addition / subtraction operators ± 1～5 , and there are 4 modes:

[0116] 1. For the sin, cos, tan, and atan functions, ±5 is not used, and ± 1～4 = (-1, -σ, -1, -σ).

[0117] 2. For the sinh, cosh, tanh, atanh, exp, log, sqrt, and sigmoid functions, do not use ±5, ± 1～4 = (+1, +σ, +1, -σ).

[0118] 3. For the asin and acos functions, ± 1～5 = (-1, -σ, -1, +σ, +1).

[0119] 4. For the asinh and acosh functions, ± 1～5 = (+1, +σ, +1, +σ, -1).

[0120] When the non - linear operation accelerator sub - module performs operations, it uses the SRC register and DST register from the control sub - module as the starting addresses for memory read / write, uses the RS1 register from the processor core as the read / write sequence length, and passes the read / write starting address and read / write sequence length to the memory access sub - module.

[0121] Analyze the non - linear operation accelerator sub - module proposed in the present invention from the aspects of calculation accuracy and error. To conform to the data width of the existing accelerators in Table 4, the experiment uses 32 - bit fixed - point numbers (20 - bit binary decimals) for calculation. ​ For the 16 non - linear function schematic diagrams and their corresponding errors in the non - linear operation accelerator sub - module provided by the present invention, the results are as ​ shown. The non - linear operation accelerator sub - module will cause precision loss when the function value or derivative value tends to infinity. This problem is caused by the error of the data type and is unavoidable. In addition, the non - linear operation accelerator sub - module can achieve a calculation accuracy of 16 - bit binary.

[0122] Compare the accelerator in the background technology with the non - linear operation accelerator sub - module proposed in the present invention from the aspects of area and power indicators:

[0123] Method 1: A single - function accelerator for x 1 / N calculation proposed by Y. Luo et al.

[0124] Method 2: A single - function accelerator for x 1 / N or x N calculation proposed by S. Mopuri et al.

[0125] Method 3: A multi - function accelerator for calculating 25 non - linear functions proposed by H. Chen et al. Its design uses 8 independent CORDIC calculation units with different modes and does not perform multi - mode resource reuse.

[0126] Table 4 Comparison of area and power indicators between the present invention and existing accelerators

[0127]

[0128]

[0129] Compared with existing accelerators, the present invention focuses on reducing the overall area and power. The computing speed is 750 MOPS / s, which is lower than that of existing accelerators. The difference in computing speed needs to be considered in the following area and power comparison.

[0130] Compared with Method 1 and Method 2 of existing single - function accelerators, the estimated area of the present invention in a 40 - nm process is 107139 μm 2 = 52498 μm 2 ·(40 nm / 28 nm) 2 , which is close to the area index of existing single - function accelerators. And the present invention can handle 16 common non - linear functions, having more functions than existing single - function accelerators in terms of functionality.

[0131] Compared with the 25 non - linear functions implemented by Method 3 of existing multi - function accelerators, the 16 functions implemented by the present invention are fewer. However, considering that the additional 9 functions mainly come from the nested composition of the 16 functions already implemented by the present invention, the two accelerators are roughly the same in terms of functionality. The present invention reduces the area and power to 28.0% and 22.5% at the cost of reducing the computing speed to 37.5%, and increases and by 33.9% and 66.7% respectively.

[0132] The area and power of the processor core with the RISC - V architecture used in the test of the present invention are 24414 μm 2 and 4.344 mW respectively. The area and power of the floating - point unit (FPU) are 36688 μm 2 and 3.785 mW respectively. The overall area and power of the coprocessor designed by the present invention during calculation using 16 - bit fixed - point numbers are 41874 μm 2 and 4.489 mW respectively, which are only 37.2% and 10.4% higher than the average area and power of the processor core and the floating - point unit respectively. Compared with directly using the RISC - V processor for calculation, the coprocessor of the present invention can accelerate linear operations by 2.24 to 20.65 times; compared with the RISC - V processor using a lower - precision piece - wise linear fitting algorithm, the coprocessor of the present invention can accelerate non - linear operations by about 14 times and accelerate several lightweight neural networks used as test cases by 4.15 to 10.46 times.

Claims

1. A lightweight neural network coprocessor based on the ROCC interface, characterized in that: It includes a control sub-module, a memory access sub-module, and an accelerator sub-module; the accelerator sub-module includes a linear operation accelerator sub-module and a non-linear operation accelerator sub-module; The control sub-module and the memory access sub-module provide instruction decoding control and memory access support for the accelerator sub-module. The control sub-module is connected to the processor core through the ROCC instruction interface, receives ROCC instructions from the processor core and data from source register RS1 and source register RS2, and outputs the operation result of the accelerator sub-module back to the destination register of the processor core; at the same time, the memory access sub-module is connected to the L1 data cache through the ROCC memory interface to read and write the memory data in the L1 data cache; Linear operation accelerator sub-module: It performs multiplication-addition linear operations at the vector level. This module includes an internal data cache and multiple parallel-running multiplication-addition units; the internal data cache is used to pre-cache the data of the read data stream from the memory access sub-module and read it multiple times, write the data of the read data stream and scalar values into multiple multiplication-addition units, and perform vector multiplication-addition summation operations for the linear part in the neural network according to the function selection signal from the control sub-module; Accumulate and sum up the operation results of all multiplication-addition units, and output the multiplication-addition operation results to the destination register of the processor core or output them to the write data stream of the access sub-module, and the access sub-module writes them back to the memory in the L1 data cache; during the operation, use the SRC register and DST register from the control sub-module as the starting addresses for memory read / write, use the source register RS1 as the scalar operand, use the source register RS2 as the read / write sequence length, and pass the read / write starting address and read / write sequence length to the memory access sub-module; Non-linear operation accelerator sub-module: This module includes a controller, an input unit, an output unit, and multiple cascaded CORDIC units; based on the CORDIC algorithm, it batch-executes 16 non-linear function operations to perform CORDIC calculations for the non-linear part in neural network operations; during the operation, use the SRC register and DST register from the control sub-module as the starting addresses for memory read / write, use the RS1 register as the read / write sequence length, and pass the read / write starting address and read / write sequence length to the memory access sub-module; Controller: Receive the function selection signal from the control sub-module, decode it, and provide corresponding mode selection signals to each unit; Input unit: Calculate the initial values of the 4 variables XYZT required for CORDIC unit iteration according to the input value x of the data in the read data stream of the memory access sub-module, and perform 6 input mode switches; Output unit: Calculate the required function value according to the XYZ values generated by CORDIC unit iteration and output it to the write data stream of the memory access sub-module, and perform 8 output mode switches.

2. The lightweight neural network coprocessor based on the ROCC interface according to claim 1, wherein: The CORDIC unit integrates 16 non-linear functions: sin, cos, tan, asin, acos, atan, sinh, cosh, tanh, asinh, acosh, atanh, exp, log, sqrtx, sigmoid; these 16 non-linear functions are divided into 8 CORDIC calculation modes: circular rotation mode, circular vector mode, hyperbolic rotation mode, hyperbolic vector, asin mode, acos mode, asinh mode, acosh mode, and CORDIC iterative calculation is performed by switching these 8 CORDIC calculation modes.

3. The lightweight neural network coprocessor based on the ROCC interface according to claim 1, wherein: The types of vector multiply-accumulate summation operations for the linear part in the neural network performed by the linear operation accelerator sub-module include vector ± vector, vector ⊙ vector, vector ± scalar, vector · scalar, vector inner product, vector cumulative summation, scalar · vector ± vector, vector ⊙ vector ± scalar; The multiply-accumulate unit: is used to perform the multiply-accumulate operation of three operands A×B±C, determines the ± operator according to the function selection signal from the control sub-module, and selects three operands A, B, and C from four sources:

1. No-effect operation: that is, multiply by 1.0 and add 0.0 operations; 2. Read data stream: data from the memory read data stream in the memory access sub-module; 3. Scalar value: scalar data from the source register RS1 in the ROCC instruction; 4. Cache data: data from the read data stream pre-input to the internal data cache.

4. The lightweight neural network coprocessor based on the ROCC interface according to claim 1 or 2, wherein: The switching relationships between the 6 input modes of the input unit, the 8 output modes of the output unit, and the 8 CORDIC calculation modes of the CORDIC unit and the 16 non-linear functions are shown in the following table:

5. The lightweight neural network coprocessor based on the ROCC interface according to claim 2, wherein: The formulas for calculating the 8 CORDIC modes in the CORDIC unit are as follows: The iterative direction σ and the five plus-minus operators ± in the formula 1～5 have the following values in the table below:

6. The lightweight neural network coprocessor based on the ROCC interface according to claim 1, wherein The control sub-module: receives the ROCC instruction issued by the processor core through the ROCC instruction interface, decodes the FUNCT field of the ROCC instruction, and according to the sub-module function selection signal generated after the instruction decoding, selects and enables one of the two accelerator sub-modules to execute the current calculation task, and transfers the source register RS1 and source register RS2 carried by the ROCC instruction, as well as the SRC register and DST register, a total of four operands, to the accelerator sub-module for operation; after the operation of the accelerator sub-module is completed, the operation result of the accelerator sub-module is sent back to the RD register of the processor core through the ROCC instruction interface.

7. The lightweight neural network coprocessor based on the ROCC interface according to claim 6, characterized in that: When the accelerator sub-module performs an operation on more than two operands, first write the addresses required for the operation into the source address register and destination address register inside the control sub-module through a write instruction for source register RS1 and source register RS2, and then start the corresponding accelerator sub-module through the subsequent received ROCC instruction to perform the operation.

8. The lightweight neural network coprocessor based on the ROCC interface according to claim 1, characterized in that The memory access sub-module is used to process memory read and write operations through the ROCC memory interface. When reading memory data, it sends a read request containing the address of the desired data to the request end of the ROCC memory interface. After the ROCC memory interface obtains the desired data, the response end of the ROCC memory interface receives a response signal containing the previously sent read request and the read data. When writing memory data, it is necessary to send a write request containing the data and the data address to the request end of the ROCC memory interface. After the ROCC memory interface completes the write operation, the response end of the ROCC memory interface will present a response signal identical to the request to confirm the completion of the write. Inside the memory access sub-module, there are a read FIFO and a write FIFO, as well as a read address counter and a write address counter for recording the memory request addresses. When the accelerator sub-module is started and needs to perform memory read and write operations, it will obtain the desired memory read data stream through the read FIFO, write the memory write data stream into the write FIFO, and the read address counter and write address counter will receive the read / write start address and read / write sequence length from the accelerator sub-module, initialize both the read address counter and the write address counter to the read / write start address, and send a memory operation request to the ROCC memory interface.

9. The lightweight neural network coprocessor based on the ROCC interface according to claim 8, wherein: The logic for the memory access sub-module to send memory read and write operation requests to the ROCC memory interface is as follows: If the write FIFO is not empty, preferentially read a data from the write FIFO and send a write request to increment the write address counter. If the write FIFO is empty and the total amount of data already in the read FIFO and in-flight data for which read requests have been sent but responses have not been received is less than the maximum capacity of the read FIFO, send a read request to the ROCC memory interface to increment the read address counter. If the read address counter reaches the read start address and read sequence length positions, and at the same time the write address counter reaches the write start address and write sequence length positions, then the memory read / write operation ends. At the same time, listen for the read data from the response end of the ROCC memory interface and fill the read data into the read FIFO.

10. A lightweight neural network coprocessor based on the ROCC interface according to claim 1, characterized in that: The processor core is a RISC-V architecture processor.

Images