A coprocessor based on RISC-V architecture and its processing system

Abstract

This application relates to a coprocessor based on the RISC-V architecture and its processing system. The coprocessor includes a decoding unit, a control unit, an arithmetic unit, a cross-clock transfer unit, and a cache unit. The decoding unit receives the values ​​of the source registers sent by the main processor and decodes them to output operation parameters. The control unit controls the working process of the arithmetic unit based on the operation parameters. The cross-clock transfer unit communicates with the main processor, including the transmission of configuration parameters and feedback signals, thereby supporting the main coprocessor to run in different clock domains. The cache unit is used to cache part of the weight data and all bias data required in a single operation, which can be used to implement the operation of extended instructions, while not occupying the main processor's storage space, greatly improving the efficiency of neural network operation.

Description

Technical Field

[0001] This application relates to the field of computers, specifically to a coprocessor based on the RISC-V architecture and its processing system. Background Technology

[0002] Edge computing is a distributed computing paradigm that brings computing, storage, and networking services closer to the data source or user to reduce latency, improve response speed, and enhance data security. This computing approach is particularly important for applications that require real-time processing of large amounts of data, such as the Internet of Things (IoT), smart manufacturing, smart cities, and autonomous vehicles. In edge computing models, data does not need to be transmitted to remote data centers for processing; instead, it is processed instantly at or near the location where the data is generated. This places dual demands on processors, requiring them to possess both high computing power and low power consumption and low cost.

[0003] The drawback of existing technologies is that traditional processor systems often face problems of low efficiency and large storage space consumption when performing neural network operations. Summary of the Invention

[0004] The purpose of this application is to overcome the shortcomings of the prior art and provide a coprocessor based on the RISC-V architecture, which can be used to implement extended instruction operations without occupying the main processor's storage space, thus greatly improving the efficiency of neural network operations.

[0005] The objective of this application is achieved through the following technical solution:

[0006] The coprocessor includes:

[0007] The decoding unit receives the value of the source register sent by the main processor and decodes it to output the operation parameters.

[0008] The control unit and the arithmetic unit control the working process of the arithmetic unit based on the operating parameters.

[0009] The cross-clock transfer unit communicates with the main processor, including the transmission of configuration parameters and feedback signals, thereby enabling the main coprocessor to operate in different clock domains;

[0010] The cache unit is used to cache some of the weight data and all the bias data required in a single operation.

[0011] Unlike traditional computing models, this application is based on the principle of distributed computing. It places the computationally intensive and memory-intensive neural network operations on a branch processor, which is to construct a coprocessor to assist the main processor in performing extended instructions. This frees up the computing power and storage space of the main processor and greatly improves computing efficiency.

[0012] For example, the decoding unit decodes the value of the source register sent by the main processor, reads the operands of the source register, breaks them down, determines the specific operation represented by the current instruction, and obtains the operation parameters required for the operation and sends them to the control unit.

[0013] For example, the control unit includes a memory management module and an arithmetic management module;

[0014] in,

[0015] The computation management module dispatches computation tasks to the computation unit according to the operation parameters given by the decoding unit, and controls the working process of the computation unit by receiving feedback signals from the computation unit;

[0016] The memory management module is responsible for managing data transfer between the coprocessor and external storage units, including memory address calculation and allocation, and memory read / write state transitions.

[0017] For example, the arithmetic unit adopts a pipelined architecture consisting of booth-encoded multipliers and Wallace-type adders, and the computation is performed in a non-blocking manner.

[0018] For example, the operation unit includes a line cache module, a convolution operation module, a pooling module, an activation function calculation module, a quantization module, and a data temporary storage module. Each module is completely decoupled and has its own dedicated enable signal port and parameter input port.

[0019] For example, the line buffer module consists of two FIFOs and a parallel output 3×3 sliding window, used to convert serial data input into parallel multi-source data output.

[0020] For example, the convolution operation module includes two parts: a 1×1 convolution module and a 3×3 convolution module. In a single operation, each module can perform operations on four output channels simultaneously. During the output channel operation, the four input channels are also calculated simultaneously.

[0021] For example, the data buffer module is used to cache the accumulated results of all channels that have not yet been completed, and to superimpose the existing results with the cached data and then store them. When all channels have completed the operation, the memory data is read out and added to the bias data.

[0022] For example, the quantization module is used to quantize the convolutional result (over 8 bits) and the relevant input parameters, and convert the output result back into an 8-bit integer.

[0023] For example, the activation function calculation module supports ReLU function operations and LeakyReLU function operations, where the selection of the activation function can be achieved through configuration parameters.

[0024] For example, the pooling module has a built-in row cache unit suitable for pooling computation and related control logic, which converts the input serial data into two rows of parallel data. The control logic is responsible for sampling and outputting the input data according to the specific parameter configuration.

[0025] For example, the caching unit consists of a weight data caching module and a bias data caching module, which are used to cache weight data and bias data, respectively.

[0026] When performing convolution and quantization operations, the corresponding weight and bias data are effectively output according to the preset reading order.

[0027] A second aspect of this application also provides a processing system based on a RISC-V architecture coprocessor, the system comprising:

[0028] Main processor;

[0029] and at least one external memory;

[0030] And coprocessors;

[0031] The coprocessor responds to the extended instructions of the main processor and completes the computational tasks based on the extended instructions. External memory serves as the memory medium for the coprocessor to access, process, and process data.

[0032] Furthermore, the main processor includes a nice module for processing extended instructions, and the coprocessor connects to the nice module to complete communication.

[0033] The beneficial effects of this application are:

[0034] 1) Utilizing the instruction extensibility feature of the RISC-V instruction set architecture, this coprocessor is directly allocated and controlled by a dedicated extended instruction set. The model can be built at the instruction level, which reduces the number of instructions required and avoids the problems of redundant instructions and high clock cycle consumption that occur in traditional processors when processing neural networks.

[0035] 2) Integrates multiple neural network operators, allowing users to configure the corresponding source register internal parameters according to their own needs to enable and configure the corresponding operators.

[0036] 3) The arithmetic units are all composed of self-designed high-performance multipliers or high-performance adders, and the dependence on DSP is relatively low. Therefore, this design can be adapted to a variety of FPGA platforms.

[0037] 4) In edge computing scenarios, due to performance and resource limitations, the deployed models are mostly lightweight models. The model parameters are quantized to convert floating-point parameters into integer parameters, thereby avoiding the high resource consumption and high computational complexity caused by floating-point operations.

[0038] 5) Supports integer operations, which means that before building the model, the model needs to be quantized to integers, usually using the INT8 quantization type.

[0039] 6) Supports cross-clock domain transfer and is compatible with RISC-V processors with various upper clock frequencies.

[0040] 7) The technical solution of the present invention has high portability and can significantly improve the computational efficiency of the processor when facing neural network tasks. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the system architecture of an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the extended instruction signal flow according to an embodiment of this application;

[0043] Figure 3 The calculation process is as follows, according to the embodiments of this application;

[0044] Figure 4 This is a schematic diagram of the cache module architecture as described in this application.

[0045] Figure 5 This is a schematic diagram illustrating the output of the cache module in this application.

[0046] Figure 6 This is a schematic diagram of the convolution module architecture in an embodiment of this application;

[0047] Figure 7 This is a schematic diagram of the computing unit architecture in an embodiment of this application;

[0048] Figure 8 This is a schematic diagram of the data temporary storage module architecture in an embodiment of this application;

[0049] Figure 9 This is a schematic diagram of the quantization module architecture in an embodiment of this application;

[0050] Figure 10 This is a schematic diagram of the activation function calculation module architecture in an embodiment of this application;

[0051] Figure 11 This is a schematic diagram of the pooling module architecture in an embodiment of this application;

[0052] Figure 12This is a schematic diagram of the computation of one branch of the yolov3-tiny model network in an embodiment of this application;

[0053] Figure 13 Examples of embodiments in this application Figure 12 The diagram shows the time distribution of calculation operations for each branch. Detailed Implementation

[0054] The technical solution of this application is described in further detail below with reference to specific embodiments, but the scope of protection of this application is not limited to the following description.

[0055] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the specific embodiments of this application are described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of this application. However, this application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of this application. Therefore, this application is not limited to the specific embodiments disclosed below.

[0056] In the description of this application, it should be understood that if terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" appear, these terms indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application.

[0057] Furthermore, where the terms "first" and "second" appear, these terms are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, where the term "multiple" appears, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0058] In this application, unless otherwise expressly specified and limited, the terms "installation," "connection," "joining," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0059] In this application, unless otherwise expressly specified and limited, the use of descriptions such as "above" or "below" the second feature indicates that the first and second features are in direct contact or indirect contact via an intermediate medium. Furthermore, "above," "on top of," and "over" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. Similarly, "below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0060] It should be noted that if an element is referred to as being "fixed to" or "set on" another element, it can be directly on the other element or there may be an intervening element. If an element is considered to be "connected to" another element, it can be directly connected to the other element or there may be an intervening element. If so, the terms "vertical," "horizontal," "upper," "lower," "left," "right," and similar expressions used in this application are for illustrative purposes only and do not represent the only possible implementation.

[0061] like Figure 1 As shown, a coprocessor based on the RISC-V architecture includes a decoding unit, a control unit, an arithmetic unit, a cross-clock transfer unit, and a cache unit.

[0062] The decoding unit receives the value of the source register sent by the main processor and decodes it to output the operation parameters. The control unit controls the working process of the arithmetic unit based on the operation parameters. The cross-clock transmission unit communicates with the main processor, including the transmission of configuration parameters and feedback signals, thereby supporting the main coprocessor to run in different clock domains. The cache unit is used to cache part of the weight data and all bias data required in a single operation.

[0063] The decoding unit decodes the value of the source register sent by the main processor, reads the operands of the source register, breaks them down, determines the specific operation represented by the current instruction, obtains the parameters required for the operation, and sends them to the subsequent control unit.

[0064] The control unit consists of a memory management module and an arithmetic management module. The arithmetic management module dispatches computation tasks to the arithmetic unit according to the parameters given by the decoding module and controls the working process of the arithmetic unit by receiving feedback signals from the arithmetic unit. The memory management module is responsible for the management of data transmission between the coprocessor and the external storage unit, including the calculation and allocation of memory addresses and the switching of memory read and write states.

[0065] The cross-clock domain transmission unit is responsible for communication between the main processor and the coprocessor, including the transmission of configuration parameters and feedback signals. This allows the main and coprocessors to run in different clock domains, so that even when the main processor's clock frequency is not high, the coprocessor can still work at a relatively high clock frequency, thus ensuring its high-performance characteristics.

[0066] The cache unit is responsible for caching some of the weight data and all the bias data required in a single operation. This unit consists of a weight data cache module and a bias data cache module. When performing convolution and quantization operations, the weight and bias data cache unit effectively outputs the corresponding weight data and bias data in a certain reading order. The function of this unit is to reduce the frequency of access to external storage units and reduce the impact of memory access on the overall operation efficiency.

[0067] As a specific embodiment, this example integrates a coprocessor onto a Hummingbird e203 main processor to form a processing system. The Hummingbird e203 main processor includes an instruction fetch module, a decoding module, an execution module, and a nice module. The coprocessor interfaces with the extended instruction processing module in the Hummingbird e203 main processor, i.e. Figure 1 The nice modules above are connected to enable communication between the main and coprocessors, allowing the execution of computational tasks specified by extended instructions. In this example, the coprocessor uses onboard DDR external memory as its memory medium to access and process data.

[0068] In the initial stage, the instruction fetch module of the Hummingbird e203 main processor reads the instruction to be processed and sends it to the decode module. After receiving the instruction, the decode module determines the type of the current instruction and the specific operation. After completing the decoding operation, the execution module performs memory access and operation according to the instruction type and the required operands. If the current instruction is not an extended instruction, the execution module will perform corresponding operation processing according to the given information and data.

[0069] like Figure 2As shown, if it is an extension instruction, the nice module will be enabled to process it, and the data of the registers to be read by the current instruction will be transferred to the nice module. The coprocessor will wait for the coprocessor to complete the operation indicated by the current extension instruction. Here, the execution process of the extension instruction is a blocking process, rather than a non-blocking process like a pipeline architecture. The nice module can only be released after the current instruction is completed, so that the next extension instruction can be received.

[0070] To improve transmission stability and meet timing requirements, the nice module encodes all operations represented by extended instructions, assigning each operation a unique ID. Upon receiving an instruction, the nice module generates a 32-bit ID based on the operation it represents. This ID, along with the data in the provided registers, is then transmitted to the coprocessor. The instructions adapted to this coprocessor are categorized into three types: configuration, reset, and startup. The typical usage order is reset first, then configuration, and finally startup.

[0071] Data transmitted to the coprocessor first undergoes clock domain transfer via a cross-clock transfer unit. After processing, it is input to the coprocessor's decoding unit. The decoding unit determines the current operation type based on the given ID identifier. If it is a configuration type, the operands transmitted along with the data are split and partially processed before being transmitted to the control and arithmetic units. This includes information such as the starting address for reading / writing data and the selection of convolution and pooling types. If it is one of the other two types, no operand configuration is required, and the operation is executed directly.

[0072] After receiving data from the decoding unit, the control unit performs static parameter configuration. The enabling of the operator and the form of operation are determined by this parameter configuration.

[0073] like Figure 3 The diagram illustrates the general computation process. When the operation is initiated, the control unit allocates and manages the computation units and reads / writes to memory in real time based on the computation status. The computation units and cache units are responsible for task computation and the caching of weight bias parameters, respectively, and both are managed and monitored by the control unit.

[0074] The control unit first determines whether the activation function used in this execution is a LeakyReLU function based on the static parameter configuration. If the activation function is LeakyReLU, the quantized static LeakyReLU lookup table data is loaded into the LeakyReLU cache module in the activation function calculation module of the computation unit. Then, all the bias data and some weight data required for this operation are placed into the bias cache module and weight cache module in the cache unit. After the above actions are completed, the control unit reads the weight data from the cache unit and transmits it to the computation unit. The convolution module in the computation unit will output a loading completion flag signal after the weight loading is completed. After receiving the signal, the control unit reads the feature input map data in memory into the computation unit. If the current configuration convolution type is 3×3 convolution, the input data enters the row cache module. The serial data is processed by the row cache module and outputs parallel sliding window data. The row cache module supports padding and stride selection functions, and the selection of all its functions can be determined by the parameter configuration. The convolution module can simultaneously operate on four of the four input channels out of four output channels. If the number of input channels is greater than four, the control unit will enable the data buffer module to temporarily store some of the convolution operation results. After all input channels have completed the operation, the control unit reads the data from the bias buffer module in the buffer unit, adds it to all the data in the buffer module, and then outputs it to the quantization module. The quantization module performs integer quantization processing on the data after the convolution operation according to the quantization parameters provided in the configuration, and outputs the processed data to the activation function calculation module. If the configuration selects LeakyReLU function calculation, the input quantized data will use its own value to look up the value in the LeakyReLU buffer module in the buffer unit, that is, it will use its own value as the buffer address to retrieve the value stored at the corresponding address. If ReLU function calculation is used, the quantized data will be compared with the zero-point data. If it is less than or equal to the zero-point value, the zero-point value will be output; if it is greater than the zero-point value, the original data will be output. After the activation function calculation module processes the data, if pooling is required for the current calculation, the data will enter the pooling module. The specific pooling operation is also determined by the parameter configuration. The pooling operation of this invention only supports 2×2 max pooling, supports adjustable step size and zeropad function. After the data is input, row caching is also performed, and then a parallel 2×2 sliding window is output. The sliding window data is compared and then the pooled data is output.

[0075] Once all operations are complete, the control unit outputs an end flag signal, which is processed by the cross-clock transmission unit and sent to the main processor, releasing the nice module to process and execute the next extended instruction.

[0076] The following section explains the working principle and design architecture of the internal modules of the computing unit.

[0077] The computation unit comprises a line cache module, a convolution operation module, a pooling module, an activation function calculation module, a quantization module, and a data temporary storage module. Each module is completely decoupled, with dedicated enable signal ports and parameter input ports, and can be configured with corresponding instructions as needed. The entire computation unit is built upon a high-performance multiply-accumulate unit composed of a booth-encoded multiplier and a Wallace-type adder. This unit adopts a pipelined architecture, and computation is performed in a non-blocking manner, greatly improving the overall data throughput.

[0078] like Figure 4 As shown, the row buffer module mainly consists of two FIFOs and a sliding window output control module. The row buffer module converts serial data input into parallel multi-row data output, outputting a 3×3 sliding window in parallel. This module also supports padding, supporting up to two rows (top and bottom) and two columns (left and right). The padding value can be configured by parameters. The two FIFOs are buffer sequence 1 and buffer sequence 2, as shown in the diagram. During operation, input data is used as input to sequence 2, and the output of sequence 2 is used as input to sequence 1. When the third row is input, sequence 1, sequence 2, and the input data form a parallel output. Generally, without padding, the initial formation and output of the 3×3 sliding window requires 3 clock cycles. Due to its pipelined design, subsequent sliding windows are generated on the rising edge of each clock cycle. If padding is enabled, taking zero as the padding value as an example, the input feature map size is n×n. Figure 5 As shown in the diagram, when the second line of input is received, Sequence 2 begins to read out data, forming two parallel output lines with the input data. The top row and left column of the sliding window are filled. When the input reaches the end of the feature map, the right column of the sliding window is filled, and so on. The bottom output is similar to the top output, and the final output is also as shown. Figure 5 As shown in the image.

[0079] like Figure 6In this architecture, the convolution module mainly consists of four output channel computation units, specifically divided into 1×1 and 3×3 convolution modules. During a single computation, each module can simultaneously perform operations on all four output channels, and during output channel computation, the four input channels are also computed simultaneously. The 3×3 convolution module uses line buffering for parallel multiplication-accumulation computation, with 144 parallel multiplication operations. Similarly, in the 1×1 convolution module, the number of parallel multiplication operations is 16. Each output channel computation unit consists of four sliding window computation units, one adder, and one saturation truncation. A single output channel computation unit can simultaneously perform convolution operations on the feature maps of all four input channels. The entire convolution module is designed based on a pipelined structure. The sliding window computation units also consist of 1×1 and 3×3 sliding window computation units. The 1×1 sliding window computation unit has a simpler structure, consisting of only a booth-encoded multiplier, while the 3×3 sliding window computation unit has a more complex architecture, as shown in the diagram. Figure 7 As shown, the sliding window computation unit mainly consists of a multiplier array and a multi-layer adder. The multiplier array comprises nine booth-coded multipliers, performing parallel multiplication of all data within the sliding window with their respective weights. The multi-layer adder consists of two 4-2 Wallace tree adders, one 5-2 Wallace tree adder, and one ordinary adder. For the nine products obtained from the multiplier array, the multi-layer adder selects one 4-2 Wallace tree adder and one 5-2 Wallace tree adder as the first layer. The second layer uses a 4-2 Wallace tree adder to add the two partial sums and two carry data. The third layer uses an ordinary adder to add the partial sums and carry data generated in the second layer, outputting the multiply-accumulated result of the sliding window computation unit. After the four input channels complete the computation, the four multiply-accumulated results are also added by a 4-2 Wallace tree adder, resulting in channel superposition before output. After a series of multiplication and accumulation calculations, the output width of each channel will exceed 8 bits, theoretically requiring 32 bits. However, 32-bit data would drastically increase hardware resource consumption in subsequent processing. Extensive experimental observations have shown that the quantized model, after multiplication and accumulation using INT8, is often far smaller than the maximum value that INT32 can represent. In fact, multiplying INT8 by INT8 to obtain INT16 for accumulation only requires INT18 to ensure that most multiplication and accumulation values ​​do not overflow. Therefore, to conserve resources, the output result is processed by a saturation truncation circuit to become INT18 data. Simultaneously, four 18-bit data points are converted into a single 72-bit data point for output to the next processing module.

[0080] like Figure 8The data buffering module's main function is to cache the accumulated results of all channels before completion, and to superimpose the existing results with the input data before storing them. Once all channels have completed their computation, the memory data is read out and added to the bias data. The data buffering module is primarily responsible for caching and processing the data after convolution operations. When the number of input channels is greater than four, the coprocessor cannot complete the computation of all input channels at once. Therefore, this module needs to cache the computation results of the currently completed input channels. When the computation results of subsequent input channels are input to the buffering module, this module reads out the cached input channel computation results, superimposes them with the subsequent input channel computation results, and then caches them. Once all input channels have completed their computation, the memory data is read out and added to the bias data.

[0081] The data buffering module consists of a FIFO and an adder, responsible for buffering some of the completed calculation results in the output channels. A single operation can only complete the calculation of feature maps for four input channels. When there are more than four input channels, a single operation cannot complete all the calculation tasks for a single output channel. The function of this module is to temporarily store the completed convolution calculation results. When there is input data, the data is read from the FIFO and added to the input data. If the calculation of a single output channel is completed, the data is sent to the bias addition unit and added to the bias data, then output to the next operation unit. If the calculation tasks of a single output channel are not completed, meaning there are still input channels that have not been calculated, the data will re-enter the FIFO, waiting for the next read.

[0082] like Figure 9As shown, similar to the convolution module, the quantization module consists of four quantization operation units. These units quantize the convolutional result (over 8 bits wide) and the relevant input parameters, converting the output back into an 8-bit integer. Each quantization operation unit contains two saturation truncation units, one quantization multiplier, and one shifter. This quantization operation unit is the hardware implementation of PyTorch's quantization inference, and its quantized result is guaranteed to be largely consistent with the result after PyTorch quantization. The quantization factor used in the hardware quantization process is the same as that used in the software-generated model. However, in the software, this quantization factor is a floating-point number and cannot be calculated on the coprocessor. Therefore, this floating-point quantization factor is split into two coefficients: an integer M coefficient and an integer S coefficient. This operation must be performed in the software-generated model parameters. The hardware quantization process transforms the floating-point multiplication of the quantization factor into a process of first multiplying it by the M coefficient and then shifting it according to the S coefficient, thus achieving a smooth transition. After completing the convolution and bias data addition operations, the input data has a bit width of 72 bits. Starting from the least significant bit, every 18 bits represent the result of one output channel, and one input data contains the results of four output channels. Four quantization units are responsible for the results of the four output channels. The operation process can be summarized as follows: INT18 type data is scaled back to INT8 type data. Specifically, the 18-bit unquantized data is saturated and truncated, changing from INT18 to INT16. Then, it is multiplied by the M coefficient of type UINT16 in the quantization factor to obtain an INT32 type data. Next, this INT32 data is shifted according to the shift factor in the quantization factor to obtain a new INT16 type data. Finally, this INT16 data undergoes saturation truncation from INT16 to INT8 to obtain an INT8 type output result.

[0083] like Figure 10 The diagram shows the activation function calculation module. This module supports ReLU and LeakyReLU functions. The LeakyReLU function is implemented using a lookup table, where the data is the quantized data from all possible data points during model training. Due to quantization, the ReLU function also requires zero-point data; the input data is compared with the zero-point data, and the corresponding result is output. The activation function can be selected through configuration parameters.

[0084] For example, the activation function calculation module consists of four ReLU function calculation units, four LeakyReLU function calculation units, and a function selector. The function selector, controlled by a selection signal from the coprocessor's decoding unit, is responsible for selecting the activation function type. Each ReLU function calculation unit is essentially a comparator; if the input data value is less than the output zero, the output data value is the output zero value; otherwise, the output data value equals the input data value. The LeakyReLU function calculation process is a lookup table process, using the input data value as an address to retrieve the data from the corresponding memory location and output it as the output data. Both architectures are relatively simple.

[0085] like Figure 11 The image shows the pooling module, which consists of four pooling computation units with a size distribution of 2×2. It is a max pooling type and supports configurable stride and feature map edge padding. The module has a dedicated row cache unit for pooling computation and related control units that convert the input serial data into two rows of parallel data. The control unit is responsible for sampling the input data according to specific parameter configurations. The sampled data is sent to the sliding window processing module to output 2×2 sliding window data.

[0086] The pooling computation unit consists of a pooling row cache module, a pooling sliding window module, and a pooling comparator. The pooling row cache unit is composed of a FIFO (FIFO file system), similar to the row cache module in convolution, but because the pooling size only supports 2×2, only one FIFO is used in this unit. The specific execution process is as follows: the first row of input data enters the FIFO. When the second row of data is input, the FIFO outputs the first row of input data, forming a parallel output with the current second row of input data. Simultaneously, the second row of input data is also input into the FIFO. The pooling sliding window unit consists of a controller and a 2×2 output window. The controller controls the data entering the 2×2 output window based on stride and padding parameters. For example, when the stride is set to 2, the controller uses a row counter to determine whether the parallel input data meets the stride requirement and controls the input enable. If the stride is odd, data input is allowed; if it is even, input enable is disabled. Simultaneously, the input data is sampled every other cycle and sent to the output window. When the stride is 1, input is always enabled, and data is sampled and sent to the output window every cycle. This module supports padding, but only for strides of 1 and bottom-right padding. The implementation is similar to convolution padding: when the last column of data is at the far left of the output window, zero-point data is added to the right side of the sliding window to achieve the effect of bottom-right padding. Once the last row of data has completely entered the FIFO, the pooling sliding window unit issues a read enable, causing the FIFO to output the last row of data, which appears at the top of the sliding window. The bottom of the window is then filled with zero-point data, completing the bottom-right padding of the 2×2 output window. The comparison unit consists of four comparison channels, also designed for pipelined operation. One comparison channel is responsible for comparing one 2×2 window of data. The specific implementation is as follows: First, the 32-bit wide sliding window data is split into four input channels, each with an 8-bit input width. Then, in the first clock cycle, the comparison channels perform pairwise comparisons to obtain two comparison numbers. In the second clock cycle, the two comparison numbers are compared to obtain the comparison result of the 2×2 window data.

[0087] This invention will be integrated into the Hummingbird e203 main processor core and deployed on the Gowin FPGA platform. The FPGA chip used is model GW5AST-LV138FPG676A. To test the computing power of this invention, the Hummingbird e203 main processor with this invention will undergo speed testing in a specific convolutional neural network model calculation scenario. The computing speed is represented by the program execution time. In this test, the e203's clock frequency is 10MHz, and the coprocessor's clock frequency is 100MHz. The table below compares the resource usage before and after integrating this invention.

[0088]

[0089]

[0090] like Figure 12 As shown, this scene represents the computation of one branch of the YOLOv3-Tiny model network, which consists of 10 layers. The specific parameter types and output feature map sizes for each layer can be seen in the figure. The specific runtime is shown in the figure. Figure 13 As shown, the computation time for the entire branch is 7.347ms.

[0091] The above description is merely a preferred embodiment of this application. It should be understood that this application is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or the technology or knowledge in related fields. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of this application should be within the protection scope of the appended claims.

Claims

1. A coprocessor based on RISC-V architecture, characterized in that, The coprocessor includes: The decoding unit receives the value of the source register sent by the main processor and decodes it to output the operation parameters. The system includes a control unit and a computation unit. The control unit controls the working process of the computation unit based on the operating parameters. The computation unit adopts a pipelined architecture consisting of a booth-encoded multiplier and a Wallace-type adder, and the calculation is performed in a non-blocking manner. The cross-clock transmission unit communicates with the main processor, including the transmission of configuration parameters and feedback signals, thereby supporting the main coprocessor to operate in different clock domains; A caching unit is used to cache some weight data and all bias data required in a single operation; the caching unit consists of a weight data caching module and a bias data caching module, which are used to cache weight data and bias data respectively; when performing convolution and quantization operations, the corresponding weight data and bias data are effectively output according to a preset reading order; The operation unit includes a line cache module, a convolution operation module, a pooling module, an activation function calculation module, a quantization module, and a data temporary storage module. Each module is completely decoupled and has a dedicated enable signal port and parameter input port. The line buffer module consists of two FIFOs and a parallel output 3×3 sliding window, which is used to convert serial data input into parallel multi-line data output. The convolution operation module includes two parts: a 1×1 convolution module and a 3×3 convolution module. In a single operation, each module can perform operations on four output channels simultaneously. During the output channel operation, the four input channels are also calculated simultaneously. The pooling module has a built-in row cache unit suitable for pooling calculations and related control logic, which converts the input serial data into two rows of parallel data. The control logic is responsible for sampling and outputting the input data according to the specific parameter configuration. The activation function calculation module supports ReLU function operations and LeakyReLU function operations, wherein the selection of the activation function is achieved through configuration parameters; The quantization module is used to quantize the convolutional result (over 8 bits) and the relevant input parameters, and convert the output result back into an 8-bit integer. The data temporary storage module is used to cache the accumulated results of all channels that have not yet been completed, and to superimpose the existing results with the cached data and then store them. When all channels have completed the calculation, the memory data is read out and added to the bias data.

2. The coprocessor based on the RISC-V architecture according to claim 1, characterized in that, The decoding unit decodes the value of the source register sent by the main processor, reads the operands of the source register, splits them, determines the specific operation represented by the current instruction, obtains the operation parameters required for the operation, and sends them to the control unit.

3. The coprocessor based on the RISC-V architecture according to claim 1, characterized in that, The control unit includes a memory management module and a computation management module; in, The computation management module dispatches computation tasks to the computation unit according to the operation parameters given by the decoding unit, and controls the working process of the computation unit by receiving feedback signals from the computation unit. The memory management module is responsible for managing data transmission between the coprocessor and external storage units, including calculating and allocating memory addresses and switching between memory read and write states.

4. A processing system based on a RISC-V architecture coprocessor, characterized in that, The system includes: Main processor; At least one external memory; And the coprocessor as described in any one of claims 1-3; The coprocessor responds to the extended instructions of the main processor and completes the computational tasks based on the extended instructions. The external memory serves as the memory medium for the coprocessor to access, process, and process data.

5. The processing system based on a RISC-V architecture coprocessor according to claim 4, characterized in that, The main processor includes a nice module for processing extended instructions, and the coprocessor is connected to the nice module to complete communication.

Images