Multi-precision vector computing device based on RISC-V

Abstract

This application relates to the field of digital signal processing technology and discloses a RISC-V-based multi-precision vector operation device. The device includes: an instruction decoding and scheduling module for decoding custom instructions and standard RVV extended instructions, connected to a vector loading and storage module and a channel scheduler, and sending the corresponding instructions; a vector loading and storage module for data interaction with external memory and vector registers via multicast or sequential processing; and a vector operation unit including a channel scheduler, an arithmetic unit, a vector register, and a systolic array unit, wherein the systolic array unit includes an operand requester, a data buffer, and a processing unit array for multi-precision, multi-mode operations of the vector operation unit. This application utilizes non-intrusive extended instructions of standard RVV to perform various operations, optimizes the storage and arithmetic unit structure to reduce redundant access frequency, and improves data utilization and arithmetic unit efficiency.

Description

Technical Field

[0001] The present application relates to the field of digital signal processing technology, and in particular to a multi-precision vector operation device based on RISC-V. Background Art

[0002] As the amount and complexity of information increases, the exponentially growing computing requirements in machine learning and artificial intelligence applications have become one of the great challenges for hardware design. Matrix multiplication is one of the cornerstones of machine learning and artificial intelligence, and is indispensable in applications such as scientific computing, graphics, and digital signal processing. As algorithm iteration and development speed continue to accelerate, overly specialized accelerators may face the problem of being eliminated. At the same time, even in computing centers, hardware resources, area resources, and computing time are still very important design factors.

[0003] The fifth generation (V) instruction set architecture of Reduced Instruction Set Computer-V (RISC-V) has become an important solution for both embedded systems and high-throughput servers with its simple, open source features and support for custom instructions. The large-scale deployment requirements of Deep Neural Network (DNN) have brought about a large amount of storage and computing costs. Although the emergence of the RISC-V Vector Extension (RVV) instruction set has solved the problem to a certain extent, the storage mapping and computing structure still have problems such as too frequent memory access and limited throughput.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present application, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] In order to provide a basic understanding of some aspects of the disclosed embodiments, a brief summary is given below. The summary is not an extensive review, nor is it intended to identify key / critical elements or delineate the scope of protection of these embodiments, but rather serves as a prelude to the detailed description that follows.

[0006] The present disclosure provides a multi-precision vector operation device based on RISC-V, the device comprising:

[0007] The instruction decoding and scheduling module is used to decode custom instructions and standard RVV extended instructions, and connect with the vector load storage module and the channel scheduler to send corresponding instructions;

[0008] Vector load and store module, used to interact with external memory and vector registers via multicast or sequential mode;

[0009] The vector operation unit includes a channel scheduler, an arithmetic unit, a vector register and a systolic array unit, wherein the systolic array unit includes an operand requester, a data buffer and a processing unit array, which is used for multi-precision and multi-mode operations of the vector operation unit.

[0010] In some embodiments, each processing unit includes 16 4-bit multiplication and addition units, and the multiplication and addition units are combined with each other. In one clock cycle, a single processing unit dynamically supports a maximum of 1 16-bit precision, or 4 8-bit precision, or 16 4-bit precision multiplication and addition operations.

[0011] In some embodiments, the vector operation unit is used to pre-process the matrix and weight to be processed along the input dimension;

[0012] For a convolution task with a kernel size of k×k, the processing unit array is first reconstructed based on the target parallelism;

[0013] If the task requirement is to complete an output of size j×j in one clock cycle, the processing unit array is reconstructed based on the task requirement so that each processing unit channel has a private j×j processing unit array to perform the computing task;

[0014] For a general convolution operation with a stride of 1, the vector load storage module requests a single-channel matrix to be processed with a size of (k+j-1)×(k+j-1) from the external memory each time.

[0015] In some embodiments, the vector operation unit is further used to preload weight data into the processing unit array;

[0016] In the first convolution operation, due to the limitation of the processing unit array dimension, the (k+j-1)×k elements of the j k×k blocks of the (k+j-1)×(k+j-1) to-be-processed matrix [0:k-1] columns are sent to the vector calculation module buffer in a column-first manner, and then the matrix operation is completed by the processing unit array in a systolic array manner.

[0017] In some embodiments, the next operation (k+j-1)×k data are pre-fetched from the register file into the vector calculation module buffer.

[0018] In some embodiments, for high-abstraction level tasks, the operation module adopts a dimension priority strategy to continue to pre-fetch data along the preset x dimension in the current matrix channel to be processed, so as to pre-fetch the corresponding elements of the [1:k] column of the current channel, wherein the high-abstraction level task is a task corresponding to a convolution kernel size greater than a threshold.

[0019] In some embodiments, the operation module adopts a channel priority strategy, jumps to the next matrix channel to be processed, and pre-fetches the same position in the new channel to pre-fetch (k+j-1)×k elements of the [0:k-1] column for the next operation, wherein the high-precision task is a task corresponding to a convolution kernel size smaller than a threshold.

[0020] In some embodiments, the apparatus further comprises a custom instruction storage module;

[0021] The custom instruction storage module stores four custom instructions and the RVV standard extended instruction set, which are used to control the working mode of the vector operation device;

[0022] The four custom instructions include a vector configuration instruction, an external memory to register load data instruction, a data write back to the external memory instruction and an instruction based on data parallelism;

[0023] Among them, the vector configuration instruction is used to configure data precision and data flow strategy. The instruction is encoded in the zimm and uimm domains:

[0024] Data parallelism-based instructions are used to call vector computing modules to perform matrix operations.

[0025] The RISC-V-based multi-precision vector operation device provided in the embodiments of the present disclosure can achieve the following technical effects:

[0026] By completing various operations through non-intrusive extended instructions based on standard RVV, the storage and operator structure are optimized to reduce the frequency of redundant access, improve data utilization and operator efficiency. At the same time, a high-parallelism hybrid data flow mode is designed based on the characteristics of convolution kernels of different tasks. This not only has scalable and high-parallelism data processing capabilities, but also the hybrid data flow and multi-precision mode support enables it to target a variety of computing tasks with different requirements, improve the computing efficiency of the computing unit and reduce resource consumption.

[0027] The above general description and the following description are exemplary and explanatory only and are not intended to limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] One or more embodiments are exemplarily described by corresponding drawings, which do not limit the embodiments. Elements with the same reference numerals in the drawings are shown as similar elements, and the drawings do not constitute a scale limitation, and wherein:

[0029] Figure 1 It is a hardware architecture diagram of a RISC-V-based multi-precision vector operation device provided in an embodiment of the present disclosure;

[0030] Figure 2 is a schematic diagram of a calculation data standardization provided by an embodiment of the present disclosure;

[0031] Figure 3 is a schematic diagram of a data flow strategy provided by an embodiment of the present disclosure;

[0032] Figure 4 It is a schematic diagram of a custom extended instruction set provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0033] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0034] The terms "first", "second", etc. in the embodiments of the present disclosure are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchanged where appropriate, so as to describe the embodiments of the present disclosure described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions.

[0035] Unless otherwise stated, the term "plurality" means two or more.

[0036] In the embodiment of the present disclosure, the character " / " indicates that the preceding and following objects are in an "or" relationship. For example, A / B indicates: A or B.

[0037] The term "and / or" is a description of the association relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or, A and B.

[0038] The term "correspondence" may refer to an association relationship or a binding relationship. The correspondence between A and B means that there is an association relationship or a binding relationship between A and B.

[0039] The RISC-V-based multi-precision vector operation device provided by the embodiments of the present disclosure is described below with reference to the accompanying drawings.

[0040] Figure 1 This is a hardware architecture diagram of a RISC-V-based multi-precision vector operation device provided in an embodiment of the present disclosure.

[0041] like Figure 1As shown, a multi-precision vector operation device based on RISC-V may include:

[0042] The instruction decoding and scheduling module is used to decode custom instructions and standard RISC-V Vector Extension (RVV) instructions, and connect with the vector load and store module and the channel scheduler to send the corresponding instructions;

[0043] Vector load and store module, used to interact with external memory and vector registers via multicast or sequential mode;

[0044] The vector operation unit includes a channel scheduler, an arithmetic unit, a vector register and a systolic array unit, wherein the systolic array unit includes an operand requester, a data buffer and a processing unit array, which is used for multi-precision and multi-mode operations of the vector operation unit.

[0045] In the present disclosure, various operations are completed by non-intrusive extended instructions based on standard RVV, and the storage and operator structures are optimized to reduce the redundant access frequency and improve the data utilization and operator efficiency.

[0046] In some embodiments, each processing unit includes 16 4-bit multiplication and addition units, and the multiplication and addition units are combined with each other. In one clock cycle, a single processing unit dynamically supports a maximum of 1 16-bit precision, or 4 8-bit precision, or 16 4-bit precision multiplication and addition operations.

[0047] In some embodiments, the vector operation unit is used to pre-process the matrix and weight to be processed along the input dimension;

[0048] For a convolution task with a kernel size of k×k, the processing unit array is first reconstructed based on the target parallelism;

[0049] If the task requirement is to complete an output of size j×j in one clock cycle, the processing unit array is reconstructed based on the task requirement so that each processing unit channel has a private j×j processing unit array to perform the computing task;

[0050] For a general convolution operation with a stride of 1, the vector load storage module requests a single-channel matrix to be processed with a size of (k+j-1)×(k+j-1) from the external memory each time.

[0051] In some embodiments, the vector operation unit is further used to preload weight data into the processing unit array;

[0052] In the first convolution operation, due to the limitation of the processing unit array dimension, the (k+j-1)×k elements of the j k×k blocks of the (k+j-1)×(k+j-1) to-be-processed matrix [0:k-1] columns are sent to the vector calculation module buffer in a column-first manner, and then the matrix operation is completed by the processing unit array in a systolic array manner.

[0053] In some embodiments, the next operation (k+j-1)×k data are pre-fetched from the register file into the vector calculation module buffer.

[0054] In some embodiments, for high-abstraction level tasks, the operation module adopts a dimension priority strategy to continue to pre-fetch data along the preset x dimension in the current matrix channel to be processed, so as to pre-fetch the corresponding elements of the [1:k] column of the current channel, wherein the high-abstraction level task is a task corresponding to a convolution kernel size greater than a threshold, wherein the x dimension is the row direction.

[0055] In some embodiments, for high-precision tasks, the operation module adopts a channel priority strategy, jumps to the next matrix channel to be processed, and pre-fetches the same position in the new channel to pre-fetch (k+j-1)×k elements of the [0:k-1] column for the next operation, wherein the high-precision task is a task corresponding to a convolution kernel size smaller than a threshold.

[0056] In some embodiments, Figure 1 The device may also include a custom instruction storage module;

[0057] The custom instruction storage module stores four custom instructions and the RVV standard extended instruction set, which are used to control the working mode of the vector operation device;

[0058] The four custom instructions include a vector configuration instruction, an external memory to register load data instruction, a data write back to the external memory instruction and an instruction based on data parallelism;

[0059] Among them, the vector configuration instruction is used to configure the data precision and data flow strategy. The instruction is encoded in the zero-extended immediate (zimm) and unsigned immediate (uimm) fields:

[0060] Data parallelism-based instructions are used to call vector computing modules to perform matrix operations.

[0061] Figure 2 is a schematic diagram of a normalized operation data provided by an embodiment of the present disclosure, Figure 3 is a schematic diagram of a data flow strategy provided by an embodiment of the present disclosure, Figure 4It is a schematic diagram of a custom extended instruction set provided by an embodiment of the present disclosure.

[0062] Combination Figure 2-Figure 4 ,right Figure 1 The processing logic of the multi-precision vector operation device based on RISC-V is further explained in the following.

[0063] Specifically, the instruction decoding and scheduling module is used to decode custom instructions and standard RVV extension instructions. The vector load storage module is used to exchange data with external memory and vector registers in a multicast or sequential manner. The main computing components of the entire vector operation unit are composed of a channel scheduler, an arithmetic logic unit (ALU), a vector register, and a synchronous array unit (SAU). SAU is the core component of the entire vector operation unit that supports multi-precision and multi-mode operations. It consists of an operand requester, a data buffer, and a processing element (PE) array. It should be noted that the PE array is a reconfigurable processing unit array, and its specific topology and dimensions can be adjusted according to computing requirements. Each PE consists of 16 4-bit multiply-accumulate units (MAC) units, which can be combined with each other and flexibly scheduled. Therefore, in one clock cycle, a single PE can dynamically support up to 1 16-bit precision, or 4 8-bit precision, or 16 4-bit precision MAC operations.

[0064] In order to make full use of limited computing resources, the technical solution disclosed in this disclosure standardizes the data accuracy on the computing end, such as Figure 2 As shown. Specifically, all matrices and weights to be processed are preprocessed along the input dimension. For 16-bit, 8-bit, and 4-bit precision modes, adjacent 1, 4, and 16 operands are combined into a basic operation element. In the following data flow discussion, the basic unit of data flow: element refers to the operation element standardized based on the data precision.

[0065] For convolution computing tasks, the technical solution disclosed in this disclosure divides them into two categories based on the size of the convolution kernel: high-abstraction tasks with larger convolution kernels and high-precision tasks with smaller convolution kernels. For high-abstraction tasks, since their convolution kernels are larger, more data will be available for reuse in the sliding of the convolution block, while the data reuse rate of high-precision tasks is relatively lower. In this context, the technical solution disclosed in this disclosure designs different data flow strategies for the two tasks to flexibly handle different computing tasks.

[0066] For a convolution task with a convolution kernel size of k×k, the PE array is first reconstructed based on the target parallelism. If the task requires the completion of j×j outputs in one clock cycle, the PE array is reconstructed based on this requirement so that each PE channel has a private j×j PE array to perform the calculation task. For a general convolution operation with a step size of 1, the vector load storage module requests a single channel size of (k+j-1)×(k+j-1) to be processed matrix from the external storage each time. Note that PE channels and matrix channels to be processed are two completely different concepts. PE channels are determined by hardware resources and parallelism requirements, and only affect the efficiency of the operator. There is no direct relationship between the two.

[0067] Specific data flow such as Figure 3 As shown, the weight data is preloaded into the PE array. In the first convolution operation, limited by the PE array dimension, the (k+j-1)×k elements of the j k×k blocks of the (k+j-1)×(k+j-1) to-be-processed matrix [0:k-1] column are sent to the vector calculation module buffer in a column-first manner, and then the PE array completes the matrix operation in a systolic array manner.

[0068] During the calculation process of the first convolution operation, (k+j-1)×k data of the next operation will be pre-fetched from the register file into the calculation module buffer.

[0069] For high-level abstraction tasks, the operation module adopts a dimension-first strategy, that is, it continues to pre-fetch data along the x dimension in the current matrix channel to be processed, that is, it pre-fetches the corresponding elements of the [1:k] column of the current channel. Compared with the elements that have been accessed in the first convolution operation, all the elements of [1:k-1] have been loaded and can be reused. Therefore, for the second convolution operation, the dimension-first strategy only needs to pre-fetch the data of the next column, which ensures high data reuse and reduces the demand for expensive storage resources of the cache. After the calculation is completed, the process results of a single channel will be sent back to the buffer for storage, waiting for subsequent data from all channels.

[0070] For high-precision tasks, the operation module adopts a channel priority strategy, that is, it jumps to the next matrix channel to be processed and pre-fetches the (k+j-1)×k elements of the same position in the new channel, that is, the [0:k-1] column for the next operation. Although under this strategy, the required data for the next operation needs to be pre-fetched and data reuse is not utilized, the data access pressure is not too large due to the small size of the convolution kernel in high-precision tasks. At the same time, since the data at the same position in different channels has a higher correlation, the channel priority strategy can output the calculation results faster, reduce the memory usage of the process results, and avoid unnecessary output consumption.

[0071] In order to support the multi-precision and multi-data flow strategy operation mode of this arithmetic unit, this solution proposes four custom instructions, which together with the RVV standard extended instruction set control the working mode of the arithmetic unit. Figure 4 As shown, the Vector Configuration Instruction (VCCFG) is used to configure data precision and data flow strategy. The instruction is encoded in the zimm and uimm domains. Vector Load (VCLD) and Vector Store (VCST) are used to load data from external memory to registers or write data back to external memory. Vector Compare (VCCOMP) is a customized data parallelism-based instruction used to call the vector calculation module to perform matrix operations.

[0072] The multi-precision vector operation device based on RISC-V provided by the embodiment of the present disclosure completes various operations through non-intrusive extended instructions based on standard RVV, optimizes the storage and operator structure to reduce the frequency of redundant access, and improves data utilization and operator efficiency. At the same time, a high-parallelism hybrid data flow mode is designed based on the characteristics of convolution kernels of different tasks. This not only has scalable, high-parallelism data processing capabilities, but also the hybrid data flow and multi-precision mode support enables it to target a variety of computing tasks with different requirements, improves the computing efficiency of the computing unit and reduces resource consumption.

[0073] The above description and the accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, individual components and functions are optional, and the order of operations may vary. Parts and features of some embodiments may be included in or replace parts and features of other embodiments. As used in the description of the embodiments, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of one or more associated listings. In addition, when used in this application, the term "comprise" and its variants "comprises" and / or including (comprising) refer to the existence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the existence or addition of one or more other features, wholes, steps, operations, elements, components and / or these groups. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the existence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same or similar parts between the embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

[0074] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software may depend on the specific application and design constraints of the technical solution. Technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present disclosure. Technicians can clearly understand that for the convenience and simplicity of description, the specific working process of the systems, devices and units described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0075] In the embodiments disclosed herein, the disclosed methods and products (including but not limited to devices, equipment, etc.) can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units can be only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to implement this embodiment. In addition, each functional unit in the embodiment of the present disclosure may be integrated in a processing unit, or each unit may exist physically alone, or two or more units may be integrated in one unit.

[0076] The flowchart and block diagram in the accompanying drawings show the possible architecture, functions and operations of the system, method and computer program product according to the embodiment of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of the code, and a part of the module, program segment or code contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. In the description corresponding to the flowchart and block diagram in the accompanying drawings, the operations or steps corresponding to different boxes can also occur in a different order from the order disclosed in the description, and sometimes there is no specific order between different operations or steps. For example, two consecutive operations or steps can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, which can depend on the functions involved. Each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a dedicated hardware-based system that performs the specified functions or actions, or may be implemented by a combination of dedicated hardware and computer instructions.

Claims

1. A multi-precision vector computing device based on RISC-V, characterized in that: The device comprises: The instruction decoding and scheduling module is used to decode custom instructions and standard RVV extended instructions, and connect with the vector load storage module and the channel scheduler to send corresponding instructions; Vector load and store module, used to interact with external memory and vector registers via multicast or sequential mode; A vector operation unit, comprising a channel scheduler, an arithmetic unit, a vector register and a systolic array unit, wherein the systolic array unit comprises an operand requester, a data buffer and a processing unit array, and is used for multi-precision and multi-mode operations of the vector operation unit; Wherein, the vector operation unit is used to preprocess the matrix and weight to be processed along the input dimension; For a convolution task with a kernel size of k×k, the processing unit array is first reconstructed based on the target parallelism; If the task requirement is to complete an output of size j×j in one clock cycle, the processing unit array is reconstructed based on the task requirement so that each processing unit channel has a private j×j processing unit array to perform the computing task; For a general convolution operation with a stride of 1, the vector load storage module requests a single-channel matrix of size (k+j-1)×(k+j-1) to be processed from the external memory each time; The vector operation unit is also used to preload weight data into the processing unit array; In the first convolution operation, limited by the dimension of the processing unit array, the (k+j-1)×k elements of the j k×k blocks of the (k+j-1)×(k+j-1) to-be-processed matrix [0:k-1] column are sent to the vector calculation module buffer in a column-first manner, and then the matrix operation is completed by the processing unit array in a systolic array manner; The device also includes a custom instruction storage module; The custom instruction storage module stores four custom instructions and the RVV standard extended instruction set, which are used to control the working mode of the vector operation device; The four custom instructions include a vector configuration instruction, an external memory to register load data instruction, a data write back to the external memory instruction and an instruction based on data parallelism; Wherein, the vector configuration instruction is used to configure data precision and data flow strategy, and the vector configuration instruction is encoded in the zimm and uimm domains; Data parallelism-based instructions are used to call vector computing modules to perform matrix operations.

2. The device according to claim 1, characterized in that Each processing unit includes 16 4-bit multiplication and addition units, which are combined with each other. In one clock cycle, a single processing unit dynamically supports up to 1 16-bit precision, or 4 8-bit precision, or 16 4-bit precision multiplication and addition operations.

3. The device according to claim 1, characterized in that The next operation (k+j-1)×k data will be pre-fetched from the register file into the vector calculation module buffer.

4. The device according to claim 1, characterized in that For high-abstraction-level tasks, the operation module adopts a dimension priority strategy and continues to pre-fetch data along the preset x dimension in the current matrix channel to be processed, so as to pre-fetch the corresponding elements of the [1:k] column of the current channel, wherein the high-abstraction-level task is a task corresponding to a convolution kernel size greater than a threshold.

5. The device according to claim 1, characterized in that For high-precision tasks, the operation module adopts a channel priority strategy, jumps to the next matrix channel to be processed, and pre-fetches the same position in the new channel to pre-fetch (k+j-1)×k elements of the [0:k-1] column for the next operation, wherein the high-precision task is a task corresponding to a convolution kernel size smaller than a threshold.

Images