Data processing device, data processing system, and chip
By using the combination of RISC-V extended instructions and multiple data processing units on the MCU platform, the problems of AI processing complexity and power consumption of the MCU platform are solved, and efficient data processing and computing resource multiplexing is achieved, suitable for TinyML inference.
Patent Information
- Application Number
- CN202411147979.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-20
- Publication Date
- 2024-11-08
AI Technical Summary
When implementing high-performance AI processing and low-power edge computing on the MCU platform, heterogeneous computing methods increase the complexity of software implementation, and the coordination capabilities of software and hardware are insufficient, especially in convolutional computing and quantitative inverse quantization, etc. There is a problem of low utilization rate of computing units.
Using RISC-V-based extension instructions, data transmission and parameter configuration are realized through the data transfer module and configuration module, data calculation processing is carried out in combination with multiple data processing units, and data processing efficiency and multiplexing rate of computing resources are improved through the data transmission interface. , simplify the software and hardware interface.
It improves the coordination ability between software and hardware, improves instruction efficiency and computing efficiency, simplifies the software and hardware interface, improves the utilization rate of computing units and data processing efficiency, and is suitable for TinyML inference.
Smart Images

Figure CN118916309A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of data processing, and in particular, to a data processing device, a data processing system and a chip. Background Art
[0002] With the continuous advancement of technology, some emerging application scenarios have also put forward new requirements for MCU (Microcontroller Unit), which requires certain AI (Artificial Intelligence) functions. Running artificial intelligence and machine learning based on the MCU platform and developing edge computing with higher performance and lower power consumption are becoming industry hotspots. For example, integrating NPU (Neural Network Processing Unit) in MCU to improve AI processing capabilities. However, this heterogeneous computing method increases the complexity of software implementation, and the coordination ability between software and hardware is insufficient. Summary of the invention
[0003] In view of this, embodiments of the present application provide a data processing device, a data processing system, and a chip.
[0004] On the one hand, an embodiment of the present application provides a data processing device, including:
[0005] A decoding module, used for receiving extended instructions based on RISC-V (Reduced Instruction Set Computer-Five, an open source instruction set architecture); wherein the extended instructions include: data movement instructions and configuration instructions;
[0006] A data transfer module, used for receiving the data transfer instruction from the decoding module, and accessing the corresponding data transfer interface based on the data transfer instruction to perform data transfer;
[0007] A configuration module, used to receive the configuration instruction from the decoding module and perform parameter configuration based on the configuration instruction;
[0008] A data processing module, used for performing data calculation and processing according to the data moving instruction and the configuration instruction;
[0009] One or more data transmission interfaces are respectively connected to the data processing module and the data moving module, and are used to transmit corresponding data according to the data moving instruction.
[0010] In some embodiments, the data processing module includes multiple data processing units, and different data processing units among the multiple data processing units are used to perform different data operations on the data.
[0011] In some embodiments, the data processing device further includes:
[0012] A data operation module is connected to the data processing module and is used to execute the data operation in the data operation processing of the data processing module.
[0013] In some embodiments, the data operation module includes one or more of the following units:
[0014] A multiplication array unit, used to implement the multiplication operation in the data operation;
[0015] An addition array unit, used to implement the addition operation in the data operation;
[0016] A format conversion unit is used to implement the format conversion of the data.
[0017] In some embodiments, the data operation module includes: reusing the ALU (Arithmetic and Logic Unit) in the CPU (Central Processing Unit).
[0018] In some embodiments, the data processing module includes one or more of the following data processing units:
[0019] A dequantization unit, used for reading data and control information through the data transmission interface, and requesting to use the data operation module to perform data operation;
[0020] A dot product unit, used for receiving the data provided by the dequantization unit and the control information, reading an enable signal through the data transmission interface, and requesting to use the data operation module to perform data operation;
[0021] an accumulation unit, configured to receive the data provided by the dot product unit and the control information, read the cache according to the control information, and request to use the data operation module to perform data operation;
[0022] A bias unit, configured to receive the data provided by the accumulating unit and the control information, read an enable signal through the data transmission interface according to the control information, and request to use the data operation module to perform data operation;
[0023] an activation unit, configured to receive the data provided by the bias unit and the control information, and perform activation processing according to the control information;
[0024] The quantization unit is used to receive the data and the control information provided by the activation unit, request the data operation module to perform data processing, and output the processed data to the data moving module through the data transmission interface.
[0025] In some embodiments, the data transmission interface includes:
[0026] A first interface connecting the data moving module and the dequantization unit;
[0027] A second interface connecting the data moving module and the dot product unit;
[0028] A third interface connecting the data moving module and the bias unit;
[0029] A fourth interface connecting the data moving module and the quantization unit;
[0030] a fifth interface connecting the data moving module and the quantization unit;
[0031] And a sixth interface connected to the data migration module.
[0032] In some embodiments, the first interface, the second interface, the third interface, the fourth interface and the sixth interface are respectively used to read the first data on the external memory through the data moving module; the fifth interface is used to write the second data processed by the quantization unit to the external memory through the data moving module.
[0033] In some embodiments, the first data includes one or more of the following:
[0034] address parameters; eigenvalue parameters; weight parameters; bias parameters; quantization coefficients; and operation data for the data processing.
[0035] In some embodiments, during the data processing by the data operation module, the weight parameter, the bias parameter or the quantization coefficient remains unchanged within a preset time period.
[0036] In some embodiments, the data migration module loads the characteristic value parameters by broadcasting or row sliding;
[0037] The broadcasting method includes: loading the same characteristic value parameters to multiple data processing units of the data processing module;
[0038] The row sliding method includes: sequentially loading and shifting characteristic value parameters of designated bits to different data processing units.
[0039] On the other hand, an embodiment of the present application further provides a data processing system, including:
[0040] RISC-V based CPU;
[0041] An NPU connected to the CPU; wherein the NPU includes: any of the above-mentioned data processing devices.
[0042] In some embodiments, the data processing system further comprises:
[0043] A bus unit connecting the CPU and the NPU;
[0044] A memory connected to the bus unit; wherein the memory includes: an on-chip memory and / or an off-chip memory.
[0045] In some embodiments, the NPU includes:
[0046] VCU (Vector Computing Unit);
[0047] LSU (Lift Station Unit, data handling unit); and
[0048] TCU (Convolutional Computation Processing Unit); wherein the TCU includes the data processing device.
[0049] In some embodiments, the NPU further comprises:
[0050] decoder; cache memory; and buffer memory.
[0051] On the other hand, an embodiment of the present application further provides a chip, which includes any of the data processing systems described above.
[0052] The embodiment of the present application utilizes RISC-V extended instructions to realize data movement and parameter configuration, utilizes the hardware structure of the data processing module to realize data calculation processing, and realizes data transmission between the data processing module and the data movement module through one or more data transmission interfaces. On the one hand, RISC-V extended instructions have the characteristics of high efficiency, strong scalability and wide application range, so it can effectively improve the coordination ability of software and hardware, improve instruction efficiency, and simplify the software and hardware interface. On the other hand, through the hardware implementation of the above-mentioned data transmission interface and data movement module, it is possible to effectively improve data movement and computing efficiency and improve the reuse rate of computing resources. BRIEF DESCRIPTION OF THE DRAWINGS
[0053] Figure 1 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;
[0054] Figure 2A schematic diagram of the structure of another data processing device provided in an embodiment of the present application;
[0055] Figure 3A A schematic diagram of the structure of another data processing device provided in an embodiment of the present application;
[0056] Figure 3B A schematic diagram of two eigenvalue loading methods provided in an embodiment of the present application;
[0057] Figure 4 A schematic diagram of the structure of another data processing system provided in an embodiment of the present application;
[0058] Figure 5 A schematic diagram of the structure of another data processing system provided in an embodiment of the present application;
[0059] Figure 6 A schematic diagram of the structure of a chip provided in an embodiment of the present application. DETAILED DESCRIPTION
[0060] The following will be combined with the embodiments of the present application and the accompanying drawings to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0061] In the following description, a large number of specific details are given to provide a more thorough understanding of the present application. However, it is obvious to those skilled in the art that the present application can be implemented without one or more of these details. In other examples, in order to avoid confusion with the present application, some technical features known in the art are not described; that is, all features of the actual embodiments are not described here, and well-known functions and structures are not described in detail.
[0062] The purpose of the terms used herein is only to describe specific embodiments and is not intended to be limiting of the present application. When used herein, the singular forms "one", "an" and "said / the" are also intended to include plural forms, unless the context clearly indicates otherwise. It should also be understood that the terms "consisting of" and / or "comprising", when used in this specification, determine the presence of the features, integers, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, parts and / or groups. When used herein, the term "and / or" includes any and all combinations of the relevant listed items.
[0063] In order to thoroughly understand the present application, detailed steps and detailed structures will be presented in the following description to illustrate the technical solution of the present application. The preferred embodiments of the present application are described in detail below, but in addition to these detailed descriptions, the present application may also have other implementation methods.
[0064] The modularity and scalability of RISC-V make it easy to implement DSA (domain specific architecture) that combines MCU with AI in embedded scenarios based on its architecture.
[0065] RISC-V is an ISA (Instruction Set Architecture) based on RISC (Reduced Instruction Set Computer). This instruction set can be freely applied to any purpose, allowing anyone to design, manufacture and use it. It is suitable for modern computer devices, including cloud computers, mobile phones and tiny embedded systems, and has the advantages of simple and efficient architecture, strong scalability and wide application range.
[0066] As an important part of AI computing, the convolution operator and convolution hardware acceleration unit can be integrated into the RISC-V kernel or implemented as a RISC-V coprocessor. However, there are still many deficiencies in the design of software-hardware collaboration, such as the implementation of "depthwise" (deep convolution), quantization and dequantization, etc. However, these aspects are very necessary for TinyML (Tiny Machine Learning). In addition, in embedded scenarios, there are higher requirements for area and power consumption, and the convolution computing units designed based on RISC-V often have the problem of low computing unit utilization.
[0067] Based on this, the embodiment of the present application provides a data processing device, such as Figure 1 As shown, the data processing device 100 includes:
[0068] The decoding module 110 is used to receive extended instructions based on RISC-V; wherein the extended instructions include: data movement instructions and configuration instructions;
[0069] The data moving module 120 is used to receive the data moving instruction from the decoding module, and access the corresponding data transmission interface based on the data moving instruction to perform data transmission;
[0070] A configuration module 130, configured to receive the configuration instruction from the decoding module and perform parameter configuration based on the configuration instruction;
[0071] A data processing module 140, configured to perform data calculation and processing according to the data moving instruction and the configuration instruction;
[0072] One or more data transmission interfaces 150 are respectively connected to the data processing module 140 and the data moving module 120, and are used to transmit corresponding data according to the data moving instruction.
[0073] In an embodiment of the present application, RISC-V-based extended instructions are used, which include two types of instructions, namely the above-mentioned data movement instructions and configuration instructions. The decoding module receives instructions from the CPU extension interface, decodes the control signals corresponding to the above-mentioned data movement instructions and configuration instructions, and provides them to the data movement module 120 and the configuration module 130 respectively. Exemplarily, the above-mentioned extended instructions can be R-type extended instructions, that is, instructions without immediate numbers in the instruction format, and all operands are stored in registers. Exemplarily, the specific instruction format of the above-mentioned extended instructions is shown in Table 1 below:
[0074]
[0075]
[0076] Table 1
[0077] Among them, taking a 32-bit instruction as an example, the above table indicates the information represented by each bit of the instruction from MSB (Most Significant Bit) to LSB (Least Significant Bit): func7 represents the function of the instruction, rs1 represents the first source register, rs2 represents the second source register, rd represents the read address, rd_en represents whether rd is valid; rs1_en represents whether rs1 is valid, rs2_en represents whether rs2 is valid; opcode represents the operation code. Us (User Setting) means that the value in the specified domain is a fixed value. It can be seen from Table 1 above that the fixed value "0" is set in the "rd_en" domain and the "rd" domain, and the fixed value "Custom" is set in the "opcode" domain.
[0078] The 7-bit data in the above func7 can represent various functions. For details, see the following
[0079] Table 2:
[0080] func7 rs1 en1 rs2 en2 1 cfg0 cfg1 2 cfg2 cfg2 3 cfg4 cfg3 4 cfg6 cfg4 5 cfg8 cfg5 6 7 8 9 10 Weight_addr Bias_addr 11 Feature_addr Output_addr 12 Other_addr0 Other_addr1
[0081] Table 2
[0082] In the above table, "cfg" 0 to 8 represent configuration information, "Weight_addr" represents the weight parameter address; "Bias_addr" represents the bias parameter address; "Feature_addr" represents the feature value address; "Output_addr" represents the output address, and "Other_addr" represents other addresses. The parameters in the above table are only examples. In actual applications, they can be set according to requirements.
[0083] The data moving module 120 reads various required parameters from the external memory through an access interface, such as a dual data access interface connected to the external memory, according to the control signal corresponding to the data moving instruction provided by the decoding module, and then writes them to the corresponding data transmission interface 150.
[0084] The configuration module 130 may be a configuration register (CSR register), which may include several registers for storing various parameters, and the configuration module 130 may generate control signals to various processing units in the data processing module 140 according to the values of these registers.
[0085] In an embodiment of the present application, the data transmission interface 150 can be a FIFO (First Input First Output), for example, OAFIFO (Output Address FIFO), ODFIFO (Output Data FIFO), WFIFO (Weight FIFO), QFIFO (Quantization FIFO), and the like. Figure 1 and Figure 2 In the figure, the number of data transmission interfaces 150 is only an example. In practical applications, according to the actual needs of data processing, the number and type of data transmission interfaces 150 may be different from those in the figure, and this application does not limit it.
[0086] The data moving module 120 can call various parameters or data according to the data moving instruction, and provide them to the data processing module 140 through the corresponding data transmission interface 150, or output the data processed by the data processing module 140 to the external memory.
[0087] The above configuration instructions can be used to implement the configuration of various parameters, for example, to update the parameters of various custom registers. Exemplarily, it can be used to configure the convolution size, output image size, etc. Through these configuration information, the hardware control of the convolution calculation process can be completed. The data movement instruction can include information on data movement, such as loading instructions for eigenvalues, data transmission instructions, and can also include instructions for starting calculations, loading instructions for weight parameters, bias parameters, etc. In addition, it can also include clearing instructions for old weight parameters and old bias parameters. This efficient extended instruction can effectively simplify the hardware and software interface and improve the utilization of the computing unit.
[0088] In some embodiments, Figure 2 As shown, the data processing module 140 includes a plurality of data processing units 141, and different data processing units among the plurality of data processing units are used to perform different data operations on the data.
[0089] These data processing units may be various operations or processing units for implementing convolution processing. Each data processing unit may be used to implement data processing once and provide the processed data to the next processing unit.
[0090] It should be noted that different processing units can be connected to the data moving module 120 through the above different data transmission interfaces 150 so as to load the required parameters or data when the processing module performs data processing, for example, weight parameters, bias parameters, eigenvalue parameters, etc.
[0091] In this way, data processing efficiency can be effectively improved and power consumption caused by data transmission can be saved.
[0092] In an exemplary embodiment, multiple data transmission interfaces 150 and multiple data processing units 141 may have a one-to-one correspondence, that is, each data transmission interface is connected to a data processing unit. In another embodiment, one data transmission interface 150 may be connected to multiple data processing units 141, or multiple data transmission interfaces 150 may be connected to the same data processing unit.
[0093] In some embodiments, continue to refer to Figure 2 , the data processing device 100 further includes:
[0094] The data operation module 160 is connected to the data processing module 140 and is used to execute the data operation in the data operation processing of the data processing module 140 .
[0095] In the embodiment of the present application, the data processing performed by different data processing units 141 may include some of the same data operations, such as array multiplication operations, addition operations, etc. Therefore, a data operation module 160 may be set for these operations, and the module includes a hardware structure for performing fixed data operations. Different data processing units 141 can call the data operation module 160.
[0096] Specifically, when different data processing units 141 perform each step of processing, they can respectively use the same hardware structure of the data operation module 160 to perform operations such as multiplication or addition. Therefore, under high performance, each step of data processing can be implemented in the form of a pipeline.
[0097] In this way, the reuse rate of data computing resources can be effectively improved and the efficiency of data processing can be improved.
[0098] In some embodiments, continue to refer to Figure 2 The data operation module 160 includes one or more of the following units:
[0099] A multiplication array unit 161, used to implement the multiplication operation in the data operation;
[0100] An addition array unit 162, used to implement the addition operation in the data operation;
[0101] The format conversion unit 163 is used to implement the format conversion of the data.
[0102] Different data processing units 141 can be connected to the above-mentioned units in the data operation module 160 respectively. The multiplication array unit 161 performs multiplication operation on the received data array separately and returns the operation result to the corresponding data processing unit 141; the addition array unit 162 performs addition operation on the received data array separately and returns the operation result to the corresponding data processing unit 141; the format conversion unit 163 performs format conversion processing on the received data array separately and returns the operation result to the corresponding data processing unit 141.
[0103] Specifically, the multiplication array unit 161 may include several multiplication units; the addition array unit 162 may include several addition units; and the format conversion unit 163 is for format conversion from high bit to low bit data in order to achieve data quantization accuracy.
[0104] In some embodiments, the data operation module includes: reusing the ALU in the CPU.
[0105] That is to say, when a certain operation (such as addition or multiplication) is required in the data processing performed by the data processing module 140, the data to be processed can be directly transmitted to the ALU in the CPU, and the ALU performs the corresponding operation and then transmits it back to the corresponding data processing unit 141. In this way, the hardware structure can be further simplified and the reuse rate of computing resources can be improved.
[0106] In some embodiments, Figure 3A As shown, the data processing module 140 includes one or more of the following data processing units:
[0107] The dequantization unit 301 is used to read data and control information through the data transmission interface 150 and request to use the data operation module 160 to perform data operation;
[0108] A dot product unit 302, configured to receive the data and the control information provided by the dequantization unit 301, read an enable signal through the data transmission interface 150, and request to use the data operation module 160 to perform data operation;
[0109] The accumulation unit 303 is used to receive the data and the control information provided by the dot product unit 302, read the cache 304 according to the control information, and request to use the data operation module 160 to perform data operation; the cache 304 is used to temporarily store the accumulated data;
[0110] A bias unit 305, configured to receive the data and the control information provided by the accumulating unit 303, read an enable signal through the data transmission interface 150 according to the control information, and request to use the data operation module 160 to perform data operation;
[0111] an activation unit 306, configured to receive the data and the control information provided by the bias unit 305, and perform activation processing according to the control information;
[0112] The quantization unit 307 is used to receive the data and the control information provided by the activation unit 306 , request the data operation module 160 to process the data, and output the processed data to the data moving module 120 through the data transmission interface 150 .
[0113] Specifically, in an embodiment of the present application, the above-mentioned decoding module 110 decodes the corresponding control signal after receiving the instruction from the CPU expansion interface and sends it to the data moving module 120 and the configuration module 130. The data moving module reads the characteristic value, weight parameter, bias parameter and other data from the external memory through the dual data access interface according to the control signal provided by the decoding module, and then writes it to each FIFO. Alternatively, the data moving module 120 reads the output information transmitted by the designated data transmission interface 150 according to the control signal of the decoding module 110, and writes it to the external memory. Exemplarily, the data moving module 120 writes the output information, such as the address and other information, into the OAFIFO, waits for the data processing module 140 to perform the corresponding processing, and then transmits the output data to the ODFIFO. At this time, the data moving module 120 detects that the ODFIFO is not empty, then reads the OAFIFO and ODFIFO China mainland data, and writes it to the external memory, thereby realizing the output of the data.
[0114] The dequantization unit 301 reads the data and corresponding control information provided by the designated data transmission interface 150, and requests the use of the data operation module 160 to perform the required operations. After completing the dequantization process, the output data and control information are sent to the next step of the dot product unit 302. Here, dequantization is the inverse operation relative to quantization, and quantization processing refers to the adjustment of the numerical range, for example, the continuous signal is taken as a finite number of discrete values; dequantization processing can reduce the distortion caused by quantization, so that the model performance is close to the unquantized state. Exemplarily, the data transmission interface 150 connected to the dequantization unit can be a FFIFO (Feature FIFO, eigenvalue FIFO) interface.
[0115] The dot product unit 302 receives the data and control information provided by the inverse quantization unit 301, and sends a read enable signal to the designated data transmission interface 150 according to the control information to read the required data, and requests to use the data operation module 160. After the calculation is completed, the data is sequentially sent to the next step accumulation unit 303. Here, the data transmission interface 150 connected to the dot product unit 302 can be a WFIFO interface.
[0116] The accumulation unit 303 receives the data and control information provided by the dot product unit 302, and reads and writes the cache 304 according to the control information. The cache 304 is used to temporarily store the accumulated data, thereby reducing the data transfer of the intermediate values, thereby reducing the loss caused by data transfer. The accumulation unit 303 also implements the operation by using the data operation module 160, and after completing the operation, the result is sent to the bias unit 305 of the next step.
[0117] The bias unit 305 receives the data and control information provided by the accumulation unit 303, and reads the data of the specified data transmission interface 150, such as BFIFO, including bias parameters, etc. At the same time, the bias unit 305 can also request to use the calculation unit, and after completing the calculation, send the data to the next activation unit 306 in sequence. The activation unit 306 can be implemented by an activation function, such as RELU (Rectified Linear Unit). After completing the activation process, the bias unit 305 sends the data to the quantization unit 307.
[0118] The quantization unit 307 receives the data processed by the bias unit 305 and requests the use of the data operation module 160 to output the processed data through the ODFIFO.
[0119] In this way, the convolution operation acceleration processing of the data is completed by using the data processing module 140. In the embodiment of the present application, the quantization unit and the dequantization unit are added, which can reduce the requirements for external storage capacity. In addition, it should be noted that the above process is to explain in detail the processing method and data transmission path between each unit. In practical applications, the data processing module 140 may also only include some of the above processing units, and output after completing some of the processing functions. Alternatively, the data processing module 140 may also include other processing units other than the above processing units, and set the order and function of each processing unit according to actual needs. In other words, although the data transmission and connection relationship between the various processing units are described in the above embodiments, in practical applications, these data transmission and connection relationships can be set according to needs, and the settings of each unit in the data processing module 140 belong to the protection scope covered by this application.
[0120] In some embodiments, continue to refer to Figure 3A , the data transmission interface 150 includes:
[0121] A first interface 151 connecting the data moving module 120 and the dequantization unit;
[0122] A second interface 152 connecting the data moving module 120 and the dot product unit;
[0123] A third interface 153 connecting the data moving module 120 and the bias unit;
[0124] A fourth interface 154 connecting the data moving module 120 and the quantization unit;
[0125] A fifth interface 155 connecting the data moving module 120 and the quantization unit;
[0126] And a sixth interface 156 connected to the data migration module 120 .
[0127] Exemplarily, the first interface 151 is a FIFO; the second interface 152 is a WFIFO; the third interface 153 is a BFIFO; the fourth interface 154 is a QFIFO; the fifth interface 155 is an ODFIFO; and the sixth interface 156 is an OAFIFO.
[0128] Through the above-mentioned different transmission interfaces, each data processing unit 141 can respectively realize data transmission with the data moving module 120, thereby simplifying the complexity of the transmission interface and improving the transmission efficiency. In addition, compared with other solutions, the embodiment of the present application adds a multi-channel data request path FIFO, so as to support the "per channel" quantization solution.
[0129] In some embodiments, the first interface 151, the second interface 152, the third interface 153, the fourth interface 154 and the sixth interface 156 are respectively used to read the first data on the external memory through the data moving module; the fifth interface 155 is used to write the second data processed by the quantization unit to the external memory through the data moving module.
[0130] The data transmission direction of the above-mentioned first interface 151, second interface 152, third interface 153, fourth interface 154 and sixth interface 156 is that the data transfer module transmits data to the data processing unit, and the first data transmitted is the various parameters and data provided by the external memory to the data processing module for data processing; the data transmission direction of the fifth interface 155 is that the data processing unit transmits data to the data transfer module. As the final output port, the second data transmitted is the data output to the external memory.
[0131] In some embodiments, the first data includes one or more of the following:
[0132] address parameters; eigenvalue parameters; weight parameters; bias parameters; quantization coefficients; and operation data for the data processing.
[0133] In some embodiments, the second data may include a result of data processing, and may also include address data and other related identification information.
[0134] In some embodiments, during the data processing by the data operation module, the weight parameter, the bias parameter or the quantization coefficient remains unchanged within a preset time period.
[0135] Here, the weight parameters, bias parameters, quantization coefficients and other parameters are loaded in a "block fixed" way to achieve data reuse. The so-called "block fixed" means that during the calculation process, the weight parameters remain unchanged for a certain period of time, thereby further reducing the energy loss caused by data transfer during the calculation process. In addition, the hardware also clears the "discarded value" of the weight parameter FIFO according to the control information.
[0136] In some embodiments, the data migration module loads the characteristic value parameters by broadcasting or row sliding;
[0137] The broadcasting method includes: loading the same characteristic value parameters to multiple data processing units of the data processing module;
[0138] The row sliding method includes: sequentially loading and shifting characteristic value parameters of designated bits to different data processing units.
[0139] For the loading of characteristic value parameters, the present application embodiment provides the above two optional methods, such as Figure 3B As shown, there are two modes, namely the broadcast mode and the row sliding mode. The solid arrows represent the hardware connection, and the dotted arrows represent the data transmission mode. Specifically, in the broadcast mode, the characteristic value data (Fdata) is synchronously provided to each data processing unit 310, and then transmitted through the data transmission unit 320. In the row sliding mode, the characteristic value data (Fdata) is transmitted to the data processing unit 310, and each data transmission unit 310 sequentially shifts the data by one or more bits before transmitting it to the next data processing unit 310, and each data processing unit transmits it through the data transmission unit 320.
[0140] The row sliding method can be applied to depthwise convolution calculations, while the broadcasting method is more applicable to ordinary convolution calculations. Therefore, using the above two methods, there is no need to change the arrangement format of data in memory, which can effectively improve the data reuse rate of eigenvalue parameters.
[0141] For example, the loading of the above-mentioned characteristic values and weight values can be reflected by the following code:
[0142]
[0143] In the above code, the "load_wdata" function is used to load the weight value, that is, the weight parameter, and the "load_fdata" function is used to load the eigenvalue, that is, the eigenvalue parameter. The purpose of this code is to load the weight value and eigenvalue on each data block of specified height and width.
[0144] Through the above technical solution of the present application, the following effects can be achieved:
[0145] First, an embodiment of the present application provides an implementation of a convolutional computing acceleration unit based on the RISCV architecture with efficient extended instructions, high reuse and low bandwidth, and suitable for TinyML reasoning.
[0146] Second, compared with the related art, the extended instructions provided by the embodiment of the present application are highly efficient. Specifically, the embodiment of the present application hides instructions such as calculation and cache data clearing in the data loading instructions, and according to the configuration instructions, the hardware implements the calculation instructions such as accumulator reading and writing, output data, quantization and dequantization, activation, etc. This simplifies the software and hardware interface and improves the utilization rate of the computing unit.
[0147] Third, the embodiments of the present application have the characteristics of high reuse and low bandwidth. Specifically, the above embodiments provide a method for loading data such as eigenvalue parameters, weight parameters, bias parameters, and quantization coefficients, so that high data reuse and low bandwidth can be achieved under the condition of low area loss.
[0148] Fourth, the above technical solution of the embodiment of the present application is applicable to TinyML reasoning and supports "depthwise" and "per channel quantization".
[0149] Based on the same inventive concept, the embodiment of the present application also provides a data processing system, such as Figure 4 As shown, the data processing system 400 includes:
[0150] RISCV-based CPU 401;
[0151] An NPU 402 connected to the CPU; wherein the NPU 402 comprises: any one of the above-mentioned data processing devices 100;
[0152] A bus unit 403 connecting the CPU 401 and the NPU 402;
[0153] A memory connected to the bus unit 403; wherein the memory includes: an on-chip memory 404 and / or an off-chip memory 405. The CPU and NPU are connected to the bus unit 403 via a parallel interface M; the on-chip memory 404 is connected to the bus unit 403 via a serial interface S; and the off-chip memory is connected to the bus unit 403 via an SPI (Serial Peripheral Interface) or w XIP (Execute-In-Place).
[0154] In some embodiments, Figure 5 As shown, the NPU includes:
[0155] VCU (Vector Computing Unit);
[0156] LSU (Lift Station Unit, data handling unit); and
[0157] TCU (Convolutional Computation Processing Unit); wherein the TCU includes the data processing device.
[0158] In some embodiments, Figure 5 As shown, the NPU also includes:
[0159] Decoder 501; cache memory 502; and buffer memory 503.
[0160] In the embodiment of the present application, a high-speed cache memory is added at the data access interface, which can further improve the data reuse rate.
[0161] Based on the same inventive concept, the present application embodiment also provides a chip, such as Figure 6 As shown, the chip 600 includes:
[0162] RISC-based CPU 601;
[0163] An NPU 602 connected to the CPU; wherein the NPU 602 includes: any one of the above-mentioned data processing devices 100;
[0164] A bus unit 603 connected to the CPU 601 and the NPU 602;
[0165] An on-chip memory 604 connected to the bus unit 603 .
[0166] The above-mentioned data processing system and chip belong to the same inventive concept as the above-mentioned data processing device provided in the embodiments of the present application. Their specific implementation and scheme effects can be referred to the detailed description in the above-mentioned embodiments, which will not be repeated here.
[0167] A person skilled in the art can understand that all or part of the steps of implementing the above method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: mobile storage devices, read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.
[0168] Alternatively, if the above-mentioned integrated unit of the present application is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium includes: a mobile storage device, ROM, a disk or an optical disk, and other media that can store program codes.
[0169] The various embodiments / implementations provided in this application can be combined with each other without causing any contradiction.
[0170] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.
Claims
1. A data processing device, It is characterized in that include: A decoding module, used for receiving extended instructions based on the open source instruction set architecture RISC-V; wherein the extended instructions include: data movement instructions and configuration instructions; A data transfer module, used for receiving the data transfer instruction from the decoding module, and accessing the corresponding data transfer interface based on the data transfer instruction to perform data transfer; A configuration module, used to receive the configuration instruction from the decoding module and perform parameter configuration based on the configuration instruction; A data processing module, used for performing data calculation and processing according to the data moving instruction and the configuration instruction; One or more data transmission interfaces are respectively connected to the data processing module and the data moving module, and are used to transmit corresponding data according to the data moving instruction.
2. The data processing device according to claim 1, It is characterized in that The data processing module includes a plurality of data processing units, and different data processing units among the plurality of data processing units are used to perform different data operations on the data.
3. The data processing device according to claim 1 or 2, It is characterized in that Also includes: A data operation module is connected to the data processing module and is used to execute the data operation in the data operation processing of the data processing module.
4. The data processing device according to claim 3, It is characterized in that The data operation module includes one or more of the following units: A multiplication array unit, used to implement the multiplication operation in the data operation; An addition array unit, used to implement the addition operation in the data operation; A format conversion unit is used to implement the format conversion of the data.
5. The data processing device according to claim 3 or 4, It is characterized in that The data operation module includes: an arithmetic logic unit ALU in the central processing unit CPU is reused.
6. The data processing device according to claim 3, It is characterized in that The data processing module includes one or more of the following data processing units: A dequantization unit, used for reading data and control information through the data transmission interface, and requesting to use the data operation module to perform data operation; A dot product unit, used for receiving the data provided by the dequantization unit and the control information, reading an enable signal through the data transmission interface, and requesting to use the data operation module to perform data operation; an accumulation unit, configured to receive the data provided by the dot product unit and the control information, read the cache according to the control information, and request to use the data operation module to perform data operation; The cache is used to temporarily store accumulated data; A bias unit, configured to receive the data provided by the accumulating unit and the control information, read an enable signal through the data transmission interface according to the control information, and request to use the data operation module to perform data operation; an activation unit, configured to receive the data provided by the bias unit and the control information, and perform activation processing according to the control information; The quantization unit is used to receive the data and the control information provided by the activation unit, request the data operation module to perform data processing, and output the processed data to the data moving module through the data transmission interface.
7. The data processing device according to claim 6, It is characterized in that The data transmission interface comprises: A first interface connecting the data moving module and the dequantization unit; A second interface connecting the data moving module and the dot product unit; A third interface connecting the data moving module and the bias unit; A fourth interface connecting the data moving module and the quantization unit; a fifth interface connecting the data moving module and the quantization unit; And a sixth interface connected to the data migration module.
8. The data processing device according to claim 7, It is characterized in that The first interface, the second interface, the third interface, the fourth interface and the sixth interface are respectively used to read the first data on the external memory through the data migration module; The fifth interface is used to write the second data processed by the quantization unit into the external memory through the data moving module.
9. The data processing device according to claim 8, It is characterized in that The first data includes one or more of the following: Address parameters; Eigenvalue parameters; weight parameters; bias parameters; quantization coefficients; and operational data for the data processing.
10. The data processing device according to claim 9, It is characterized in that During the data processing by the data operation module, the weight parameter, the bias parameter or the quantization coefficient remains unchanged within a preset time length.
11. The data processing device according to claim 9, It is characterized in that The data migration module loads the characteristic value parameters by broadcasting or row sliding; The broadcasting method includes: loading the same characteristic value parameters to multiple data processing units of the data processing module; The row sliding method includes: sequentially loading and shifting characteristic value parameters of designated bits to different data processing units.
12. A data processing system, It is characterized in that include: Central processing unit CPU based on the Kaiyuan instruction set architecture RISC-V; A neural network processing unit NPU connected to the CPU; wherein the NPU comprises: a data processing device as described in any one of claims 1 to 11.
13. The data processing system according to claim 12, It is characterized in that Also includes: A bus unit connecting the CPU and the NPU; A memory connected to the bus unit; wherein the memory includes: an on-chip memory and / or an off-chip memory.
14. The data processing system according to claim 12, It is characterized in that The NPU includes: Vector computing unit VCU; Data handling unit LSU; and Convolution calculation processing unit TCU; wherein the TCU includes the data processing device.
15. The data processing system according to claim 12, It is characterized in that The data processing system is a chip.
Citation Information
Cited By
Processor based on RISC-V instruction set architecture, design method of processor, data processing method, chip and system
CN120029970A
