Data processing device, chip product, computer equipment and data processing method

Abstract

The embodiment of the invention discloses a data processing device, a chip product, computer equipment and a data processing method. The data processing device comprises an instruction decoding module, a data moving module and a plurality of calculation units; wherein the instruction decoding module is used for receiving a calculation instruction based on open source instruction set architecture RISC-V extension; the data moving module is used for responding to the calculation instruction and determining a prediction configuration parameter to be used by a target calculation unit corresponding to the calculation instruction; the calculation unit is used for utilizing a target calculation unit corresponding to the calculation instruction to obtain configuration parameters to be used by the target calculation unit when the calculation instruction is executed; and if the configuration parameter hits the predicted configuration parameter, performing unit configuration by using the predicted configuration parameter so as to perform data operation processing.

Description

Technical Field

[0001] This invention relates to, but is not limited to, the field of data processing technology, and particularly to a data processing apparatus, a chip product, a computer device, and a data processing method. Background Technology

[0002] With the development of artificial intelligence technology, neural networks have been widely used in fields such as image recognition and natural language processing. Because neural network computation involves operations such as convolution, pooling, and activation functions, it exhibits high parallelism and regularity. In related technologies, neural network computation is typically performed using general-purpose processor extended vector instructions to support neural network calculations.

[0003] However, in actual use, neural network calculations usually require multiple basic instructions, which increases computational overhead. Moreover, when reusing data, the data in memory needs to be accessed repeatedly and frequently, which limits the computational speed of neural networks and results in low computational efficiency. Summary of the Invention

[0004] In view of this, embodiments of this application provide at least one data processing apparatus, a chip product, a computer device, and a data processing method.

[0005] The technical solution of this invention is implemented as follows:

[0006] On one hand, embodiments of this application provide a data processing apparatus, which includes an instruction decoding module, a data movement module, and multiple computing units; wherein:

[0007] The instruction decoding module is used to receive computation instructions based on the open-source instruction set architecture RISC-V extension;

[0008] The data movement module is used to determine the prediction configuration parameters to be used by the target computing unit corresponding to the computing instruction in response to the computing instruction.

[0009] The computing unit is configured to, when executing the computing instruction, obtain the configuration parameters to be used by the target computing unit corresponding to the computing instruction; if the configuration parameters match the predicted configuration parameters, then the unit is configured using the predicted configuration parameters to perform data processing.

[0010] In some embodiments, the data movement module is configured to:

[0011] In response to the calculation instruction, the system determines the prediction configuration parameters to be used by the target computing unit corresponding to the calculation instruction, and loads the prediction configuration parameters from memory; it receives a data read instruction issued by the target computing unit when executing the calculation instruction, the data read instruction including the configuration parameters to be used by the target computing unit; and it sends the parameters of the hit parameter items in the prediction configuration parameters to the target computing unit.

[0012] In some embodiments, the data movement module includes a prefetch controller, a data flow controller, and a buffer unit;

[0013] The prefetch controller is configured to determine the data access mode corresponding to the computation instruction based on the instruction type and instruction parameters of the computation instruction; determine the predicted configuration parameters to be used by the target computation unit based on the data access mode, and load the predicted configuration parameters from memory into the cache unit; the data flow controller is configured to select the target parameter item that the configuration parameter matches from among the multiple parameter items included in the predicted configuration parameters, and transmit the parameters of the target parameter item in the predicted configuration parameters to the target computation unit; and reload the parameters of other parameter items that the configuration parameter does not match from memory, and transmit the parameters of the other parameter items to the target computation unit; the cache unit is configured to store the configuration parameters of the target computation unit.

[0014] In some embodiments, when the computation instruction is a sparse computation type, the prefetch controller is configured to load non-zero index data from memory and load the prediction configuration parameters into the cache unit based on the index data;

[0015] The data flow controller is used to perform zero-value data filtering for each parameter item and transmit the filtered parameter items to the target computing unit.

[0016] In some embodiments, the computing unit includes a convolution computing engine; the convolution computing engine includes a first buffer, a first multiply-accumulate array, and a window sliding controller;

[0017] The window sliding controller is configured to obtain an input feature map and convolution parameters for performing convolution calculations according to the calculation instructions; generate a first window position sequence of the output feature map according to the input feature map and the convolution parameters; and, while the first multiply-accumulate array performs data operations on the current window, preload the calculation data of the next window adjacent to the current window into the first buffer according to the first window position sequence; the first buffer is configured to store the calculation data of the next window adjacent to the current window preloaded by the window sliding controller; the first multiply-accumulate array is configured to use each multiply-accumulate unit in the first multiply-accumulate array to perform multiply-accumulate operations in parallel on the calculation data of the current window read from the first buffer to obtain the data operation result of the current window.

[0018] In some embodiments, the first buffer includes an input feature map buffer, a convolution kernel buffer, and a first result accumulation buffer; the input feature map buffer is used to store the input feature map of the next window adjacent to the current window when the first multiply-accumulate array performs data operations on the current window; the convolution kernel buffer is used to store the convolution kernel parameters of the next window adjacent to the current window when the first multiply-accumulate array performs data operations on the current window; the first result accumulation buffer is used to sum the received and stored multiply-accumulate partial results corresponding to each window to obtain the convolution calculation result.

[0019] In some embodiments, the computation unit further includes a pooling processing unit; the pooling processing unit includes a window controller, a second buffer, and a pooling array; the window controller is configured to obtain an input feature map and pooling parameters for performing pooling processing according to the computation instructions; generate a second window position sequence of an output feature map according to the input feature map and the pooling parameters; and, when the pooling array performs data computation on the current window, preload the computation data of the next window adjacent to the current window into the second buffer according to the second window position sequence; the second buffer is configured to store the computation data of the next window adjacent to the current window preloaded by the window controller; the pooling array is configured to use each pooling unit in the pooling array to perform pooling processing on the computation data of the current window read from the second buffer in parallel to obtain the data computation result of the current window.

[0020] In some embodiments, the second buffer includes an input buffer and an output buffer; the output buffer includes multiple buffers; the input buffer is used to store the input feature map of the next window adjacent to the current window when the pooling array performs data operations on the current window; the output buffer is used to store the data operation results of each window using a first output buffer until the remaining amount of the first output buffer is less than or equal to a preset threshold, then switch to using a second output buffer to store the data operation results of each window, and transmit the data operation results of each window stored in the first output buffer to a specified address, wherein the first output buffer is any one of the multiple output buffers, and the second output buffer is any one of the multiple output buffers other than the first output buffer.

[0021] In some embodiments, the computing unit further includes an activation function unit, which includes a function selector and a plurality of function processing units; the function selector is configured to select and enable a target function processing unit corresponding to the instruction type from the plurality of function processing units according to the instruction type of the computing instruction, wherein each function processing unit corresponds to a different instruction type.

[0022] In some embodiments, the computation unit further includes a matrix multiplication unit, which includes a third buffer, a second multiply-accumulate array, and a matrix dimension controller. The matrix dimension controller is used to obtain the input matrix and the array parameters of the second multiply-accumulate array according to the computation instructions; to divide the input matrix into multiple sub-matrices according to the dimension parameters of the input matrix and the array parameters; and to preload the next sub-matrix adjacent to the current sub-matrix into the third buffer while the second multiply-accumulate array performs data operations on the current sub-matrix. The third buffer is used to store the next sub-matrix adjacent to the current sub-matrix preloaded by the matrix dimension controller. The second multiply-accumulate array is used to perform multiply-accumulate operations on the current sub-matrix read from the third buffer using each multiply-accumulate unit in the second multiply-accumulate array to obtain the data operation result of the current sub-matrix.

[0023] In some embodiments, the third buffer includes a matrix buffer and a second result accumulation buffer; the matrix buffer is used to store the next submatrix adjacent to the current submatrix when the data operation of the current submatrix is ​​performed by the second multiply-accumulate array; the second result accumulation buffer is used to accumulate and merge the corresponding multiply-accumulate results of each received and stored submatrix in matrix dimension order to obtain the output matrix output by the matrix multiplication unit.

[0024] In some embodiments, the computing unit further includes a quantization processing unit, which includes a quantization parameter register, a quantizer, and a dequantizer. The quantization parameter register stores configuration parameters corresponding to quantization and dequantization operations; it transmits a first configuration parameter corresponding to the quantization operation to the quantizer and a second configuration parameter corresponding to the dequantization operation to the dequantizer. The quantizer obtains first data to be quantized according to the computing instructions; it performs quantization conversion operations on the first data according to the first configuration parameters to obtain a quantized data operation result. The dequantizer obtains second data to be dequantized according to the computing instructions; it performs dequantization conversion operations on the second data according to the second configuration parameters to obtain a dequantized data operation result.

[0025] On the other hand, embodiments of this application provide a data processing method, characterized in that the method is applied to a data processing device, the data processing device including a decoding module, a data movement module, and multiple computing units; the method includes:

[0026] The instruction decoding module receives computation instructions based on the open-source instruction set architecture RISC-V extension;

[0027] In response to the calculation instruction, the data movement module determines the prediction configuration parameters to be used by the target calculation unit corresponding to the calculation instruction;

[0028] When executing the calculation instruction, the configuration parameters to be used by the target calculation unit corresponding to the calculation instruction are obtained; if the configuration parameters match the predicted configuration parameters, the unit is configured using the predicted configuration parameters to perform data operation processing.

[0029] In some embodiments, configuring the unit using the predicted configuration parameters includes: in response to the calculation instruction, determining the predicted configuration parameters to be used by the target computing unit corresponding to the calculation instruction, and loading the predicted configuration parameters from memory; receiving a data read instruction issued by the target computing unit when executing the calculation instruction, the data read instruction including the configuration parameters to be used by the target computing unit; and sending the parameters of the hit parameter items in the predicted configuration parameters to the target computing unit.

[0030] In some embodiments, determining the predicted configuration parameters to be used by the target computing unit corresponding to the computing instruction includes: determining the data access mode corresponding to the computing instruction based on the instruction type and instruction parameters of the computing instruction; and determining the predicted configuration parameters to be used by the target computing unit based on the data access mode.

[0031] Sending the parameters of the matched parameter items in the prediction configuration parameters to the target computing unit includes: selecting the target parameter item matched by the configuration parameters from among the multiple parameter items included in the prediction configuration parameters; and transmitting the parameters of the target parameter item in the prediction configuration parameters to the target computing unit.

[0032] The method further includes: reloading the parameters of other parameter items for which the configuration parameters were not matched from the memory; and transferring the parameters of the other parameter items to the target computing unit.

[0033] On the other hand, embodiments of this application provide a chip product, which includes any of the data processing devices described in embodiments of this application.

[0034] In another aspect, embodiments of this application provide a computer device, which includes the chip product described in embodiments of this application.

[0035] In this embodiment of the application, a data processing apparatus is provided, comprising an instruction decoding module, a data movement module, and multiple computing units; wherein: the instruction decoding module is configured to receive computing instructions based on an extension of the open-source instruction set architecture RISC-V; the data movement module is configured to, in response to the computing instructions, determine the predicted configuration parameters to be used by the target computing unit corresponding to the computing instructions; the computing units are configured to, when executing the computing instructions, obtain the configuration parameters to be used by the target computing unit corresponding to the computing instructions; if the configuration parameters match the predicted configuration parameters, then the predicted configuration parameters are used to configure the units for data processing. In this way, on the one hand, neural network computation operations implemented with hundreds of basic arithmetic instructions in related technologies can be realized through RISC-V-based extended computation instructions. The entire computation process can be triggered with only a single extended instruction, thereby reducing the overhead of instruction decoding and scheduling. On the other hand, the configuration parameters of the target computing unit can be predicted in advance by the data movement module, and the parameter preparation can be completed before the execution of the computation instructions. This avoids the problem of the computing unit pausing and waiting for the configuration parameters to be loaded during the instruction execution process. The parameter configuration stage can be parallelized with the instruction decoding and data preparation stages, saving the execution time of a single instruction. Moreover, loading the configuration parameters in advance can avoid frequent repeated reading of data from memory during the computation process. When the configuration parameters match the predicted parameters, the computing unit can directly start the operation using the preloaded cached data without waiting for memory data transfer, which can improve data reuse efficiency and shorten data transfer latency.

[0036] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this application. Attached Figure Description

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0038] Figure 1 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 1 ;

[0039] Figure 2 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 2 ;

[0040] Figure 3 This is a schematic diagram illustrating the implementation flow of a data processing method provided in an embodiment of this application;

[0041] Figure 4 This is an overall architecture diagram of a neural network instruction set extension system provided in an embodiment of this application;

[0042] Figure 5 This is a schematic diagram of the architecture of a convolutional computation engine provided in an embodiment of this application;

[0043] Figure 6 This is a schematic diagram of the architecture of a pooling processing unit provided in an embodiment of this application;

[0044] Figure 7 This is a schematic diagram of the architecture of an activation function unit provided in an embodiment of this application;

[0045] Figure 8 This is a schematic diagram of the architecture of a matrix multiplication unit provided in an embodiment of this application;

[0046] Figure 9 This is a schematic diagram of the architecture of a quantization processing unit provided in an embodiment of this application;

[0047] Figure 10 This is a schematic diagram of the architecture of a data prefetching and caching unit provided in an embodiment of this application;

[0048] Figure 11 This is a schematic diagram of the execution architecture of a sparse computing method provided in an embodiment of this application;

[0049] Figure 12 This is a schematic diagram of the structure of a chip product provided in an embodiment of this application;

[0050] Figure 13 This is a schematic diagram of the structure of a calculator device provided in an embodiment of this application;

[0051] Figure 14This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application are further described in detail below with reference to the accompanying drawings and embodiments. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0053] In the following description, references to "some embodiments" are made, which describe a subset of all possible embodiments. However, it is understood that "some embodiments" may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict. The terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0054] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. The terminology used herein is for descriptive purposes only and is not intended to limit the scope of this application.

[0055] This application provides a data processing apparatus that can be used in computational scenarios involving neural network calculations.

[0056] Figure 1 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 1 ,like Figure 1 As shown, the data processing apparatus 100 provided in this application embodiment may include the following components: an instruction decoding module 101, a data movement module 102, and multiple computing units 103. The instruction decoding module 101 may be connected to each of the multiple computing units 103, and the data movement module 102 may be connected to both the instruction decoding module 101 and the multiple computing units 103.

[0057] In this embodiment, the instruction decoding module 101 can be used to receive computation instructions based on the open-source instruction set architecture RISC-V extension.

[0058] The computation instructions can refer to specialized instructions designed for neural network operations, such as convolution, pooling, and activation function instructions. These instructions can be extended from the standard RISC-V instruction set and used to control hardware such as the computing unit 103 to perform neural network operations. It should be noted that the format of the computation instructions can be compatible with RISC-V to ensure a smooth transition in the software ecosystem. For example, computation instructions can include, but are not limited to, NNCONV, NNGEMM, and NNRELU instructions. NNCONV is used to trigger convolution calculations, NNGEMM is used for matrix multiplication, and NNRELU is used to trigger ReLU activation functions, etc.

[0059] By introducing computational instructions, neural network operations can be executed directly at the hardware level without relying on a large number of general-purpose arithmetic instructions, thereby significantly improving execution efficiency.

[0060] In practical implementation, the instruction decoding module 101, as an extension component of the RISC-V processor architecture, can be connected to the instruction fetch unit of the RISC-V core. After the instruction fetch unit reads the instruction stream from memory or the instruction cache, it can pass the instruction stream sequentially to the instruction decoding module 101. This instruction stream can include standard RISC-V instructions, such as add (addition) and lw (load) instructions, as well as extended computation instructions, such as NNCONV (convolution computation instruction).

[0061] After receiving the instruction stream, the instruction decoding module 101 can distinguish between standard RISC-V instructions and extended computation instructions based on the opcode. Specifically, the instruction decoding module 101 can perform an opcode matching operation on each instruction in the received instruction stream, checking the lower 7 bits (opcode field) of each instruction to determine whether the opcode field matches the opcode range of the extended computation instructions.

[0062] If the opcode field of an instruction matches the opcode of an extended computation instruction, the encoding format corresponding to the instruction can be further determined based on the opcode. This encoding format can include, but is not limited to, R-type, I-type, S-type, and NN-type formats. For example, when the opcode field is NNLD, it corresponds to the I-type format; when the opcode field is NNCONV, it corresponds to the newly added NN-type format.

[0063] It should be noted that if an instruction opcode is detected as an extended instruction but the format does not match, for example, if an NN type instruction is found to be missing the rs3 field, it can be marked as an illegal instruction and exception handling can be triggered, thus avoiding hardware errors.

[0064] After filtering out the valid extended instructions (NNE-RV), the instruction decoding module 101 can temporarily store the original instruction data in the internal buffer register.

[0065] In this embodiment, the data movement module 102 can be used to determine the predicted configuration parameters to be used by the target computing unit corresponding to the computing instruction in response to the computing instruction.

[0066] The predicted configuration parameters refer to the configuration information required by the target computing unit, which is predicted by the data movement module 102 based on the instruction type and parameters, before the computing unit 103 executes the computing instruction, and is pre-loaded into the cache. For example, before executing the convolution instruction, the system predicts the configuration information required by the convolution computing engine, such as the computing mode parameters, precision configuration parameters (e.g., FP32, INT8), sparse mode parameters, cache and data routing parameters, based on parameters such as the input feature map size, convolution kernel size, and stride, and stores the configuration information in a specific register. Storing the predicted configuration information in a specific register allows for quick retrieval during subsequent calculations, avoiding repeated judgments and delays.

[0067] When the instruction decoding module 101 receives an extended calculation instruction, it can synchronously send an indication signal to the data movement module 102. This indication signal may include the instruction type identifier and key field snapshots of the calculation instruction, rather than the complete instruction. This is because a complete instruction includes 32-bit fields such as opcode, function, register address, and immediate value, while the data movement module 102 only needs the instruction type and core parameters to initiate prefetching. This eliminates the need to parse fields unrelated to data prefetching, such as function4 and rd, reducing parsing latency. Furthermore, since a complete instruction requires 32 bits for bus transmission, while a key field snapshot typically only requires 32 bits, bus bandwidth usage can be reduced, further lowering transmission latency.

[0068] In one possible implementation, after the data movement module 102 receives the indication signal, it can determine the computation unit 103 matching the computation instruction and the configuration parameters of the computation unit 103 based on an internally pre-set mapping table. This mapping table can include the correspondence between the computation instruction, the computation unit 103, and the configuration parameters of the computation unit 103. Each computation instruction can correspond to a different computation unit 103, and each computation unit 103 executing the computation instruction has different configured parameters. For example, when the computation instruction is NNCONV (convolution computation instruction), the convolution computation engine matching the convolution computation instruction can be automatically associated, and the configuration parameters predicted by the convolution computation engine can be further determined to include at least one of the following: kernel size (1x1 / 3x3 / 5x5), stride (1 / 2 / 4), padding mode (VALID / SAME), and buffer partition of the input feature map.

[0069] In another possible implementation, due to the strong parameter continuity of neural network computation, configuration parameters of the same instruction type from recent times can be reused as prediction values, and updated only when the instruction field changes explicitly. For example, in a depthwise separable convolutional layer, 16 consecutive NNDEPTHCONV (depthwise separable convolution) instructions share parameters such as stride and padding. After the data movement module 102 parses the first instruction, it can directly cache these parameters, and subsequent instructions can directly reuse them as prediction parameters, thus reducing parsing overhead.

[0070] It should be noted that, to avoid the prediction parameters exceeding the hardware capabilities of the computing unit 103, the data movement module 102 can also verify the prediction parameters based on the hardware specifications of each computing unit 103 to ensure that the computing unit 103 can execute the calculation instructions using the configuration parameters. For example, if the convolution calculation engine only supports 1×1, 3×3, and 5×5 convolution kernels, but the kernel size for instruction parsing is 7×7, the prediction parameters determined by the data movement module 102 can be automatically corrected to "not supported" and trigger an instruction exception.

[0071] In related technologies, the computing unit 103 needs to wait for the instructions to be fully decoded and the parameters to be loaded before it can start the configuration, which causes the configuration and calculation to be serialized, increasing the execution cycle. However, in this embodiment, the data movement module 102 can determine the predicted configuration parameters in advance and send a pre-configuration signal to the target computing unit synchronously, so that the computing unit 103 can complete the hardware initialization (such as switching the precision of the multiply-accumulate array and configuring the parameters of the window sliding controller) during the data loading period, which can reduce the configuration delay of the computing unit 103.

[0072] In this embodiment of the application, when executing a calculation instruction, the configuration parameters to be used by the target calculation unit corresponding to the calculation instruction can be obtained by the calculation unit 103; if the configuration parameters match the predicted configuration parameters, the unit can be configured using the predicted configuration parameters to perform data operation processing.

[0073] The target computation unit can refer to the corresponding hardware module selected according to the type of computation instruction, such as a convolution computation engine, a pooling processing unit, or an activation function unit. Each computation unit 103 has an independent configuration register set for storing the parameters required for the current computation.

[0074] When the data movement module 102 obtains the corresponding target computing unit based on the computing instructions, it can send an activation signal to the target computing unit. This activation signal can be used to wake up the hardware circuitry of the target computing unit in a low-power state. After receiving the activation signal, the target computing unit can obtain the configuration parameters to be used when executing the computing instructions.

[0075] In one possible implementation, considering that the extended instructions essentially encapsulate a complete neural network operation into a single instruction, the computation instructions themselves can carry configuration parameters. For example, a convolution computation instruction can be represented as: NNCONV rd,rs1,rs2,rs3, where the rs3 register can encapsulate parameters such as stride, padding, and kernel size. Similarly, a matrix multiplication instruction can be represented as: NNGEMM rd,rs1,rs2,rs3, where the rs3 register can encapsulate parameters such as matrix dimensions M / N / K. Therefore, the target computation unit can receive the core parameters from the computation instructions from the instruction decoding module 101.

[0076] In another possible implementation, considering that some configurations may include multiple sets of subdivision parameters that cannot be encapsulated by the limited fields of a 32-bit instruction, for example, the quantization parameters of the quantization processing unit may include scaling factor (FP32), zero point (INT32), quantization mode (symmetric / asymmetric), precision identifier (INT8 / INT4), etc., totaling 10 bytes of data, which far exceeds the capacity of the 5-bit address field of the rs2 register in the instruction. Therefore, they will be stored in the corresponding dedicated registers. Thus, the target computation unit can also read extended configuration parameters from the dedicated register group (nnscr0-nnscr3) of the neural network. For example, when executing the quantization computation instruction, the scaling factor, zero point, and other parameters can be read from the nnscr1 register; and when executing the sparse computation instruction, the sparse format (CSR / COO), zero threshold, and other parameters can be read from the nnscr2 register.

[0077] After the target computing unit obtains the configuration parameters to be used, it can compare the actual obtained configuration parameters with the predicted parameters in the pre-configuration buffer. If the configuration parameters match the predicted configuration parameters, the pre-configuration parameters can be directly reused, skipping the hardware initialization process. If they do not match, the configuration can be started based on the actual parameters. Then, the target computing unit can execute computation instructions based on the configured parameters to perform data operations.

[0078] By adopting the above technical solution, on the one hand, the neural network computation operations implemented by hundreds of basic arithmetic instructions in related technologies can be realized through the computation instructions based on RISC-V extensions. The entire computation process can be triggered with only a single extended instruction, thereby reducing the overhead of instruction decoding and scheduling. On the other hand, the configuration parameters of the target computing unit can be predicted in advance by the data movement module 102, and the parameter preparation can be completed before the execution of the computation instructions. This avoids the problem of the computing unit 103 pausing and waiting for the configuration parameters to be loaded during the instruction execution process. The parameter configuration stage can be parallelized with the instruction decoding and data preparation stages, saving the execution time of a single instruction. Moreover, loading the configuration parameters in advance can avoid frequent repeated reading of data from memory during the computation process. When the configuration parameters match the predicted parameters, the computing unit 103 can directly start the operation using the preloaded cached data without waiting for memory data transfer, which can improve data reuse efficiency and shorten data transfer latency.

[0079] In some embodiments, the data movement module 102 may respond to a calculation instruction, determine the prediction configuration parameters to be used by the target computing unit corresponding to the calculation instruction, and load the prediction configuration parameters from memory; then it may receive a data read instruction issued by the target computing unit when executing the calculation instruction; and send the parameters of the hit parameter items in the prediction configuration parameters to the target computing unit.

[0080] The data read instruction includes the configuration parameters to be used by the target computing unit.

[0081] In this embodiment, the data movement module 102 can generate the predicted configuration parameters of the target computation unit based on a built-in mapping table that establishes the correspondence between the representation computation instructions, the computation unit 103, and the corresponding configuration parameters. For example, after receiving the indication signal of the NNCONV instruction, the data movement module 102 can automatically associate the predicted configuration parameters of the convolution computation engine using the mapping table. These predicted configuration parameters may include, but are not limited to, parameters such as stride, padding mode, computation precision, and feature map cache partitioning. Then, the data movement module 102 can load the predicted configuration parameters from memory or a dedicated parameter area and store them in the pre-configured parameter buffer.

[0082] After the target computing unit, such as the convolutional computation engine or quantization processing unit, is activated by the instruction decoding module 101, it can send a data reading instruction to the data movement module 102 to initiate hardware configuration. The configuration parameters to be used in this data reading instruction are essentially a list of parameter requirements generated by the computing unit 103 based on the instruction decoding result. For example, it may require information such as stride, padding, and precision parameters, rather than complete parameter values, for matching and verification with the prediction parameters of the data movement module 102.

[0083] The data movement module 102 can compare the predicted parameters in the pre-configured buffer with the parameter requirement list in the data read instruction. If the predicted configuration parameters are matched, the pre-loaded parameter items are directly sent from the buffer or special register to the target computing unit. If the predicted configuration parameters are matched, a reloading process can be triggered to read the correct parameters from memory and distribute them. At the same time, the pre-configured buffer can be updated according to the reloaded parameters.

[0084] By employing the above technical solution, the pre-loading of predicted configuration parameters can effectively reduce the waiting time during configuration parameter reading and improve data processing efficiency. Simultaneously, by identifying and transmitting hit parameter items, unnecessary parameter transmissions are avoided, saving bandwidth resources and further enhancing system performance.

[0085] Figure 2 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application. Figure 2 ,like Figure 2 As shown, the data movement module 102 includes a prefetch controller 1021, a data flow controller 1022, and a buffer unit 1023.

[0086] Optionally, the prefetch controller 1021 can be used to determine the data access mode corresponding to the computation instruction based on the instruction type and instruction parameters of the computation instruction; determine the predicted configuration parameters to be used by the target computation unit based on the data access mode; and load the predicted configuration parameters from memory into the cache unit 1023.

[0087] The prefetch controller 1021 continuously monitors the instruction pipeline. When it identifies neural network-specific instructions (such as NNCONV, NNGEMM, etc.), it first distinguishes different computation types, such as convolution, pooling, and matrix multiplication, based on the opcode of the identified computation instruction. Each type of computation operation corresponds to different data access characteristics. For example, convolution (NNCONV) exhibits a sliding window access pattern on the input feature map, and the weight data shows high reuse characteristics. As another example, matrix multiplication (NNGEMM) requires block traversal of two matrix data blocks, and the access pattern is characterized by strong regularity and large data block size.

[0088] Once the computation type is obtained, the prefetch controller 1021 can further decode the immediate values ​​or register operands embedded in the instruction word to extract key computation parameters. Then, based on the instruction type and instruction parameters of the computation instruction, the prefetch controller 1021 uses an internally preset pattern matching algorithm to derive the data access pattern required for subsequent computations. This data access pattern is used to define all key attributes of data access, such as the sequence, block size, step size, number of reuses, and temporal locality.

[0089] When the prefetch controller 1021 obtains the data access mode, it can determine the predicted configuration parameters to be used by the target computing unit based on the determined precise access mode.

[0090] Specifically, the data processing device may also be equipped with one or more prefetch strategy lookup tables. The prefetch strategy lookup table can map the data access mode to a set of executable prediction configuration parameters. The prediction configuration parameters may include, but are not limited to, prefetch depth, prefetch stride, and data layout format. Prefetch depth may refer to the amount of data prefetched into the cache in advance, such as the data of several future convolution windows or matrix blocks. Prefetch stride may refer to the data block size of each prefetch operation and the increment of the next prefetch starting address, which strictly corresponds to the computation stride. Data layout format can specify the arrangement format of data in the cache to ensure that it can be directly and efficiently utilized by the computation unit 103 (such as a systolic array), which may involve data rearrangement or transposition.

[0091] The aforementioned predicted configuration parameters can be pre-stored in a specific area of ​​the system memory in the form of a data structure. The prefetch controller 1021 can load the required parameter set from the specified address into the control register or configuration buffer inside the cache unit 1023 of the data movement module 102 by initiating a dedicated memory read transaction.

[0092] Optionally, the data flow controller 1022 can be used to select the target parameter item that the configuration parameter hits from among multiple parameter items included in the prediction configuration parameters, and transmit the parameter of the target parameter item in the prediction configuration parameters to the target computing unit; and reload the parameters of other parameter items that the configuration parameter does not hit from memory, and transmit the parameters of the other parameter items to the target computing unit.

[0093] In actual use, when the data flow controller 1022 receives the set of predicted configuration parameters generated by the prefetch controller 1021 and loaded into the cache unit 1023, it compares each parameter item in the predicted configuration parameters provided by the prefetch controller 1021 with the configuration parameters to determine the target parameter item that the configuration parameter hits among the multiple parameter items included in the predicted configuration parameters. The parameter value of the hit target parameter item is then transmitted in real time to the corresponding configuration register of the target computing unit through the internal configuration bus (such as APB or a custom bus) so that it takes effect immediately.

[0094] The cache unit 1023 can be used to store the configuration parameters of the target computing unit.

[0095] By adopting the above technical solution, the prefetch controller 1021 analyzes the data access mode and predicts the loading of configuration parameters, which enables more accurate preloading of relevant configuration parameters and improves the hit rate. Simultaneously, the data flow controller 1022 can flexibly adjust the transmission strategy based on whether the configuration parameters are hit, ensuring timely supply of the required parameters, while the cache unit 1023 provides a fast access channel for frequently accessed parameters, improving the response speed of data access.

[0096] In some embodiments, when the computation instruction is of the sparse computation type, the prefetch controller 1021 can be used to load non-zero value index data from memory and load the prediction configuration parameters to the cache unit 1023 according to the index data; the data flow controller 1022 can be used to perform zero value data filtering for each parameter item and transmit the parameters of the filtered parameter items to the target computation unit.

[0097] In this embodiment, when a computation instruction is identified as a sparse computation type (e.g., the instruction opcode corresponds to NNSPARSE_CONV or NNSPARSE_GEMM), the data movement module 102 can enable a collaborative workflow optimized for sparse data, which can be implemented in the following ways:

[0098] The prefetch controller 1021 first loads index data of non-zero values ​​from memory. This index data can be stored in a compressed sparse format (such as a row offset array and column index array in CSR format, or a coordinate list in COO format), explicitly identifying the location information of all non-zero elements in the sparse tensor. Then, the prefetch controller 1021 parses this index data to accurately deduce the actual memory address range and data block distribution pattern required for subsequent computations. Based on this derivation, the prefetch controller 1021 retrieves the matching prediction configuration parameters from memory and loads them into the cache unit 1023. These matching prediction configuration parameters are pre-set specifically for the sparse computation mode and may include the non-zero element block size, the memory access jump step size, and loop control parameters specific to sparse matrix operations.

[0099] The dataflow controller 1022 can receive a complete set of predicted configuration parameters from the cache unit 1023 and perform a zero-value data filtering operation for each parameter item. This filtering process does not process the actual computational data but is applied to the control flow parameters themselves. The dataflow controller 1022 examines all parameter items and, based on pre-loaded index information, identifies and filters out those parameter items that would cause the target computation unit to perform invalid access or computation on zero-value data. For example, it filters out memory pointers whose target address is a zero-value region, or skips multiplication control signals associated with zero-value operands in multiply-accumulate operations.

[0100] After filtering is completed, the data flow controller 1022 can transmit the filtered parameter items (i.e. those parameters that only point to or control operations on non-zero value data) to the target computing unit (such as a sparse computing engine).

[0101] This ensures that the target computing unit only acquires and processes valid configuration information related to non-zero values ​​throughout the entire execution process, thereby completely avoiding any form of memory access or arithmetic operation on zero-value data, and improving execution efficiency and energy efficiency in sparse computing scenarios.

[0102] In some embodiments, the computing unit 103 includes a convolution computing engine. The convolution computing engine includes a first buffer, a first multiply-accumulate array, and a window sliding controller.

[0103] Optionally, the window sliding controller can be used to obtain the input feature map and the convolution parameters for performing convolution calculation according to the calculation instructions; generate a first window position sequence of the output feature map according to the input feature map and the convolution parameters; and, while performing data operations on the current window in the first multiply-accumulate array, preload the calculation data of the next window adjacent to the current window into the first buffer according to the first window position sequence.

[0104] First, the window sliding controller decodes the received calculation instruction to obtain the address and dimension information of the input feature map to be processed in memory, as well as the convolution parameters required to perform this convolution operation. These convolution parameters may include, but are not limited to, the kernel size, the stride, the padding method, and the target size of the output feature map.

[0105] After obtaining the input feature map and convolution parameters, the window sliding controller can perform address sequence calculation based on the parsed input feature map and convolution parameters. Specifically, according to the rules of the convolution kernel sliding on the input feature map, it can calculate the input window positions corresponding to all points on the output feature map, thereby generating an ordered sequence of first window positions. This sequence is used to determine the precise memory address or cache address of the data required by each window on the input feature map during the convolution operation.

[0106] After obtaining the first window position sequence, the window sliding controller can pipeline and parallelize data loading and computation operations based on this sequence. Specifically, while the first multiply-accumulate array is performing data operations on the current window (i.e., performing a multiply-accumulate operation), the window sliding controller works in parallel, predicting and obtaining the next window position to be involved in the computation (i.e., the next window adjacent to the current window) based on the generated first window position sequence. Subsequently, it can initiate a loading request for the computational data required for that next window, preloading this data from memory or the previous level cache into the first buffer.

[0107] This allows for the overlap of computation and data handling. Once the first multiply-accumulate array completes the computation for the current window, the data for the next window is already ready in the first buffer, allowing the next round of computation to begin immediately. This effectively masks data access latency and significantly improves the throughput and overall computational efficiency of the convolution computation engine.

[0108] Optionally, the first buffer can be used to store the computed data of the next window adjacent to the current window, which is preloaded by the window sliding controller.

[0109] The first buffer can be used to store specific data initiated by the window sliding controller and preloaded from memory, specifically the calculated data of the next window that is spatially adjacent to the current window.

[0110] In one possible implementation, the first buffer includes an input feature map buffer, a convolution kernel buffer, and a first result accumulation buffer.

[0111] Specifically, the input feature map buffer can be used to store the input feature map of the next window adjacent to the current window when the data operation of the current window is performed on the first multiply-accumulate array.

[0112] While the first multiply-accumulate array is performing data operations on the current window, the buffer can store in parallel the input feature map data block corresponding to the next window adjacent to the current window, which is pre-fetched from memory by the window sliding controller.

[0113] In this way, at the end of the current computation cycle, it can be ensured that the input data required for the next computation cycle has been stored in the on-chip cache, thereby achieving parallel overlap of computation and data loading and reducing memory access latency.

[0114] The kernel buffer can be used to store the kernel parameters of the next window adjacent to the current window when the data operation of the first multiply-accumulate array is performed on the current window.

[0115] While the first multiply-accumulate array performs the multiply-accumulate operation for the current window, the kernel buffer can store the kernel parameters prepared for the next window's computation. Considering that kernel weights are typically reused across multiple window computations, this buffer can also enable weight reuse, for example, keeping weights resident across the entire input channel dimension. This further reduces the number of accesses to the memory system.

[0116] The first result accumulation buffer can be used to sum the multiply-accumulate partial results corresponding to each received and stored window to obtain the convolution calculation result.

[0117] Since convolution operations typically involve summing over multiple input channels, the first multiply-accumulate array may compute only a portion of the sums for a subset of channels or a window at a time. A first result accumulation buffer can be used to receive and temporarily store the multiply-accumulate partial results from each window's computation. These partial results from different computation cycles or channels can then be accumulated and summed to generate the complete, final convolution result for that window. This decomposes a large-scale convolution computation into multiple small-scale, pipelined operations, enabling high-throughput computation.

[0118] Optionally, the first multiply-accumulate array can be used to perform multiply-accumulate operations in parallel on the computational data of the current window read from the first buffer using each multiply-accumulate unit in the first multiply-accumulate array, so as to obtain the data computation result of the current window.

[0119] The first multiply-accumulate array consists of multiple multiply-accumulate units arranged in parallel.

[0120] Specifically, the first multiply-accumulate array can synchronously read all the computational data required for the current computation window from the first buffer (i.e., from the input feature map buffer and convolution kernel buffer mentioned above). Subsequently, the first multiply-accumulate array can initiate its highly parallel computation process. Each multiply-accumulate unit in the first multiply-accumulate array is assigned a specific data pair (one input data element and one weight parameter). Then, under unified control signal scheduling, each multiply-accumulate unit performs its respective multiplication operation in parallel, and then accumulates the product results into a designated accumulator.

[0121] In this way, the first multiply-accumulate array can complete the multiply-accumulate operation of all data points in the current window within a short time period through parallel computation, thereby obtaining the complete data operation result representing the convolution result of the window.

[0122] By employing the above technical solution, precise control of the convolution window through a sliding window controller and pre-loading of data for the next window in the first buffer achieves pipelined data processing, reducing idle waiting time and improving computational throughput. Simultaneously, the first multiply-accumulate array supports parallel computation, making the entire convolution operation more efficient.

[0123] In some embodiments, the computing unit 103 further includes a pooling processing unit. The pooling processing unit includes a window controller, a second buffer, and a pooling array.

[0124] Optionally, the window controller can be used to obtain the input feature map and the pooling parameters for performing pooling processing according to the calculation instructions; generate a second window position sequence of the output feature map according to the input feature map and the pooling parameters; and, while the pooling array performs data operations on the current window, preload the calculation data of the next window adjacent to the current window into the second buffer according to the second window position sequence.

[0125] The window controller first decodes the received pooling instruction (e.g., NNMAXPOOL or NNAVGPOOL). For example, it can parse and obtain the starting address and dimension information of the input feature map in memory, as well as the pooling parameters required to perform the current pooling operation, based on the operand and immediate value information contained in the instruction encoding. The input feature map and pooling parameters are obtained directly from the instruction field or from the configuration register specified by the instruction. The pooling parameters specifically include the pooling window size (e.g., 2x2, 3x3), stride, padding method, and pooling type (max pooling or average pooling).

[0126] The window controller can execute an address generation algorithm for the pooling sliding window based on the parsed input feature map dimensions and pooling parameters. Specifically, it can calculate the sequence of starting memory addresses of the input windows corresponding to all points on the output feature map according to the geometric rules of the pooling window sliding on the input feature map, thereby generating the second window position sequence. The second window position sequence can be used to define the location of the input data required for each computation window in memory or cache during the entire pooling operation.

[0127] In one possible implementation, an address generator can be used to load predefined operation parameters from the received pooling instruction or its associated configuration register to generate an address sequence. The predefined operation parameters may include, but are not limited to, the dimensional information (height and width) of the input feature map, the size (height and width) of the pooling window, the sliding step size, the padding size, and the starting base address of the input feature map data in memory.

[0128] Finally, the window controller can deeply pipeline pooling computations and data loading operations to reduce memory access latency. Specifically, while the pooling array is performing data operations on the current window (i.e., finding the maximum value or averaging), the window controller can work in parallel. Based on a pre-calculated sequence of second window positions, it accurately predicts and locates the next window to participate in the computation (i.e., the next window spatially adjacent to the current window). Subsequently, it sends a request to the memory subsystem or the parent cache to load all the computational data required for that next window, preloading this data into the second buffer.

[0129] The second buffer is used to store the computational data of the next window adjacent to the current window, which is preloaded by the window controller.

[0130] Specifically, the second buffer includes an input buffer and an output buffer; the output buffer includes multiple buffers.

[0131] The input buffer stores the input feature map of the next window adjacent to the current window while the pooled array is performing data operations on the current window. The output buffer uses a first output buffer to store the data operation results of each window until the remaining space in the first output buffer is less than or equal to a preset threshold. Then, it switches to a second output buffer to store the data operation results of each window, and transfers the data operation results of each window stored in the first output buffer to a specified address. The first output buffer can be any one of multiple output buffers, and the second output buffer can be any one of multiple output buffers other than the first output buffer. This multi-level output buffer design allows for automatic switching to another buffer when one buffer is nearing full, avoiding computational stagnation caused by waiting for writes. This double-buffering mechanism improves the continuity of data processing, thereby enhancing overall performance.

[0132] This ensures that pooling computation and data transport overlap in time. Once the pooling array completes the computation for the current window, the data for the next window is already ready in the second buffer, and the pooling processing unit can immediately and seamlessly begin the next round of computation without waiting for data transport.

[0133] Optionally, the pooling array can be used to perform pooling processing in parallel on the computational data of the current window read from the second buffer using each pooling unit in the pooling array, so as to obtain the data computation result of the current window.

[0134] The pooling array consists of multiple pooling units arranged in parallel.

[0135] For example, the pooling array can synchronously read all the computational data required for the current window to be computed from the second buffer, i.e., the input feature map data block covered by a complete pooling window. The pooling array can then initiate its highly parallel computation process. Specifically, each pooling unit in the pooling array can be configured to perform a specific pooling operation (such as maximum lookup or average calculation). Under unified control signal scheduling, each pooling unit can perform its respective operation in parallel on the current window data obtained from the second buffer.

[0136] For max pooling, the pooling unit is typically implemented using a comparison tree structure to quickly find the maximum value in the window of data. For average pooling, the pooling unit is implemented using an addition tree structure to sum all elements in the window, followed by a division operation (which can be implemented using bit shifting).

[0137] In this way, the pooling array can complete the pooling operation of the entire window data in a short time through parallel computing, thereby obtaining the final data operation result of the window.

[0138] By employing the above technical solution, the pooling processing unit achieves efficient pooling operations through the management of the pooling window by the window controller and the preloading of data for the next window by the second buffer. The parallel processing capability of the pooling array further improves the speed of pooling computation.

[0139] In some embodiments, the calculation unit 103 further includes an activation function unit. The activation function unit includes a function selector and a plurality of function processing units.

[0140] Optionally, a function selector is used to select and enable the target function processing unit corresponding to the instruction type from multiple function processing units based on the instruction type of the computation instruction.

[0141] Each function processing unit corresponds to a different instruction type.

[0142] In this embodiment, the function selector continuously monitors the instruction pipeline. When it identifies a neural network activation function instruction (e.g., NNRELU, NNSIGMOID, or NNTANH), it initiates its decoding and routing functions. It can be used to select and activate a target function processing unit that perfectly corresponds to the activation function type explicitly specified by the instruction opcode from among multiple function processing units integrated within the unit.

[0143] Each function processing unit is pre-designed and optimized to perform a specific, unique activation function computation. For example, one processing unit might specifically employ comparison and selection logic to implement the ReLU function, while another might efficiently compute the Sigmoid function using a combination of lookup tables and polynomial approximation. Therefore, each function processing unit can correspond to different instructions.

[0144] In this way, upon receiving an instruction, the function selector can directly activate the corresponding hardware computation path through hard-wired decoding logic without requiring complex software judgment or branch prediction. The selected target function processing unit then begins to work, applying its dedicated nonlinear transformation to the input data, while other unselected processing units remain in a static, low-power state.

[0145] Optionally, the function processing unit can be used to perform data processing using the target function processing unit corresponding to the instruction type.

[0146] By employing the above technical solution, the intelligent selection of different activation functions through a function selector can quickly invoke the most suitable processing unit, avoiding the complex conditional judgments and branch jumps found in general-purpose processors. This design not only improves the execution efficiency of activation functions but also enhances the system's flexibility.

[0147] In some embodiments, the calculation unit 103 further includes a matrix multiplication unit.

[0148] The matrix multiplication unit includes a third buffer, a second multiplication accumulation array, and a matrix dimension controller.

[0149] Optionally, the matrix dimension controller can obtain the input matrix and the array parameters of the second multiply-accumulate array according to the calculation instructions; based on the dimension parameters and array parameters of the input matrix, the input matrix is ​​divided into multiple sub-matrices, and while the data operation of the current sub-matrix is ​​performed in the second multiply-accumulate array, the next sub-matrix adjacent to the current sub-matrix is ​​preloaded into the third buffer.

[0150] In this embodiment, the matrix dimension controller first decodes the received matrix multiplication instruction (e.g., the NNGEMM instruction). Based on the operand and immediate value information contained in the matrix multiplication instruction encoding, it parses and obtains the starting address and dimension information of the input matrix to be processed in memory (e.g., the dimension of matrix A is M×K, and the dimension of matrix B is K×N), as well as the fixed hardware array parameters of the second multiply-accumulate array it controls, such as the size of the systolic array or the set of parallel multiply-accumulate units (e.g., a fixed 16x16 array).

[0151] The matrix dimension controller executes a matrix partitioning algorithm based on the parsed input matrix dimensions (M, K, N) and a fixed array size. It divides large input matrices that exceed the hardware's computational capabilities into multiple submatrices (or data blocks) with sizes matching the second-stage multiplication and accumulation array. This partitioning must follow the mathematical rules of matrix multiplication to ensure that the results of operations between submatrices can be correctly combined into the final product matrix through accumulation.

[0152] Finally, the matrix dimension controller can deeply pipeline computation and data loading. Specifically, while the second multiply-accumulate array performs data operations on the current submatrix pair (i.e., performs submatrix multiplication and accumulation operations), the matrix dimension controller works in parallel. Based on the submatrix sequence generated by its block strategy, it can accurately predict and locate the next submatrix pair to be involved in the computation (i.e., the subsequent block adjacent to the current operation block in the matrix rows and columns). Subsequently, it can initiate a load request to the memory subsystem for the data required for the next submatrix pair, preloading this data into the third buffer.

[0153] This ensures that matrix multiplication calculations and data transfer completely overlap in time. Once the second multiplication accumulation array completes the operation on the current submatrix pair, the data for the next set of submatrixes is already ready in the third buffer, and the calculation unit 103 can immediately and seamlessly begin the next round of calculations without waiting for data transfer.

[0154] Optional, a third buffer is used to store the next submatrix adjacent to the current submatrix, preloaded by the matrix dimension controller.

[0155] The third buffer includes a matrix buffer and a second result accumulation buffer.

[0156] Specifically, the matrix buffer is used to store the next submatrix adjacent to the current submatrix when the data operation of the current submatrix is ​​performed by the second multiply-accumulate array; the second result accumulation buffer is used to accumulate and merge the corresponding multiply-accumulate results of each received and stored submatrix in matrix dimension order to obtain the output matrix output by the matrix multiplication unit.

[0157] Optionally, a second multiply-accumulate array is used to perform multiply-accumulate operations on the current submatrix read from the third buffer using each multiply-accumulate unit in the second multiply-accumulate array to obtain the data operation result of the current submatrix.

[0158] Specifically, the second multiply-accumulate array can synchronously read all the data required for the current submatrix pair to be computed from the third buffer. This can include a sub-block decomposed from matrix A and a sub-block decomposed from matrix B. The second multiply-accumulate array can then initiate its highly parallel computation process. Each multiply-accumulate unit in the second multiply-accumulate array can be assigned a specific computational task, such as processing data pairs at specific row and column positions in the input submatrix. Under unified control signal scheduling, each multiply-accumulate unit performs its respective multiplication operation in parallel, and then accumulates the product results into a designated accumulator register for storing partial sums.

[0159] In this way, the second multiply-accumulate array can complete the multiply-accumulate operation on the current submatrix pair in a short time, thereby obtaining the data operation result (i.e., the partial sum matrix block) representing the intermediate result of the operation of the sub-block.

[0160] By employing the above technical solution, efficient matrix multiplication is achieved through the reasonable block division of large matrices by the matrix dimension controller and the preloading of the next submatrix by the third buffer. The parallel processing capability of the second-multiplication accumulation array further accelerates the calculation speed.

[0161] In some embodiments, the computing unit 103 further includes a quantization processing unit.

[0162] The quantization processing unit includes a quantization parameter register, a quantizer, and an inverse quantizer.

[0163] Optionally, the quantization parameter register can be used to store configuration parameters corresponding to quantization and dequantization operations; and transmit the first configuration parameter corresponding to the quantization operation to the quantizer, and the second configuration parameter corresponding to the dequantization operation to the dequantizer.

[0164] The quantization parameter register stores configuration parameters related to quantization and dequantization operations.

[0165] In this embodiment, when the quantization processing unit performs an operation, the quantization parameter register can route the parameter set corresponding to the instruction type to the corresponding calculation unit 103 according to the type of the current calculation instruction: when performing a quantization operation (converting a floating-point number to a fixed-point number), the quantization parameter register can transmit the first configuration parameter corresponding to the quantization operation (e.g., the scaling factor and zero point used for quantization) to the quantizer. When performing a dequantization operation (converting a fixed-point number back to a floating-point number), the quantization parameter register can transmit the second configuration parameter corresponding to the dequantization operation (e.g., the scaling factor used for dequantization, which may be different from that used during quantization) to the dequantizer.

[0166] In this way, through centralized parameter management and a feedforward parameter distribution mechanism, the quantization processing unit can achieve efficient and flexible numerical conversion. This ensures that the quantizer and dequantizer can acquire the required precise parameters with extremely low latency, without frequent access to main memory or parameter recalculation during computation, thus significantly reducing the latency and power consumption of quantization inference.

[0167] Optionally, the quantizer can be used to obtain first data to be quantized according to the calculation instructions; perform quantization conversion operation on the first data according to the first configuration parameters to obtain the quantized data operation result; and the dequantizer can be used to obtain second data to be dequantized according to the calculation instructions; perform dequantization conversion operation on the second data according to the second configuration parameters to obtain the dequantized data operation result.

[0168] By adopting the above technical solution, the efficient data conversion through the quantization processing unit can fully leverage the hardware potential and improve computational throughput in low-precision computing scenarios. Simultaneously, the quantization parameter register settings make quantization and dequantization operations more flexible, adaptable to various application scenarios.

[0169] Based on the foregoing embodiments, this application provides a data processing method. The method is applied to a data processing apparatus, which includes a decoding module, a data movement module, and multiple computing units, such as... Figure 3 As shown, the method may include the following steps S301 to S303.

[0170] Step S301: The instruction decoding module receives computation instructions based on the open-source instruction set architecture RISC-V extension;

[0171] In step S302, the data movement module responds to the calculation command and determines the prediction configuration parameters to be used by the target calculation unit corresponding to the calculation command.

[0172] In step S303, when executing the calculation instruction, the configuration parameters to be used by the target calculation unit are obtained using the target calculation unit corresponding to the calculation instruction; if the configuration parameters match the predicted configuration parameters, the unit is configured using the predicted configuration parameters to perform data operation processing.

[0173] By adopting the above technical solution, on the one hand, the neural network computation operations implemented by hundreds of basic arithmetic instructions in related technologies can be realized through RISC-V-based extended computation instructions. The entire computation process can be triggered with only a single extended instruction, thereby reducing the overhead of instruction decoding and scheduling. On the other hand, the configuration parameters of the target computing unit can be predicted in advance by the data movement module, and the parameter preparation can be completed before the execution of the computation instructions. This avoids the problem of the computing unit pausing and waiting for the configuration parameters to be loaded during the instruction execution process. The parameter configuration stage can be parallelized with the instruction decoding and data preparation stages, saving the execution time of a single instruction. Moreover, loading the configuration parameters in advance can avoid frequent repeated reading of data from memory during the computation process. When the configuration parameters match the predicted parameters, the computing unit can directly start the operation using the preloaded cached data without waiting for memory data transfer, which can improve data reuse efficiency and shorten data transfer latency.

[0174] In some embodiments, step S303 can be implemented by steps S3031 to S3033:

[0175] Step S3031: In response to the calculation instruction, determine the prediction configuration parameters to be used by the target calculation unit corresponding to the calculation instruction, and load the prediction configuration parameters from memory.

[0176] Optionally, the data access mode corresponding to the computation instruction is determined based on the instruction type and instruction parameters of the computation instruction; and the predicted configuration parameters to be used by the target computation unit are determined based on the data access mode.

[0177] Step S3032: Receive a data read instruction issued by the target computing unit when executing the computing instruction. The data read instruction includes the configuration parameters to be used by the target computing unit.

[0178] Step S3033: Send the parameters of the matched parameter items in the predicted configuration parameters to the target computing unit.

[0179] Optionally, the method further includes selecting the target parameter item that matches the configuration parameter from among the multiple parameter items included in the predicted configuration parameters; transferring the parameter of the target parameter item in the predicted configuration parameters to the target computing unit; and reloading the parameters of other parameter items that do not match the configuration parameter from memory; and transferring the parameters of the other parameter items to the target computing unit.

[0180] By employing the above technical solution, the pre-loading of predicted configuration parameters can effectively reduce the waiting time during configuration parameter reading and improve data processing efficiency. Simultaneously, by identifying and transmitting hit parameter items, unnecessary parameter transmissions are avoided, saving bandwidth resources and further enhancing system performance.

[0181] In this application, specific implementation methods can be referred to the description of any of the foregoing data processing devices. The descriptions of the above method embodiments are similar to those of the above device embodiments and have similar beneficial effects. For technical details not disclosed in the method embodiments of this application, please refer to the description of the device embodiments of this application for understanding.

[0182] The following describes the application of the data processing device provided in the embodiments of this application in a real-world scenario, mainly involving a neural network instruction set extension for RISC-V.

[0183] With the rapid development of artificial intelligence technology, neural networks have become one of the most mainstream machine learning methods. Neural network computation is highly parallel and systematic, mainly including operations such as convolution, pooling, activation functions, and fully connected layers. These operations are computationally intensive, especially convolution and matrix multiplication, which consume most of the computational resources during neural network inference and training.

[0184] RISC-V, as an open-source instruction set architecture, has gained widespread attention and application. Its openness and modular design make it very suitable for customized extensions. However, the standard RISC-V instruction set has the following shortcomings when handling neural network computations:

[0185] 1. The lack of dedicated instructions for neural network computation results in a large number of basic arithmetic instructions being required when performing neural network operations, leading to low execution efficiency.

[0186] 2. While existing SIMD extensions (such as "V" vector extensions) provide some parallel computing capabilities, they are not optimized for specific computational patterns of neural networks, such as the sliding of convolutional windows and the processing of channel dimensions of feature maps.

[0187] 3. Commonly used activation functions in neural networks (such as ReLU, Sigmoid, Tanh, etc.) require multiple basic instructions to be combined, which increases the number of instructions and the execution cycle.

[0188] 4. The lack of dedicated support for fixed-point number operations in quantized neural networks prevents the full utilization of hardware potential in low-precision computing scenarios.

[0189] 5. Existing architectures lack efficient mechanisms when processing sparse neural networks, and cannot skip zero-value calculations, resulting in a waste of computing resources.

[0190] In related technologies, on the one hand, while some dedicated processors for neural network computation exist, such as Google's TPU and NVIDIA's Tensor Core, which significantly improve neural network computation efficiency through customized hardware architectures and instruction sets, these solutions are mostly closed architectures, making them difficult to customize and extend flexibly. On the other hand, although there are methods to add neural network acceleration instructions to general-purpose processor architectures, such as ARM's MVE (M-Profile Vector Extension) and SVE (Scalable Vector Extension) which provide some vector computation capabilities, these extensions are not specifically designed for neural networks and still have efficiency bottlenecks when handling specific operations such as convolution.

[0191] Therefore, in view of the above reasons, the following technical problems are proposed in the embodiments of this application:

[0192] 1. The standard RISC-V instruction set is inefficient when performing neural network calculations, requiring a large number of basic instructions to complete a single neural network operation.

[0193] 2. The lack of hardware-level support for specific computational patterns of neural networks (such as convolution and pooling) means that these operations need to be implemented through software loops, which limits performance.

[0194] 3. Activation functions in neural networks are computationally complex and inefficient to implement on standard instruction sets.

[0195] 4. Low-precision calculations in quantized neural networks (such as INT8 and INT4) lack dedicated instruction support, making it impossible to fully utilize hardware resources.

[0196] 5. Zero-value handling in sparse neural network computation is inefficient and cannot effectively skip unnecessary calculations.

[0197] 6. Existing architectures struggle to effectively handle data reuse issues in neural networks, leading to frequent memory accesses and becoming a performance bottleneck.

[0198] In this application embodiment, to address the above-mentioned technical problems, a data processing apparatus, a chip product, a computer device, and a data processing method are provided. The data processing apparatus may include the following technical effects:

[0199] 1. By introducing dedicated neural network instructions, the number of instructions required to perform neural network operations is significantly reduced, thereby improving instruction efficiency.

[0200] 2. Provides hardware-level support for operations such as convolution and pooling, improving the execution speed of these operations by 5-10 times.

[0201] 3. Provides dedicated activation function instructions, simplifying activation function calculation from multiple instructions to a single instruction, thereby improving execution efficiency.

[0202] 4. Supports neural network computation with multiple precisions (FP32, FP16, INT8, INT4, etc.), enabling higher computational throughput in low-precision scenarios.

[0203] 5. The introduction of sparse computation support can effectively skip zero-value calculations, which can improve the performance of sparse neural networks by 2-4 times.

[0204] 6. Optimize data access patterns, reduce memory access frequency, lower memory bandwidth pressure, and improve energy efficiency.

[0205] 7. Maintain compatibility with the underlying RISC-V architecture, enabling a smooth transition of the existing software ecosystem to the new architecture.

[0206] 8. Compared to dedicated AI accelerators, this solution offers greater flexibility and programmability, enabling it to adapt to evolving neural network algorithms.

[0207] In this application embodiment, a Neural Network Extension for RISC-V (NNE-RV) is proposed. While maintaining compatibility with the basic RISC-V architecture, this extension introduces a series of instructions and registers specifically for neural network computation, which significantly improves the computational efficiency of neural networks.

[0208] Figure 4 This is an overall architecture diagram of a neural network instruction set extension system provided in an embodiment of this application, such as... Figure 4 As shown, the neural network instruction set extension system 400 may include a neural network dedicated register set 401, a neural network computing unit 402 (i.e., the computing unit in the above embodiment), a data prefetching and caching unit 403 (i.e., the data movement module in the above embodiment), and an instruction decoding and scheduling unit 404.

[0209] The neural network dedicated register group 401 includes, but is not limited to, vector registers and special-purpose registers. The vector registers can be 16 vector registers (nn0-nn15) of 128 bits each, used to store vector data in neural network computation. The special-purpose registers can be 4 special-purpose registers (nnscr0-nnscr3), used to store configuration and status information for neural network computation. The neural network computation unit 402 may include, but is not limited to, a convolution computation engine, a pooling processing unit, an activation function unit, a matrix multiplication unit, and a quantization processing unit. The data prefetching and caching unit 403 may include a feature map cache, a weight cache, and predictive configuration parameters to be used by the target computation unit corresponding to the computation instruction, using intelligent prefetching logic. The instruction decoding and scheduling unit 404 may include a neural network instruction decoder and an operation scheduler.

[0210] In this embodiment of the application, the neural network instruction set extension may include the following extended computation instructions:

[0211] 1. Data move command.

[0212] Specifically, this includes: NNLD rd,rs1,imm instructions, used to load data from memory into neural network registers; NNST rs2,rs1,imm instructions, used to store data from neural network registers into memory; and NNMV rd,rs instructions, used to move data between neural network registers.

[0213] 2. Convolution operation instructions.

[0214] Specifically, the instructions include: NNCONV rd,rs1,rs2,rs3, which are used to perform convolution operations, where rs1 points to the input feature map, rs2 points to the convolution kernel, rs3 contains the convolution parameters (stride, padding, etc.), and the result is stored in rd; NNDEPTHCONV rd,rs1,rs2,rs3, which are used to perform depthwise separable convolutions; and NNPOINTCONV rd,rs1,rs2, which are used to perform pointwise convolutions (1x1 convolutions).

[0215] 3. Pooling operation instructions.

[0216] Specifically, it includes: NNMAXPOOL rd,rs1,rs2 instructions, used to perform max pooling operations, where rs2 contains the pooling window size and stride; and NNAVGPOOL rd,rs1,rs2 instructions, used to perform average pooling operations.

[0217] 4. Activation function instructions.

[0218] Specifically, these include: NNRELU rd,rs instructions, used to execute the ReLU activation function; NNSIGMOID rd,rs instructions, used to execute the Sigmoid activation function; NNTANH rd,rs instructions, used to execute the Tanh activation function; and NNLEAKYRELU rd,rs,rs2 instructions, used to execute the Leaky ReLU activation function, where rs2 contains the slope parameter.

[0219] 5. Matrix operation instructions.

[0220] Specifically, it includes: the NNGEMM rd,rs1,rs2,rs3 instruction, used to perform general matrix multiplication, where rs3 contains matrix dimension information; and the NNMAC rd,rs1,rs2 instruction, used to perform vector multiplication and accumulation operations.

[0221] 6. Quantization processing instructions.

[0222] Specifically, it includes: the NNQUANT rd,rs1,rs2 instruction, which is used to quantize floating-point data into integers, where rs2 contains the quantization parameters; and the NNDEQUANT rd,rs1,rs2 instruction, which is used to convert quantized integers back into floating-point numbers.

[0223] 7. Sparse computation instructions.

[0224] Specifically, it includes: the NNSPARSE_CONV rd,rs1,rs2,rs3,rs4 instruction, used to perform sparse convolution, where rs4 contains sparse mode information; and the NNSPARSE_GEMM rd,rs1,rs2,rs3,rs4 instruction, used to perform sparse matrix multiplication.

[0225] Based on the above instructions, it can be concluded that the instruction encoding of the NNE-RV extension follows the RISC-V instruction format specification, adopts 32-bit fixed-length encoding, and defines multiple instruction formats according to different instruction types. The opcode field of all extended instructions uses the RISC-V custom instruction space (custom-0 / 1 / 2 / 3) to ensure that there is no conflict with the standard RISC-V instructions.

[0226] The R-type instruction format is mainly used for register-to-register operations, such as activation function instructions like NNRELU and NNSIGMOID. Its structure includes a 7-bit function code (funct7) (used to distinguish different instructions under the same opcode), a 5-bit source register 2 address (rs2), a 5-bit source register 1 address (rs1), a 3-bit additional function code (funct3), a 5-bit destination register address (rd), and a 7-bit opcode.

[0227] Type I instruction format is applicable to immediate operations and load operations, such as NNLD (load data from memory into a neural network register). This format contains a 12-bit immediate value imm[11:0], a 5-bit rs1 base register address, a 3-bit funct3 additional function code, a 5-bit rd destination register address, and a 7-bit opcode.

[0228] The S-type instruction format is used for storage operations, such as NNST (which stores data from the neural network register into memory). Its structure consists of the high 7 bits of the immediate value imm[11:5], the 5-bit rs2 source register address (the stored data), the 5-bit rs1 base register address, the 3-bit funct3 additional function code, the low 5 bits of the immediate value imm[4:0], and the 7-bit opcode.

[0229] In addition, the NN instruction format is a new format specifically defined for neural network operations. It supports three source operands and is suitable for complex neural network operations such as NNCONV and NNGEMM. This format includes a 4-bit function code (used to distinguish different neural network operations), a 5-bit source register 3 address (used to pass additional parameters, such as convolution parameters, matrix dimensions, etc.), a 5-bit source register 2 address (such as convolution kernel weights), a 5-bit source register 1 address (such as input feature maps), a 5-bit destination register address (stores the calculation results), and a 7-bit opcode.

[0230] Figure 5 This is a schematic diagram of the architecture of a convolutional computation engine provided in an embodiment of this application, as shown below. Figure 5 As shown, the convolution calculation engine 500 includes, but is not limited to, modules such as input feature map buffer 501, convolution kernel buffer 502, multiply-accumulate array 503 (i.e., the first multiply-accumulate array in the above embodiment), result accumulation buffer 504, and window sliding controller 505.

[0231] Specifically, the input feature map buffer 501 stores the currently processed input feature map data. It employs a double-buffered design, enabling parallel computation and data loading to improve overall efficiency. The convolution kernel buffer 502 stores the convolution kernel weights and supports convolution kernels of different sizes, such as 1x1, 3x3, and 5x5. The multiply-accumulate array 503 is the core of the computation, consisting of 8x8 multiply-accumulate units. Each unit supports multiply-accumulate operations with multiple precisions, including FP32, FP16, INT8, and INT4, and can be configured with different data flow modes, such as fixed weights or fixed inputs. The result accumulation buffer 504 accumulates part of the convolution results and supports accumulation operations with different precisions. The window sliding controller 505 controls the sliding of the convolution window on the input feature map, implementing different strides and padding strategies.

[0232] In actual execution, the workflow of the convolutional computation engine 500 can be triggered by the NNCONV (convolutional computation instruction) instruction, which includes the input feature map address, convolution kernel address, output feature map address, and convolutional parameters such as stride and padding. After the instruction is triggered, the window sliding controller 505 can calculate the position sequence of the sliding window based on these convolutional parameters; then, the input feature map data and convolution kernel weights are loaded from memory into their respective buffers; next, the multiply-accumulate array 503 starts to perform convolutional computation, multiplying and accumulating the window data of the input feature map with the convolution kernel weights, and storing the calculation result in the result accumulation buffer 504, completing the calculation of one output feature point; then, the window slides to the next position, repeating the above calculation and storage process until the calculation of the entire output feature map is completed; finally, the final result is written back to the specified output address.

[0233] Figure 6 This is a schematic diagram of the architecture of a pooling processing unit provided in an embodiment of this application, as shown below. Figure 6 As shown, the pooling processing unit 600 includes modules such as an input buffer 601, a maximum value calculation unit 602, an average value calculation unit 603, a window controller 604, and an output buffer 605; wherein, the maximum value calculation unit 602 and the average value calculation unit 603 are the pooling arrays in the above embodiments.

[0234] Specifically, firstly, the input buffer 601 is responsible for storing the feature map data to be pooled, which is loaded from the input memory. The window controller 604 plays a core control role throughout the process. It is used to calculate the position sequence of the pooling window on the feature map based on the pooling parameters (provided by the pooling parameters, including information such as window size and stride). It can also perform pooling operations with different stride lengths. Through functions such as window sliding control and stride implementation, it transmits control signals such as window position to the input buffer, guiding the input buffer to provide the corresponding window data.

[0235] Once the input buffer provides window data, the operation selector sends a control signal to either the maximum value calculation unit 602 or the average value calculation unit 603, based on the pooling type (determined by the pooling parameters), to select whether to perform maximum pooling or average pooling. The maximum value calculation unit 602 calculates the maximum value within the pooling window and supports pooling windows of different sizes, such as 2x2 and 3x3. The average value calculation unit 603 is responsible for calculating the average value within the pooling window; it is equipped with a configurable divider and also supports different window sizes.

[0236] After the calculation is complete, the maximum value result of the maximum value calculation unit 602 or the average value result of the average value calculation unit 603 is stored in the output buffer. The output buffer 605 supports a double buffering mechanism, which enables parallel computation and data transfer, improving overall efficiency. Finally, the pooling result is transferred from the output buffer 605 to the output memory, completing the output of the entire pooling operation.

[0237] In practical implementation, the workflow of the pooling processing unit 600 is triggered by instructions such as NNMAXPOOL (max pooling instruction) or NNAPGPOOL (average pooling instruction). These instructions contain key information such as the input feature map address, the output feature map address, and pooling parameters. After the instruction is triggered, the window controller 604 calculates the window position sequence based on the pooling parameters. Next, the input feature map data is loaded from memory into the input buffer. Then, depending on the instruction type, the corresponding maximum value calculation unit 602 or average value calculation unit 603 is activated to process the data within the window. Afterward, the calculation result is stored in the output buffer. Subsequently, the window slides to the next position, repeating the above data processing and storage steps until the calculation of the entire output feature map is completed. Finally, the result is written back to the specified output address.

[0238] Figure 7 This is a schematic diagram of the architecture of an activation function unit provided in an embodiment of this application, such as... Figure 7 As shown, the activation function unit 700 includes multiple function processing units and modules such as function selector 701; the function processing units may include, but are not limited to, ReLU processing unit 702, Sigmoid processing unit 703 and Tanh processing unit 703.

[0239] Specifically, firstly, after the input data enters the activation function unit 700, the function selector 701 determines which processing unit to use subsequently. The function selector 701 relies on instruction decoding. Instruction decoding parses the instructions that trigger the activation function operation (such as NNRELU, NNSIGMOID, or NNTANH), obtains information such as instruction type and function parameters, and then the function selector 701 selects the function type and configures the function parameters based on this information, and then sends a signal to the corresponding processing unit to select the function. The ReLU processing unit 702 implements the ReLU activation function, with the formula f(x) = max(0,x). It also supports the Leaky ReLU variant, with the formula f(x) = x (when x > 0), otherwise αx. Configuration parameters such as α participate in the calculation process. The Sigmoid processing unit 703 implements the Sigmoid activation function, with the formula f(x) = 1 / (1 + ex). To improve computational efficiency, a combination of lookup tables and linear interpolation can be used. A preliminary result is obtained through the lookup table, followed by linear interpolation to obtain a more accurate result. The Tanh processing unit 704 implements the Tanh activation function, with the formula f(x) = (ex - ex) / (ex + ex). It also uses a lookup table and linear interpolation method, first searching the lookup table and then using linear interpolation to obtain the final result.

[0240] In practical implementation, when the corresponding processing unit receives the function selection signal from the function selector 701, it applies the corresponding activation function to the input data. The ReLU processing unit 702 calculates directly according to the function formula, while the Sigmoid processing unit 703 and the Tanh processing unit 704 use lookup tables and linear interpolation for efficient calculation. After the calculation is completed, the ReLU processing unit 702 outputs the ReLU result, the Sigmoid processing unit 703 outputs the Sigmoid result, and the Tanh processing unit 704 outputs the Tanh result. These results are finally summarized into output data and written back to the specified output address, thus completing the entire activation function operation process.

[0241] Figure 8 This is a schematic diagram of the architecture of a matrix multiplication unit provided in an embodiment of this application, such as... Figure 8 As shown, the matrix multiplication unit 800 includes, but is not limited to, multiple buffers (i.e., the third buffer in the above embodiment), a multiply-accumulate array 801 (i.e., the second multiply-accumulate array in the above embodiment), and a matrix dimension controller 802, etc. Among them, the buffers include, but are not limited to, matrix buffer 803 (matrix A buffer, matrix B buffer) and result accumulation buffer 804 (i.e., the second result accumulation buffer in the above embodiment).

[0242] Specifically, matrix A buffer 803 is used to store the first input matrix. It has flexible data storage capabilities, supporting both row-major and column-major storage formats to adapt to different data access needs and facilitate subsequent calculations. Correspondingly, matrix B buffer 803 is responsible for storing the second input matrix, also supporting either row-major or column-major storage formats, ensuring that both input matrices can be temporarily saved in the most suitable way for computation.

[0243] The multiply-accumulate array 801, serving as the core of the computation, employs a 16x16 cell array structure. This scale design enables efficient handling of numerous multiply-accumulate operations in matrix multiplication. It supports computational operations of varying precision and can process various data types according to actual needs. Furthermore, its pulsating array architecture allows data to flow orderly within the array and be reused, significantly improving data utilization efficiency and reducing unnecessary data transmission overhead. The result accumulation buffer 804 plays a crucial role in temporarily storing some computational results during the computation process. It supports accumulation operations of varying precision, ensuring that intermediate results from staged computations are accurately accumulated, laying the foundation for the generation of the final result.

[0244] The matrix dimension controller 802 is used to control the calculation order of matrix multiplication, ensuring that the operation process is logically ordered. It can also flexibly handle matrix multiplication tasks of different dimensions. When encountering a large matrix, it will decompose it into small blocks suitable for multiply-accumulate array processing, making the calculation process more efficient.

[0245] In practice, the entire matrix multiplication unit is initiated by the NNGEMM (matrix multiplication instruction) instruction, which contains the addresses of the two input matrices, the address of the output matrix, and crucial matrix dimension information. Upon triggering the instruction, the matrix dimension controller 802 first begins working based on this dimension information, decomposing the large matrix into appropriate smaller blocks. Subsequently, the input matrix data is loaded from memory into the corresponding matrix A buffer 803 and matrix B buffer 803. Next, the multiply-accumulate array 801 initiates computation, performing matrix multiplication operations on the data in the buffers and storing a portion of the results in the result accumulation buffer 804. For large matrices, this loading and computation process is repeated until all smaller blocks are processed, completing the entire matrix multiplication calculation. Finally, the complete calculation result is written back to the output address specified by the instruction, thus completing the entire matrix multiplication operation.

[0246] Figure 9 This is a schematic diagram of the architecture of a quantization processing unit provided in an embodiment of this application, as shown below. Figure 9 As shown, the quantization processing unit 900 includes modules such as a quantizer 901, an inverse quantizer 902, and a quantization parameter register 903.

[0247] Specifically, quantizer 901 quantizes floating-point data (like FP32 and FP16) into integers (such as INT8 and INT4). In terms of quantization methods, it supports symmetric and asymmetric quantization, as well as channel-based and tensor-based quantization, to adapt to the data quantization needs of different scenarios, allowing floating-point data to be stored or used in computations more efficiently as integers. Dequantizer 902 is responsible for converting the quantized integers back to floating-point numbers. Furthermore, it supports various quantization modes corresponding to the quantizer, ensuring that the quantized and dequantized data accurately reproduces the original floating-point data characteristics, guaranteeing accuracy during the conversion process.

[0248] The function of the quantization parameter register 903 is to store the parameters required for quantization and dequantization, such as scaling factor and zero point.

[0249] This parameter is crucial for the accuracy of quantization and dequantization operations. Moreover, it supports dynamic updates and can be adjusted in a timely manner according to different quantization needs or data characteristics, providing accurate parameter support for quantizers and dequantizers and ensuring the smooth operation of the entire quantization process.

[0250] In practical implementation, when quantization or dequantization is triggered by the NNQUANT or NNDEQUANT instructions, the instructions include the input data address, the output data address, and the quantization parameter address. First, the quantization parameters (such as scaling factor, zero point, etc.) are loaded from the quantization parameter address into the quantization parameter register 903. This register is used to store the parameters required for quantization and dequantization, and supports dynamic updates of these parameters, providing necessary parameter support for subsequent operations.

[0251] Next, depending on the instruction type, either quantizer 901 or dequantizer 902 is activated. The quantizer is responsible for quantizing floating-point data (such as FP32, FP16) into integers (such as INT8, INT4). Quantizer 901 supports symmetric and asymmetric quantization, and can also perform channel-based and tensor-based quantization to adapt to different data quantization requirements. Dequantizer 902 is responsible for converting quantized integers back to floating-point numbers, and supports various quantization modes corresponding to the quantizer, ensuring that the dequantization process can accurately restore the data.

[0252] After activating the corresponding quantizer 901 or dequantizer 902, quantization or dequantization operations are applied to the input data (from the activation input or input data from other modules). After the operation is completed, the calculation result is written back to the specified output address, thus completing the entire quantization or dequantization process. This enables on-demand conversion of data between floating-point and integer forms to meet different computational or storage requirements.

[0253] Figure 10This is a schematic diagram of the architecture of a data prefetching and caching unit provided in an embodiment of this application, as shown below. Figure 10 As shown, the data prefetching and caching unit 1000 includes modules such as feature map cache 1001, weight cache 1002, prefetch controller 1003, and data flow controller 1004.

[0254] Specifically, the feature map cache 1001 stores the currently processed feature map data and adopts a multi-level cache structure. This structure can optimize data locality, allowing the feature map data to be accessed by the computing unit more efficiently. The weight cache 1002 is responsible for storing the weight parameters of the neural network and supports different cache replacement strategies to flexibly update the cache content according to the actual situation, ensuring that the cache stores the weight data that is more needed.

[0255] The prefetch controller 1003 predicts future data access scenarios based on the neural network's computational pattern. Before executing neural network instructions, it makes predictions based on the instruction type and parameters, and then preloads data from memory into the feature map cache or weight cache. This allows the computational unit to prioritize reading data from the cache when needed, reducing the number of memory accesses and improving data access efficiency.

[0256] The data flow controller 1004 is primarily responsible for controlling the flow of data between different computing units, while optimizing data reuse and reducing memory access. It allocates data rationally according to computing needs, enabling data to be reused in multiple computing processes, further improving overall computing efficiency.

[0257] The workflow of the entire data prefetching and caching unit is as follows: before executing neural network instructions, the prefetch controller 1003 predicts the data access pattern; then, based on the prediction results, it preloads data into the feature map cache or weight cache; subsequently, the computation unit reads data from the cache first; the data flow controller 1004 controls the flow of data between computation units to maximize data reuse; the cache updates its content according to the replacement strategy to maintain a high cache hit rate, thereby efficiently providing data support for neural network computation.

[0258] Figure 11 This is a schematic diagram of the execution architecture of a sparse computing method provided in an embodiment of this application, such as... Figure 11 As shown, the neural network instruction set extension system can also provide hardware support for sparse neural networks, effectively skipping zero-value calculations and improving computational efficiency.

[0259] When performing sparse neural network computations, dedicated sparse computation instructions, such as NNSPARSE_CONV or NNSPARSE_GEMM, can be used. These instructions carry the address and format information of the sparse data and can be used to start the entire processing pipeline.

[0260] First, the sparse format controller receives sparsity parameters (such as row pointers in CSR format, coordinate information, and thresholds in COO format) and the original dense data from the instructions. Based on these parameters, the sparse format controller parses and generates specific sparse format configuration signals, determining how the data is compressed, stored, and accessed.

[0261] Next, the sparse format manager, based on the configuration information issued by the controller, is actually responsible for processing the input sparse data. It supports various mainstream sparse storage formats such as CSR (Compressed Sparse Lines) and COO (Coordinate Format), and can efficiently parse out non-zero values ​​and their corresponding indexes / position information from the data. This process transforms the massive, dense data stream into two refined data streams: "non-zero values" containing only valid data and "indexes" indicating their positions.

[0262] After format parsing, valid data (non-zero values ​​and their indices) can be temporarily stored in a sparse buffer. The purpose of the sparse buffer is to reorganize and align the data to ensure that the computing unit can obtain non-zero data in an efficient and continuous manner, avoiding access bottlenecks caused by data sparsity.

[0263] The computation unit can obtain non-zero value data streams from the buffer unit, while the zero-value skipping logic processes index information in parallel. The zero-value skipping logic detects and predicts in real time whether an upcoming multiply-accumulate operation contains zero-value operands (whether input activation values ​​or weights). Once confirmed, it immediately sends a control signal to the computation unit to dynamically disable (skip) the associated multiplier array and accumulation operation. In other words, the computation unit only performs valid multiply-accumulate operations on non-zero values, thereby avoiding a large amount of unnecessary computation, saving power, and significantly improving computational efficiency.

[0264] Finally, the computation results are output. Dynamic sparsity can be supported through hardware instructions (such as NNPRUNE), allowing small weights to be reset to zero and the sparse format to be updated during inference based on a threshold. This enables the system to adaptively maintain high sparsity, thereby achieving continuous high-performance computing. Therefore, through hardware-level collaboration of sparse format control, dedicated data management, and intelligent zero-value skipping, efficient acceleration of the entire sparse computation process is achieved.

[0265] In this application embodiment, in order to ensure the efficient use of the NNE-RV extension, this application embodiment constructs a complete and multi-layered software support system, covering the entire process from low-level compilation to high-level model optimization, providing developers with a convenient and efficient development and optimization toolchain.

[0266] At the compiler support level, this application extends the LLVM / GCC compiler to natively support the NNE-RV instruction set. To meet different development needs, the compiler provides both an inline assembly interface, allowing developers to directly call NNE-RV instructions for low-level programming, and automatic vectorization and instruction selection optimization capabilities, which can automatically map high-level neural network operations to corresponding NNE-RV instructions. Simultaneously, by providing compiler built-in functions (Intrinsics), the calling method of NNE-RV instructions is simplified, lowering the barrier to entry for developers.

[0267] The library supports providing a concise and easy-to-use high-level API by encapsulating NNE-RV instructions. To adapt to the mainstream deep learning ecosystem, the library also includes interface layers with frameworks such as TensorFlow and PyTorch, enabling seamless integration. In addition, the library has optimized implementations of commonly used neural network layers such as convolutional layers, pooling layers, and fully connected layers based on NNE-RV instructions, and supports calculations with different precisions such as FP32, FP16, INT8, and INT4 to meet diverse performance and accuracy requirements.

[0268] The model compilation tool can convert trained neural network models into optimized code using NNE-RV instructions. For model optimization, the tool supports model quantization, which can convert floating-point models into low-precision integer models to improve computational efficiency. It also implements model pruning to generate sparse neural networks to make full use of sparse computation instructions. The accompanying model performance analysis tool can help developers accurately identify performance bottlenecks in model operation and provide direction for further optimization.

[0269] The simulators and debugging tools can extend mainstream RISC-V simulators such as QEMU and Spike to enable them to simulate the execution of NNE-RV instruction sets; the developed performance analysis tools can help developers locate system performance bottlenecks, while instruction-level debugging support provides convenience for debugging neural network applications; in addition, by implementing accuracy analysis tools, the impact of optimization operations such as quantization on model accuracy can be effectively evaluated, helping developers find a balance between performance and accuracy.

[0270] It should be noted that there are multiple alternative technical paths for the technical solution of this application.

[0271] Optionally, at the instruction encoding level, besides using a custom RISC-V instruction space, NNE-RV instructions can be encoded in other ways. One approach is to rely on RISC-V's standard extension mechanism to define entirely new standard extensions (such as the "N" extension) and integrate them into the RISC-V standard ecosystem. Another approach is to utilize RISC-V's reserved instruction space to identify neural network instructions through specific prefix instructions, thereby achieving differentiated instruction identification. In addition, a double-word (64-bit) instruction format can be adopted, providing more operands and function fields with a wider bit width to adapt to the needs of complex neural network instructions.

[0272] In terms of computing unit design, finer-grained SIMD design can be adopted to improve data parallel processing capabilities by supporting vector operations of different widths; reconfigurable computing arrays can also be introduced to dynamically adjust the connection method of computing units according to different neural network operations, thereby enhancing the flexibility of the architecture; for matrix multiplication operations, the dedicated design concept of tensor cores can be used to achieve targeted optimization of computational efficiency; at the same time, analog computing methods can be used to further improve the overall energy efficiency ratio.

[0273] In terms of storage hierarchy optimization, dedicated on-chip memory for neural networks can be used to achieve efficient data access through explicit software management; and a near-memory computing architecture can be adopted to deploy computing units close to storage units, fundamentally reducing the migration of data between storage and computing modules; in addition, a multi-level cache structure can be built, and dedicated caches can be designed for different types of neural network data such as feature maps and weights, thereby improving the data hit rate through differentiated caching strategies.

[0274] In terms of sparse computation support, the scope of support for sparse storage formats can be expanded to be compatible with more types such as CSC (Compressed Sparse Columns) and BSR (Block Sparse Rows) to adapt to different sparse data characteristics. On the other hand, a dynamic sparsification mechanism can be introduced to automatically adjust the sparse computation strategy based on the real-time characteristics of the data during operation, thereby achieving adaptive optimization. At the same time, the implementation of a sparse representation based on bitmaps can further compress the amount of index data and reduce the overhead of index maintenance and access.

[0275] Therefore, the Neural Network Instruction Set Extension (NNE-RV) for RISC-V architecture proposed in this application can significantly improve the performance of RISC-V architecture in neural network computing scenarios by introducing dedicated neural network instructions and computing units. While strictly maintaining compatibility with the basic RISC-V architecture, this extension performs in-depth optimization for the unique computing patterns of neural networks, incorporating core operations such as convolution, pooling, and activation functions into the hardware acceleration scope, thereby improving computational efficiency at the architectural level.

[0276] Furthermore, to adapt to diverse application needs, the NNE-RV extension supports multi-precision neural network computation from FP32 to INT4, and can be flexibly adjusted according to different accuracy and performance requirements of the scenario. At the same time, it provides hardware-level support for sparse neural networks, and can further reduce invalid computation overhead and enhance computational efficiency advantages through a mechanism that effectively skips zero-value calculations.

[0277] To ensure the efficient implementation and use of the extensions, this application also includes a complete software support system covering key components such as compilers, libraries, model compilation tools, and debugging tools. The compiler provides the foundation for instruction calls, the neural network basic operation library seamlessly integrates with mainstream deep learning frameworks, the model compilation tools can optimize and convert trained models, and the simulators and debugging tools provide support for development and optimization, collectively creating a convenient and efficient development environment for developers.

[0278] In practical application verification, the NNE-RV extension demonstrates significant advantages on typical neural network models. Compared to the standard RISC-V implementation, it offers improved feasibility and energy efficiency, effectively enhancing the core competitiveness of the RISC-V architecture in the field of artificial intelligence. The core innovation of this application's embodiments lies in constructing a complete neural network-specific instruction set and computing unit system. High-efficiency computation is achieved through deep adaptation and optimization of neural network computing modes, while a comprehensive software support system ensures the ease of use and practicality of the extension, forming a closed-loop technology that combines hardware acceleration and software support.

[0279] This application provides a chip product, such as... Figure 12 As shown, the chip product 1200 includes a data processing device 100 as described above.

[0280] This application provides a computer device, such as... Figure 13 As shown, the calculator device 1300 includes the aforementioned chip product 1200.

[0281] It should be noted that the descriptions of the various embodiments above tend to emphasize the differences between them, while their similarities or commonalities can be referred to interchangeably. The descriptions of the above embodiments of the device, storage medium, computer program, and computer program product are similar to the descriptions of the above method embodiments and have similar beneficial effects. For technical details not disclosed in the embodiments of the device, storage medium, computer program, and computer program product of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0282] Figure 14 This is a schematic diagram of the hardware entity of a computer device provided in an embodiment of this application, such as... Figure 14As shown, the hardware entity of the computer device 1400 includes a processor 1401 and a memory 1402, wherein the memory 1402 stores a computer program that can run on the processor 1401, and the processor 1401 executes the program to implement the steps in the method of any of the above embodiments.

[0283] The memory 1402 stores computer programs that can run on the processor. The memory 1402 is configured to store instructions and applications that can be executed by the processor 1401. It can also cache data to be processed or already processed (e.g., image data, audio data, voice communication data, and video communication data) in the processor 1401 and various modules in the computer device 1400. It can be implemented by flash memory or random access memory (RAM).

[0284] The processor 1401 executes the steps of the task execution method described above when executing the program. The processor 1401 typically controls the overall operation of the computer device 1400.

[0285] This application provides a computer storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the task execution method described above.

[0286] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0287] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0288] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various terminals that include one or any combination of the above-mentioned memories, such as mobile phones, computers, tablet devices, personal digital assistants, etc.

[0289] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above steps / processes do not imply a sequential order of execution; the execution order of each step / process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above embodiments of this application are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0290] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0291] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0292] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0293] Furthermore, in the various embodiments of this application, all functional units can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units. Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0294] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence or the part that contributes to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, magnetic disks, or optical disks.

[0295] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A data processing apparatus, characterized by, The data processing apparatus comprises an instruction decoding module, a data moving module and a plurality of computing units; wherein: The instruction decoding module is configured to receive a computing instruction based on an open source instruction set architecture RISC-V extension; The data moving module is configured to determine, in response to the computing instruction, a predicted configuration parameter to be used by a target computing unit corresponding to the computing instruction; The computing unit is configured to, when executing the computing instruction, acquire, by the target computing unit corresponding to the computing instruction, a configuration parameter to be used by the target computing unit; and if the configuration parameter hits the predicted configuration parameter, perform unit configuration by using the predicted configuration parameter to perform data operation processing.

2. The apparatus of claim 1, wherein, The data moving module is configured to: determine, in response to the computing instruction, a predicted configuration parameter to be used by a target computing unit corresponding to the computing instruction, and load the predicted configuration parameter from a memory; receive a data reading instruction issued by the target computing unit when executing the computing instruction, wherein the data reading instruction comprises a configuration parameter to be used by the target computing unit; send a parameter of a parameter item hit in the predicted configuration parameter to the target computing unit.

3. The apparatus of claim 2, wherein, The data moving module comprises a prefetch controller, a data flow controller and a cache unit; The prefetch controller is configured to determine a data access mode corresponding to the computing instruction according to an instruction type and an instruction parameter of the computing instruction; determine a predicted configuration parameter to be used by the target computing unit according to the data access mode, and load the predicted configuration parameter from a memory to the cache unit; The data flow controller is configured to select a target parameter item in which the configuration parameter hits from a plurality of parameter items included in the predicted configuration parameter, transmit a parameter of the target parameter item in the predicted configuration parameter to the target computing unit, and reload parameters of other parameter items in which the configuration parameter does not hit from the memory, and transmit the parameters of the other parameter items to the target computing unit; The cache unit is configured to store the configuration parameter of the target computing unit. In the case that the computing instruction is of a sparse computing type, 4. The apparatus of claim 3, wherein, The prefetch controller is configured to load index data of non-zero values from a memory, and load the predicted configuration parameter to the cache unit according to the index data; The data flow controller is configured to perform zero value data filtering on each of the parameter items, and transmit parameters of filtered parameter items to the target computing unit. The computing unit comprises a convolution computing engine; the convolution computing engine comprises a first buffer, a first multiply-accumulate array and a window sliding controller; 5. The apparatus of any one of claims 1-4, wherein, The window sliding controller is configured to obtain an input feature map and convolution parameters for performing convolution calculation according to the computing instruction; generate a first window position sequence of an output feature map according to the input feature map and the convolution parameters; and in the case that the first multiply-accumulate array performs data operation of a current window, load calculation data of a next window adjacent to the current window to the first buffer in advance according to the first window position sequence. ​ The first buffer is configured to store calculation data of a next window adjacent to the current window, which is pre-loaded by the window sliding controller. The first multiply-accumulate array is configured to perform multiply-accumulate operation on the calculation data of the current window read from the first buffer by each multiply-accumulate unit in the first multiply-accumulate array in parallel to obtain a data operation result of the current window.

6. The apparatus of claim 5, wherein, The first buffer includes an input feature map buffer, a convolution kernel buffer and a first result accumulation buffer. The input feature map buffer is configured to store input feature maps of a next window adjacent to the current window in a case where the first multiply-accumulate array performs data operation on the current window. The convolution kernel buffer is configured to store convolution kernel parameters of a next window adjacent to the current window in a case where the first multiply-accumulate array performs data operation on the current window. The first result accumulation buffer is configured to sum up multiply-accumulate partial results corresponding to each window received and stored to obtain a convolution calculation result.

7. The apparatus of any one of claims 1-4, wherein, The calculation unit further includes a pooling processing unit, and the pooling processing unit includes a window controller, a second buffer and a pooling array. The window controller is configured to obtain input feature maps and pooling parameters for performing pooling processing according to the calculation instruction, generate a second window position sequence of output feature maps according to the input feature maps and the pooling parameters, and pre-load calculation data of a next window adjacent to the current window to the second buffer according to the second window position sequence in a case where the pooling array performs data operation on the current window. The second buffer is configured to store calculation data of a next window adjacent to the current window, which is pre-loaded by the window controller. The pooling array is configured to perform pooling processing on the calculation data of the current window read from the second buffer by each pooling unit in the pooling array in parallel to obtain a data operation result of the current window.

8. The apparatus of claim 7, wherein, The second buffer includes an input buffer and an output buffer, and the output buffer includes a plurality of output buffers. The input buffer is configured to store input feature maps of a next window adjacent to the current window in a case where the pooling array performs data operation on the current window. The output buffer is configured to store a data operation result of each window by using a first output buffer until a remaining amount of the first output buffer is less than or equal to a preset threshold value, switch to store a data operation result of each window by using a second output buffer, and transmit the data operation result of each window stored in the first output buffer to a designated address, where the first output buffer is any one of a plurality of output buffers, and the second output buffer is any one of the plurality of output buffers except the first output buffer.

9. The apparatus of any one of claims 1-4, wherein, The calculation unit further includes an activation function unit, and the activation function unit includes a function selector and a plurality of function processing units. The function selector is configured to select a target function processing unit corresponding to an instruction type of the computing instruction from a plurality of function processing units; and each function processing unit corresponds to a different instruction type. The function processing unit is configured to perform data processing by using the target function processing unit corresponding to the instruction type.

10. The apparatus of any one of claims 1-4, wherein, The computing unit further comprises a matrix multiplication unit, the matrix multiplication unit comprising a third buffer, a second multiply-accumulate array, and a matrix dimension controller; The matrix dimension controller is configured to obtain an input matrix and array parameters of the second multiply-accumulate array according to the computing instruction; divide the input matrix into a plurality of sub-matrices according to dimension parameters of the input matrix and the array parameters, and pre-load a next sub-matrix adjacent to a current sub-matrix to the third buffer in a case that the second multiply-accumulate array performs data operation on the current sub-matrix; The third buffer is configured to store the next sub-matrix adjacent to the current sub-matrix pre-loaded by the matrix dimension controller; The second multiply-accumulate array is configured to perform multiply-accumulate operation on the current sub-matrix read from the third buffer by using each multiply-accumulate unit in the second multiply-accumulate array to obtain a data operation result of the current sub-matrix.

11. The apparatus of claim 10, wherein, The third buffer comprises a matrix buffer and a second result accumulation buffer; The matrix buffer is configured to store the next sub-matrix adjacent to the current sub-matrix in a case that the second multiply-accumulate array performs data operation on the current sub-matrix; The second result accumulation buffer is configured to accumulate and merge corresponding multiply-accumulate partial results of each sub-matrix received and stored in matrix dimension order to obtain an output matrix output by the matrix multiplication unit.

12. The apparatus of any one of claims 1-4, wherein, The computing unit further comprises a quantization processing unit, the quantization processing unit comprising a quantization parameter register, a quantizer, and a dequantizer; The quantization parameter register is configured to store configuration parameters corresponding to quantization and dequantization operations; transmit a first configuration parameter corresponding to a quantization operation to the quantizer, and transmit a second configuration parameter corresponding to a dequantization operation to the dequantizer; The quantizer is configured to obtain first data to be quantized according to the computing instruction; perform quantization conversion operation on the first data according to the first configuration parameter to obtain a data operation result after quantization; The dequantizer is configured to obtain second data to be dequantized according to the computing instruction; perform dequantization conversion operation on the second data according to the second configuration parameter to obtain a data operation result after dequantization.

13. A data processing method, characterized by, The method is applied to a data processing device, the data processing device comprising a decoding module, a data moving module, and a plurality of computing units; the method comprising: The instruction decoding module receives a computing instruction based on an open source instruction set architecture RISC-V extension; The data moving module determines, in response to the computing instruction, a predicted configuration parameter to be used by a target computing unit corresponding to the computing instruction; and In the execution of the computing instruction, a configuration parameter to be used by a target computing unit corresponding to the computing instruction is acquired; if the configuration parameter hits the predicted configuration parameter, the target computing unit is configured by using the predicted configuration parameter to perform data operation processing.

14. The method of claim 13, wherein, The configuration of the computing unit by using the predicted configuration parameter comprises: In response to the computing instruction, a predicted configuration parameter to be used by a target computing unit corresponding to the computing instruction is determined, and the predicted configuration parameter is loaded from the memory; A data read instruction issued by the target computing unit in the execution of the computing instruction is received, and the data read instruction comprises a configuration parameter to be used by the target computing unit; Parameters of a parameter item hit in the predicted configuration parameter are sent to the target computing unit.

15. The method of claim 14, wherein, The determination of the predicted configuration parameter to be used by the target computing unit corresponding to the computing instruction comprises: According to an instruction type and an instruction parameter of the computing instruction, a data access mode corresponding to the computing instruction is determined; According to the data access mode, a predicted configuration parameter to be used by the target computing unit is determined.

16. The method of claim 14, wherein, The sending of the parameters of the parameter item hit in the predicted configuration parameter to the target computing unit comprises: A target parameter item hit by the configuration parameter is selected from a plurality of parameter items included in the predicted configuration parameter; Parameters of the target parameter item in the predicted configuration parameter are transmitted to the target computing unit. The method further comprises: Parameters of other parameter items not hit by the configuration parameter are reloaded from the memory; The parameters of the other parameter items are transmitted to the target computing unit.

17. A chip product, characterized by The chip product comprises the data processing apparatus according to any one of claims 1 to 12.

18. A computer device, comprising: The computer device comprises the chip product according to claim 17.

Images