Data processing system and chip

Abstract

The application provides a data processing system and a chip, and relates to the technical field of data processing. The system comprises a master control unit, an instruction processing unit and a data processing unit. The master control unit is used for issuing a data processing instruction to the instruction processing unit. The instruction processing unit is used for analyzing the data processing instruction to determine the operator type indicated by the data processing instruction, and in the case that the operator type is a non-matrix multiplication-addition operation type, converting the data processing instruction into a data processing micro-instruction of a matrix multiplication-addition operation dimension according to a mapping rule corresponding to the operator type, and writing the data processing micro-instruction into a micro-instruction queue. The data processing unit is used for executing a data processing operation of the matrix multiplication-addition operation dimension according to the data processing micro-instruction in the micro-instruction queue. The chip hardware complexity is effectively reduced, the resource utilization rate and the overall computing efficiency are improved while multiple operators are uniformly accelerated.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data processing system and chip. Background Technology

[0002] Artificial intelligence (AI) chips are core hardware specifically designed to accelerate AI model computation. Deep neural networks and other AI models involve massive matrix and tensor operations during training and inference, placing extremely high demands on the parallel computing capabilities of chips.

[0003] Currently, AI chips typically employ dedicated hardware units to process different types of operators. Specifically, for different operator types such as convolution and fully connected layers, the chip integrates corresponding dedicated computing units (such as convolution engines and matrix multiplication / accumulation units). During runtime, the system assigns computational tasks to the appropriate hardware units for execution based on the operator type.

[0004] However, this heterogeneous design requires the chip to integrate multiple computing units, significantly increasing the chip area. At the same time, when running a specific model, a large number of hardware units are idle, resulting in low resource utilization. In addition, the switching between operators introduces additional pipeline overhead, further restricting the overall computing efficiency. Summary of the Invention

[0005] This application provides a data processing system and chip to solve the problems of high hardware complexity, low resource utilization and limited computing efficiency in related technologies. The data processing system of this application uses an instruction processing unit to uniformly convert data processing instructions of different operators into micro-instructions of matrix multiplication and addition operation dimensions according to the mapping rules corresponding to the operator type. The data processing unit then executes matrix operations in parallel based on the micro-instruction queue, thereby achieving unified acceleration of multiple operators while effectively reducing chip hardware complexity, improving resource utilization and overall computing efficiency.

[0006] A first aspect of this application provides a data processing system, which includes: a main control unit, an instruction processing unit, and a data processing unit;

[0007] The main control unit is used to issue data processing instructions to the instruction processing unit;

[0008] The instruction processing unit is used to parse the data processing instruction to determine the operator type indicated by the data processing instruction. If the operator type is not a matrix multiplication and addition operation type, the data processing instruction is converted into a data processing micro-instruction of the matrix multiplication and addition operation dimension according to the mapping rule corresponding to the operator type, and the data processing micro-instruction is written into the micro-instruction queue.

[0009] The data processing unit is used to perform data processing operations on the matrix multiplication and addition dimensions according to the data processing microinstructions in the microinstruction queue.

[0010] In some embodiments of this application, converting data processing instructions into data processing micro-instructions of matrix multiplication and addition operation dimensions according to the mapping rules corresponding to the operator type includes: mapping the data processing parameters in the data processing instructions to matrix dimension parameters of matrix multiplication and addition operations according to the mapping rules corresponding to the operator type; performing block processing on matrix multiplication and addition operations according to the matrix dimension parameters and the preset matrix calculation scale to determine the block dimension information; and generating data processing micro-instructions according to the block dimension information and the queue status of the micro-instruction queue.

[0011] In some embodiments of this application, mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of matrix multiplication and addition operations according to the mapping rules corresponding to the operator type includes: when the operator type is ordinary convolution, mapping the spatial size of the output feature map indicated in the data processing instruction to the number of rows of the left matrix, mapping the product of the number of input channels indicated in the data processing instruction and the convolution kernel size to the number of columns of the left matrix and the number of rows of the right matrix, and mapping the number of output channels indicated in the data processing instruction to the number of columns of the right matrix.

[0012] In some embodiments of this application, mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of matrix multiplication and addition operations according to the mapping rules corresponding to the operator type includes: when the operator type is transposed convolution, mapping the spatial size of the output feature map indicated in the data processing instruction to the number of rows of the left matrix, mapping the product of the number of output channels indicated in the data processing instruction and the convolution kernel size to the number of columns of the left matrix and the number of rows of the right matrix, and mapping the number of input channels indicated in the data processing instruction to the number of columns of the right matrix.

[0013] In some embodiments of this application, mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of matrix multiplication and addition operations according to the mapping rules corresponding to the operator type includes: when the operator type is convolution kernel gradient calculation, mapping the product of the convolution kernel size indicated in the data processing instruction and the number of input channels to the number of rows of the left matrix, mapping the product of the batch number indicated in the data processing instruction and the error map space size to the number of columns of the left matrix and the number of rows of the right matrix, and mapping the number of output channels indicated in the data processing instruction to the number of columns of the right matrix.

[0014] In some embodiments of this application, mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of the matrix multiplication and addition operation according to the mapping rules corresponding to the operator type includes: when the operator type is left matrix transpose, mapping the original number of rows of the left matrix indicated in the data processing instruction to the number of columns of the left matrix in the matrix multiplication and addition operation, and mapping the original number of columns of the left matrix indicated in the data processing instruction to the number of rows of the left matrix in the matrix multiplication and addition operation; when the operator type is right matrix transpose, mapping the original number of rows of the right matrix indicated in the data processing instruction to the number of columns of the right matrix in the matrix multiplication and addition operation, and mapping the original number of columns of the right matrix indicated in the data processing instruction to the number of rows of the right matrix in the matrix multiplication and addition operation.

[0015] In some embodiments of this application, the data processing microinstructions include loading microinstructions, loading flags, computation microinstructions, and storage microinstructions; the microinstruction queue includes a loading microinstruction queue, a loading flag queue, a computation microinstruction queue, and a storage microinstruction queue; generating data processing microinstructions and writing them into the microinstruction queue based on the block dimension information and the queue state of the microinstruction queue includes: generating loading microinstructions and loading flags based on the block dimension information and the queue state of the loading microinstruction queue, and writing the loading microinstructions and loading flags into the loading microinstruction queue and the loading flag queue; generating computation microinstructions based on the block dimension information and the queue state of the computation microinstruction queue, and writing the computation microinstructions into the computation microinstruction queue; and generating storage microinstructions based on the block dimension information and the queue state of the storage microinstruction queue, and writing the storage microinstructions into the storage microinstruction queue.

[0016] In some embodiments of this application, the data processing unit includes: an on-chip storage unit, a load control unit, a local data cache unit, a matrix operation unit, and a storage control unit; the load control unit is used to read load micro-instructions from the load micro-instruction queue and send the data to be processed loaded from the on-chip storage unit according to the load micro-instructions to the local data cache unit; the local data cache unit is used to read load flags from the load flag queue and write the received data to be processed into the local cache according to the load flags; and to read calculation micro-instructions from the calculation micro-instruction queue and send the matrix operation data read from the local cache according to the calculation micro-instructions to the matrix operation unit; the matrix operation unit is used to perform matrix multiplication and addition operations on the received matrix operation data to obtain the calculation result, and read storage micro-instructions from the storage micro-instruction queue and send the calculation result to the storage control unit according to the storage micro-instructions; the storage control unit is used to write the calculation result into the on-chip storage unit.

[0017] In some embodiments of this application, the data processing unit further includes a data post-processing unit connected between the matrix operation unit and the storage control unit, used to perform data processing operations on the calculation results of the matrix operation unit, so as to send the processed post-processed data to the storage control unit.

[0018] A third aspect of this application provides a chip comprising a data processing system as described in the first aspect of the present application.

[0019] In summary, the data processing system proposed in this application, by uniformly mapping non-matrix multiplication-addition operators to standard matrix multiplication-addition operations, can be compatible with the execution of multiple AI operators through a collaborative architecture of the main control unit, instruction processing unit, and data processing unit. This eliminates the need for dedicated hardware circuits for different operators, significantly reducing hardware design complexity and chip area overhead. Simultaneously, by relying on a micro-instruction queue to achieve efficient scheduling of data loading, computation, and storage, it fully unifies and reuses chip computing resources, greatly improving computing power utilization and overall computational efficiency, and enhancing the versatility and scalability of the AI ​​chip.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0022] Figure 1 A schematic diagram of a data processing system provided in an embodiment of this application;

[0023] Figure 2 A schematic diagram of a data processing unit provided in an embodiment of this application;

[0024] Figure 3 This is a schematic diagram of a specific data processing system provided in an embodiment of this application. Detailed Implementation

[0025] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0026] Artificial Intelligence (AI) chips are core hardware specifically designed to accelerate AI model computation. Deep neural networks and other AI models involve massive matrix and tensor operations during training and inference, placing extremely high demands on the parallel computing capabilities of chips. Matrix multiplication and addition operations, as a fundamental computational mode in AI models, directly impact the overall performance of the chip.

[0027] Currently, AI chips typically employ dedicated hardware units to handle different types of operators. Specifically, for different operator types such as convolution, pooling, and fully connected layers, the chip integrates corresponding dedicated computing units (such as convolution engines, pooling units, and matrix multiplication and addition units). During runtime, the system assigns the computational tasks to the appropriate hardware units for execution based on the operator type.

[0028] However, the above solution has the following drawbacks:

[0029] 1) Large chip area. Due to the need to integrate multiple heterogeneous computing units, the hardware size of the chip increases significantly, leading to higher manufacturing costs;

[0030] 2) Low resource utilization. When running a specific AI model, often only some hardware units are active, while others remain idle for extended periods, resulting in a waste of hardware resources;

[0031] 3) Limited computational efficiency. Switching between different operators requires additional pipeline overhead (such as pipeline emptying, data reloading, etc.), which further restricts the improvement of overall computational performance.

[0032] To address the problems existing in related technologies, this application proposes a data processing system that unifies various non-matrix multiplication and addition operators into matrix multiplication and addition operations and generates corresponding microinstructions for execution, thereby simplifying the hardware architecture while improving the chip's versatility, computational efficiency, and scalability.

[0033] The data processing system and chip provided in this application will be described in detail below with reference to the accompanying drawings.

[0034] Figure 1 This is a schematic diagram of a data processing system provided in an embodiment of this application. Figure 1 As shown, the system includes:

[0035] The main control unit, instruction processing unit, and data processing unit;

[0036] The main control unit is used to issue data processing instructions to the instruction processing unit;

[0037] The instruction processing unit is used to parse the data processing instruction to determine the operator type indicated by the data processing instruction. If the operator type is not a matrix multiplication and addition operation type, the data processing instruction is converted into a data processing micro-instruction of the matrix multiplication and addition operation dimension according to the mapping rule corresponding to the operator type, and the data processing micro-instruction is written into the micro-instruction queue.

[0038] The data processing unit is used to perform data processing operations on the matrix multiplication and addition dimensions according to the data processing microinstructions in the microinstruction queue.

[0039] Among them, data processing instructions are task commands issued by the main control unit, such as calculating a normal convolution, calculating the gradient of the convolution kernel, or performing a matrix transpose. For example, the main control unit can issue corresponding data processing instructions when specific operators (such as normal convolution, transposed convolution, convolution kernel gradient calculation, or matrix transpose) need to be executed, according to the execution order of the model computation graph.

[0040] Operator type refers to the specific operation type of a data processing task. In this application, operator types can be divided into two categories: matrix multiplication-addition operation type and non-matrix multiplication-addition operation type. Matrix multiplication-addition operation type refers to operations that directly use matrix multiplication and addition as their core (such as ordinary matrix multiplication and matrix multiplication-addition); non-matrix multiplication-addition operation type refers to operations that cannot be directly performed using matrix multiplication and addition (such as ordinary convolution, transposed convolution, convolution kernel gradient calculation, matrix transposition, etc.). The instruction processing unit of this application can identify the operator type indicated by the instruction by parsing the opcode or specific fields in the instruction. For example, when the instruction processing unit reads a data processing instruction whose opcode field is 0x01, it determines that the operator type corresponding to the instruction is ordinary convolution; if the opcode field is 0x02, it determines that the operator type is transposed convolution, and so on, with different opcode values ​​corresponding to different operator types.

[0041] Mapping rules are pre-defined transformation rules for different non-matrix multiplication and addition operators. The core is to convert the parameters of non-matrix multiplication and addition operations (such as the channels of convolution and the size of the convolution kernel) into the parameters of matrix multiplication and addition operations (M, K, N dimensions).

[0042] The data processing microinstructions for matrix multiplication and addition are low-level execution instructions generated by the instruction processing unit after conversion. They are the commands that the data processing unit needs to execute at each stage. It is important to note that regardless of the original operator, this application will ultimately convert it into matrix multiplication and addition microinstructions with a unified format. This allows the data processing unit to focus on executing only one type of instruction, simplifying hardware design.

[0043] The microinstruction queue is a temporary storage queue (essentially FIFO, first-in, first-out) for the generated data processing microinstructions, equivalent to an instruction waiting area. In this application, after the instruction processing unit generates data processing microinstructions, it first writes them into the microinstruction queue. The data processing unit then reads the microinstructions from the queue sequentially for execution, avoiding instruction congestion and execution chaos, and achieving an orderly connection between issuing and executing instructions.

[0044] In some embodiments of this application, the instruction processing unit and the main control unit interact via an instruction FIFO. When deciding whether to issue a new data processing instruction, the main control unit checks the state of the instruction FIFO and whether there is an address dependency or conflict between the new instruction and an already executed instruction. Only if the instruction FIFO is not full and the new instruction has no address dependency or conflict with previous instructions in on-chip storage will the new instruction be written into the instruction FIFO. The instruction processing unit reads the data processing instructions issued by the main control unit from the instruction FIFO and parses them, generating microinstructions required by each unit module in the data processing unit based on the parsing results.

[0045] Specifically, the instruction processing unit first maps the data processing parameters in the data processing instruction to matrix dimension parameters for matrix multiplication and addition operations according to the mapping rules corresponding to the operator type. These matrix dimension parameters include the number of rows M, columns K, and columns N of the left matrix. Next, based on the mapped matrix dimension parameters and the preset matrix computation unit size, the instruction processing unit performs block processing on the matrix multiplication and addition operations to obtain block dimension information. This block dimension information includes the number of row blocks α, column blocks β, and the number of blocks in the K dimension of the output matrix.

[0046] The block processing procedure is as follows:

[0047] If the size of the left matrix is ​​M×K and the size of the right matrix is ​​K×N, then the size of the output matrix is ​​M×N.

[0048] The output matrix is ​​divided into α rows and β columns according to the size of the matrix computation unit (0 < α). <M,0<β<N);

[0049] The input left matrix is ​​divided into α rows and γ columns according to the size of the matrix computation unit (0 < α). <M,0<γ<K);

[0050] The input right matrix is ​​divided into β rows and γ columns (0 < β) according to the size of the matrix computation unit. <N,0<γ<K)。

[0051] Assuming the matrix multiplication unit can compute matrix multiplications of size a×b (left matrix block) and b×c (right matrix block) per clock cycle, the output matrix is ​​divided into α rows and β columns (α=ceil(M / a), β=ceil(N / c)), the input left matrix is ​​divided into α rows and γ columns (γ=ceil(K / b)), and the input right matrix is ​​divided into β rows and γ columns. This determines the block dimension information, including the number of row blocks α, the number of column blocks β, and the number of blocks in dimension K γ of the output matrix. Based on the above block division, the total number of computation clock cycles is t=(M×N×K) / (a×b×c), thus determining that the left matrix data loading bandwidth is at least (M×K) / t, the right matrix data loading bandwidth is at least (N×K) / t, and the output matrix data storage bandwidth is at least (M×N) / t.

[0052] After completing block processing, the instruction processing unit can generate corresponding data processing micro-instructions based on the block dimension information and the queue status of each micro-instruction queue. Data processing micro-instructions include load micro-instructions, load flags, computation micro-instructions, and storage micro-instructions, with corresponding micro-instruction queues including a load micro-instruction queue, a load flag queue, a computation micro-instruction queue, and a storage micro-instruction queue. Specifically, the instruction processing unit generates load-related micro-instructions and data flags based on the status of the load micro-instruction queue and the load flag queue, writing them when both the load micro-instruction queue and the load flag queue are not full; it generates computation control-related micro-instructions based on the status of the computation micro-instruction queue, writing them when the computation micro-instruction queue is not full; and it generates storage-related micro-instructions based on the status of the storage micro-instruction queue, writing them when the storage micro-instruction queue is not full. Through this method, the instruction processing unit writes load micro-instructions and load flags to the load micro-instruction queue and the load flag queue, writes computation micro-instructions to the computation micro-instruction queue, and writes storage micro-instructions to the storage micro-instruction queue, thereby achieving parallel execution and seamless integration of the load, computation, and storage three-stage pipeline.

[0053] Furthermore, regarding the mapping rules mentioned above, the mapping rules for different operator types can be the same or different. The specific mapping methods are as follows:

[0054] For a common convolution operator, which is often used in convolutional or fully connected layers, it can be represented as Y=X. W+b.

[0055] The mathematical formula for ordinary convolution is:

[0056] ;

[0057] Where Y is the output feature map, containing the batch number n, the number of output channels co, the output height ho, and the output width wo; X is the input feature map, containing the number of input channels ci, the input height hi, and the input width wi; W is the convolution kernel, containing the number of output channels co, the number of input channels ci, the kernel height ky, and the width kx; b (patial_sum) is the intermediate sum or calculation result bias; sy is the height direction step, and sx is the width direction step.

[0058] In this application, the spatial dimensions ho×wo of the output feature map are mapped to the number of rows M of the left matrix, the product ci×ky×kx of the number of input channels and the kernel size is mapped to the number of columns K of the left matrix and the number of rows K of the right matrix, and the number of output channels co is mapped to the number of columns N of the right matrix.

[0059] The transposed convolution operator is primarily used for upsampling scenarios, such as restoring high-resolution feature maps from low-resolution features in semantic segmentation and generative models (GANs). It can be understood as the inverse process of regular convolution. Its mathematical expression is the same as regular convolution, but the roles of input and output are reversed; the input is the output of a regular convolution, and the output is the input of a regular convolution. The convolution kernel can be the original kernel or its transpose. Its expression is:

[0060] ;

[0061] The output feature map Y contains the batch number n, the number of output channels ci, the output height hi, and the output width wi; the input feature map X contains the number of input channels co, the input height ho, and the input width wo; the dimensions of the convolution kernel W are the same as those of a regular convolution, i.e., the kernel height ky and the width kx; b (patial_sum) is the intermediate sum or the bias of the calculation result; sy is the step size in the height direction, and sx is the step size in the width direction.

[0062] In this application, the spatial dimensions hi×wi of the output feature map are mapped to the number of rows M of the left matrix, the product co×ky×kx of the number of output channels and the kernel size is mapped to the number of columns K of the left matrix and the number of rows K of the right matrix, and the number of input channels ci is mapped to the number of columns N of the right matrix.

[0063] The convolution kernel gradient calculation operator is used to calculate the gradient of the loss function with respect to the weights W, i.e. L / W. The computation process is essentially a convolution of the input feature map and the error map, where the error map is the product of the gradient of the loss function with respect to the current layer's output and the gradient of the activation function. The computation process is as follows:

[0064] 1) L / A: The gradient of the loss with respect to the output of the current layer (obtained through backpropagation from the next layer);

[0065] 2) A / Z: The gradient of the activation function with respect to the linear output;

[0066] 3) L / Z=( L / A) ( A / Z): The gradient of the current layer's linear output;

[0067] 4) L / W=( L / Z) X T .

[0068] Where L represents the loss function, used to measure the error between the model's prediction and the true label; A represents the output of the activation function, i.e., the feature map after non-linear transformation; Z represents the output of the linear layer (i.e., the result calculated without the activation function), usually the direct output of a convolutional or fully connected layer; T represents the transpose operation, such as X T This represents the transpose of the input feature map X.

[0069] The mathematical formula is:

[0070] ;

[0071] The input feature map X includes the batch number n, the number of input channels ci, the input height hi, and the input width wi; the error map... It includes the number of output channels co, the height of the error map ho, and the width of the error map wo; the kernel gradient output includes the number of output channels co, the number of input channels ci, the height of the kernel ky, and the width kx; patial_sum: intermediate sum or the bias of the calculated result.

[0072] In this application, the product of the convolution kernel size and the number of input channels, ky×kx×ci, is mapped to the number of rows M of the left matrix; the product of the batch number and the error map space size, n×ho×wo, is mapped to the number of columns K of the left matrix and the number of rows K of the right matrix; and the number of output channels, co, is mapped to the number of columns N of the right matrix.

[0073] For the matrix transpose operator, the calculation process for transposing the left matrix, the right matrix, or both matrices simultaneously is the same as that for ordinary matrix calculations. Only the dimensions need to be interchanged, and the corresponding transpose operation is completed when the data is written to or read from the local cache.

[0074] In this application, when the operator type is left matrix transpose, the original number of rows of the left matrix is ​​mapped to the number of columns of the left matrix in the matrix multiplication-addition operation, and the original number of columns of the left matrix is ​​mapped to the number of rows of the left matrix in the matrix multiplication-addition operation. When the operator type is right matrix transpose, the original number of rows of the right matrix is ​​mapped to the number of columns of the right matrix in the matrix multiplication-addition operation, and the original number of columns of the right matrix is ​​mapped to the number of rows of the right matrix in the matrix multiplication-addition operation. When the data processing instruction instructs that both the left and right matrices be transposed simultaneously, the dimensions of the left and right matrices are interchanged according to the above method.

[0075] In some embodiments of this application, a three-stage pipeline planning for matrix multiplication is also implemented, dividing the control flow into three independent parts: Load, Calc, and Store, thereby improving instruction execution efficiency and ultimately enhancing overall performance. Specifically, the data processing microinstructions include load microinstructions, load flags, calculation microinstructions, and store microinstructions, and the corresponding microinstruction queues include a load microinstruction queue, a load flag queue, a calculation microinstruction queue, and a store microinstruction queue.

[0076] The instruction processing unit generates corresponding data processing micro-instructions and writes them to the corresponding queues based on the block dimension information and the queue status of each micro-instruction queue. Specifically, the instruction processing unit generates load micro-instructions and load flags based on the block dimension information and the queue status of the load micro-instruction queue, and writes the load micro-instructions and load flags to the load micro-instruction queue and load flag queue, respectively; it generates computation micro-instructions based on the block dimension information and the queue status of the computation micro-instruction queue, and writes the computation micro-instructions to the computation micro-instruction queue; and it generates storage micro-instructions based on the block dimension information and the queue status of the storage micro-instruction queue, and writes the storage micro-instructions to the storage micro-instruction queue. Through the parallel execution mechanism of the above three-stage pipeline, the decoupling and parallelism of the loading, computation, and storage stages are achieved, effectively reducing the waiting time between each stage, thereby improving instruction efficiency and overall performance.

[0077] The information contained in the loading microinstruction varies depending on the operator type. For ordinary matrices (including ordinary matrices, left matrix transposes, right matrix transposes, or simultaneous transposes of both matrices), the loading microinstruction includes the following information: the coordinates (m_start, n_start, k_start) of the top-left corner of each block after the output matrix is ​​divided; the operator type flag of the current instruction, indicating whether it is ordinary convolution, transposed convolution, convolution kernel gradient calculation, ordinary matrix, left matrix transpose, or right matrix transpose, where both left and right matrix transpose flags are valid to indicate simultaneous transpose of both matrices; the instruction start flag and instruction end flag, used for the control flow of the data loading unit; the data type flag, indicating the data types supported by this computing device, affecting the data loading unit to generate corresponding data loading requests based on bandwidth requirements; and the starting addresses of the left and right matrix data in the on-chip memory unit.

[0078] For a standard convolution operator, generating microinstructions requires mapping the matrix block coordinates back to the coordinates of the output feature map and the input convolution kernel. Specifically, after the output matrix is ​​divided into blocks, the matrix data coordinates corresponding to the top-left corner of each block are mapped back to the coordinates of the output feature map, including the batch number n_start, the starting position of the output feature map height ho_start, the starting position of the output feature map width wo_start, and the starting position of the output channel co_start, where wo_start = M%wo, ho_start = ceil(M / wo), and co_start = N. Simultaneously, these coordinates are also mapped back to the coordinates of the input convolution kernel, including the starting position of the output channel co_start, the starting position of the input channel ci_start, the starting position of the convolution kernel width kx_start, and the starting position of the height ky_start, where co_start = N, ci_start = K%ci, kx_start = ceil(K / ci)%kx, and ky_start = ceil(K / (ci×kx))%ky. In addition, loading microinstructions also includes operator type flags, instruction start flags, instruction end flags, data type flags, and the starting addresses of the input feature map data and convolution kernel data in the on-chip storage unit.

[0079] For the convolution kernel gradient calculation operator, the generation of loading microinstructions requires back-mapping the matrix block coordinates back to the coordinates of the output weight gradient and the input error map. Specifically, after the output matrix is ​​divided into blocks, the matrix data coordinates corresponding to the top left corner of each block are back-mapping back to the coordinates corresponding to the output weight gradient, including the output channel start position co_start, the input channel start position ci_start, the convolution kernel width start position kx_start, and the height start position ky_start, where ci_start=M%ci, kx_start=ceil(M / ci)%kx, ky_start=ceil(M / (ci×kx))%ky, and co_start=N; at the same time, these coordinates are also back-mapping back to the coordinates of the input error map, including the batch number start position n_start, the error map height start position ho_start, and the width start position wo_start, where wo_start=K%wo, ho_start=ceil(K / wo)%ho, and n_start=ceil(K / (wo×ho))%N. Loading microinstructions also includes operator type flags, instruction start flags, instruction end flags, data type flags, and the starting addresses of input feature map data and input error map data in on-chip storage units.

[0080] Understandably, during the data loading process, this application can calculate the position information of the output blocks based on the matrix, and deduce the required input feature map and the reading position of the convolution kernel through the mapping rules, thereby realizing the loading logic of determining the input information from the output information.

[0081] During the generation of loading microinstructions, the instruction processing unit also generates corresponding loading flags and writes them to the loading flag queue. Loading flags are used to indicate the characteristics of the loaded data to the local data cache unit, enabling the local data cache unit to identify the data characteristics and reorganize the data before writing it to the local cache (such as a register set). Loading flags include, but are not limited to, the following information: operator type (indicating ordinary convolution, transposed convolution, convolution kernel gradient calculation, ordinary matrix, left matrix transpose, or right matrix transpose); whether there is intermediate or result bias; and data type, such as 4-bit, 8-bit, 16-bit, or 32-bit.

[0082] For the generation of computational microinstructions, this application provides two granularity generation methods. The first method generates computational instructions at the granularity of the matrix multiplication size that can be computed by the matrix operation unit per clock cycle. The instruction content includes: the α and γ values ​​of the input left matrix corresponding to the current computational output matrix; the β and γ values ​​of the input right matrix corresponding to the current computational output matrix; the valid data size (a_valid, b_valid) in the current input left matrix block, used to mask invalid data when the height of the left matrix block is less than a or the width is less than b, to prevent errors in the calculation result; the valid data size (b_valid, c_valid) in the current input right matrix block, used to mask invalid data when the height of the right matrix block is less than b or the width is less than c; the accumulation pointer identifier (PC) of the current input data calculation result, used to determine which intermediate sum to accumulate the calculation result with, the accumulation pointer is composed of information of the α and β values; and the operator type of the current computation. The second approach generates computation instructions at the granularity of matrix multiplications that can be computed by the matrix operation unit over multiple clock cycles (i.e., burst-mode long instructions). The instructions include: calculating the α and γ values ​​of the first left matrix corresponding to the output matrix over multiple clock cycles; calculating the β and γ values ​​of the first right matrix corresponding to the output matrix over multiple clock cycles; the left matrix input size M×K, based on which the computation unit generates valid data information; the right matrix input size N×K, based on which the computation unit generates valid data information; and the type of operator currently being computed. Using burst-mode long instructions reduces the overhead of handshaking or interaction with the main control unit when executing multiple instructions, thereby improving multi-instruction execution performance.

[0083] For the generation of stored microinstructions, the instruction processing unit generates corresponding computation result feature information based on the input instruction information. Specifically, the stored microinstructions include, but are not limited to, the following: when the operator type is matrix operation, the size of the output matrix is ​​M×N; when the operator type is ordinary convolution or transposed convolution, the size of the output feature map (n, co, ho, wo); when the operator type is convolution kernel gradient calculation, the size of the output convolution kernel gradient (co, ci, ky, kx); and the starting address of the output data in the on-chip storage unit.

[0084] In some embodiments of this application, such as Figure 2 As shown, this application provides a schematic diagram of a data processing unit. (Refer to...) Figure 2 The data processing unit includes an on-chip storage unit, a load control unit, a local data cache unit, a matrix operation unit, and a storage control unit.

[0085] The load control unit reads load microinstructions from the load microinstruction queue, loads data to be processed from the on-chip memory according to the load microinstructions, and sends the loaded data to be processed to the local data cache unit. The local data cache unit reads load flags from the load flag queue, writes the received data to be processed into the local cache according to the load flags, and reads computation microinstructions from the computation microinstruction queue, reads matrix operation data from the local cache according to the computation microinstructions, and sends it to the matrix operation unit. The matrix operation unit performs matrix multiplication and addition operations on the received matrix operation data, obtains the calculation result, reads storage microinstructions from the storage microinstruction queue, and sends the calculation result to the storage control unit according to the storage microinstructions. The storage control unit writes the calculation result into the on-chip memory unit.

[0086] Optionally, the data processing unit further includes a data post-processing unit connected between the matrix operation unit and the storage control unit. This data post-processing unit performs data processing operations on the calculation results of the matrix operation unit and sends the processed post-processed data to the storage control unit.

[0087] The loading control unit reads data from the on-chip storage unit, preprocesses the data according to the loading microinstruction information, and then forwards the processed data to the local data cache unit. Specifically, the loading control unit can perform at least one of the following operations as needed: data alignment operation, that is, right-aligning or left-aligning each batch of read data according to the protocol with the local cache unit and the characteristic information of the read data; invalid data removal operation, that is, removing invalid data from the read data according to the protocol with the local cache unit and the characteristic information of the read data, where invalid data may be located in the high, low, or middle bits of the data; data reassembly or splicing operation, that is, recombining or splicing the read data according to the interface protocol with the local cache unit, generating characteristic information of the combined or spliced ​​data, and forwarding it to the local cache unit.

[0088] The local data cache unit can be designed to cache data for one or more clock cycles based on the computational characteristics of the matrix operation unit. It can be implemented using register banks or on-chip memory units (such as single-port SRAM or dual-port SRAM). Depending on the computational characteristics of the matrix operation unit, the local data cache unit can be designed as a single cache unit storing all data, or as two independent cache units storing the left matrix input data and right matrix input data (or feature maps, error maps, convolution kernels, etc.) respectively. The local data cache unit reads the load flag information, receives data forwarded by the load control unit, and writes the received data into the local cache unit according to the load flag and data characteristics. Simultaneously, the local data cache unit reads the computation microinstructions, retrieves the data required for computation from the local cache unit according to the computational characteristics, and forwards it to the matrix operation unit.

[0089] The matrix operation unit can be implemented using a systolic array or a broadcast array for matrix multiplication. This unit supports matrix multiplication of a left matrix block (size a×b) and a right matrix block (size b×c) within one clock cycle, and supports inputting intermediate sums, partial sums, or biases, accumulating the matrix multiplication result with the corresponding intermediate sum. Furthermore, the matrix operation unit receives and parses stored microinstructions, and forwards the calculation results to the data post-processing unit according to the instruction information.

[0090] The data post-processing unit receives the calculation result data output by the matrix operation unit and can perform at least one of the following operations: activation function operation (such as ReLU, Leaky ReLU), quantization processing at the granularity of each data point, quantization processing at the granularity of a block of data, Softmax operation, pooling operation, and then sends the processed post-processed data to the storage control unit.

[0091] The storage control unit receives and parses the storage micro-instructions, and writes the calculation results back to the on-chip storage unit based on the characteristics and address information of the calculated output data.

[0092] In summary, the data processing system of this application uses an instruction processing unit to uniformly convert different AI operators (such as ordinary convolution, transposed convolution, convolution kernel gradient calculation, matrix transpose, etc.) into data processing micro-instructions of the matrix multiplication and addition dimension according to their respective mapping rules. It employs a three-stage pipeline architecture (loading, computation, and storage) and a corresponding micro-instruction queue mechanism to achieve parallel execution and seamless integration of micro-instructions. Therefore, this system only requires a small amount of added control logic to support multiple AI operators, significantly reducing chip hardware complexity and area overhead. It avoids the resource idleness problem caused by dedicated hardware units processing different operators separately, and effectively improves computational efficiency and hardware resource utilization through parallel execution of the three-stage pipeline. It also possesses scalability and flexible operator adaptation capabilities.

[0093] Furthermore, for ease of understanding, based on Figure 1 and Figure 2 The illustrated embodiments, such as Figure 3 As shown, this application provides a schematic diagram of a specific data processing system.

[0094] In this application, the data processing system includes: a host unit, an instruction processing unit, and a data processing unit.

[0095] The main control unit and the instruction processing unit interact through an instruction FIFO and use a handshake signal to achieve synchronous instruction issuance and reception.

[0096] The instruction processing unit is responsible for parsing the data processing instructions issued by the main control unit, and generating load microinstructions, load flags, calculation microinstructions and storage microinstructions according to the parsing results, and writing them into the corresponding load microinstruction queue (Load microinstruction FIFO), load flag queue (Load flag FIFO), calculation microinstruction queue (Calculation microinstruction FIFO) and storage microinstruction queue (Store microinstruction FIFO) respectively.

[0097] The data processing unit includes an on-chip memory unit, a load control unit, a local data cache unit, a matrix operation unit, a data post-processing unit, and a storage control unit. Specifically, the load control unit reads load micro-instructions from the load micro-instruction queue, loads the data to be processed from the on-chip memory unit according to the load micro-instructions, and sends the data to the local data cache unit. The local data cache unit writes the received data into its local cache according to the load flag in the load flag queue, and reads matrix operation data from the local cache according to the calculation micro-instructions, sending it to the matrix calculation unit. The matrix calculation unit performs matrix multiplication and addition operations on the received matrix operation data, and sends the calculation result to the data post-processing unit for post-processing. After post-processing, the data post-processing unit sends the processed data to the storage control unit. The storage control unit reads storage micro-instructions from the storage micro-instruction queue, writes the final result back to the on-chip memory unit according to the storage micro-instructions, and returns a write-complete signal (write_done).

[0098] Through the above architecture, the system achieves a unified mapping from different AI operators to matrix multiplication and addition operations, as well as parallel execution of the three-stage pipeline of loading, computation, and storage. The specific execution logic of each unit can be found in [reference needed]. Figure 1 The embodiments shown will not be described in detail here.

[0099] The above embodiments of this application describe the system provided in the embodiments of this application. To implement the functions of the methods provided in the above embodiments of this application, the electronic device may include a hardware structure and software modules, and may implement the above functions in the form of a hardware structure, software modules, or a hardware structure plus software modules. One of the above functions may be executed in the form of a hardware structure, software modules, or a hardware structure plus software modules.

[0100] Embodiments of this application also propose an electronic device comprising, as described above. Figures 1 to 3 The data processing system described in the illustrated embodiment.

[0101] Embodiments of this application also propose a chip comprising the components described above. Figures 1 to 3 The data processing system described in the illustrated embodiment.

[0102] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0103] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0104] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0105] It should be understood that various parts of the embodiments of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0106] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0107] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0108] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A data processing system, characterized by include: The main control unit, instruction processing unit, and data processing unit; The main control unit is used to issue data processing instructions to the instruction processing unit; The instruction processing unit is used to parse the data processing instruction to determine the operator type indicated by the data processing instruction, and when the operator type is not a matrix multiplication and addition operation type, it converts the data processing instruction into a data processing micro-instruction of the matrix multiplication and addition operation dimension according to the mapping rule corresponding to the operator type, and writes the data processing micro-instruction into the micro-instruction queue. The data processing unit is used to perform data processing operations on the matrix multiplication and addition dimension according to the data processing micro-instructions in the micro-instruction queue; The step of converting the data processing instruction into a data processing micro-instruction of the matrix multiplication and addition operation dimension according to the mapping rule corresponding to the operator type, and writing the data processing micro-instruction into the micro-instruction queue includes: According to the mapping rules corresponding to the operator type, the data processing parameters in the data processing instruction are mapped to the matrix dimension parameters of matrix multiplication and addition operations; Based on the matrix dimension parameters and the preset matrix calculation scale, the matrix multiplication and addition operations are divided into blocks to determine the block dimension information. Based on the block dimension information and the queue status of the microinstruction queue, the data processing microinstruction is generated and written into the microinstruction queue.

2. The system of claim 1, wherein, The step of mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of matrix multiplication and addition operations according to the mapping rule corresponding to the operator type includes: When the operator type is ordinary convolution, the spatial size of the output feature map indicated in the data processing instruction is mapped to the number of rows of the left matrix, the product of the number of input channels indicated in the data processing instruction and the convolution kernel size is mapped to the number of columns of the left matrix and the number of rows of the right matrix, and the number of output channels indicated in the data processing instruction is mapped to the number of columns of the right matrix.

3. The system of claim 1, wherein, The step of mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of matrix multiplication and addition operations according to the mapping rule corresponding to the operator type includes: When the operator type is transposed convolution, the spatial size of the output feature map indicated in the data processing instruction is mapped to the number of rows of the left matrix, the product of the number of output channels indicated in the data processing instruction and the kernel size is mapped to the number of columns of the left matrix and the number of rows of the right matrix, and the number of input channels indicated in the data processing instruction is mapped to the number of columns of the right matrix.

4. The system according to claim 1, characterized in that, The step of mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of matrix multiplication and addition operations according to the mapping rule corresponding to the operator type includes: When the operator type is convolution kernel gradient calculation, the product of the convolution kernel size indicated in the data processing instruction and the number of input channels is mapped to the number of rows in the left matrix, the product of the batch number indicated in the data processing instruction and the error map space size is mapped to the number of columns in the left matrix and the number of rows in the right matrix, and the number of output channels indicated in the data processing instruction is mapped to the number of columns in the right matrix.

5. The system according to claim 1, characterized in that, The step of mapping the data processing parameters in the data processing instruction to the matrix dimension parameters of matrix multiplication and addition operations according to the mapping rule corresponding to the operator type includes: When the operator type is left matrix transpose, the original number of rows of the left matrix indicated in the data processing instruction is mapped to the number of columns of the left matrix in the matrix multiplication and addition operation, and the original number of columns of the left matrix indicated in the data processing instruction is mapped to the number of rows of the left matrix in the matrix multiplication and addition operation. When the operator type is right matrix transpose, the original row number of the right matrix indicated in the data processing instruction is mapped to the column number of the right matrix in the matrix multiplication and addition operation, and the original column number of the right matrix indicated in the data processing instruction is mapped to the row number of the right matrix in the matrix multiplication and addition operation.

6. The system according to claim 1, characterized in that, The data processing microinstructions include loading microinstructions, loading flags, computation microinstructions, and storage microinstructions; the microinstruction queue includes a loading microinstruction queue, a loading flag queue, a computation microinstruction queue, and a storage microinstruction queue. The step of generating the data processing micro-instruction based on the block dimension information and the queue state of the micro-instruction queue, and writing the data processing micro-instruction into the micro-instruction queue, includes: Based on the block dimension information and the queue status of the loading microinstruction queue, the loading microinstruction and the loading flag are generated, and the loading microinstruction and the loading flag are written into the loading microinstruction queue and the loading flag queue. Based on the block dimension information and the queue status of the computation microinstruction queue, the computation microinstruction is generated and written into the computation microinstruction queue; Based on the block dimension information and the queue status of the storage microinstruction queue, the storage microinstruction is generated and written into the storage microinstruction queue.

7. The system according to claim 6, characterized in that, The data processing unit includes: On-chip storage unit, load control unit, local data cache unit, matrix operation unit, and storage control unit; The loading control unit is used to read loading micro-instructions from the loading micro-instruction queue and send the data to be processed loaded from the on-chip storage unit according to the loading micro-instructions to the local data cache unit; The local data cache unit is used to read load flags from the load flag queue, write the received data to be processed into the local cache according to the load flags; and read computation micro-instructions from the computation micro-instruction queue, and send the matrix operation data read from the local cache according to the computation micro-instructions to the matrix operation unit. The matrix operation unit is used to perform matrix multiplication and addition operations on the received matrix operation data to obtain the calculation result, read the stored microinstructions from the stored microinstruction queue, and send the calculation result to the storage control unit according to the stored microinstructions; The storage control unit is used to write the calculation results into the on-chip storage unit.

8. The system according to claim 7, characterized in that, The data processing unit further includes: A data post-processing unit is connected between the matrix operation unit and the storage control unit, and is used to perform data processing operations on the calculation results of the matrix operation unit so as to send the processed post-processed data to the storage control unit.

9. A chip, characterized in that, include: The data processing system as described in any one of claims 1 to 8.

Images