Matrix data processing method and device, computer equipment, readable storage medium and program product
By extracting and calculating matrix data according to the row direction of the matrix in the data processing system, and using registers to cache intermediate results, the problem of low resource utilization in matrix multiplication calculation is solved, achieving more efficient computation and bandwidth utilization.
Patent Information
- Application Number
- CN202511018453.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-23
- Publication Date
- 2025-11-11
AI Technical Summary
In existing technologies, the utilization rate of computational resources is low during matrix multiplication, resulting in low computational efficiency.
By extracting the first and second matrix data in the storage unit of the data processing system based on the size, storage address and row direction of the matrix data to be processed, and performing calculations according to the row direction of the matrix, and using registers to cache multiple intermediate cache results, a balance between hardware pipeline design and resource overhead can be achieved.
It improved data reuse and bandwidth utilization, optimized the use of hardware resources, and enhanced computing efficiency and system performance.
Smart Images
Figure CN120929253A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for processing matrix data. Background Technology
[0002] With the widespread development of artificial intelligence technology, more and more model applications have emerged, among which the most widely used is the Large Language Model (LLM). Large language models undergo continuous iteration, and during this iteration, the number of computational parameters increases explosively, thus creating a demand for hardware-accelerated computing. In the entire hardware computing process, matrix multiplication is one of the most crucial parts, and its computational efficiency is closely related to the model's inference speed.
[0003] In related technologies, calculations are generally performed between matrices through nested loops. Therefore, each matrix needs to fetch data multiple times to complete the calculation, resulting in low utilization of computing resources. Summary of the Invention
[0004] Therefore, it is necessary to provide a method, apparatus, computer equipment, computer-readable storage medium, and computer program product for processing matrix data that can improve data reuse rate and bandwidth utilization, in order to address the above-mentioned technical problems.
[0005] In a first aspect, this application provides a method for processing matrix data, applied to a computing engine in a data processing system, the method comprising:
[0006] The system receives matrix data processing instructions sent by the processor, parses the matrix data processing instructions, and obtains the size and storage address of the matrix data to be processed.
[0007] In the storage unit of the data processing system, first matrix data and second matrix data are extracted based on the size, storage address, and row direction of the matrix data to be processed; and calculations are performed based on the first matrix data and the second matrix data to obtain matrix calculation results.
[0008] In one embodiment, the data processing system extracts first matrix data and second matrix data in its storage unit based on the size, storage address, and row direction of the matrix data to be processed; and performs calculations based on the first matrix data and the second matrix data to obtain matrix calculation results, including:
[0009] In the storage unit, after extracting the matrix data at position j in row i of the first matrix, the matrix data in row j of the second matrix is extracted sequentially according to the row direction of the matrix arrangement.
[0010] Based on the matrix data at position j in row i of the first matrix, calculations are performed with the matrix data corresponding to each position in row j of the second matrix to obtain the intermediate cache result at each position in row i of the output matrix;
[0011] The calculation results for each position in the i-th row of the output matrix are obtained by superimposing the multiple intermediate cache results corresponding to each position in the i-th row of the output matrix.
[0012] Based on the calculation results of each position in each row of the output matrix, the matrix calculation result of the first matrix data and the second matrix data is obtained.
[0013] In one embodiment, the i-th row is the first row; the first row contains n positions; the calculation results for each position of the i-th row of the output matrix are obtained by superimposing multiple intermediate cache results corresponding to each position of the i-th row of the output matrix, including:
[0014] Based on the first intermediate cache result, the second intermediate cache result, ..., the nth intermediate cache result corresponding to the n positions in the first row of the output matrix, a superposition calculation is performed to obtain the superposition result corresponding to each position.
[0015] The superposition results corresponding to each position are determined, which are the calculation results of the n positions in the first row of the output matrix.
[0016] In one embodiment, the data processing system further includes a processor, a storage unit, and a target number of registers; the data stored at unit addresses in the storage unit is of the target size; and the target number of registers is the same as the number of columns in the second matrix data.
[0017] In one embodiment, the j-th position is a first position; the matrix data based on the j-th position of the i-th row of the first matrix is calculated with the matrix data corresponding to each position in the j-th row of the second matrix to obtain the j-th intermediate cache result of each position in the i-th row of the output matrix, including:
[0018] At the first moment, the matrix data of the first position of the first row of the first matrix is extracted from the storage unit, and the matrix data of the first position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the first position of the first row of the output matrix;
[0019] At the second moment, the matrix data of the second position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the second position of the first row of the output matrix;
[0020] At the third moment, the matrix data of the third position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the third position of the first row of the output matrix.
[0021] In one embodiment, the method further includes:
[0022] The matrix calculation result is returned to the storage unit.
[0023] Secondly, this application also provides a matrix data processing apparatus, applied to a computing engine in a data processing system, comprising:
[0024] The first receiving module is used to receive matrix data processing instructions sent by the processor, parse the matrix data processing instructions, and obtain the size and storage address of the matrix data to be processed.
[0025] The first processing module is used to extract first matrix data and second matrix data in the storage unit of the data processing system based on the size, storage address and row direction of the matrix data to be processed; and to perform calculations based on the first matrix data and the second matrix data to obtain matrix calculation results.
[0026] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0027] The system receives matrix data processing instructions sent by the processor, parses the matrix data processing instructions, and obtains the size and storage address of the matrix data to be processed.
[0028] In the storage unit of the data processing system, first matrix data and second matrix data are extracted based on the size, storage address, and row direction of the matrix data to be processed; and calculations are performed based on the first matrix data and the second matrix data to obtain matrix calculation results.
[0029] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0030] The system receives matrix data processing instructions sent by the processor, parses the matrix data processing instructions, and obtains the size and storage address of the matrix data to be processed.
[0031] In the storage unit of the data processing system, first matrix data and second matrix data are extracted based on the size, storage address, and row direction of the matrix data to be processed; and calculations are performed based on the first matrix data and the second matrix data to obtain matrix calculation results.
[0032] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0033] The system receives matrix data processing instructions sent by the processor, parses the matrix data processing instructions, and obtains the size and storage address of the matrix data to be processed.
[0034] In the storage unit of the data processing system, first matrix data and second matrix data are extracted based on the size, storage address, and row direction of the matrix data to be processed; and calculations are performed based on the first matrix data and the second matrix data to obtain matrix calculation results.
[0035] The aforementioned matrix data processing method, apparatus, computer device, computer-readable storage medium, and computer program product include the following method: receiving a matrix data processing instruction sent by a processor; parsing the matrix data processing instruction to obtain the size and storage address of the matrix data to be processed; extracting first matrix data and second matrix data from the storage unit of the data processing system based on the size, storage address, and row direction of the matrix data to be processed; and performing calculations based on the first matrix data and second matrix data to obtain a matrix calculation result. By employing this method, data at each position of the second matrix data can be obtained from the storage unit of the data processing system according to the row direction of the matrix, improving the data reuse rate of the first matrix data during matrix calculation, increasing the bandwidth utilization of the data processing system, and achieving a balance between hardware pipeline design and hardware resource overhead in the data processing system by adding registers to cache multiple intermediate cache results. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 This is an application environment diagram of a matrix data processing method in one embodiment;
[0038] Figure 2 This is a flowchart illustrating a method for processing matrix data in one embodiment;
[0039] Figure 3 This is a flowchart illustrating the steps for obtaining matrix calculation results in one embodiment;
[0040] Figure 4 This is a flowchart illustrating the steps for obtaining the calculation result in one embodiment;
[0041] Figure 5 This is a flowchart illustrating the steps for obtaining intermediate cache results in one embodiment;
[0042] Figure 6 This is a schematic diagram of the computing engine in one embodiment;
[0043] Figure 7a This is a schematic diagram of the structure of a storage unit storing data in one embodiment;
[0044] Figure 7b This is a schematic diagram of the computation timing from cycle_0 to cycle_n+1 in one embodiment;
[0045] Figure 8 This is a structural block diagram of a matrix data processing device in one embodiment;
[0046] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation
[0047] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0048] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0049] The matrix data processing method provided in this application embodiment can be applied to, for example... Figure 1In the application environment shown, the data processing system 100 may include a processor CPU 200, a storage unit (cache) 300, and a computing engine 400. The CPU 200 can send work instructions to the computing engine 400, which can parse the work instructions, retrieve data from the storage unit 300, perform calculations, obtain the calculation results, and write the calculation results back to the cache.
[0050] The data processing system can be integrated into terminals or servers. Terminals can be, but are not limited to, various personal computers, laptops, smartphones, tablets, drones, low-altitude aircraft, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection equipment. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. Servers can be standalone physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers providing cloud computing services, etc.
[0051] In one exemplary embodiment, such as Figure 2 As shown, a method for processing matrix data is provided, which can be applied to... Figure 1 Taking the computing engine 400 in the example, the explanation includes:
[0052] Step 202: Receive the matrix data processing instruction sent by the processor, parse the matrix data processing instruction, and obtain the size and storage address of the matrix data to be processed.
[0053] The processor, which can be the CPU in a data processing system, generates matrix data processing instructions (i.e., work instructions) based on the needs of the actual application scenario and sends these instructions to the computing engine. The matrix data to be processed can be at least two matrices specified by the matrix data processing instructions, such as a first matrix and a second matrix. The matrix data processing instructions can be used to instruct a multiplication operation between the first matrix and the second matrix.
[0054] Specifically, the processor generates matrix data processing instructions based on the needs of the actual application scenario and sends these instructions to the computing engine. After receiving the matrix data processing instructions, the computing engine can parse them to determine the matrix data to be processed, such as a first matrix and a second matrix. The computing engine can also determine the size of the first matrix and its storage address in the cache (storage unit), as well as the size of the second matrix and its storage address in the cache (storage unit).
[0055] Step 204: In the storage unit of the data processing system, extract the first matrix data and the second matrix data based on the size, storage address, and row direction of the matrix data to be processed. Then, perform calculations based on the first matrix data and the second matrix data to obtain the matrix calculation result.
[0056] The data processing system also includes a storage unit, such as a storage unit for storing cached data. The matrix data to be processed can be a first matrix and a second matrix indicated by a matrix data processing instruction. The size of the matrix data can be the dimension or scale of the matrix, including the number of rows and columns. The matrix calculation result can be the result obtained by performing operations on the first matrix data and the second matrix data, including one or more of the following: sum matrix, difference matrix, and product matrix. When the matrix data processing instruction instructs the first matrix and the second matrix to perform a multiplication operation, the matrix calculation result is a product matrix.
[0057] Specifically, after receiving a matrix data processing instruction, the computing engine can extract the matrix data for calculation from the storage unit. Based on the size and storage address of the first matrix data, the engine extracts the first matrix data and, according to the row direction of the matrix arrangement, extracts the elements at each position of the first matrix data and the elements at each position of the second matrix data for calculation, thereby obtaining the matrix calculation result of the matrix data processing instruction.
[0058] Optionally, the specific calculation process may include: the calculation engine can extract the elements of each position in each row of the first matrix data according to the matrix arrangement row direction, and after extracting the elements of each position in each row of the first matrix data, for each position of the element, extract the elements of multiple positions in each row of the second matrix data according to the matrix arrangement row direction, and perform calculations based on the elements of the first matrix data and the elements of the second matrix data to obtain the matrix calculation result.
[0059] In this context, the matrix in the matrix arrangement row direction refers to the first matrix and the second matrix. The matrix arrangement row direction can be the arrangement order of the matrix data of each matrix after being split into sub-matrices of a target size. Optionally, the target size of the sub-matrices can be 4×4. This embodiment does not limit the specific value of the target size of the sub-matrices. Those skilled in the art can determine it based on the actual application scenario.
[0060] In the above matrix data processing method, the matrix data processing instructions sent by the processor are received, parsed, and the size and storage address of the matrix data to be processed are obtained. In the storage unit of the data processing system, first matrix data and second matrix data are extracted based on the size, storage address, and row direction of the matrix data to be processed. Calculations are then performed based on the first and second matrix data to obtain the matrix calculation result. By adopting this method, data at each position of the second matrix data can be obtained from the storage unit of the data processing system according to the row direction of the matrix, improving the data reuse rate of the first matrix data during matrix calculation, increasing the bandwidth utilization of the data processing system, and achieving a balance between hardware pipeline design and hardware resource overhead in the data processing system by adding registers to cache multiple intermediate cache results.
[0061] In one embodiment, such as Figure 3 As shown, the specific implementation process of step "extracting the first matrix data and the second matrix data in the storage unit of the data processing system based on the size, storage address, and row direction of the matrix data to be processed, and performing calculations based on the first matrix data and the second matrix data to obtain the matrix calculation result" may include:
[0062] Step 302: In the storage unit of the data processing system, after extracting the matrix data at position j in row i of the first matrix, extract the matrix data in row j of the second matrix sequentially according to the row direction of the matrix arrangement.
[0063] Where i can be a positive integer, i is less than or equal to m, and m can be the number of rows in the first matrix; j is a positive integer, j is less than or equal to n, and n can be the number of columns in the first matrix; optionally, the dimensions of the first matrix and the second matrix can be the same.
[0064] Specifically, in the storage unit, the computing engine can first extract the matrix data at position j in row i of the first matrix, that is, extract the element at position j in row i of the first matrix. In this way, after the computing engine obtains the element at position j in row i of the first matrix, it can extract the elements at each position in row j of the second matrix in the storage unit. This allows for the reuse of the element at position j in row i of the first matrix, thereby improving bandwidth utilization.
[0065] In one example, after the computing engine obtains the first element at position j in row i of the first matrix, it can extract the second element at each position j in row j of the second matrix from the storage unit. That is, after extracting the first element at position j in row i of the first matrix, n elements in row j of the second matrix are extracted according to the row direction of the matrix arrangement. In other words, after extracting the elements of the first matrix once, the data of the second matrix is extracted n times.
[0066] Step 304: Based on the matrix data at position j in row i of the first matrix, calculate the matrix data corresponding to each position in row j of the second matrix to obtain the intermediate cache result at each position in row i of the output matrix.
[0067] Specifically, after the computing engine extracts the first element at position j in row i of the first matrix, it performs a product operation with the second element at position n in row j of the second matrix to obtain the j-th intermediate cache result corresponding to position n in row i of the output matrix data, and stores each intermediate cache result in an additionally configured register.
[0068] Optionally, after the calculation engine obtains the first element at position j in row i of the first matrix, it can extract the second element at each position (n positions) in row j of the second matrix in the storage unit. In this way, the calculation engine can calculate the calculation results between the first element and the n second elements at different times, obtain n calculation results, and determine the n calculation results as the j-th intermediate cache results corresponding to the n positions in row i of the output matrix.
[0069] Step 306: Based on the multiple intermediate cache results corresponding to each position of the i-th row of the output matrix, perform superposition processing to obtain the calculation results of each position of the i-th row of the output matrix; based on the calculation results of each position of each row of the output matrix, obtain the matrix calculation results of the first matrix data and the second matrix data.
[0070] The calculation results of each position in the i-th row of the output matrix are the feature elements of each position in the output matrix.
[0071] Specifically, for each row, the calculation engine can obtain n intermediate cache results corresponding to the n positions of the i-th row by executing the method described in the above embodiments multiple times. For each position, the calculation engine performs superposition calculation on the n intermediate cache results corresponding to that position to obtain the superposition result, and determines the superposition result as the calculation result of that position, that is, the feature element corresponding to that position.
[0072] In this way, the calculation engine performs multiplication operations on the first matrix data and the second matrix data, and the resulting product matrix can be the output matrix. That is to say, the calculation engine can determine the calculation result of each position in each row of the output matrix, which is the matrix calculation result of the first matrix data and the second matrix data.
[0073] In this embodiment, by extracting the elements of the first matrix once and then extracting the elements of the second matrix multiple times for multiple calculations, the elements of the extracted first matrix are reused, and the multiple intermediate cache results are stored in registers, thus achieving a balance between bandwidth utilization and the hardware resources of the data processing system.
[0074] In one embodiment, such as Figure 4 As shown, the i-th row is the first row. The first row contains n positions. Specifically, the n positions in the first row can be the number of columns in the first matrix. Optionally, the dimensions of the first matrix and the second matrix are the same. The row dimension and column dimension of the first matrix can be the same. The following describes the processing method of the above matrix data in detail, taking the first row as an example:
[0075] Accordingly, the specific implementation process of the step "adding together the multiple intermediate cache results corresponding to each position in the i-th row of the output matrix to obtain the calculation results for each position in the i-th row of the output matrix" may include:
[0076] Step 402: Based on the first intermediate cache result, the second intermediate cache result, ..., the nth intermediate cache result corresponding to the n positions in the first row of the output matrix, perform superposition calculation to obtain the superposition result corresponding to each position.
[0077] Step 404: Determine the superposition result corresponding to each position, which is the calculation result of n positions in the first row of the output matrix.
[0078] Specifically, the calculation engine can re-execute the method described in the above embodiments to obtain n intermediate cache results corresponding to the n positions in the first row of the output matrix. For each position, the calculation engine can perform superposition calculation on the first intermediate cache result, the second intermediate cache result, ..., the nth intermediate cache result corresponding to that position to obtain the superposition result corresponding to that position. In this way, the terminal can determine the superposition result at that position as the calculation result corresponding to that position, that is, the feature element of that position in the output matrix.
[0079] In this embodiment, calculations between multiple matrices are achieved by superimposing the obtained intermediate cache results.
[0080] In one embodiment, the data processing system further includes a processor, storage units, and a target number of registers. Data stored per unit address in the storage units is of the target size. The target number of registers is the same as the number of columns in the second matrix data.
[0081] Specifically, the storage unit can be a space used to store cached data, such as a cache. In this embodiment, a target number of registers can be configured in the data processing system to store each intermediate cached result separately, and the target number can be the number of columns in the second matrix.
[0082] In this embodiment, by configuring multiple registers, a balance can be achieved between bandwidth utilization and the hardware resources of the data processing system.
[0083] In one embodiment, the j-th position is the first position. Optionally, the first row of the first matrix has 3 positions, meaning the first and second matrices have a total of 3 columns. Figure 5 As shown, the specific implementation process of the step "based on the matrix data at position j in row i of the first matrix, calculate with the matrix data corresponding to each position in row j of the second matrix to obtain the intermediate cached result at each position in row i of the output matrix" can include:
[0084] Step 502: At the first moment, extract the matrix data of the first position of the first row of the first matrix from the storage unit, and extract the matrix data of the first position of the first row of the second matrix from the storage unit to obtain the first intermediate cache result of the first position of the first row of the output matrix.
[0085] Specifically, in this embodiment, the first position in the first row is used as an example. At the first moment, the computing engine can extract the first element of the first position in the first row of the first matrix from the storage unit. At the same time, the computing engine can extract the second element of the first position in the first row of the second matrix from the storage unit. The computing engine can perform a product operation on the first element of the first matrix and the second element of the second matrix to obtain the product result, and determine the product result as the first intermediate cache result of the first position in the first row of the output matrix.
[0086] Step 504: At the second moment, extract the matrix data of the second position of the first row of the second matrix from the storage unit to obtain the first intermediate cache result of the second position of the first row of the output matrix.
[0087] Specifically, at the second moment, the computing engine can extract the second element of the second position of the first row of the second matrix from the storage unit, multiply the first element with the second element of the second position to obtain the product result, and determine the product result as the second intermediate cache result of the first position of the first row of the output matrix.
[0088] Step 506: At the third moment, extract the matrix data of the third position of the first row of the second matrix from the storage unit to obtain the first intermediate cache result of the third position of the first row of the output matrix.
[0089] Specifically, at the third moment, the computing engine can extract the second element at the third position of the first row of the second matrix from the storage unit, multiply the first element with the second element at the third position to obtain the product result, and determine the product result as the second intermediate cache result at the third position of the first row of the output matrix.
[0090] In this way, at the fourth time, the terminal can extract the matrix data of the second position of the first row of the first matrix from the storage unit, and extract the matrix data of the first position of the second row of the second matrix from the storage unit to obtain the second intermediate cache result of the first position of the first row of the output matrix; at the fifth time, it can extract the matrix data of the second position of the second row of the second matrix from the storage unit to obtain the second intermediate cache result of the second position of the first row of the output matrix; at the sixth time, it can extract the matrix data of the third position of the second row of the second matrix from the storage unit to obtain the second intermediate cache result of the third position of the first row of the output matrix.
[0091] Based on this, at the seventh moment, the terminal can extract the matrix data of the third position of the first row of the first matrix from the storage unit, and extract the matrix data of the first position of the second row of the second matrix from the storage unit to perform a product operation, thereby obtaining the third intermediate cache result of the first position of the first row of the output matrix; at the eighth moment, the terminal can extract the matrix data of the second position of the second row of the second matrix from the storage unit to obtain the third intermediate cache result of the second position of the first row of the output matrix; at the ninth moment, the terminal can extract the matrix data of the third position of the second row of the second matrix from the storage unit to obtain the third intermediate cache result of the third position of the first row of the output matrix.
[0092] The calculation engine can superimpose the first, second, and third intermediate cache results corresponding to the first, second, and third positions of the output matrix to obtain the calculation results of the first, second, and third positions in the first row of the output matrix.
[0093] In one embodiment, the method further includes:
[0094] The matrix calculation result is returned to the storage unit.
[0095] Specifically, after the computing engine obtains the matrix calculation result between the first matrix and the second matrix, that is, after the computing engine obtains the output matrix, it can return the output matrix to the storage unit, that is, store it in the storage unit.
[0096] In this embodiment, by returning the calculated output matrix to the storage unit, a balance between the performance, resources, and stability of the data processing system can be guaranteed.
[0097] The following describes in detail the specific implementation steps of the above matrix data processing method, with reference to a specific embodiment:
[0098] The CPU sends a matrix multiplication instruction to the computation engine; the computation engine parses the instruction, obtains the size and storage address of the matrix, and retrieves data from the cache based on this information; the computation engine performs calculations on the retrieved data; and writes the calculation results back to the cache unit.
[0099] like Figure 6 As shown, it includes an insn decoder (instruction decoding module), a data fetch unit, a MAC array (MAC computation array), and a data write unit; the insn decoder is used to parse the working instruction content; the data fetch unit is used to fetch data from the cache and pass it to the computation array; the MAC array is used to perform matrix multiplication calculations; and the data write unit is used to write the calculation results back to the cache.
[0100] like Figure 7a As shown, in this embodiment, to improve the efficiency of the module retrieving data from the cache unit, the matrix data can be split into 4×4 blocks and then stored in the cache. Each time the computing engine accesses the cache, it can retrieve 4×4 data items from a single address. The size of the matrix data can be N rows and M columns. The matrix data is divided into multiple sub-matrix data according to the 4×4 block size, that is, the target size of the sub-matrix data can be 4×4. The first row contains m sub-matrix data, and each sub-matrix data is stored in address 0, address 1, address 2, address 3, ..., address m respectively; correspondingly, the first sub-matrix data in the second row is stored in address m+1; correspondingly, the last sub-matrix data in the last row is stored in address n. That is to say, there are a total of n sub-matrix data in the matrix data, which are stored in n addresses respectively.
[0101] In this embodiment, the matrix data processing method adjusts the data retrieval and calculation order of Matrix B (the second matrix). During cycle 0-cycle 2, Matrix A consistently uses data at point (0,0), while Matrix B retrieves data column-wise instead of row-wise. Based on this, it can be seen that Matrix A can retrieve data from the cache every three time intervals, reducing bandwidth to one-third. Furthermore, since the matrix multiplication results calculated in each iteration are in different columns, the data processing system hardware can add two sets of registers to cache intermediate calculation results.
[0102] If the size of Matrix B is KxN, theoretically, this method can reduce the read bandwidth of Matrix A to at most 1 / N of its original value. However, it also requires an additional (N-1) sets of registers to cache intermediate results, meaning the number of registers needed is the same as the number of columns in Matrix B. It can be seen that reducing bandwidth consumption comes at the cost of increased hardware resources. Considering the trade-off between bandwidth consumption and resource overhead, the design also takes into account the latency of the floating-point adder to set the matrix data reuse rate.
[0103] In hardware design, higher operating frequencies necessitate the insertion of registers at a certain level during computation to optimize hardware timing. The diagram below illustrates this using a single-level register insertion as an example. It shows that the calculation results of dat1 and dat2 are only obtained after two cycles, and subsequent accumulations must wait until the adder completes the previous results before computation can continue. This situation results in a loss of hardware computational performance. This embodiment addresses this problem by combining the latency of the floating-point adder with the data reuse rate of Matrix A. This allows for fully pipelined computation with only one set of floating-point adders, while also reducing matrix read bandwidth. Using M=12, K=12, and N=32 as an example, Matrix B can be split into four columns.
[0104] like Figure 7b The diagram shown can be a timing diagram of matrix data calculation at the first time step (Cycle 0), the second time step (Cycle 1), the third time step (Cycle 2), and up to the nth time step (Cycle n+1). At the first time step (cycle 0): the calculation of Matrix A (0,0) block and Matrix B (0,0) block is completed, and Output Matrix (0,0)_0 is obtained, which is the red data block; that is, the 0th data (0,0)_0 at position (0,0) in the matrix data of the output matrix, and placed in the register.
[0105] The second time step (cycle 1): Complete the calculation of Matrix A (0,0) block and Matrix B (1,0) block to obtain Output Matrix (1,0)_0, which is the green data block. Combine the calculation results of the first time step to obtain Output Matrix (0,0)_0 and (1,0)_0, which are the 0th intermediate calculation cache result at position (0,0) and the 0th intermediate calculation cache result at position (1,0);
[0106] The third time step (cycle 2): Complete the calculation of Matrix A(0,0) block and Matrix B(2,0) block, and obtain Output Matrix(2,0)_0, which is the blue data block;
[0107] Fourth time step (cycle 3): Complete the calculation of Matrix A(0,0) block and Matrix B(3,0) block, and obtain Output Matrix(3,0)_0;
[0108] Fifth time step (cycle4): Complete the calculation of Matrix A(1,0) block and Matrix B(0,1) block, and obtain Output Matrix(0,0)_1. The result will be accumulated with Output Matrix(0,0)_0.
[0109] The sixth time step (cycle 5): Complete the calculation of Matrix A(1,0) block and Matrix B(1,1) block to obtain Output Matrix(1,0)_1, and its result will be accumulated with Output Matrix(1,0)_0;
[0110] This process continues. At cycle_n, the calculations for Matrix A(2,0) and Matrix B(3,2) are completed, resulting in Output Matrix(3,0)_2, which is then added to the previously cached intermediate calculation results. At this point, the matrix results for (0,0), (1,0), (2,0), and (3,0) of the Output Matrix have been calculated and accumulated. At cycle_n+1, the calculations for Matrix A(0,0) and Matrix B(4,0) are performed.
[0111] It can be seen that for Matrix A, the data retrieved from the cache each time can be reused 4 times. Here, it is set to 4. The floating-point adder used in this embodiment has a latency of 4 cycles, so the data reuse rate and the calculation latency of the floating-point adder match perfectly. If the latency of the floating-point adder changes subsequently, it can be flexibly adjusted accordingly.
[0112] The matrix data processing method in this embodiment proposes a novel design optimization method: mainly controlling the data retrieval and calculation order of matrix B, thereby improving the data reuse rate of matrix A, so as to improve the bandwidth utilization rate. At the same time, this method can take into account the hardware pipeline design and hardware resource consumption.
[0113] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0114] Based on the same inventive concept, this application also provides a matrix data processing apparatus for implementing the matrix data processing method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more matrix data processing apparatus embodiments provided below can be found in the limitations of the matrix data processing method described above, and will not be repeated here.
[0115] In one exemplary embodiment, such as Figure 8 As shown, a matrix data processing device 800 is provided, which is applied to a computing engine in a data processing system, including:
[0116] The first receiving module 802 is used to receive matrix data processing instructions sent by the processor, parse the matrix data processing instructions, and obtain the size and storage address of the matrix data to be processed.
[0117] The first processing module 804 is used to extract first matrix data and second matrix data in the storage unit of the data processing system based on the size, storage address and row direction of the matrix data to be processed; and to perform calculations based on the first matrix data and the second matrix data to obtain matrix calculation results.
[0118] In one embodiment, the first processing module is specifically used for:
[0119] In the storage unit, after extracting the matrix data at position j in row i of the first matrix, the matrix data in row j of the second matrix is extracted sequentially according to the row direction of the matrix arrangement.
[0120] Based on the matrix data at position j in row i of the first matrix, calculations are performed with the matrix data corresponding to each position in row j of the second matrix to obtain the intermediate cache result at each position in row i of the output matrix;
[0121] The calculation results for each position in the i-th row of the output matrix are obtained by superimposing the multiple intermediate cache results corresponding to each position in the i-th row of the output matrix.
[0122] Based on the calculation results of each position in each row of the output matrix, the matrix calculation result of the first matrix data and the second matrix data is obtained.
[0123] In one embodiment, the i-th row is the first row; the first processing module is further specifically used for:
[0124] Based on the first intermediate cache result, the second intermediate cache result, ..., the nth intermediate cache result corresponding to the n positions in the first row of the output matrix, a superposition calculation is performed to obtain the superposition result corresponding to each position.
[0125] The superposition results corresponding to each position are determined, which are the calculation results of the n positions in the first row of the output matrix.
[0126] In one embodiment, the data processing system further includes a processor, a storage unit, and a target number of registers; the data stored at unit addresses in the storage unit is of the target size; and the target number of registers is the same as the number of columns in the second matrix data.
[0127] In one embodiment, the j-th position is a first position; the first processing module is further specifically used for:
[0128] At the first moment, the matrix data of the first position of the first row of the first matrix is extracted from the storage unit, and the matrix data of the first position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the first position of the first row of the output matrix;
[0129] At the second moment, the matrix data of the second position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the second position of the first row of the output matrix;
[0130] At the third moment, the matrix data of the third position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the third position of the first row of the output matrix.
[0131] In one embodiment, the device further includes:
[0132] The return module is used to return the matrix calculation result to the storage unit.
[0133] Each module in the aforementioned matrix data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.
[0134] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 9 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores matrix data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements a method for processing matrix data.
[0135] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0136] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0137] The system receives matrix data processing instructions sent by the processor, parses the matrix data processing instructions, and obtains the size and storage address of the matrix data to be processed.
[0138] In the storage unit of the data processing system, first matrix data and second matrix data are extracted based on the size, storage address, and row direction of the matrix data to be processed; and calculations are performed based on the first matrix data and the second matrix data to obtain matrix calculation results.
[0139] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0140] In the storage unit, after extracting the matrix data at position j in row i of the first matrix, the matrix data in row j of the second matrix is extracted sequentially according to the row direction of the matrix arrangement.
[0141] Based on the matrix data at position j in row i of the first matrix, calculations are performed with the matrix data corresponding to each position in row j of the second matrix to obtain the intermediate cache result at each position in row i of the output matrix;
[0142] The calculation results for each position in the i-th row of the output matrix are obtained by superimposing the multiple intermediate cache results corresponding to each position in the i-th row of the output matrix.
[0143] Based on the calculation results of each position in each row of the output matrix, the matrix calculation result of the first matrix data and the second matrix data is obtained.
[0144] In one embodiment, the i-th row is the first row; the first row contains n positions; when the processor executes the computer program, it further implements the following steps:
[0145] Based on the first intermediate cache result, the second intermediate cache result, ..., the nth intermediate cache result corresponding to the n positions in the first row of the output matrix, a superposition calculation is performed to obtain the superposition result corresponding to each position.
[0146] The superposition results corresponding to each position are determined, which are the calculation results of the n positions in the first row of the output matrix.
[0147] In one embodiment, the data processing system further includes a processor, a storage unit, and a target number of registers; the data stored at unit addresses in the storage unit is of the target size; and the target number of registers is the same as the number of columns in the second matrix data.
[0148] In one embodiment, the j-th position is a first position, and the processor further performs the following steps when executing the computer program:
[0149] At the first moment, the matrix data of the first position of the first row of the first matrix is extracted from the storage unit, and the matrix data of the first position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the first position of the first row of the output matrix;
[0150] At the second moment, the matrix data of the second position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the second position of the first row of the output matrix;
[0151] At the third moment, the matrix data of the third position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the third position of the first row of the output matrix.
[0152] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0153] The matrix calculation result is returned to the storage unit.
[0154] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.
[0155] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0156] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A method for processing matrix data, characterized in that, The method, applied to a computing engine in a data processing system, includes: The system receives matrix data processing instructions sent by the processor, parses the matrix data processing instructions, and obtains the size and storage address of the matrix data to be processed. In the storage unit of the data processing system, first matrix data and second matrix data are extracted based on the size, storage address, and row direction of the matrix data to be processed; and calculations are performed based on the first matrix data and the second matrix data to obtain matrix calculation results.
2. The method according to claim 1, characterized in that, In the storage unit of the data processing system, the first matrix data and the second matrix data are extracted based on the size, storage address and row direction of the matrix data to be processed. The matrix calculation results are obtained based on the first matrix data and the second matrix data, including: In the storage unit, after extracting the matrix data at position j in row i of the first matrix, the matrix data in row j of the second matrix is extracted sequentially according to the row direction of the matrix arrangement. Based on the matrix data at position j in row i of the first matrix, calculations are performed with the matrix data corresponding to each position in row j of the second matrix to obtain the intermediate cache result at each position in row i of the output matrix; The calculation results for each position in the i-th row of the output matrix are obtained by superimposing the multiple intermediate cache results corresponding to each position in the i-th row of the output matrix. Based on the calculation results of each position in each row of the output matrix, the matrix calculation result of the first matrix data and the second matrix data is obtained.
3. The method according to claim 2, characterized in that, The i-th row is the first row; the first row contains n positions; the calculation results of each position in the i-th row of the output matrix are obtained by superimposing multiple intermediate cache results corresponding to each position, including: Based on the first intermediate cache result, the second intermediate cache result, ..., the nth intermediate cache result corresponding to the n positions in the first row of the output matrix, a superposition calculation is performed to obtain the superposition result corresponding to each position. The superposition results corresponding to each position are determined, which are the calculation results of the n positions in the first row of the output matrix.
4. The method according to claim 1, characterized in that, The data processing system also includes a processor, a storage unit, and a target number of registers; the data stored at each address in the storage unit is of the target size; the target number of registers is the same as the number of columns in the second matrix data.
5. The method according to claim 3, characterized in that, The j-th position is the first position; the matrix data based on the j-th position of the i-th row of the first matrix is calculated with the matrix data corresponding to each position in the j-th row of the second matrix to obtain the j-th intermediate cache result of each position in the i-th row of the output matrix, including: At the first moment, the matrix data of the first position of the first row of the first matrix is extracted from the storage unit, and the matrix data of the first position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the first position of the first row of the output matrix; At the second moment, the matrix data of the second position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the second position of the first row of the output matrix; At the third moment, the matrix data of the third position of the first row of the second matrix is extracted from the storage unit to obtain the first intermediate cache result of the third position of the first row of the output matrix.
6. The method according to claim 1, characterized in that, The method further includes: The matrix calculation result is returned to the storage unit.
7. A matrix data processing apparatus, characterized in that, A computing engine applied in a data processing system, the device comprising: The first receiving module is used to receive matrix data processing instructions sent by the processor, parse the matrix data processing instructions, and obtain the size and storage address of the matrix data to be processed. The first processing module is used to extract first matrix data and second matrix data in the storage unit of the data processing system based on the size, storage address and row direction of the matrix data to be processed; and to perform calculations based on the first matrix data and the second matrix data to obtain matrix calculation results.
8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.