Data reading method, device, computer equipment, storage medium and program product
By splitting the matrix into row matrix blocks and column matrix blocks, and computed in parallel in shared memory, the problem of global memory read and write delay in general matrix multiplication is solved, and data processing efficiency and hardware utilization are improved.
Patent Information
- Application Number
- CN202211434973.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-11-16
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-11-16
Smart Images

Figure CN118051168B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computer technology, and in particular to a data reading method, device, computer equipment, storage medium and program product. Background Art
[0002] General Matrix Multiplication (GEMM) is widely used in scientific computing, deep learning and other fields, and is the core computing unit of deep learning. With the improvement of graphics processing unit (GPU) performance and the emergence of new matrix multiplication hardware computing units (such as Tensor Core), the "memory wall" phenomenon is becoming more and more obvious.
[0003] For general matrix multiplication, important delays usually include global memory read delay, shared memory (ShareMemory) write delay, shared memory (ShareMemory) read delay, synchronization overhead caused by thread execution speed differences, etc. Traditionally, GPUs cover up memory delays in two ways: thread-level parallelism and instruction-level parallelism. For GEMM, the block algorithm usually requires more registers and shared memory, resulting in a limit on the number of threads that can be scheduled by the hardware, which cannot effectively cover the read and write delays of global memory, resulting in low data processing efficiency. Summary of the invention
[0004] Based on this, it is necessary to provide a data reading method, device, computer equipment, computer-readable storage medium and computer program product that can effectively cover the global memory read and write delays in response to the above technical problems.
[0005] In a first aspect, the present application provides a data reading method. The method comprises:
[0006] In response to a data read request, splitting a first matrix to be processed into a plurality of row matrix blocks, and splitting a second matrix to be processed into a plurality of column matrix blocks;
[0007] The thread reads the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; while the thread processes the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory;
[0008] After the thread completes the write operation to the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; while the thread processes the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread into the first shared memory;
[0009] The next row matrix block of the current row matrix block among the multiple row matrix blocks is used as the current row matrix block of the next iteration loop, and the next column matrix block of the current column matrix block among the multiple column matrix blocks is used as the current column matrix block of the next iteration loop, and the thread is returned to the step of reading the current row matrix block and the current column matrix block to continue executing until the loop end condition is reached, and the multiple first calculation results and the multiple second calculation results are obtained;
[0010] Accumulate the multiple first calculation results and the multiple second calculation results to obtain an output matrix.
[0011] In one embodiment, if the current column matrix block is the first column matrix block and the current row matrix block is the first row matrix block, the method further includes:
[0012] The thread reads the first row matrix block and the first column matrix block;
[0013] After the thread completes the read operation, the thread writes the first row matrix block and the first column matrix block into the first shared memory; while the thread is processing the first shared memory, the thread reads the next row matrix block and the next column matrix block, and simultaneously performs synchronization processing on the thread.
[0014] In one embodiment, the threads calculate the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain the first calculation result, including:
[0015] Splitting the history row matrix block written in the previous iteration loop in the first shared memory into a plurality of sub-row matrix blocks, and splitting the history column matrix block into a plurality of sub-column matrix blocks;
[0016] The thread reads the current sub-row matrix block and the current sub-column matrix block in the first shared memory, and the thread calculates the historical sub-row matrix block and the historical sub-column matrix block read in the previous iteration loop in the first shared memory in parallel to obtain a first intermediate result; after the thread completes the calculation operation on the first shared memory, the thread reads the next sub-row matrix block of the current sub-row matrix block, reads the next sub-column matrix block of the current sub-column matrix block, and the thread calculates the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a second intermediate result;
[0017] The next sub-row matrix block of the current sub-row matrix block among the multiple sub-row matrix blocks is used as the current sub-row matrix block of the next iteration loop, and the next sub-column matrix block of the current sub-column matrix block among the multiple sub-column matrix blocks is used as the current sub-column matrix block of the next iteration loop, and the thread is returned to the step of reading the current sub-row matrix block and the current sub-column matrix block and continuing to execute until the loop end condition is reached, and a plurality of first intermediate results and a plurality of second intermediate results are obtained;
[0018] Accumulate the plurality of first intermediate results and the plurality of second intermediate results to obtain a first calculation result.
[0019] In one embodiment, if the current sub-column matrix block is the first sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-row matrix block is the first sub-row matrix block of the historical column matrix block in the first shared memory, the method further includes:
[0020] The thread reads the first sub-row matrix block of the history row matrix block and the first sub-column matrix block of the current history column matrix block in the first shared memory;
[0021] When the thread executes a read operation on the first shared memory, the thread reads the next sub-row matrix block and the next sub-column matrix block in the first shared memory, and the thread performs matrix block multiplication on the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a first intermediate result.
[0022] In one embodiment, if the current sub-row matrix block is the last sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, before the threads perform parallel calculations on the historical sub-row matrix blocks and historical sub-column matrix blocks read in the last iteration loop in the first shared memory, the method further includes:
[0023] A synchronization operation is performed on threads that read sub-row matrix blocks and sub-row matrix blocks in the first shared memory; and a synchronization operation is performed on threads that write sub-row matrix blocks and sub-row matrix blocks in the first shared memory.
[0024] In one embodiment, if the current sub-row matrix block is the last sub-row matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, then after the threads calculate the historical sub-row matrix block and the historical sub-column matrix block read in the last iteration loop in the first shared memory in parallel, the method further includes:
[0025] The process of the thread performing matrix block multiplication on the last sub-row matrix block of the historical sub-row matrix block and the last sub-column matrix block of the historical sub-column matrix block is delayed until the thread calculates the previous row matrix block and the previous column matrix block written to the second shared memory, and is executed during the process of the thread reading the first sub-row matrix block of the previous row matrix block of the current row matrix block and the first sub-column matrix block of the previous column matrix block of the current column matrix block in the second shared memory.
[0026] In one embodiment, the threads calculate in parallel the historical sub-row matrix blocks and the historical sub-column matrix blocks read in the last iteration loop in the first shared memory to obtain the first intermediate result, including:
[0027] The thread stores each row element of the history sub-row matrix block read in the last iteration loop in the first shared memory into a vector register; the thread stores a single element of each row element of the history sub-column matrix block read in the last iteration loop in the first shared memory into a scalar register in parallel;
[0028] The thread performs multiplication and accumulation processing on a single element in the scalar register and each row of elements in the vector register to obtain a first intermediate result.
[0029] In a second aspect, the present application also provides a data reading device. The device comprises:
[0030] A block division module, configured to divide the first matrix to be processed into a plurality of row matrix blocks and the second matrix to be processed into a plurality of column matrix blocks in response to a data read request;
[0031] The first reading module uses a thread to read the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; when the thread processes the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory;
[0032] A second reading module is used for, after the thread completes the write operation to the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; when the thread processes the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread into the first shared memory;
[0033] an iteration module, used for taking the next row matrix block of the current row matrix block among the multiple row matrix blocks as the current row matrix block of the next iteration loop, taking the next column matrix block of the current column matrix block among the multiple column matrix blocks as the current column matrix block of the next iteration loop, and returning to the step of the thread reading the current row matrix block and the current column matrix block to continue execution until the loop end condition is reached, thereby obtaining multiple first calculation results and multiple second calculation results;
[0034] The output module is used to accumulate the multiple first calculation results and the multiple second calculation results to obtain an output matrix.
[0035] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:
[0036] In response to a data read request, splitting a first matrix to be processed into a plurality of row matrix blocks, and splitting a second matrix to be processed into a plurality of column matrix blocks;
[0037] The thread reads the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; while the thread processes the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory;
[0038] After the thread completes the write operation to the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; while the thread processes the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread into the first shared memory;
[0039] The next row matrix block of the current row matrix block among the multiple row matrix blocks is used as the current row matrix block of the next iteration loop, and the next column matrix block of the current column matrix block among the multiple column matrix blocks is used as the current column matrix block of the next iteration loop, and the thread is returned to the step of reading the current row matrix block and the current column matrix block to continue executing until the loop end condition is reached, and the multiple first calculation results and the multiple second calculation results are obtained;
[0040] Accumulate the plurality of first calculation results and the plurality of second calculation results to obtain an output matrix.
[0041] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0042] In response to a data read request, splitting a first matrix to be processed into a plurality of row matrix blocks, and splitting a second matrix to be processed into a plurality of column matrix blocks;
[0043] The thread reads the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; while the thread processes the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory;
[0044] After the thread completes the write operation to the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; while the thread processes the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread into the first shared memory;
[0045] The next row matrix block of the current row matrix block among the multiple row matrix blocks is used as the current row matrix block of the next iteration loop, and the next column matrix block of the current column matrix block among the multiple column matrix blocks is used as the current column matrix block of the next iteration loop, and the thread is returned to the step of reading the current row matrix block and the current column matrix block to continue executing until the loop end condition is reached, and the multiple first calculation results and the multiple second calculation results are obtained;
[0046] Accumulate the plurality of first calculation results and the plurality of second calculation results to obtain an output matrix.
[0047] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:
[0048] In response to a data read request, splitting a first matrix to be processed into a plurality of row matrix blocks, and splitting a second matrix to be processed into a plurality of column matrix blocks;
[0049] The thread reads the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; while the thread processes the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory;
[0050] After the thread completes the write operation to the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; while the thread processes the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread into the first shared memory;
[0051] The next row matrix block of the current row matrix block among the multiple row matrix blocks is used as the current row matrix block of the next iteration loop, and the next column matrix block of the current column matrix block among the multiple column matrix blocks is used as the current column matrix block of the next iteration loop, and the thread is returned to the step of reading the current row matrix block and the current column matrix block to continue executing until the loop end condition is reached, and the multiple first calculation results and the multiple second calculation results are obtained;
[0052] Accumulate the plurality of first calculation results and the plurality of second calculation results to obtain an output matrix.
[0053] The above-mentioned data reading method, device, computer equipment, storage medium and program product split the first matrix to be processed into multiple row matrix blocks, split the second matrix to be processed into multiple column matrix blocks, the thread reads the current row matrix block and the current column matrix block, and the thread parallelly calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory. In the process of the thread processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory. In the above-mentioned method, through the process of the thread calculating the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory, and the process of the thread writing the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory, the read latency of the thread covering the global memory to read the current row matrix block and the current column matrix block is effectively reduced, the read and write latency of the global memory is improved, and the data processing efficiency is improved, and higher hardware utilization is achieved. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 An application environment diagram of a data reading method in an embodiment;
[0055] Figure 2 A schematic diagram of a flow chart of a data reading method in an embodiment;
[0056] Figure 3 A schematic diagram of matrix division in one embodiment;
[0057] Figure 4 A schematic diagram of reading and writing of a double buffer mechanism in another embodiment;
[0058] Figure 5 A timing diagram for loop execution of row matrix blocks and column matrix blocks in one embodiment;
[0059] Figure 6 A timing regularity diagram of reading, calculating and writing a plurality of row matrix blocks and a plurality of column matrix blocks if the number of row matrix blocks and column matrix blocks are both even numbers in one embodiment;
[0060] Figure 7 A timing regularity diagram of reading, calculating and writing a plurality of row matrix blocks and a plurality of column matrix blocks if the number of row matrix blocks and the number of column matrix blocks are both odd numbers in one embodiment;
[0061] Figure 8 A schematic diagram of a process for calculating a first calculation result in one embodiment;
[0062] Fig. 9 A schematic diagram of the partitioning and pre-fetching of row matrix blocks and column matrix blocks in one embodiment;
[0063] Fig.10 A timing diagram of cyclic execution of a sub-row matrix block and a sub-column matrix block in one embodiment;
[0064] Fig.11 A timing regularity diagram of reading and calculating a plurality of sub-row matrix blocks and a plurality of sub-column matrix blocks if the number of sub-row matrix blocks and the number of sub-column matrix blocks are both even numbers in one embodiment;
[0065] Fig.12 A timing regularity diagram of reading and calculating a plurality of sub-row matrix blocks and a plurality of sub-column matrix blocks if the number of sub-row matrix blocks and the number of sub-column matrix blocks are both odd numbers in one embodiment;
[0066] Fig.13 A flowchart of a conventional shared memory read and write synchronization in one embodiment;
[0067] Fig.14 A flowchart of delayed synchronous reading and writing of a shared memory in one embodiment;
[0068] Fig.15 A schematic diagram of a process for calculating a first intermediate result in one embodiment;
[0069] Fig.16 A schematic diagram of a process of multiplying a sub-row matrix block and a sub-column matrix block, and data sharing in one embodiment;
[0070] Fig.17 A schematic diagram of an overall scheme of data reading in one embodiment;
[0071] Fig.18 is a structural block diagram of a data reading device in one embodiment;
[0072] Fig.19 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION
[0073] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0074] The data reading method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the computer device 102 obtains the data read request initiated by the terminal 104, and splits the first matrix to be processed in the global memory into multiple row matrix blocks, and splits the second matrix to be processed into multiple column matrix blocks; the thread reads the current row matrix block and the current column matrix block, and the thread parallelly calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory to obtain a first calculation result; in the process of the thread processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory; after the thread completes the write operation on the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread writes The previous row matrix block and the previous column matrix block in the second shared memory are calculated to obtain a second calculation result; in the process of the thread processing the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread to the first shared memory; the next row matrix block of the current row matrix block in the multiple row matrix blocks is used as the current row matrix block of the next iteration loop, and the next column matrix block of the current column matrix block in the multiple column matrix blocks is used as the current column matrix block of the next iteration loop, and the step of returning to the thread to read the current row matrix block and the current column matrix block continues to execute until the loop end condition is reached, and multiple first calculation results and multiple second calculation results are obtained; the multiple first calculation results and the multiple second calculation results are accumulated to obtain an output matrix. Among them, the computer device 102 can be but not limited to a terminal and a server, and the computer device 102 includes a graphics processor or a hardware CPU chip, etc. The terminal can be but not limited to various personal computers, laptops, smart phones, tablets, Internet of Things devices, portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The server may be implemented as an independent server or a server cluster consisting of multiple servers.
[0075] In one embodiment, Figure 2 As shown, a data reading method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate, including the following steps:
[0076] Step 202: In response to a data read request, split the first matrix to be processed into a plurality of row matrix blocks, and split the second matrix to be processed into a plurality of column matrix blocks.
[0077] The first matrix is the right matrix in matrix multiplication, the second matrix is the left matrix in matrix multiplication, and the elements of the matrix are image data, scientific data, AI data or other data stored in the global memory. Figure 3 As shown, the product of the second matrix with a dimension of M×K multiplied by the first matrix with a dimension of K×N is equal to the matrix with a dimension of M×N. The first matrix is split in the horizontal direction to obtain multiple row matrix blocks B (Tile B) with a dimension of k×n, and the second matrix is split in the vertical direction to obtain multiple column matrix blocks A (Tile A) with a dimension of m×k. The number of row matrix blocks split from the first matrix is equal to the number of column matrix blocks split from the second matrix; the product of a column matrix block A with a dimension of m×k multiplied by a row matrix block B with a dimension of k×n is equal to a matrix slice with a dimension of m×n in the output matrix C. Each column matrix block A is subjected to matrix block multiplication and accumulation processing with the row matrix block B at the corresponding position to obtain multiple matrix slices with a dimension of m×n. The multiple matrix slices with a dimension of m×n are accumulated to obtain an output matrix C with a dimension of m×n.
[0078] Step 204, the thread reads the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; while the thread is processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory.
[0079] A group of computer devices calculates an output matrix C, and threads in the group of computer devices 102 cooperate to read column matrix block A and row matrix block B, and pre-fetch Figure 3 The next column matrix block A and row matrix block B in the K direction. The historical row matrix block and historical column matrix block of this loop are the current row matrix block and current column matrix block written into the first shared memory by the previous iteration loop thread.
[0080] The first shared memory and the second shared memory are two shared memories of the shared memory, which alternately store the current row matrix block B and the current column matrix block A read by the thread and the next row matrix block B and the next column matrix block A read by the thread. The read and write timing of the first shared memory (first buffer) and the second shared memory (second buffer) of this embodiment is as follows: Figure 4As shown. This embodiment adopts a double-buffered Ping-Pong operation mechanism. When the thread of the computer device reads the current row matrix block and the current column matrix block, and the thread performs matrix block multiplication on the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel, the thread of the computer device writes the next row matrix block B and the next column matrix block A read by the thread into the second shared memory; when the thread writes the next pre-fetched row matrix block and column matrix block to the first shared memory, the thread performs matrix block multiplication on the historical row matrix block and the historical column matrix block stored in the second shared memory; the loop is repeated until the loop stop condition is met. It should be noted that the initial writing order of the first shared memory and the second shared memory can also be to write to the second shared memory first and then to the first shared memory. This embodiment takes the writing of the first row matrix block and the first column matrix block to the first shared memory as an example.
[0081] In some embodiments, when the current column matrix block is the first column matrix block and the current row matrix block is the first row matrix block, the data reading method further includes the following steps:
[0082] Step 1, the thread reads the first row matrix block and the first column matrix block.
[0083] in, Figure 5 Loop through the timing diagram for the row matrix block and column matrix block, Figure 5 From left to right in the figure is the code execution timeline in the GPU processor, and from top to bottom are global memory read, synchronization and check memory read preparation instructions, shared memory write instructions, thread loop calculation process and final data output process. Instructions of different functional categories are distributed in the vertical direction in sequence, and the instruction execution delay can be observed through the instruction "tailing" area, and the masking of global memory read and shared memory read and write delays can be clearly seen. Figure 5 Reading column matrix block A and reading row matrix block B respectively indicate that the thread reads the row matrix block and the column matrix block, and the sequence numbers of the currently read row matrix block and column matrix block are determined according to the order of global memory reading.
[0084] Step 2, after the thread completes the read operation, the thread writes the first row matrix block and the first column matrix block into the first shared memory; while the thread is processing the first shared memory, the thread reads the next row matrix block and the next column matrix block, and simultaneously performs synchronization processing on the thread.
[0085] Alternatively, if Figure 5As shown, after the thread of the computer device completes reading the current row matrix block and the current column matrix block, the thread of the computer device writes the first row matrix block and the first column matrix block into the first shared memory, and in the delay corresponding to the "tailing" area of the thread writing the first row matrix block and the first column matrix block into the first shared memory, the thread reads the second row matrix block and the second column matrix block; in the delay corresponding to the "tailing" area of the thread reading the second row matrix block and the second column matrix block, the synchronization thread performs synchronization processing on the threads in parallel; after the thread completes reading the second row matrix block and the second column matrix block, the thread writes the second row matrix block and the second column matrix block into the second shared memory; in the delay corresponding to the "tailing" area of the thread writing the second row matrix block and the second column matrix block into the second shared memory, the thread reads the third row matrix block and the third column matrix block, and the thread calculates the first row matrix block and the first column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; after the thread completes the processing on the first shared memory, the thread writes the third row matrix block and the third column matrix block to the first shared memory.
[0086] In some embodiments, when the current row matrix block is any row matrix block after the second row matrix block, and the current column matrix block is any column matrix block after the second column matrix block, the thread reads the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; while the thread is processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory.
[0087] Optionally, taking the current row matrix block as the third row matrix block and the current column matrix block as the third column matrix block as an example, Figure 5 As shown, after the thread of the computer device reads the second row matrix block and the second column matrix block, the thread of the computer device reads the third row matrix block and the third column matrix block, and the thread calculates in parallel the first row matrix block and the first column matrix block written in the previous iteration loop in the first shared memory to obtain a first calculation result; while the thread is processing the first shared memory, the thread writes the second row matrix block and the second column matrix block to the second shared memory.
[0088] Step 206, after the thread completes the write operation on the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; while the thread is processing the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread to the first shared memory.
[0089] Among them, after the thread of this loop reads the current row matrix block and the current column matrix block and writes them to the first shared memory, the current row matrix block and the current column matrix block written in the first shared memory serve as the historical row matrix block and the historical column matrix block involved in the next iteration.
[0090] Optionally, taking the current row matrix block as the third row matrix block and the current column matrix block as the third column matrix block as an example, Figure 5 As shown, after the thread of the computer device writes the second row matrix block and the second column matrix block into the second shared memory, the thread reads the fourth row matrix block and the fourth column matrix block, and the thread performs matrix block multiplication on the second row matrix block and the second column matrix block written into the second shared memory to obtain a second calculation result; while the thread is processing the second shared memory, the thread writes the fourth row matrix block and the fourth column matrix block read by the thread into the first shared memory.
[0091] Step 208, use the next row matrix block of the current row matrix block among the multiple row matrix blocks as the current row matrix block of the next iteration loop, use the next column matrix block of the current column matrix block among the multiple column matrix blocks as the current column matrix block of the next iteration loop, and return to the thread to continue executing the step of reading the current row matrix block and the current column matrix block until the loop end condition is reached, and multiple first calculation results and multiple second calculation results are obtained.
[0092] The loop end condition may be receiving a loop interrupt instruction, or the execution of each row matrix block and each column matrix block in the first shared memory and the second shared memory is completed. In this embodiment, the plurality of row matrix blocks are marked as row matrix block B1 to row matrix block Bn in the splitting order, and the plurality of column matrix blocks are marked as column matrix block A1 to column matrix block An in the splitting order.
[0093] This embodiment takes the writing of the first row matrix block and the first column matrix block into the first shared memory as an example. In the process of alternately storing the row matrix blocks and the column matrix blocks in the first shared memory and the second shared memory, the row matrix blocks and the column matrix blocks marked as odd numbers are written into the first shared memory, and the row matrix blocks and the column matrix blocks marked as even numbers are written into the second shared memory. When the current row matrix block is any row matrix block after the second row matrix block, and the current column matrix block is any column matrix block after the second column matrix block, a loop cycle may be executed according to steps 204 and 206.
[0094] In some embodiments, if the number of row matrix blocks and column matrix blocks is an even number, the timing rules of reading, calculating and writing multiple row matrix blocks and multiple column matrix blocks are as follows: Figure 6 As shown, Figure 6 The timing steps in the dotted box are one cycle. When the current row matrix block is the last row matrix block and the current column matrix block is the last column matrix block, the thread writes the last row matrix block and the last column matrix block read in the previous iteration loop in the second shared memory into the second shared memory. In the thread's write operation, the thread performs matrix block multiplication on the second-to-last row matrix block and the second-to-last column matrix block in parallel to obtain a first calculation result; after the thread completes the calculation operation, the thread performs matrix block multiplication on the last row matrix block and the last column matrix block to obtain a second calculation result.
[0095] In some embodiments, if the number of row matrix blocks and column matrix blocks is an odd number, the timing rules of reading, calculating and writing multiple row matrix blocks and multiple column matrix blocks are as follows: Figure 7 As shown, Figure 7 The timing steps in the dotted box are one cycle. When the current row matrix block is the last row matrix block and the current column matrix block is the last column matrix block, the last row matrix block and the last column matrix block are in the last cycle. Therefore, after the last row matrix block and the last column matrix block have passed through one cycle, the thread performs matrix block multiplication on the last row matrix block and the last column matrix block to obtain a first calculation result.
[0096] Optionally, taking the first row matrix block and the first column matrix block being written into the first shared memory, the number of row matrix blocks and the number of column matrix blocks being an even number, the current row matrix block being the third row matrix block, and the current column matrix block being the third column matrix block as an example, after the third row matrix block and the third column matrix block execute the above steps 204 and 206, as follows Figure 6As shown, the computer device uses the next row matrix block of the current row matrix block in the multiple row matrix blocks as the current row matrix block of the next iteration loop, and uses the next column matrix block of the current column matrix block in the multiple column matrix blocks as the current column matrix block of the next iteration loop, that is, the fifth row matrix block is used as the current row matrix block, and the fifth column matrix block is used as the current column matrix block, and returns to step 204 to continue execution. After executing one loop cycle, the next row matrix block of the current row matrix block of the current loop cycle is again used as the current row matrix block of the next iteration loop, and the next column matrix block of the current column matrix block of the current loop cycle is used as the current column matrix block of the next iteration loop, and returns to step 204 to continue execution until the loop end condition is reached and stops, and multiple first calculation results and multiple second calculation results are obtained.
[0097] Step 210, accumulating the plurality of first calculation results and the plurality of second calculation results to obtain an output matrix.
[0098] The output matrix is obtained by multiplying and accumulating the corresponding positions of the plurality of row matrix blocks in the first matrix and the plurality of column matrix blocks in the second matrix. That is to say, the multiplication result of a row matrix block and a column matrix block is equal to a row of the output matrix, and the plurality of first calculation results and the plurality of second calculation results are accumulated in a time series order to obtain the output matrix.
[0099] The matrix multiplication process of this embodiment is specifically embodied in a subtask of convolution acceleration calculation in deep neural network learning, and the final output matrix can be the result corresponding to a subtask of convolution acceleration calculation. Specific application scenarios can be image recognition, model training, model prediction, artificial intelligence and other scenarios.
[0100] In this embodiment, the first matrix to be processed is split into a plurality of row matrix blocks, the second matrix to be processed is split into a plurality of column matrix blocks, the thread reads the current row matrix block and the current column matrix block, the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel, and in the process of the thread processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory. In the above method, through the process of the thread calculating the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory, and the process of the thread writing the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory, the read latency of the thread reading the current row matrix block and the current column matrix block covering the global memory is effectively reduced, and the read and write latency of the global memory is achieved. Higher hardware utilization.
[0101] In the above embodiment, data is read from the global memory into the shared memory, and the shared memory shares the mutual data. The read and write delay of the shared memory is less than that of the global memory, and the purpose of reducing data reading is achieved by this method. However, in order to realize the sharing of more data, more data needs to be stored in the shared memory, resulting in the problem that the read and write delay of the shared memory cannot be effectively covered and the synchronization overhead is large. Therefore, in order to solve the above problems, the row matrix blocks and column matrix blocks of the first shared memory and the second shared memory written to the shared memory are split again in this embodiment to obtain multiple sub-row matrix blocks and multiple sub-column matrix blocks. By cyclically calculating the multiplication process of the sub-row matrix blocks and the sub-column matrix blocks, the read delay of the thread covering the shared memory to read the current sub-row matrix block and the current sub-column matrix block is effectively reduced, and the hardware utilization rate is further improved.
[0102] In one embodiment, since in the above embodiment, the thread in parallel calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory to obtain the first calculation result, the process of the thread calculating the previous row matrix block and the previous column matrix block written in the second shared memory to obtain the second calculation result is the same, therefore, this embodiment only introduces the process of the thread processing the first shared memory. Specifically, Figure 8 As shown, the threads calculate the historical row matrix blocks and historical column matrix blocks written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result, including:
[0103] Step 802: split the historical row matrix block written in the previous iteration loop in the first shared memory into a plurality of sub-row matrix blocks, and split the historical column matrix block into a plurality of sub-column matrix blocks.
[0104] The historical row matrix block and the historical column matrix block written in the previous iteration cycle in the first shared memory are the current row matrix block and the current column matrix block written in the first shared memory in the previous iteration cycle.
[0105] The dimension of the historical row matrix block is k×n, and the dimension of the historical column matrix block is m×k. The multiplication of a historical column matrix block and a historical row matrix block obtains a matrix slice of dimension m×n in the output matrix C. For example, the value of k can be 1, 4, 16 or other values. In this embodiment, k is 4. The computer device splits the historical row matrix block horizontally to obtain multiple sub-row matrix blocks B' of dimension 4×n, and splits the historical column matrix block vertically to obtain multiple sub-column matrices A' of dimension m×4. The number of sub-row matrix blocks split from the historical row matrix block is equal to the number of sub-column matrix blocks split from the historical column matrix block. A sub-column matrix block A' of dimension m×4 is multiplied with a sub-row matrix block B' of dimension 4×n to obtain a matrix slice of dimension m×n. Each sub-row matrix block B' and each sub-column matrix block A' are subjected to matrix block multiplication and accumulation processing to obtain a matrix slice of dimension m×n in the output matrix C.
[0106] like Fig. 9 As shown, this embodiment splits the historical row matrix block into 4 sub-row matrix blocks, splits the historical column matrix block into 4 sub-column matrix blocks, and uses the first shared memory and the second shared memory to store the read and pre-fetched sub-column matrix block A' and sub-row matrix block B', and uses the matrix multiplication time of the sub-column matrix block A' and the sub-row matrix block B' to cover the shared memory read latency, thereby obtaining higher performance.
[0107] Optionally, taking the current row matrix block as the third row matrix block and the current column matrix block as the third column matrix block as an example, the historical row matrix block written in the previous iteration loop in the first shared memory of the computer device is the first row matrix block, and the historical column matrix block is the second column matrix block, the first row matrix block is split into multiple sub-row matrix blocks B', and the first column matrix block is split into multiple sub-column matrix blocks A'.
[0108] Step 804, the thread reads the current sub-row matrix block and the current sub-column matrix block in the first shared memory, and the thread calculates the historical sub-row matrix block and the historical sub-column matrix block read in the previous iteration loop in the first shared memory in parallel to obtain a first intermediate result; after the thread completes the calculation operation on the first shared memory, the thread reads the next sub-row matrix block of the current sub-row matrix block, reads the next sub-column matrix block of the current sub-column matrix block, and the thread calculates the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a second intermediate result.
[0109] Among them, the historical sub-row matrix block and historical sub-column matrix block read in the last iteration loop in the first shared memory are respectively the next sub-row matrix block of the current sub-row matrix block and the next sub-column matrix block of the current sub-column matrix block of the historical row matrix block read by the last iteration loop thread.
[0110] In some embodiments, if the current sub-column matrix block is the first sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-row matrix block is the first sub-row matrix block of the historical column matrix block in the first shared memory, the data reading method further includes the following steps:
[0111] Step 1: The thread reads the first sub-row matrix block of the historical row matrix block and the first sub-column matrix block of the current historical column matrix block in the first shared memory.
[0112] in, Fig.10 Loop through the timing diagram for the sub-row matrix block and the sub-column matrix block, Fig.10 From left to right is the timeline of the GPU processor executing instructions, from top to bottom are global memory read, shared memory write instruction, shared memory read instruction, synchronization and check memory read preparation instruction, thread loop calculation process. Instructions of different functional categories are distributed in the vertical direction in sequence, and the instruction execution delay can be observed through the instruction "tailing" area, and the masking of global memory read and shared memory read and write delay can be clearly seen.
[0113] Step 2, when the thread executes a read operation on the first shared memory, the thread reads the next sub-row matrix block and the next sub-column matrix block in the first shared memory, and the thread performs matrix block multiplication on the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a first intermediate result.
[0114] Alternatively, if Fig.10 As shown, after the thread initiates reading of the first sub-row matrix block and the first sub-column matrix block, the thread initiates a request to pre-fetch the second sub-row matrix block and the second sub-column matrix block in the first shared memory, and after waiting for the thread to complete reading the first sub-row matrix block and the first sub-column matrix block, while the thread reads the second sub-row matrix block and the second sub-column matrix block in the first shared memory, the thread performs matrix block multiplication on the first sub-row matrix block and the first sub-column matrix block in parallel to obtain a first intermediate result.
[0115] In some embodiments, when the current sub-row matrix block is any sub-row matrix block after the second sub-column matrix block, and the current sub-column matrix block is any sub-column matrix block after the second sub-column matrix block, the thread reads the current sub-row matrix block and the current sub-column matrix block in the first shared memory, and the thread calculates the historical sub-row matrix block and the historical sub-column matrix block read in the previous iteration loop in the first shared memory in parallel to obtain a first intermediate result; after the thread completes the calculation operation on the first shared memory, the thread reads the next sub-row matrix block of the current sub-row matrix block, reads the next sub-column matrix block of the current sub-column matrix block, and the thread calculates the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a second intermediate result.
[0116] The next sub-row matrix block of the current sub-row matrix block and the next sub-column matrix block of the current sub-column matrix block read by the thread of this loop are used as the historical sub-row matrix blocks and historical sub-column matrix blocks involved in the next iteration.
[0117] Optionally, taking the current sub-row matrix block as the third sub-row matrix block and the current sub-column matrix block as the third sub-column matrix block as an example, Fig.10 As shown, the thread reads the third sub-row matrix block and the third sub-column matrix block in the first shared memory. After the thread completes reading the historical sub-row matrix block and the historical sub-column matrix block in the previous iteration loop, the thread parallelly calculates the second sub-row matrix block and the second sub-column matrix block read in the previous iteration loop in the first shared memory to obtain the first intermediate result. After the thread completes the calculation operation on the first shared memory, the thread reads the fourth sub-row matrix block and the fourth sub-column matrix block, and the thread parallelly calculates the third sub-row matrix block and the third sub-column matrix block to obtain the second intermediate result.
[0118] Step 806, use the next sub-row matrix block of the current sub-row matrix block among the multiple sub-row matrix blocks as the current sub-row matrix block of the next iteration loop, use the next sub-column matrix block of the current sub-column matrix block among the multiple sub-column matrix blocks as the current sub-column matrix block of the next iteration loop, and return to the thread to continue executing the step of reading the current sub-row matrix block and the current sub-column matrix block until the loop end condition is reached and stop, to obtain multiple first intermediate results and multiple second intermediate results.
[0119] The loop end condition may be receiving a loop interrupt instruction, or the execution of each row matrix block and each column matrix block in the first shared memory and the second shared memory is completed. In this embodiment, multiple sub-row matrix blocks are marked as sub-row matrix blocks B1'~Bn' in the splitting order, and multiple sub-column matrix blocks are marked as sub-column matrix blocks A1'~An' in the splitting order. When the current sub-row matrix block is any sub-row matrix block after the second sub-row matrix block, and the current sub-column matrix block is any sub-column matrix block after the second sub-column matrix block, a loop cycle may be executed according to steps 802 and 804.
[0120] In some embodiments, if the number of sub-row matrix blocks and sub-column matrix blocks is an even number, the timing rules for reading and calculating the multiple sub-row matrix blocks and the multiple sub-column matrix blocks are as follows: Fig.11 As shown, Fig.11 The timing steps in the dotted box are one cycle. When the current sub-row matrix block is the last sub-row matrix block and the current sub-column matrix block is the last sub-column matrix block, the last sub-row matrix block and the last sub-column matrix block are in the last cycle. Fig.11It can be seen that after the last sub-row matrix block and the last sub-column matrix block go through one cycle, the thread performs matrix block multiplication processing on the last sub-row matrix block and the last sub-column matrix block to obtain a second intermediate result.
[0121] In some embodiments, if the number of sub-row matrix blocks and sub-column matrix blocks is an odd number, the timing rules for reading and calculating the plurality of sub-row matrix blocks and the plurality of sub-column matrix blocks are as follows: Fig.12 As shown, Fig.12 The timing steps in the dotted box are one cycle. When the current sub-row matrix block is the last sub-row matrix block and the current sub-column matrix block is the last sub-column matrix block, the thread reads the last sub-row matrix block and the last sub-column matrix block in the first shared memory, and the thread performs matrix block multiplication on the second-to-last sub-row matrix block and the second-to-last sub-column matrix block in parallel to obtain the second intermediate result; after the thread completes the calculation operation, the thread performs matrix block multiplication on the last sub-row matrix block and the last sub-column matrix block again to obtain the first intermediate result.
[0122] Optionally, taking the case where the number of sub-row matrix blocks and sub-column matrix blocks are both even numbers, the current sub-row matrix block is the third sub-row matrix block, and the current sub-column matrix block is the third sub-column matrix block as an example, after the third sub-row matrix block and the third sub-column matrix block perform the above steps 802 and 804, Fig.11 As shown, the next sub-row matrix block of the current sub-row matrix block among the multiple sub-row matrix blocks is used as the current sub-row matrix block of the next iteration cycle, and the next sub-column matrix block of the current sub-column matrix block among the multiple sub-column matrix blocks is used as the current sub-column matrix block of the next iteration cycle, that is, the fifth sub-row matrix block is used as the current sub-row matrix block, and the fifth sub-column matrix block is used as the current sub-column matrix block, and the execution is returned to step 804 for further execution. After one loop cycle is executed, the next sub-row matrix block of the current sub-row matrix block of the current loop cycle is used as the current sub-row matrix block of the next iteration cycle, and the next sub-column matrix block of the current sub-column matrix block of the current loop cycle is used as the current sub-column matrix block of the next iteration cycle, and the execution is returned to step 804 for further execution, and the execution is stopped when the loop end condition is met, and multiple first intermediate results and multiple second intermediate results are obtained.
[0123] Step 808: Accumulate the multiple first intermediate results and the multiple second intermediate results to obtain a first calculation result.
[0124] The result of multiplying and accumulating the corresponding positions of the multiple sub-row matrix blocks of the current row matrix block and the multiple sub-column matrix blocks of the current column matrix block written into the first shared memory is the product result of the current row matrix block and the current column matrix block.
[0125] In this embodiment, the row matrix blocks and column matrix blocks of the first shared memory and the second shared memory written to the shared memory are split again to obtain multiple sub-row matrix blocks and multiple sub-column matrix blocks. Through a two-stage loop, the first shared memory and the second shared memory are used to implement global memory data prefetching and a dual register group is used to implement shared memory data prefetching. The matrix multiplication of the row matrix blocks and the column matrix blocks, and the matrix multiplication of the sub-row matrix blocks and the sub-column matrix blocks are used to cover the global memory and shared memory read latency, effectively reducing the read and write latency of the shared memory, reducing the number of synchronization operations and overhead, and improving the GEMM calculation efficiency.
[0126] In some embodiments, Fig.13 As shown in the figure, after the shared memory write instruction is executed, the conventional operation is to immediately execute the synchronization instruction and then execute the shared memory read instruction. This method will cause the latency of the shared memory write instruction to be unable to be covered. On the other hand, after the synchronization instruction is executed, all threads will execute to the same point. When the shared memory is read concurrently later, the latency cannot be covered by thread-level parallelism. Therefore, in order to solve the above problems, Fig.14 As shown, this embodiment executes a synchronization instruction before the penultimate sub-row matrix block and sub-column matrix block of the first shared memory perform matrix block multiplication processing, ensuring that when the matrix multiplication loop of the second shared memory starts, the read row matrix block and column matrix block are written into the first shared memory, and the rectangular multiplication processing is performed after the synchronization instruction, which can reduce the competition for shared hardware units (such as data reading units, shared memory units) caused by synchronization, more effectively reduce thread-level parallel latency, and realize parallel reading and writing. Fig.10 As shown, the execution sequence of the synchronization instruction is as follows:
[0127] If the current sub-row matrix block is the last sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, then before the threads parallelly calculate the historical sub-row matrix blocks and historical sub-column matrix blocks read in the previous iteration loop in the first shared memory, the threads that read the sub-row matrix blocks and sub-row matrix blocks in the first shared memory are synchronized to ensure that the threads synchronously read the sub-row matrix blocks and sub-row matrix blocks; the threads that write the sub-row matrix blocks and sub-row matrix blocks in the first shared memory are synchronized to ensure that the threads synchronously write the sub-row matrix blocks and sub-row matrix blocks.
[0128] In this embodiment, before the penultimate sub-row matrix block and sub-column matrix block of the first shared memory are subjected to matrix block multiplication processing, a synchronization instruction is executed to ensure that when the matrix multiplication loop of the second shared memory starts, the read row matrix block and column matrix block are both written into the first shared memory, and rectangular multiplication processing is performed after the synchronization instruction. This can reduce the competition for shared hardware units (such as data reading units and shared memory units) caused by synchronization, more effectively reduce thread-level parallel latency, and achieve parallel reading and writing.
[0129] In some embodiments, Fig.10 As shown, in the process of performing matrix block multiplication processing on the last sub-row matrix block and the last sub-column matrix block in the first shared memory, the read latency of the shared memory is not covered, and in the process of the thread reading the first sub-row matrix block and the first sub-column matrix block in the first shared memory, the latency of other threads is not covered, which wastes the latency of the thread in the first shared memory reading the first sub-row matrix block and the first sub-column matrix block; Fig.11 and Fig.12 It can also be seen that the read delay of the shared memory is not covered in the process of matrix block multiplication of the last sub-row matrix block and the last sub-column matrix block. Therefore, in order to solve the above problem, when the current sub-row matrix block is the last sub-row matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, after the thread calculates the historical sub-row matrix blocks and the historical sub-column matrix blocks read in the last iteration loop in the first shared memory in parallel, the thread delays the process of matrix block multiplication of the last sub-row matrix block of the historical sub-row matrix block and the last sub-column matrix block of the historical sub-column matrix block until the thread calculates the previous row matrix block and the previous column matrix block written to the second shared memory, and the thread reads the first sub-row matrix block of the previous row matrix block of the current row matrix block and the first sub-column matrix block of the previous column matrix block of the current column matrix block in the second shared memory. That is, Fig.10 As shown by the dotted line in the middle, the process of multiplying the last matrix is delayed to the first sub-row matrix block of the previous row matrix block of the current row matrix block and the first sub-column matrix block of the previous column matrix block of the current column matrix block in the second shared memory to cover the reading delay of the first sub-row matrix block and the first sub-column matrix block in the second shared memory. In the entire loop process, the shared memory reading delay is fully covered.
[0130] It should be noted that the calculation of the second calculation result in the second shared memory and the calculation process of the thread in the second shared memory are the same as the calculation process of the thread in the first shared memory, such as Figure 6As shown, after the thread calculates the historical row matrix block and the historical column matrix block written in the last iteration loop in the first shared memory, the thread calculates the previous row matrix block and the previous column matrix block of the current column matrix block in the second shared memory. That is to say, after the thread finishes the calculation work on the first shared memory, it then calculates the second shared memory. Therefore, this embodiment delays the calculation process of the last column matrix block of the historical row matrix block and the last sub-column matrix block of the historical column matrix block in the first shared memory to the calculation process of the second shared memory.
[0131] It should be noted that: the calculation process of the thread on the second shared memory is the same as the above embodiment, which also splits the previous row matrix block of the current row matrix block into multiple sub-row matrix blocks, and splits the previous column matrix block of the current column matrix block into multiple sub-column matrix blocks. In the process of reading the sub-row matrix block of the previous row matrix block of the current row matrix block and the sub-column matrix block of the previous column matrix block of the current column matrix block, the thread calculates the sub-row matrix block and the sub-column matrix block to obtain the second calculation result. In other words, the calculation process of the last column matrix block of the historical row matrix block and the last sub-column matrix block of the historical column matrix block in the first shared memory is delayed to the first sub-row matrix block of the previous row matrix block of the current row matrix block and the first sub-column matrix block of the previous column matrix block of the current column matrix block in the second shared memory.
[0132] It should be noted that: except for the last calculation process of the thread that does not use the delay thread, no matter the thread is calculating the historical row matrix block, the historical column matrix block, or the previous row matrix block of the current row matrix block, or the previous column matrix block of the current column matrix block, before calculating the last sub-row matrix block and the last sub-column matrix block, the matrix multiplication process of the last sub-row matrix block and the last sub-column matrix block needs to be delayed until the start of the next calculation of the thread.
[0133] Optionally, with the current row matrix block as the third row matrix block and the current column matrix block as the third column matrix block, the thread calculates the first row matrix block and the first column matrix block written in the last iteration loop of the first shared memory. During the calculation process, the first row matrix block is split into multiple sub-row matrix blocks, and the first column matrix block is split into multiple sub-column matrix blocks. Before the thread calculates the last sub-row matrix block of the first row matrix block and the last sub-column matrix block of the first column matrix block, the calculation process of the last sub-row matrix block of the first row matrix block and the last sub-column matrix block of the first column matrix block is delayed until the thread reads the first sub-row matrix block of the second row matrix block and the first sub-column matrix block of the second column matrix block in the second shared memory, so as to cover the delay of the thread reading the first sub-row matrix block of the second row matrix block and the first sub-column matrix block of the second column matrix block in the second shared memory.
[0134] This embodiment executes a delay instruction to delay the process of matrix block multiplication of the last sub-row matrix block and the last sub-column matrix block of the first shared memory until the thread reads the first sub-row matrix block and the first sub-column matrix block in the second shared memory, covering the reading delay of the first sub-row matrix block and the first sub-column matrix block in the second shared memory, thereby achieving complete coverage of the shared memory read delay.
[0135] In some embodiments, the GPU hardware executes instructions in a single instruction multiple data (SIMD) manner. 32 threads are organized together into a thread bundle. Each thread bundle has several vector registers CRF (Common Register File). The length of the vector register is 32. Threads in the thread bundle cannot access data from other threads. In order to achieve fast data sharing within the thread bundle, each GPU thread bundle has several scalar registers SRF (Scalar Register File). The scalar registers are used to achieve fast data sharing within the thread bundle. Some GPUs can share the data to be shared with other threads through hardware broadcast instructions. In order to make full use of the hardware characteristics to quickly share data and achieve efficient sub-matrix multiplication, this embodiment stores each row element of the sub-row matrix block in a vector register, and stores a single element of a row of elements of the sub-column matrix block in a scalar register. The single element in the scalar register is multiplied and accumulated with a row of elements in the vector register to obtain the multiplication result of the sub-row matrix block and the sub-column matrix block, that is, the first intermediate result or the second intermediate result. Among them, the calculation process of the first intermediate result and the second intermediate result is the same. This embodiment only introduces the calculation process of the first intermediate result. Specifically, as Fig.15As shown, the threads calculate in parallel the historical sub-row matrix blocks and the historical sub-column matrix blocks read in the last iteration loop in the first shared memory to obtain a first intermediate result, including:
[0136] In step 1502, the thread stores each row element of the historical sub-row matrix block read in the previous iteration loop in the first shared memory into a vector register; the thread concurrently stores a single element of each row element of the historical sub-column matrix block read in the previous iteration loop in the first shared memory into a scalar register.
[0137] The historical sub-column matrix block also needs to be read into the vector register and then moved to the scalar register. Fig.16 As shown, in this embodiment, the length and width of the sub-row matrix block B' are 32 and 4 respectively, the length and width of the sub-column matrix block A' are 4 and 32 respectively, and the 4 elements of a row of the sub-column matrix block A' are stored in scalar registers SR0~SR3 in sequence, and the 4 row elements of the sub-row matrix block B' are stored in R1~R4.
[0138] Optionally, in the GPU, four consecutive single elements of a row of the sub-column matrix block A' can be shared sequentially to the scalar registers SR0-SR3 through a MOVQLN instruction.
[0139] Step 1504: The thread performs multiplication and accumulation processing on a single element in the scalar register and each row of elements in the vector register to obtain a first intermediate result.
[0140] like Fig.16 As shown, the single elements stored in the scalar registers SR0~SR3 are multiplied with the elements of a row stored in the vector register respectively, and then accumulated to obtain the matrix multiplication result of the historical sub-row matrix block and the historical sub-column matrix block read in the last iteration loop in the first shared memory, that is, the first intermediate result.
[0141] Optionally, the GPU processor implements the following multiplication and accumulation calculation through an instruction FMADC+Repeat3 Modifier: R10=R10+SR0×R1+SR1×R2+SR2×R3+SR3×R4, and the FMADC instruction is: FMADC.rp3 R10,R10,SR0,R1; the Repeat3 Modifier instruction is: MOVQLN SR0 R6,sqd0x0,qdn0x1. The FMADC instruction can also be combined with the MOVQLN instruction to perform a static combine into one instruction, hiding the MOVQLN instruction overhead and increasing the proportion of effective computing instructions.
[0142] In this embodiment, each row element of the historical sub-row matrix block read in the last iteration loop in the first shared memory is stored in a vector register; a single element of each row element of the historical sub-column matrix block read in the last iteration loop in the first shared memory is stored in a scalar register, and fast data sharing within the thread bundle is achieved through the scalar register SRF inside the GPU thread bundle, thereby achieving efficient sub-matrix multiplication.
[0143] In one embodiment, Fig.17 As shown, the data reading method specifically includes the following steps:
[0144] Step 1: In response to a data read request, split the first matrix to be processed into a plurality of row matrix blocks, and split the second matrix to be processed into a plurality of column matrix blocks.
[0145] Step 2, the thread reads the first row matrix block and the first column matrix block.
[0146] Step 3, after the thread completes the read operation, the thread writes the first row matrix block and the first column matrix block into the first shared memory; while the thread is processing the first shared memory, the thread reads the second row matrix block and the second column matrix block, and simultaneously performs synchronization processing on the thread.
[0147] Step 4: After the thread completes the write operation, the thread reads the current row matrix block and the current column matrix block.
[0148] Step 5: During the process of the thread reading the current row matrix block and the current column matrix block, the historical row matrix block written in the previous iteration loop in the first shared memory is split into multiple sub-row matrix blocks, and the historical column matrix block is split into multiple sub-column matrix blocks.
[0149] Step 6, the thread reads the first sub-row matrix block of the historical row matrix block and the first sub-column matrix block of the historical column matrix block in the first shared memory. When the thread performs a read operation on the first shared memory, the thread reads the next sub-row matrix block and the next sub-column matrix block in the first shared memory. The thread performs matrix block multiplication on the first sub-row matrix block and the first sub-column matrix block in parallel to obtain a first intermediate result.
[0150] Step 7, after the thread performs matrix block multiplication processing on the first sub-row matrix block and the first sub-column matrix block in parallel, the thread reads the current sub-row matrix block and the current sub-column matrix block in the first shared memory, and the thread calculates the historical sub-row matrix block and the historical sub-column matrix block read in the previous iteration loop in the first shared memory in parallel to obtain a first intermediate result; after the thread completes the calculation operation on the first shared memory, the thread reads the next sub-row matrix block of the current sub-row matrix block, reads the next sub-column matrix block of the current sub-column matrix block, and the thread calculates the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a second intermediate result; the current sub-row matrix block is any sub-row matrix block after the second sub-column matrix block, and the current sub-column matrix block is any sub-column matrix block after the second sub-column matrix block.
[0151] Step 8: Use the next sub-row matrix block of the current sub-row matrix block in the multiple sub-row matrix blocks as the current sub-row matrix block of the next iteration loop, use the next sub-column matrix block of the current sub-column matrix block in the multiple sub-column matrix blocks as the current sub-column matrix block of the next iteration loop, and return to the thread to read the current sub-row matrix block and the current sub-column matrix block and continue to execute until the loop end condition is reached, and multiple first intermediate results and multiple second intermediate results are obtained. Accumulate the multiple first intermediate results and the multiple second intermediate results to obtain a first calculation result.
[0152] Step 9, while the thread is processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory; the current row matrix block is any row matrix block after the second row matrix block, and the current column matrix block is any column matrix block after the second column matrix block.
[0153] Step 10, after the thread completes the write operation on the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; while the thread is processing the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread to the first shared memory.
[0154] Step 11, use the next row matrix block of the current row matrix block among the multiple row matrix blocks as the current row matrix block of the next iteration loop, use the next column matrix block of the current column matrix block among the multiple column matrix blocks as the current column matrix block of the next iteration loop, and return to the thread to read the current row matrix block and the current column matrix block and continue to execute until the loop end condition is reached and stop, to obtain multiple first calculation results and multiple second calculation results.
[0155] Step 12, accumulating the multiple first calculation results and the multiple second calculation results to obtain an output matrix.
[0156] This embodiment uses matrix blocking and data sharing through shared memory and register-level data sharing to reduce memory data reading, and uses a two-level loop approach to mask the memory read latency and shared memory read and write latency through matrix multiplication time, achieving very good results. The hardware utilization rate reaches 95%, and the effective computing power reaches 90% of the GPU peak computing power.
[0157] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.
[0158] Based on the same inventive concept, the embodiment of the present application also provides a data reading device for implementing the data reading method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data reading device embodiments provided below can refer to the limitations on the data reading method above, and will not be repeated here.
[0159] In one embodiment, Fig.18 As shown, a data reading device is provided, comprising:
[0160] The block division module 100 is used for dividing the first matrix to be processed into a plurality of row matrix blocks and dividing the second matrix to be processed into a plurality of column matrix blocks in response to a data reading request;
[0161] The first reading module 200 is used for the thread to read the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; when the thread processes the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory.
[0162] The second reading module 300 is used for, after the thread completes the write operation to the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; in the process of the thread processing the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread to the first shared memory.
[0163] The iteration module 400 uses the next row matrix block of the current row matrix block among the multiple row matrix blocks as the current row matrix block of the next iteration loop, uses the next column matrix block of the current column matrix block among the multiple column matrix blocks as the current column matrix block of the next iteration loop, and returns to the thread to continue executing the step of reading the current row matrix block and the current column matrix block until the loop end condition is reached, thereby obtaining multiple first calculation results and multiple second calculation results.
[0164] The output module 500 is used to accumulate the multiple first calculation results and the multiple second calculation results to obtain an output matrix.
[0165] In one of the embodiments, if the current column matrix block is the first column matrix block, and the current row matrix block is the first row matrix block, the first reading module 200 is also used for the thread to read the first row matrix block and the first column matrix block; after the thread completes the read operation, the thread writes the first row matrix block and the first column matrix block to the first shared memory; while the thread is processing the first shared memory, the thread reads the next row matrix block and the next column matrix block, and simultaneously performs synchronization processing on the thread.
[0166] In one of the embodiments, the first reading module 200 is further used to split the historical row matrix block written in the previous iteration loop in the first shared memory into a plurality of sub-row matrix blocks, and split the historical column matrix block into a plurality of sub-column matrix blocks;
[0167] The thread reads the current sub-row matrix block and the current sub-column matrix block in the first shared memory, and the thread calculates the historical sub-row matrix block and the historical sub-column matrix block read in the previous iteration loop in the first shared memory in parallel to obtain a first intermediate result; after the thread completes the calculation operation on the first shared memory, the thread reads the next sub-row matrix block of the current sub-row matrix block, reads the next sub-column matrix block of the current sub-column matrix block, and the thread calculates the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a second intermediate result;
[0168] The next sub-row matrix block of the current sub-row matrix block among the multiple sub-row matrix blocks is used as the current sub-row matrix block of the next iteration loop, and the next sub-column matrix block of the current sub-column matrix block among the multiple sub-column matrix blocks is used as the current sub-column matrix block of the next iteration loop, and the thread is returned to the step of reading the current sub-row matrix block and the current sub-column matrix block and continuing to execute until the loop end condition is reached, and a plurality of first intermediate results and a plurality of second intermediate results are obtained;
[0169] Accumulate the plurality of first intermediate results and the plurality of second intermediate results to obtain a first calculation result.
[0170] In one embodiment, if the current sub-column matrix block is the first sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-row matrix block is the first sub-row matrix block of the historical column matrix block in the first shared memory, the first reading module 200 is further used to thread read the first sub-row matrix block of the historical row matrix block in the first shared memory, and the first sub-column matrix block of the historical column matrix block;
[0171] When the thread executes a read operation on the first shared memory, the thread reads the next sub-row matrix block and the next sub-column matrix block in the first shared memory, and the thread performs matrix block multiplication on the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a first intermediate result.
[0172] In one embodiment, if the current sub-row matrix block is the last sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, the first reading module 200 is also used to synchronize threads and threads in the first shared memory before the threads parallelly calculate the historical sub-row matrix blocks and historical sub-column matrix blocks read in the last iteration loop in the first shared memory.
[0173] In one embodiment, if the current sub-row matrix block is the last sub-row matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, the first reading module 200 is also used to use threads to parallelly calculate the historical sub-row matrix blocks and historical sub-column matrix blocks read in the last iteration loop in the first shared memory, and the thread delays the process of performing matrix block multiplication on the last sub-row matrix block of the historical sub-row matrix block and the last sub-column matrix block of the historical sub-column matrix block until the thread calculates the previous row matrix block and the previous column matrix block written to the second shared memory, and the thread reads the first sub-row matrix block of the previous row matrix block of the current row matrix block and the first sub-column matrix block of the previous column matrix block of the current column matrix block in the second shared memory.
[0174] In one embodiment, the first reading module 200 is further used to: use a thread to store each row element of the historical sub-row matrix block read in the last iteration loop in the first shared memory into a vector register; the thread parallelly stores a single element of each row element of the historical sub-column matrix block read in the last iteration loop in the first shared memory into a scalar register;
[0175] A thread is used to perform multiplication and accumulation processing on a single element in the scalar register and each row element in the vector register to obtain a first intermediate result.
[0176] Each module in the above data reading device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.
[0177] In one embodiment, a computer device is provided. The computer device may be an image processor, and its internal structure diagram may be as follows: Fig.19 As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a data reading method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.
[0178] Those skilled in the art will understand that Fig.19The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.
[0179] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.
[0180] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in each method embodiment are implemented.
[0181] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in each method embodiment when executed by a processor.
[0182] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0183] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.
[0184] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0185] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.
Claims
1. A data reading method, characterized in that: The method comprises: In response to a data read request, splitting a first matrix to be processed into a plurality of row matrix blocks, and splitting a second matrix to be processed into a plurality of column matrix blocks; The thread reads the current row matrix block and the current column matrix block, and the thread calculates the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory in parallel to obtain a first calculation result; in the process of the thread processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory; After the thread completes the write operation to the second shared memory, the thread reads the next row matrix block of the current row matrix block, reads the next column matrix block of the current column matrix block, and the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; in the process of the thread processing the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread into the first shared memory; The next row matrix block of the current row matrix block in the multiple row matrix blocks is used as the current row matrix block of the next iteration loop, and the next column matrix block of the current column matrix block in the multiple column matrix blocks is used as the current column matrix block of the next iteration loop, and the thread is returned to the step of reading the current row matrix block and the current column matrix block to continue executing until the loop end condition is reached, and a plurality of first calculation results and a plurality of second calculation results are obtained; Accumulate the plurality of first calculation results and the plurality of second calculation results to obtain an output matrix.
2. The method according to claim 1, characterized in that If the current column matrix block is the first column matrix block, and the current row matrix block is the first row matrix block, the method further includes: The thread reads a first row matrix block and a first column matrix block; After the thread completes the read operation, the thread writes the first row matrix block and the first column matrix block into the first shared memory; while the thread is processing the first shared memory, the thread reads the next row matrix block and the next column matrix block, and the thread is processed synchronously.
3. The method according to claim 1, characterized in that The threads calculate in parallel the historical row matrix blocks and the historical column matrix blocks written in the previous iteration loop in the first shared memory to obtain a first calculation result, including: Splitting the historical row matrix block written in the previous iteration loop in the first shared memory into a plurality of sub-row matrix blocks, and splitting the historical column matrix block into a plurality of sub-column matrix blocks; The thread reads a current sub-row matrix block and a current sub-column matrix block in the first shared memory, and the thread calculates in parallel the historical sub-row matrix block and the historical sub-column matrix block read in the previous iteration loop in the first shared memory to obtain a first intermediate result; after the thread completes the calculation operation on the first shared memory, the thread reads the next sub-row matrix block of the current sub-row matrix block, reads the next sub-column matrix block of the current sub-column matrix block, and the thread calculates in parallel the current sub-row matrix block and the current sub-column matrix block to obtain a second intermediate result; The next sub-row matrix block of the current sub-row matrix block in the multiple sub-row matrix blocks is used as the current sub-row matrix block of the next iteration loop, and the next sub-column matrix block of the current sub-column matrix block in the multiple sub-column matrix blocks is used as the current sub-column matrix block of the next iteration loop, and the thread is returned to the step of reading the current sub-row matrix block and the current sub-column matrix block and continuing to execute until the loop end condition is reached, and a plurality of first intermediate results and a plurality of second intermediate results are obtained; Accumulate the plurality of first intermediate results and the plurality of second intermediate results to obtain a first calculation result.
4. The method according to claim 3, characterized in that: If the current sub-column matrix block is the first sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-row matrix block is the first sub-row matrix block of the historical column matrix block in the first shared memory, the method further includes: The thread reads the first sub-row matrix block of the historical row matrix block and the first sub-column matrix block of the current historical column matrix block in the first shared memory; When the thread executes a read operation on the first shared memory, the thread reads the next sub-row matrix block and the next sub-column matrix block in the first shared memory, and the thread performs matrix block multiplication processing on the current sub-row matrix block and the current sub-column matrix block in parallel to obtain a first intermediate result.
5. The method according to claim 3, characterized in that: If the current sub-row matrix block is the last sub-column matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, before the thread calculates the historical sub-row matrix block and the historical sub-column matrix block read in the last iteration loop in the first shared memory in parallel, the method further includes: A synchronization operation is performed on threads that read sub-row matrix blocks and sub-row matrix blocks in the first shared memory; and a synchronization operation is performed on threads that write sub-row matrix blocks and sub-row matrix blocks in the first shared memory.
6. The method according to claim 3, characterized in that: If the current sub-row matrix block is the last sub-row matrix block of the historical row matrix block in the first shared memory, and the current sub-column matrix block is the last sub-column matrix block of the historical column matrix block in the first shared memory, then after the thread calculates the historical sub-row matrix block and the historical sub-column matrix block read in the last iteration loop in the first shared memory in parallel, the method further includes: The process of the thread performing matrix block multiplication processing on the last sub-row matrix block of the historical sub-row matrix block and the last sub-column matrix block of the historical sub-column matrix block is delayed until the thread calculates the previous row matrix block and the previous column matrix block written into the second shared memory, and is executed during the process of the thread reading the first sub-row matrix block of the previous row matrix block of the current row matrix block and the first sub-column matrix block of the previous column matrix block of the current column matrix block in the second shared memory.
7. The method according to claim 3, characterized in that The thread calculates in parallel the historical sub-row matrix block and the historical sub-column matrix block read in the last iteration loop in the first shared memory to obtain a first intermediate result, including: The thread stores each row element of the historical sub-row matrix block read in the last iteration loop in the first shared memory into a vector register; the thread stores a single element of each row element of the historical sub-column matrix block read in the last iteration loop in the first shared memory into a scalar register in parallel; The thread performs multiplication and accumulation processing on a single element in the scalar register and each row of elements in the vector register to obtain a first intermediate result.
8. A data reading device, characterized in that: The device comprises: A block division module, configured to divide the first matrix to be processed into a plurality of row matrix blocks and the second matrix to be processed into a plurality of column matrix blocks in response to a data read request; A first reading module uses a thread to read a current row matrix block and a current column matrix block, and the thread calculates in parallel the historical row matrix block and the historical column matrix block written in the previous iteration loop in the first shared memory to obtain a first calculation result; in the process of the thread processing the first shared memory, the thread writes the previous row matrix block of the current row matrix block and the previous column matrix block of the current column matrix block to the second shared memory; a second reading module, configured to, after the thread completes the write operation to the second shared memory, cause the thread to read the next row matrix block of the current row matrix block, read the next column matrix block of the current column matrix block, and calculate the previous row matrix block and the previous column matrix block written into the second shared memory to obtain a second calculation result; and when the thread processes the second shared memory, the thread writes the current row matrix block and the current column matrix block read by the thread into the first shared memory; an iteration module, used to use the next row matrix block of the current row matrix block in the multiple row matrix blocks as the current row matrix block of the next iteration loop, use the next column matrix block of the current column matrix block in the multiple column matrix blocks as the current column matrix block of the next iteration loop, and return to the step of the thread reading the current row matrix block and the current column matrix block to continue executing until the loop end condition is reached, thereby obtaining multiple first calculation results and multiple second calculation results; The output module is used to accumulate the multiple first calculation results and the multiple second calculation results to obtain an output matrix.
9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
