Non-blocking multi-thread matrix multiplication processing method based on state polling
By adopting a non-blocking multithreading method based on state polling in matrix multiplication, the problem of unbalanced task division and large thread synchronization overhead in non-regular matrix calculation is solved, efficient matrix multiplication calculation is realized, and CPU utilization is improved rate and performance.
Patent Information
- Application Number
- CN202411054692.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-02
- Publication Date
- 2024-11-15
AI Technical Summary
In the prior art, the processing efficiency of non-regular matrices in matrix multiplication calculation is low, mainly due to the unbalanced task division and the large overhead of thread synchronization, resulting in waste of computing resources and low CPU utilization.
The non-blocking multi-threaded matrix multiplication processing method based on state polling is adopted to traverse other threads sequentially by the current thread, and realize non-blocking thread synchronization and calculation, avoid thread waiting and improve computational efficiency.
It significantly improves the efficiency of matrix multiplication calculation, reduces thread waiting time, and improves CPU utilization, which is manifested as a significant improvement in performance acceleration ratio.
Smart Images

Figure CN118964811A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer science, and in particular to a non-blocking multi-thread matrix multiplication processing method based on state polling. Background Art
[0002] Generalized Matrix Multiplication (GEMM) is one of the most common and important operations in linear algebra, and is widely used in scientific computing, engineering computing, machine learning, and other fields. The calculation formula of general GEMM is C = α*A*B+β*C, where C, A, and B are three matrices with matrix dimensions [M, N], [M, K], and [K, N], respectively.
[0003] If the size of the matrix to be processed is relatively regular, the current matrix multiplication operation has various performance optimizations based on the matrix size. However, the prior art has given little consideration to some irregular matrix multiplications. The so-called irregular matrix multiplication means that the dimensions of the matrix are irregular, such as Figure 2 (1) shows the thin matrix A and the small matrix B, and Figure 2 As shown in (2), the fat and short matrix A and the thin and long matrix B, the main performance issues of multi-threaded matrix multiplication are unbalanced task division and thread synchronization overhead.
[0004] At present, the single core matrix multiplication task of the processor is mainly divided into three parts: block operation, pack operation and calculation operation. First, each thread is responsible for completing the pack operation of the input data required for its own task, that is, each thread needs to block the C matrix according to its own calculation task, first pack the sub-matrices of A and B, and then perform block matrix calculation operations.
[0005] Secondly, the sub-matrix B data packed by the thread itself needs to be shared with other threads. The threads need to judge whether the required data blocks are ready, and the threads need to share data. Figure 3 As shown, the two-dimensional data structure job[i].working[j] points to the data address addr. If the address is empty, it means that the packed data of thread j required by thread i to complete the computing task is not ready yet and needs to continue waiting. If the addr has a value, it means that the packed data of thread j required by thread i to complete the computing task is ready, and the data is read through the address to complete the computing operation.
[0006] Figure 4This describes the multi-threaded task division and calculation process of a large matrix multiplication. According to the matrix size, 12 threads are allocated, where mypos_m is the number of thread rows and mypos_n is the number of thread columns. The two-dimensional threads are mapped to the task of calculating the C matrix. The upper left block [0,0] of the result matrix C consists of four small blocks. The black box on the left indicates that thread 0 first packs the A_p0 and B_p0 matrices that it is responsible for, and then calculates and fills them, while sharing the packed data blocks. Threads 1, 2, and 3 multiply their own packed B blocks with the packed A matrix block shared by thread 0, and fill them into the other three small blocks of data. The upper left block [1,0] of the result matrix C also consists of four small blocks, one of which is calculated and written by thread 1, and the other three blocks are filled by threads 0, 2, and 3 respectively. The result matrix C[2,0] and C[3,0] blocks are similar. Similarly, the result matrices C[0, 1], C[1, 1], and C[2, 1] blocks are respectively handled by threads 4, 5, 6, and 7; the result matrices C[0, 2], C[1, 2], and C[2, 2] blocks are respectively handled by threads 8, 9, 10, and 11.
[0007] The existing solution uses blocking thread waiting, that is, the current thread must wait for the next adjacent thread to complete data copying before proceeding to the next step. Thread execution and synchronization between threads are carried out by mutually determining whether the data of submatrix B that the current thread and other threads are responsible for has been packaged. A blocking waiting method is adopted, that is, the current thread completes the packaging of matrix block A, and queries the flag to determine whether the data block required by the current thread is ready, and this data block needs to be completed by other adjacent threads. The way threads traverse and wait for the packaged submatrix B is sequential, and the number of traversals and waits is the number of threads with the same value of the thread dimension mypos_n, that is, thread_m. Because threads are affected by a variety of non-fixed factors during the execution process, such as the start and end time of the thread being scheduled, the delay of each thread in executing each step, system load scheduling, etc., there is a problem of long waiting time between threads. For example Figure 5 This paper describes the problem of thread waiting in the current solution. The processor where the thread is located will keep waiting because the data is not ready, which wastes computing resources and reduces CPU utilization.
[0008] Therefore, how to improve the computational efficiency of matrix multiplication is a technical problem that needs to be solved urgently. Summary of the invention
[0009] In order to address the deficiencies in the prior art, the present invention provides a non-blocking multi-threaded matrix multiplication processing method based on state polling, which completes non-blocking thread synchronization and calculation by sequentially traversing other threads through the current thread, thereby solving the problem of wasting computing resources and low computing resource utilization due to thread waiting.
[0010] The embodiments of the present invention provide the following solutions:
[0011] In a first aspect, the embodiments of the present invention provide a non-blocking multi-threaded matrix multiplication processing method based on status polling. The method includes the following steps:
[0012] S1. Initialization:
[0013] Assume that the dimension of matrix A is m*k, and the dimension of matrix B is k*n. Set the block size to be divided for matrix A and matrix B, that is, after division, nthreads_m sub-A matrices and nthreads_n sub-B matrices will be obtained; Initialize a two-dimensional thread for matrix calculation. The row dimension of the two-dimensional thread is nthreads_m, and the column dimension is nthreads_n, that is, the total number of threads nthreads = nthreads_m * nthreads_n; When m < nthreads_m * 4, perform a halving operation on nthreads_m until m ≥ nthreads_m * 4; Set the counter cnt not equal to nthreads_n;
[0014] Establish a two-dimensional index [mypos_m, mypos_n] for defining the position of each thread in the two-dimensional thread and assign a thread ID to each thread. Among them, mypos_m is the row index, mypos_n is the column index, and the thread value mypos = mypos_m * nthreas_m * mypos_n; Each thread is responsible for the division and packaging of matrix A and matrix B according to its own thread ID. The spaces for storing the copied and packaged matrix A and matrix B during the division and packaging process are the first space packedA and the second space packedB respectively. Among them, the second space packedB is divided into more than 2 second sub-spaces packed_subB;
[0015] Create a task list jobs with a size of nthreads and a matrix information list queues; Among them, the task list jobs adopts an array data structure type, and each data element is used to record the address where the divided data of the current thread and other threads are saved; The matrix information list queues adopts a list data structure, and each data element is used to record the starting position, size, data transfer address of the matrix block responsible for the current thread, and the associated next matrix information;
[0016] S2. Set the packing ordinal number j. When 0 <= j and j < times, determine that there is no other thread using the current thread current in the second space packedB, and the j-th second subspace packed_subB[j] in the second space packedB is not empty, then pack the j-th second subspace, where times is the total number of times of preset execution of packing the second subspace; When the first space packedA and the j-th second subspace packed_subB[j] responsible for the current thread are ready, start to execute the computing kernel operation. The computing kernel is the minimum function body that uses the CPU to complete the self-matrix multiplication operation for the matrix data in the packed first space packedA and the second space packedB after packing.
[0017] S3. Determine whether the operation of packing the second space packedB responsible for the current thread is completed. If not, return to step S2; otherwise, enter step S4.
[0018] S4. Assign the address of packedB to the variable job[current].working[other_thread_id] for communication between the current thread and other threads, and inform other threads that they can access the second space packedB packed by this thread. The variable job[current].working[other_thread_id] for communication between the current thread and other threads is an element in the task list jobs, and a thread synchronization data structure is adopted. Each data element in the task list jobs records the address of the block data completed by the current thread current and other threads other_thread_id with the same column dimension nthreads_n.
[0019] S5. Execute the computing kernel operation on the matrix data in the already ready first space packedA and the second space packedB.
[0020] S6. Through the task list jobs, determine whether the second space packedB responsible for the next thread with the same thread dimension nthread_n is valid. If it is valid, enter step S7 for processing; otherwise, perform an increment operation on the current variable current to traverse the packing status of the block matrix data of other threads, that is, determine whether the packed B matrix data responsible for other threads is available to itself, and repeat this step until the second space packedB is ready and available.
[0021] S7. Execute the computing kernel on the first space packedA that is responsible for packing and is already ready by itself, and the second space packedB of the data packed and ready by other threads.
[0022] S8. Determine whether cnt is equal to thread_m. If so, the process ends. Otherwise, perform accumulation operation on cnt and return to step S6.
[0023] In an optional embodiment, the two-dimensional thread described in step S1 is determined by a preset number of threads and a matrix size, wherein the row dimension nthreads_m is determined by comparing the size of the input matrix dimension m and a multiple of the number of threads, and if m is less than 8, nthreads_m=1.
[0024] In an optional embodiment, the nthreads_n thread dimension is determined by comparing the size of m with the multiple of the number of threads. If m is less than 8, nthreads_n=1.
[0025] The beneficial effects of the present invention based on its technical solution are:
[0026] The present invention provides a non-blocking multi-thread matrix multiplication processing method based on state polling. Through polling traversal, any thread with the same index mypos_n as thread_m can be non-blockingly accessed. As long as the sub-matrix that the thread is responsible for has been packed and is ready, the corresponding calculation operation can be pointed to. The invention point of the present invention is to traverse other threads in sequence through the current thread to complete non-blocking thread synchronization and calculation, solve the problem that the original solution wastes computing resources due to thread waiting, and solve the problem that the computing resource utilization rate is not high. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0028] Figure 1 A flow chart of a non-blocking multi-threaded matrix multiplication processing method based on state polling provided by the present invention.
[0029] Figure 2 Schematic diagram of irregular matrix multiplication.
[0030] Figure 3 Complete the thread synchronization diagram for a shared array.
[0031] Figure 4 Schematic diagram of matrix operation multi-threaded task division, calculation and synchronization process.
[0032] Figure 5It is a schematic diagram of the blocking access process to the packed sub-matrix B during sequential traversal between threads in the prior art.
[0033] Figure 6 It is the technical principle of a non-blocking multi-threaded matrix multiplication processing method based on status polling provided by the present invention.
[0034] Figure 7 It is a schematic diagram of thread comparison.
[0035] Figure 8 It is a schematic diagram of the average acceleration ratio of the function on server A.
[0036] Fig. 9 It is a schematic diagram of the average acceleration ratio of the function on server B. Specific embodiments
[0037] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art belong to the scope protected by the embodiments of the present invention.
[0038] The technical principle of the present invention is as Figure 6 shown
[0039] Refer to Figure 1 , the embodiments of the present invention provide a non-blocking multi-threaded matrix multiplication processing method based on status polling, and the method includes the following steps:
[0040] S1. Initialization:
[0041] Set the dimension of matrix A as m*k, the dimension of matrix B as k*n, set the block size to be divided for matrix A and matrix B, that is, after division, nthreads_m sub-A matrices and nthreads_n sub-B matrices will be obtained; initialize the two-dimensional threads for matrix calculation, the row dimension of the two-dimensional threads is nthreads_m, and the column dimension is nthreads_n, that is, the total number of threads of the two-dimensional threads nthreads = nthreads_m * nthreads_n; when m < nthreads_m * 4, perform the operation of halving nthreads_m until m ≥ nthreads_m * 4; set the counter cnt not equal to nthreads_n.
[0042] Among them, the two-dimensional thread is determined by a preset number of threads and matrix size. Among them, the row dimension nthreads_m is determined by comparing the size of the input matrix dimension m with the multiple size of the number of threads. If m is less than 8, then nthreads_m = 1. The nthreads_n thread dimension is determined by comparing the size of m with the multiple size of the number of threads. If m is less than 8, then nthreads_n = 1.
[0043] Establish a two-dimensional index [mypos_m, mypos_n] for defining the position of each thread in the two-dimensional thread and assign a thread ID to each thread. Among them, mypos_m is the row index, mypos_n is the column index, and the thread value mypos = mypos_m * nthreas_m * mypos_n; each thread is responsible for the block division and packing of matrix A and matrix B according to its own thread ID. The spaces for storing the copied and packed matrix A and matrix B during the block division and packing process are the first space packedA and the second space packedB respectively. Among them, the second space packedB is divided into more than 2 second sub-spaces packed_subB;
[0044] Create global variables, including a task list jobs with a size of nthreads and a matrix information list queues; among them, the task list jobs adopts an array data structure type, and each data element is used to record the address where the current thread saves the block data with other threads; the matrix information list queues adopts a list data structure, and each data element is used to record the starting position, size, data transfer address of the matrix block responsible for the current thread, and the associated next matrix information;
[0045] S2. Set the packing ordinal number j. When 0 <= j and j < times, determine that the current thread current is not being used by other threads in the second space packedB, and the j-th second sub-space packed_subB[j] in the second space packedB is not empty, then pack the j-th second sub-space, where times is the total number of times of preset execution of packing the second sub-space; when the first space packedA and the j-th second sub-space packed_subB[j] responsible for the current thread are ready, start executing the computing kernel operation. The computing kernel is the smallest function body that uses the CPU to complete the self-matrix multiplication operation on the matrix data in the packed first space packedA and the second space packedB;
[0046] S3. Determine whether the operation of packing the second space packedB responsible for the current thread is completed. If not, return to step S2, otherwise enter step S4;
[0047] S4. The variable job[current].working[other_thread_id] is notified to the current thread and other threads to assign the packedB address, and the other threads are notified that they can access the second space packedB packed by the current thread, wherein the variable job[current].working[other_thread_id] notified by the current thread and other threads is an element in the task list jobs, and a thread synchronization data structure is adopted. Each data element in the task list jobs records the address of the block data completed by the current thread current and other threads other_thread_id of the same column dimension nthreads_n;
[0048] S5, performing a calculation kernel operation on the matrix data in the ready first space packedA and second space packedB;
[0049] S6. Through the task list jobs, determine whether the second space packedB of the next thread with the same thread dimension nthread_n is valid. If it is valid, go to step S7 for processing; otherwise, perform an increment operation on the current variable current to traverse the packing status of the block matrix data of other threads, that is, determine whether the packed B matrix data of other threads is available to itself, and repeat this step until the second space packedB is ready and available;
[0050] S7, execute the calculation kernel on the first space packedA that is responsible for packing and is ready, and the second space packedB that is responsible for packing the ready data of other threads;
[0051] S8. Determine whether cnt is equal to thread_m. If so, the process ends. Otherwise, perform accumulation operation on cnt and return to step S6.
[0052] On multiple servers and various matrix size types, the double-precision complex function zgemm is tested, and the traditional blocking access matrix multiplication is compared with the present invention.
[0053]
[0054] (1) Performance improvement:
[0055] On machine A, the thread waiting part is timed to compare the traditional blocking access matrix multiplication and the invention. The average speedup ratios of the waiting time are 2.78, 1.34 and 1.2 respectively when the three different dimensional data are running on 24 cores, 48 cores and 96 cores. The results are as follows: Figure 7 shown.
[0056] (2) Average function speedup
[0057] Compared with traditional multiplication processing, on machine A, we measured three different dimensional data running on 24 cores, 48 cores and 96 cores. The average of 50 measurements was 1.191, 1.072 and 1.14 respectively, and the overall average speedup was 1.121. The results are as follows: Figure 8 shown.
[0058] Machine B measured the operation of three different dimensional data on 16 cores, 32 cores, and 64 cores, and measured 50 times to get the average. The average speedup ratios of 16 cores, 32 cores, and 48 cores were 1.17, 1.25, and 1.17 respectively, and the overall average speedup ratio was 1.19. The results are as follows: Fig. 9 shown.
[0059] It can be concluded from the above comparative experiments that the non-blocking multi-threaded matrix multiplication processing method based on state polling provided by the present invention has significantly improved performance compared with the traditional matrix multiplication algorithm.
[0060] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0061] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (modules, systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0062] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0063] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0064] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0065] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A non-blocking multi-threaded matrix multiplication processing method based on state polling, characterized in that: The method includes the following steps: S1. Initialization: Suppose the dimension of matrix A is m*k, and the dimension of matrix B is k*n. Set the block sizes to be divided for matrix A and matrix B, that is, after division, nthreads_m sub-A matrices and nthreads_n sub-B matrices will be obtained; Initialize the two-dimensional threads for matrix calculation. The row dimension of the two-dimensional thread is nthreads_m, and the column dimension is nthreads_n, that is, the total number of threads of the two-dimensional thread nthreads = nthreads_m * nthreads_n; When m < nthreads_m * 4, perform the operation of halving nthreads_m until m ≥ nthreads_m * 4; Set the counter cnt not equal to nthreads_n; Establish a two-dimensional index [mypos_m, mypos_n] for defining the position of each thread in the two-dimensional thread and assign a thread ID to each thread. Among them, mypos_m is the row index, mypos_n is the column index, and the thread value mypos = mypos_m * nthreas_m * mypos_n; Each thread is responsible for the block division and packing of matrix A and matrix B according to its own thread ID. The spaces for storing the copied and packed matrix A and matrix B during the block division and packing are the first space packedA and the second space packedB respectively. Among them, the second space packedB is divided into more than 2 second sub-spaces packed_subB; Create a task list jobs with a size of nthreads and a matrix information list queues; Among them, the task list jobs adopts an array data structure type, and each data element is used to record the address where the block data of the current thread and other threads are saved; The matrix information list queues adopts a list data structure, and each data element is used to record the starting position, size, data transfer address of the matrix block responsible for the current thread, and the associated next matrix information; S2. Set the packing ordinal number j. When 0 <= j and j < times, determine that the current thread current is not used by other threads in the second space packedB, and the j-th second sub-space packed_subB[j] in the second space packedB is not empty, then pack the j-th second sub-space, where times is the total number of times of preset execution of packing the second sub-space; When the first space packedA and the j-th second sub-space packed_subB[j] responsible for the current thread are ready, start to execute the calculation kernel operation. The calculation kernel is the smallest function body that uses the CPU to complete the self-matrix multiplication operation for the matrix data in the packed first space packedA and the second space packedB; S3. Determine whether the operation of packing the second space packedB responsible for the current thread is completed. If not, return to step S2, otherwise enter step S4; S4. The variable job[current].working[other_thread_id] is notified to the current thread and other threads to assign the packedB address, and the other threads are notified that they can access the second space packedB packed by the current thread, wherein the variable job[current].working[other_thread_id] notified by the current thread and other threads is an element in the task list jobs, and a thread synchronization data structure is adopted. Each data element in the task list jobs records the address of the block data completed by the current thread current and other threads other_thread_id of the same column dimension nthreads_n; S5, performing a calculation kernel operation on the matrix data in the ready first space packedA and second space packedB; S6. Through the task list jobs, determine whether the second space packedB of the next thread with the same thread dimension nthread_n is valid. If it is valid, go to step S7 for processing; otherwise, perform an increment operation on the current variable current to traverse the packing status of the block matrix data of other threads, that is, determine whether the packed B matrix data of other threads is available to itself, and repeat this step until the second space packedB is ready and available; S7, execute the calculation kernel on the first space packedA that is responsible for packing and is ready, and the second space packedB that is responsible for packing the ready data of other threads; S8. Determine whether cnt is equal to thread_m. If so, the process ends. Otherwise, perform accumulation operation on cnt and return to step S6.
2. The non-blocking multi-threaded matrix multiplication processing method based on state polling according to claim 1, characterized in that: The two-dimensional threads described in step S1 are determined by a preset number of threads and a matrix size, wherein the row dimension nthreads_m is determined by comparing the size of the input matrix dimension m with a multiple of the number of threads. If m is less than 8, nthreads_m=1.
3. The non-blocking multi-threaded matrix multiplication processing method based on state polling according to claim 1, characterized in that: The nthreads_n thread dimension is determined by comparing the size of m with the multiple of the number of threads. If m is less than 8, nthreads_n=1.
