Multi-channel matrix data processing method, device and medium based on GPGPU

Abstract

The embodiment of the present application discloses a multi-channel matrix data processing method, device and medium based on GPGPU, which belongs to the field of data processing technology. It solves the problem that the data transmission takes a long time and affects the operation speed when the traditional CPU architecture performs matrix data processing. The matrix data to be processed is cached to the global memory in row storage mode; based on the computing resources allocated by GPGPU, the active thread bundle and the data processing channel are determined, and the active thread bundle is matched with the data processing channel; through the matched data processing channel, the matrix data of the corresponding row in the global memory is cached to the matrix buffer corresponding to each data processing channel; based on the status of each matrix calculation unit, the matrix data in the matrix buffer is allocated to the corresponding calculation unit for matrix data processing of the corresponding row; the matrix data processing results of each calculation unit are stored in the global memory until the matrix operation is completed and GPGPU releases the computing resources.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a GPGPU-based multi-channel matrix data processing method, device and medium. Background Art

[0002] With the booming development of big data and artificial intelligence, matrix operations have become the core foundation of many key applications. For example, in the field of machine learning, neural network training requires complex matrix multiplication and addition operations on massive data to optimize model parameters; in image processing, image transformation, filtering and other operations also rely on efficient matrix operations; in physical simulation, matrix operations are also needed to simulate physical phenomena such as the movement and interaction of objects.

[0003] Although the traditional CPU architecture performs well in general computing, when faced with large-scale matrix operations, due to its serial processing characteristics, it usually uses a single thread to traverse the matrix elements in order and operate on the matrix elements one by one. For example, when performing matrix multiplication, it is necessary to first read a row of elements in the first matrix, and then perform multiplication operations with each column of the second matrix in turn and accumulate the results. In this process, data reading and operation operations are strictly performed in sequence, which is extremely inefficient, resulting in a long time for data transmission, which seriously affects the overall operation speed. Summary of the invention

[0004] The embodiments of the present application provide a GPGPU-based multi-channel matrix data processing method, device and medium for solving the following technical problems: the traditional CPU architecture is extremely inefficient when processing matrix data, resulting in a long data transmission time, which seriously affects the overall computing speed.

[0005] The present application embodiment adopts the following technical solutions:

[0006] The embodiment of the present application provides a multi-channel matrix data processing method based on GPGPU. It includes: responding to matrix processing task requests, caching the matrix data to be processed in the global memory in row storage mode; determining the active thread bundle and the data processing channel based on the computing resources allocated by GPGPU, and matching the active thread bundle with the data processing channel; caching the matrix data of the corresponding row in the global memory to the matrix buffer corresponding to each data processing channel through the matched data processing channel; obtaining the state of each matrix computing unit, and based on the state of each matrix computing unit, allocating the matrix data in the matrix buffer to the corresponding computing unit for matrix data processing of the corresponding row; storing the matrix data processing results of each computing unit in the global memory through the matched data processing channel until the matrix operation is completed and GPGPU releases computing resources.

[0007] The embodiment of the present application caches the matrix data to be processed into the global memory in a row storage mode to improve the continuity and locality of the data. Since the adjacent addresses in the memory store the elements of the same row of the matrix, the random seek time of memory access can be reduced and the data reading efficiency can be improved. Through the matched data processing channel, the matrix data of the corresponding row in the global memory is cached in the matrix buffer corresponding to each data processing channel, so as to realize the local storage of data, reduce the number of accesses of the computing unit to the global memory, reduce the data transmission delay, and improve the access speed of data. At the same time, different data processing channels correspond to their own buffers, avoid data conflicts, and improve the parallelism of data processing. By acquiring the state of the matrix computing unit in real time, data can be allocated to idle or low-load computing units according to their load conditions and processing capabilities, avoiding idleness or overload of computing units, improving the utilization rate of computing units, and further optimizing the parallel processing capability of the entire system.

[0008] In one implementation of the present application, matrix data of corresponding rows in the global memory are cached into matrix buffers corresponding to each data processing channel through matched data processing channels, specifically including: determining the matrix rows to be processed corresponding to the data processing channel, and loading the matrix data of corresponding rows in the global memory into the shared memory based on the matrix rows to be processed; executing the Load instruction to cache the matrix data in the shared memory into the logic block corresponding to the register file; wherein the register file is set as a multi-logic block parallel storage mechanism; and caching the matrix data in the logic block into the matrix buffer corresponding to the data processing channel through the operand collection unit corresponding to the data processing channel.

[0009] In one implementation of the present application, before caching the matrix data in the logic block into the matrix buffer corresponding to the data processing channel, the method also includes: reading the matrix operation instruction block in the instruction storage area corresponding to the global memory through the matched data processing channel; based on the read matrix operation instruction block, simultaneously reading the matrix data in different logic blocks corresponding to the register file.

[0010] In one implementation of the present application, based on the state of each matrix calculation unit, the matrix data in the matrix buffer is allocated to the corresponding calculation unit to perform matrix data processing of the corresponding row, specifically including: determining the matrix calculation unit in the idle state; allocating the matrix data in the matrix buffer to the matrix calculation unit in the idle state; converting the received matrix data into matrix blocks through the matrix calculation unit; performing matrix data processing on each matrix block respectively to obtain a result matrix block, and converting each result matrix block into row data of the corresponding column width.

[0011] In one implementation of the present application, matrix data processing is performed on each matrix block to obtain a result matrix block, specifically including: allocating the matrix block to a matrix calculation sub-unit corresponding to a matrix calculation unit; wherein the matrix calculation unit is composed of a plurality of matrix calculation sub-units and a plurality of bias addition units; the matrix calculation sub-unit is composed of a plurality of multiplication units and a plurality of addition units; after multiplying the matrix block by the multiplication unit, the processing result is sent to the addition unit for addition processing; the result after the addition processing is sent to the bias addition unit for processing to obtain a result matrix block.

[0012] In one implementation of the present application, the matrix data processing results of each computing unit are stored in the global memory through the matched data processing channel until the matrix operation is completed, specifically including: writing the matrix data processing results of the computing unit back to the logic block of the register file; executing the Store instruction through the active thread corresponding to the matched data processing channel to store the matrix data processing results in the logic block to the global memory; when it is determined that all the operation results corresponding to the matrix data to be processed are stored in the global memory, the GPGPU generates a done signal; after the done signal is detected on the host side, the operation results in the global memory are read and cached to the host memory.

[0013] In one implementation of the present application, active thread bundles are determined, specifically including: determining a task priority based on a matrix processing task request; determining a thread load based on the amount of matrix operations completed by each thread bundle per unit time, the time the thread bundle waits for resources, and the amount of data being processed by each thread bundle; screening out thread bundles whose thread loads are not greater than a preset load balancing threshold in a pool of idle thread bundles as a reference thread bundle subset; determining data locality characteristics based on the data block location and access mode required for the matrix operation task; querying each thread bundle in the reference thread bundle subset based on a preset data locality index table to determine the data block location and range currently cached by each thread bundle; determining the degree of match between each thread bundle and the data locality characteristics based on the data block location and range, so as to screen out a thread bundle list in the reference thread bundle subset based on the degree of match; and determining active thread bundles based on the task priority and the thread bundle list.

[0014] In one implementation of the present application, after determining the active thread bundle and the data processing channel, the method also includes: when it is detected that the active thread bundle is blocked, marking the matrix processing task executed by the active thread bundle as a task to be reallocated; taking the tasks corresponding to different thread bundles as nodes to construct a task association graph; performing node feature extraction and edge feature extraction on the task association graph; inputting the node features and edge features into a preset graph neural network to output the association relationship between different tasks; based on the association relationship and the priorities corresponding to different tasks, re-prioritizing the tasks to be reallocated; if the priority after reordering is an urgent task, stopping a task with the lowest priority and reallocating the released thread resources to the task to be reallocated; if the priority after reordering is not an urgent task, performing computing resource prediction on the task to be reallocated and reselecting the thread bundle for the task.

[0015] An embodiment of the present application provides a GPGPU-based multi-channel matrix data processing device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can: respond to a matrix processing task request, and cache the matrix data to be processed in a global memory in a row storage mode; determine active thread bundles and data processing channels based on the computing resources allocated by the GPGPU, and match the active thread bundles with the data processing channels; cache the matrix data of the corresponding row in the global memory into the matrix buffers corresponding to each data processing channel through the matched data processing channels; obtain the status of each matrix computing unit, and based on the status of each matrix computing unit, allocate the matrix data in the matrix buffer to the corresponding computing unit for matrix data processing of the corresponding row; store the matrix data processing results of each computing unit in the global memory through the matched data processing channels until the matrix operation is completed and the GPGPU releases the computing resources.

[0016] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, wherein the computer executable instructions are configured to: respond to a matrix processing task request, cache matrix data to be processed in a global memory in a row storage mode; determine active thread bundles and data processing channels based on computing resources allocated by a GPGPU, and match the active thread bundles with the data processing channels; cache matrix data of corresponding rows in a global memory into matrix buffers corresponding to each data processing channel through the matched data processing channels; obtain the status of each matrix computing unit, and based on the status of each matrix computing unit, allocate the matrix data in the matrix buffer to the corresponding computing unit to process the matrix data of the corresponding row; store the matrix data processing results of each computing unit in the global memory through the matched data processing channels until the matrix operation is completed and the GPGPU releases computing resources.

[0017] At least one of the above technical solutions adopted in the embodiment of the present application can achieve the following beneficial effects: the embodiment of the present application caches the matrix data to be processed into the global memory in a row storage mode to improve the continuity and locality of the data. Since the adjacent addresses in the memory store the elements of the same row of the matrix, the random seek time of memory access can be reduced and the data reading efficiency can be improved. Through the matched data processing channel, the matrix data of the corresponding row in the global memory is cached in the matrix buffer corresponding to each data processing channel, so as to realize the local storage of data, reduce the number of accesses of the computing unit to the global memory, reduce the data transmission delay, and improve the access speed of data. At the same time, different data processing channels correspond to their own buffers, avoid data conflicts, and improve the parallelism of data processing. By acquiring the state of the matrix computing unit in real time, data can be allocated to idle or low-load computing units according to their load conditions and processing capabilities, avoiding idleness or overload of computing units, improving the utilization rate of computing units, and further optimizing the parallel processing capability of the entire system. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0019] Figure 1 A flow chart of a multi-channel matrix data processing method based on GPGPU provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of a matrix data row storage format provided in an embodiment of the present application;

[0021] Figure 3 A schematic diagram of a computing core structure provided in an embodiment of the present application;

[0022] Figure 4 A schematic diagram of a single-channel matrix operation flow provided in an embodiment of the present application;

[0023] Figure 5 A schematic diagram of a matrix computing subunit architecture and data conversion provided in an embodiment of the present application;

[0024] Figure 6 A schematic diagram of the structure of a GPGPU-based multi-channel matrix data processing device provided in an embodiment of the present application.

[0025] Reference numerals:

[0026] 200 is a multi-channel matrix data processing device based on GPGPU, 201 is a processor, and 202 is a memory. DETAILED DESCRIPTION

[0027] The embodiments of the present application provide a GPGPU-based multi-channel matrix data processing method, device, and medium.

[0028] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.

[0029] Figure 1 A flow chart of a multi-channel matrix data processing method based on GPGPU is provided in an embodiment of the present application, such as Figure 1 As shown, the multi-channel matrix data processing method based on GPGPU includes the following steps:

[0030] Step 101 : respond to a matrix processing task request and cache the matrix data to be processed into a global memory in a row storage mode.

[0031] In one implementation of the present application, the host sends a matrix multiplication operation task, and the data of matrix A, matrix B and offset matrix C are cached from the CPU memory to the data storage area of ​​the global memory HBM in row storage mode. The maximum width of each row of the matrix data supports 256 bits, that is, 8 columns (the default matrix column width is 32 bits). When the maximum row width is exceeded, the row data can be sliced. At the same time, the CPU caches the instruction data of the matrix multiplication operation to the instruction storage area of ​​the global memory HBM.

[0032] It should be noted that GPGPU (General-Purpose Graphics Processing Unit), or general-purpose graphics processor, is a technology that uses a graphics processing unit (GPU) to perform general computing tasks that were originally handled by a central processing unit (CPU).

[0033] Figure 2 A schematic diagram of a matrix data row storage format provided in an embodiment of the present application is shown in FIG. Figure 2 As shown, matrix A is an 8x8 matrix consisting of row A R0 ~A R7 and column A C0 ~A C7 The matrix B is an 8x8 matrix consisting of rows B R0 ~B R7 and column B C0 ~B C7 The bias matrix C is an 8x8 matrix consisting of rows C R0 ~C R7 and column C C0 ~C C7 In order, store each row of matrix A, each row of matrix B, and each row of matrix C into global memory.

[0034] Step 102: Based on the computing resources allocated by the GPGPU, active thread warps and data processing channels are determined, and the active thread warps and the data processing channels are matched.

[0035] In one implementation of the present application, a task priority is determined based on a matrix processing task request. A thread load is determined based on the amount of matrix operations completed by each thread bundle per unit time, the time the thread bundle waits for resources, and the amount of data being processed by each thread bundle. Thread bundles whose thread loads are not greater than a preset load balancing threshold are screened out from the idle thread bundle pool as a reference thread bundle subset. Data locality characteristics are determined based on the data block location and access mode required for the matrix operation task. Based on a preset data locality index table, each thread bundle in the reference thread bundle subset is queried to determine the data block location and range currently cached by each thread bundle. Based on the data block location and range, the degree of match between each thread bundle and the data locality feature is determined, so as to screen out a thread bundle list based on the degree of match. Based on the task priority and the thread bundle list, an active thread bundle is determined.

[0036] Specifically, a priority is assigned to each task according to the nature and requirements of different matrix processing tasks. The priority of a task can be determined based on a variety of factors, such as the real-time requirements of the task, the importance of the final result, the computational complexity of the task, the amount of data, etc. The amount of matrix operations completed per unit time can reflect the processing capability of the thread bundle. The more operations completed, the more efficient the thread bundle is in processing matrix operations. However, if the load is high for a long time, the subsequent processing speed may decrease. The time the thread bundle waits for resources, a long waiting time indicates that the thread bundle may be idle or blocked due to resource competition, reflecting the efficiency of resource allocation. The amount of data being processed by the thread bundle reflects the workload of the current thread bundle. If the amount of data being processed is large, then the thread bundle may be in a high load state.

[0037] For example, Warp 1 has completed operations on 100 matrix elements in the past minute, waiting for resources for 2 seconds, and processing 1000 elements; Warp 2 has completed operations on 50 matrix elements in the past minute, waiting for resources for 5 seconds, and processing 500 elements. By combining these indicators, it can be determined that Warp 1 may be busier than Warp 2 and has a higher load.

[0038] Furthermore, in order to avoid allocating new tasks to thread bundles that are already highly loaded and affect system performance, a load balancing threshold is set. Only when the load of a thread bundle is lower than the threshold will it be selected into the reference thread bundle subset as a candidate for subsequent task allocation. The location of the data blocks in the memory that the matrix operation task needs to access and the access mode are analyzed to determine the locality of the data. The data locality in the embodiment of the present application includes temporal locality, such as data that has been recently accessed may be accessed again, and also includes spatial locality, such as when accessing a certain data, the data nearby may be accessed.

[0039] Furthermore, the embodiment of the present application is provided with a preset data locality index, and the index table stores the data block information currently cached by each thread warp. By querying the table, it is possible to understand which thread warps have cached data related to the new task.

[0040] Furthermore, the matching degree is determined by comparing the data block location and range required by the task with the data block location and range cached by the thread warp. The matching degree can be measured by calculating the size of the overlapping area, the degree of proximity of the data blocks, etc. By filtering out a list of thread warps with high matching degrees, the cached data already in these thread warps can be used to reduce data transmission and loading time. Considering the priority of the task, for high-priority tasks, thread warps are preferentially selected from the filtered thread warp list.

[0041] In one implementation of the present application, taking 4x4 as the minimum computing granularity as an example, the memory access mode is based on row data priority storage, and through a multi-level cache strategy, the distribution of data in different levels of cache is optimized, which significantly improves the efficiency and performance of matrix multiplication operations. GPGPU allocates computing resources to it according to matrix multiplication operation tasks.

[0042] Specifically, assume that this matrix multiplication operation is assigned to a computing core CE1, which has four pipeline channels VC0~VC3, supporting 4 warps to be activated and run simultaneously, and each warp contains 8 threads. Assume that there are currently active warps W0~W3, warp W1 is assigned to the VC1 channel, warp W2 is assigned to the VC2 channel, warp W3 is assigned to the VC3 channel, and warp W4 is assigned to the VC4 channel. In this way, the active warps are matched with the data processing channels.

[0043] In one implementation of the present application, when it is detected that an active thread bundle is blocked, the matrix processing task executed by the active thread bundle is marked as a task to be reallocated. The tasks corresponding to different thread bundles are used as nodes to construct a task association graph. Node feature extraction and edge feature extraction are performed on the task association graph. The node features and edge features are input into a preset graph neural network to output the association relationship between different tasks. Based on the association relationship and the priorities corresponding to different tasks, the tasks to be reallocated are re-prioritized. If the priority after reordering is an urgent task, a task with the lowest priority is stopped, and the released thread resources are reallocated to the task to be reallocated. If the priority after reordering is not an urgent task, computing resources are predicted for the task to be reallocated, and the thread bundle is reselected for the task.

[0044] Specifically, when a blocking of an active thread bundle is detected, in order to ensure the continued execution of the task, the matrix processing task being executed by the thread bundle needs to be specially processed. It is marked as a task to be reallocated so that resources can be reallocated and scheduled later. The task association graph is a data structure used to represent the dependencies and connections between different tasks. The tasks responsible for each thread bundle are regarded as nodes in the graph, and the relationships between tasks are regarded as edges in the graph. For each node, relevant features are extracted. These features may include the type of task, such as matrix multiplication, matrix addition, etc., the complexity of the task, such as the size of the matrix, the data type of the elements, and the amount of resources required for the task, such as memory, computing resources, etc. For the edges in the graph, the features of the edges are extracted, such as the amount of data transmission between tasks, the frequency of data transmission, and the time constraints between tasks, such as how long after task A is completed, task B must start.

[0045] Furthermore, the extracted node features and edge features are used as inputs to a preset graph neural network to output the associations between different tasks. A graph neural network is a neural network specifically used to process graph structured data. It uses the extracted node features and edge features as inputs to learn the complex associations between tasks through a series of graph convolutional layers, pooling layers, etc.

[0046] Furthermore, tasks marked as to be reallocated are re-evaluated and sorted according to the association between tasks and their original priorities. If a task is strongly associated with multiple critical tasks, its priority may be increased because its completion or failure will affect the execution of multiple subsequent tasks. When the reordered task to be reallocated has a higher priority (regarded as an urgent task), a resource preemption strategy is adopted to complete the task as soon as possible. By stopping a task with the lowest priority, the thread resources occupied by it are released, and these resources are reallocated to the urgent task to be allocated to ensure the smooth execution of the urgent task. When the priority of the task to be reallocated is not high, the appropriate thread bundle is reselected according to the characteristics of the task and the current system resource status through computing resource prediction. Computing resource prediction can select the most suitable thread bundle for the task based on factors such as the task's computing amount, data volume, and requirements for data locality, avoiding resource waste and performance degradation.

[0047] Step 103 : Cache the matrix data of the corresponding row in the global memory into the matrix buffers corresponding to each data processing channel through the matched data processing channel.

[0048] In one implementation of the present application, a to-be-processed matrix row corresponding to a data processing channel is determined, and matrix data of a corresponding row in a global memory is loaded into a shared memory based on the to-be-processed matrix row. A Load instruction is executed to cache the matrix data in the shared memory into a logic block corresponding to a register file; wherein the register file is set as a multi-logic block parallel storage mechanism. The matrix data in the logic block is cached into a matrix buffer corresponding to the data processing channel through an operand collection unit corresponding to the data processing channel.

[0049] Specifically, assume that this matrix multiplication operation is assigned to a computing core CE1, which has four pipeline channels V C0 ~V C3 , supports 4 thread warps to be activated and run at the same time, each thread warp contains 8 threads. Assume that there are currently active thread warps W0~W3, and the matrix A is an 8x8 matrix with row A R0 ~A R7 and column A C0 ~A C7 The matrix B is an 8x8 matrix consisting of rows B R0 ~BR7 and column B C0 ~B C7 The bias matrix C is an 8x8 matrix consisting of rows C R0 ~C R7 and column C C0 ~C C7 Thread warp W0 is assigned to the VC0 channel and reads the matrix multiplication instruction block from the instruction storage area. After being parsed by the decoding unit, the matrix A R0 , B R0 and C R0 It is loaded from global memory to shared memory, and then the execution unit executes the Load instruction to cache it into the logic block LR corresponding to the register file W0. Then, the operand collection unit in the channel caches the matrix data in the logic block into the matrix buffer corresponding to the data processing channel.

[0050] Furthermore, the thread warps W1 to W3 are respectively assigned to VC1 to VC3, and the above operations are repeated to assign row A of matrix A to R2 ~A R7 , the row B of matrix B R2 ~B R7 and row C of matrix C R2 ~C R7 Cache into the corresponding data buffer.

[0051] In one implementation of the present application, a matrix operation instruction block is read from an instruction storage area corresponding to the global memory through a matched data processing channel. Based on the read matrix operation instruction block, matrix data is simultaneously read from different logic blocks corresponding to the register file.

[0052] Specifically, after the data is cached in the logical area LR of the register file, the matrix operation instruction is executed to read operands OP1, OP2 and OP3 from different logical areas of the register file at the same time. The operands represent the row data in the A matrix, the B matrix and the offset matrix C respectively, and are forwarded to the data buffer FIFO dedicated to matrix calculation in the channel for buffering and waiting for scheduling by the resource management unit. That is, through the matched data processing channel, the operand OP1-1 (A R0 ),OP2-1(B R0 ) and OP3-1(C R0 ), and then cache the operand into the data buffer FIFO dedicated to the matrix calculation of the subsequent stage of the channel, and then W0 repeats the above operation to obtain the operand OP1-2 (A R1 ),OP2-2(B R1 ) and OP3-2(C R1 ) is cached again into the corresponding data buffer.

[0053] The matrix calculation unit PE in the embodiment of the present application adopts 4x4 as the minimum calculation granularity. The complete 4x4 matrix data can be obtained by executing the operand read instruction twice through the thread bundle W. Then, the multiplication and addition operation is completed by the matrix calculation unit PE and the operation result is written back to the register file logic area.

[0054] Figure 3 A schematic diagram of a computing core structure provided in an embodiment of the present application is shown in FIG. Figure 3 As shown, the data to be processed is loaded from the global memory to the shared memory LM, and the Load instruction is executed to cache the matrix data in the shared memory into the logic block corresponding to the register file. Through different pipeline channels VC0~VC3, the matrix multiplication operation instruction block is read from the instruction storage area, and the matrix data in the logic block is cached into the resource management unit corresponding to the data processing channel, that is, the matrix cache, through the operand collection unit corresponding to the data processing channel. Determine the matrix calculation unit in the idle state, and allocate the matrix data in the matrix cache to the matrix calculation unit in the idle state. As shown Figure 2 As shown, the matrix calculation unit may include a matrix calculation unit PE1, a matrix calculation unit PE2, a matrix calculation unit PE3 and a matrix calculation unit PE4.

[0055] Figure 4 A schematic diagram of a single-channel matrix operation flow provided in an embodiment of the present application is shown in FIG. Figure 4 As shown in the figure, taking a single-channel matrix operation as an example, the matrix data is loaded from the global memory HBM to the shared memory LM of the GPGPU, and then the execution unit executes the Load instruction to cache it in the logic block LR corresponding to the register file. The matrix multiplication operation instruction block is read from the instruction storage area, the operand is obtained through the operand collection unit in the channel, and then the operand is cached in the data cache FIFO dedicated to the matrix calculation of the subsequent stage of the channel. The resource management unit will allocate the matrix data in different channel FIFOs to the idle calculation units for matrix multiplication operations according to the status of the matrix calculation units.

[0056] Step 104 : Acquire the state of each matrix calculation unit, and based on the state of each matrix calculation unit, distribute the matrix data in the matrix buffer to the corresponding calculation unit to process the matrix data of the corresponding row.

[0057] In one implementation of the present application, a matrix calculation unit in an idle state is determined. Matrix data in a matrix buffer is allocated to a matrix calculation unit in an idle state. The received matrix data is converted into a matrix block by the matrix calculation unit. Matrix data processing is performed on each matrix block to obtain a result matrix block, and each result matrix block is converted into row data of a corresponding column width. Wherein, matrix data processing is performed on each matrix block, and the process is: the matrix block is allocated to a matrix calculation subunit corresponding to the matrix calculation unit. Wherein, the matrix calculation unit is composed of a plurality of matrix calculation subunits and a plurality of bias addition units; the matrix calculation subunit is composed of a plurality of multiplication units and a plurality of addition units. After the matrix block is multiplied by the multiplication unit, the processing result is sent to the addition unit for addition processing. The result after the addition processing is sent to the bias addition unit for processing to obtain a result matrix block.

[0058] Specifically, after the matrix data is cached into the data buffer FIFO, the resource management unit will allocate the matrix data in different channel FIFOs to the idle computing units for matrix multiplication according to the status of the matrix computing unit. Assume that the matrix computing unit PE1 receives the first two rows of data of each matrix, totaling 512 bits, {AR1, AR0}, {BR1, BR0} and {CR1, CR0} respectively. PE1 converts the row data into a 4x4 matrix block. The matrix computing unit integrates 16 matrix computing sub-units and 16 bias addition units. Each matrix computing sub-unit consists of 4 multiplication units and 3 addition units. After the matrix blocks are allocated to these matrix computing sub-units, a 4x4 result matrix block is output, and then the data is converted into 512-bit row data {R1, R0}.

[0059] Figure 5 A matrix computing subunit architecture and data conversion diagram provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the matrix calculation subunit is composed of 4 multiplication units and 3 addition units. After the matrix blocks are multiplied by the multiplication units, the processing results are sent to the addition units for addition processing.

[0060] The result after the addition process is sent to the bias addition unit for processing to obtain a result matrix block, and the matrix block is distributed to these matrix calculation sub-units and then a 4x4 result matrix block is output.

[0061] Step 105: Store the matrix data processing results of each computing unit into the global memory through the matched data processing channel until the matrix operation is completed and the GPGPU releases the computing resources.

[0062] In one implementation of the present application, the matrix data processing results of the computing unit are written back to the logic block of the register file. The active thread corresponding to the matched data processing channel executes the Store instruction to store the matrix data processing results in the logic block to the global memory. When it is determined that all the calculation results corresponding to the matrix data to be processed are stored in the global memory, the GPGPU generates a done signal. After the done signal is detected on the host side, the calculation results in the global memory are read and cached to the host memory.

[0063] Specifically, when the matrix calculation unit PE1 completes the calculation, it writes the final result {R1, R0} back to the logic block LR of the register file, and executes the Store instruction through W0 to store the calculation result in the data storage area of ​​the global memory HBM. The calculation process of different matrix calculation units is consistent with PE1 to complete the entire 8x8 matrix operation. When all the calculation results are stored in the global memory HBM, GPGPU generates a done signal. After the host detects the done signal, it reads the calculation results in the global memory and caches them in the host memory. At this point, the matrix operation ends and GPGPU releases computing resources.

[0064] Figure 6 The following is a schematic diagram of a multi-channel matrix data processing device based on GPGPU provided in an embodiment of the present application. Figure 6 As shown, a multi-channel matrix data processing device 200 based on GPGPU includes: at least one processor 201; and a memory 202 in communication with the at least one processor 201; wherein the memory 202 stores instructions that can be executed by the at least one processor 201, and the instructions are executed by the at least one processor 201, so that the at least one processor 201 can: respond to a matrix processing task request, cache the matrix data to be processed in the global memory in a row storage mode; determine the active thread bundle and the data processing channel based on the computing resources allocated by the GPGPU, and match the active thread bundle with the data processing channel; cache the matrix data of the corresponding row in the global memory to the matrix buffers corresponding to each data processing channel through the matched data processing channel; obtain the state of each matrix computing unit, and based on the state of each matrix computing unit, allocate the matrix data in the matrix buffer to the corresponding computing unit to process the matrix data of the corresponding row; store the matrix data processing results of each computing unit in the global memory through the matched data processing channel until the matrix operation is completed and the GPGPU releases the computing resources.

[0065] A non-volatile computer storage medium provided in an embodiment of the present application stores computer executable instructions, wherein the computer executable instructions are configured to: respond to a matrix processing task request, cache matrix data to be processed in a global memory in a row storage mode; determine active thread bundles and data processing channels based on computing resources allocated by a GPGPU, and match the active thread bundles with the data processing channels; cache matrix data of corresponding rows in a global memory into matrix buffers corresponding to each data processing channel through the matched data processing channels; obtain the status of each matrix computing unit, and based on the status of each matrix computing unit, allocate the matrix data in the matrix buffer to the corresponding computing unit to process the matrix data of the corresponding row; store the matrix data processing results of each computing unit in the global memory through the matched data processing channels until the matrix operation is completed and the GPGPU releases computing resources.

[0066] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0067] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application.

Claims

1. A multi-channel matrix data processing method based on GPGPU, characterized in that: The method comprises: In response to matrix processing task requests, cache the matrix data to be processed into the global memory in row storage mode; Based on the computing resources allocated by the GPGPU, determining active thread warps and data processing channels, and matching the active thread warps with the data processing channels; Cache the matrix data of the corresponding row in the global memory into the matrix buffers respectively corresponding to the data processing channels through the matched data processing channels; Acquire the state of each matrix calculation unit, and based on the state of each matrix calculation unit, distribute the matrix data in the matrix buffer to the corresponding calculation unit to process the matrix data of the corresponding row; The matrix data processing results of each computing unit are stored in the global memory through the matched data processing channel until the matrix operation is completed and the GPGPU releases the computing resources.

2. The multi-channel matrix data processing method based on GPGPU according to claim 1, characterized in that: The matching data processing channels cache the matrix data of the corresponding rows in the global memory into the matrix buffers respectively corresponding to the data processing channels, specifically including: Determine the to-be-processed matrix row corresponding to the data processing channel, and load the matrix data of the corresponding row in the global memory into the shared memory based on the to-be-processed matrix row; Execute a Load instruction to cache the matrix data in the shared memory into a logic block corresponding to a register file; wherein the register file is set as a multi-logic block parallel storage mechanism; The matrix data in the logic block is cached in the matrix buffer corresponding to the data processing channel through the operand collection unit corresponding to the data processing channel.

3. The multi-channel matrix data processing method based on GPGPU according to claim 2 is characterized in that: Before caching the matrix data in the logic block into the matrix buffer corresponding to the data processing channel, the method further includes: Reading a matrix operation instruction block in an instruction storage area corresponding to the global memory through the matched data processing channel; Based on the read matrix operation instruction block, the matrix data is simultaneously read in different logic blocks corresponding to the register file.

4. The multi-channel matrix data processing method based on GPGPU according to claim 1, characterized in that: The method of allocating the matrix data in the matrix buffer to the corresponding calculation unit to process the matrix data of the corresponding row based on the state of each matrix calculation unit specifically includes: Determine the matrix computing unit that is in an idle state; Allocating the matrix data in the matrix buffer to the matrix calculation unit in an idle state; The received matrix data is converted into a matrix block by the matrix calculation unit; Matrix data processing is performed on each of the matrix blocks to obtain a result matrix block, and each of the result matrix blocks is converted into row data of a corresponding column width.

5. The multi-channel matrix data processing method based on GPGPU according to claim 4 is characterized in that: The matrix data processing is performed on each matrix block to obtain a result matrix block, specifically including: Allocating the matrix block to the matrix calculation subunit corresponding to the matrix calculation unit; wherein the matrix calculation unit is composed of a plurality of matrix calculation subunits and a plurality of bias addition units; the matrix calculation subunit is composed of a plurality of multiplication units and a plurality of addition units; After the multiplication unit performs multiplication processing on the matrix block, the processing result is sent to the addition unit for addition processing; The result after the addition process is sent to the bias addition unit for processing to obtain the result matrix block.

6. The multi-channel matrix data processing method based on GPGPU according to claim 1, characterized in that: The matching data processing channel stores the matrix data processing results of each computing unit in the global memory until the matrix operation is completed, specifically including: Writing the matrix data processing result of the computing unit back to the logic block of the register file; Executing a Store instruction through the active thread corresponding to the matched data processing channel to store the matrix data processing result in the logic block into the global memory; When it is determined that all the operation results corresponding to the matrix data to be processed are stored in the global memory, the GPGPU generates a done signal; After the done signal is detected on the host side, the operation result in the global memory is read and cached in the host memory.

7. The multi-channel matrix data processing method based on GPGPU according to claim 1, characterized in that: The determining of the active thread warp specifically includes: Processing task requests based on the matrix to determine task priorities; Determine the thread load based on the amount of matrix operations completed by each thread warp per unit time, the time the thread warp waits for resources, and the amount of data being processed by each thread warp; Filter out thread bundles whose thread loads are not greater than a preset load balancing threshold from the idle thread bundle pool as reference thread bundle subsets; Determining data locality characteristics based on data block locations and access patterns required by the matrix operation task; Based on a preset data locality index table, query each thread warp in the reference thread warp subset to determine the location and range of the data block currently cached by each thread warp; Based on the data block location and range, determining a matching degree between each thread warp and the data locality feature, so as to filter out a thread warp list from the reference thread warp subset based on the matching degree; The active warp is determined based on the task priority and the warp list.

8. The multi-channel matrix data processing method based on GPGPU according to claim 7, characterized in that: After determining the active thread warps and data processing channels, the method further includes: When it is detected that the active warp is blocked, marking the matrix processing task executed by the active warp as a task to be reallocated; The tasks corresponding to different thread bundles are used as nodes to build a task association graph; Performing node feature extraction and edge feature extraction on the task association graph; Inputting the node features and the edge features into a preset graph neural network to output the association relationship between different tasks; Based on the association relationship and the priorities corresponding to different tasks, the tasks to be reallocated are re-prioritized; If the priority after reordering is an urgent task, then stop a task with the lowest priority and reallocate the released thread resources to the task to be reallocated; If the priority of the reordered task is not an urgent task, computing resources are predicted for the task to be reallocated, and a thread warp is reselected for the task.

9. A multi-channel matrix data processing device based on GPGPU, characterized in that: The device comprises a memory for storing computer program instructions and a processor for executing the program instructions, wherein when the computer program instructions are executed by the processor, the device is triggered to execute the method according to any one of claims 1 to 8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions can execute the method according to any one of claims 1 to 8.

Images