Batch matrix multiplication parallel optimization method and system for CNN based on Shenwei architecture

Abstract

The present invention proposes a parallel optimization method and system for batch matrix multiplication for CNN on the Shenwei architecture, which belongs to the field of artificial intelligence parallel optimization technology; it includes: converting the input feature map and convolution kernel in the convolution layer into input matrix and weight matrix respectively, and batch processing them into multiple groups of independent matrix multiplication tasks; the master core encapsulates the matrix multiplication task as a parameter structure array, and transmits it to the slave core through a single DMA. The slave core uses a dynamic row block partitioning algorithm to divide the rows of the input matrix into row block tasks based on the total number of threads and the matrix height; and performs sub-matrix multiplication calculations on the allocated independent row blocks, uses double-buffered DMA to asynchronously pre-fetch matrix sub-blocks, and performs matrix multiplication and accumulation calculations. The present invention can improve the parallel processing efficiency of batch matrix multiplication between the master and slave cores of the Shenwei processor and optimize the algorithm performance.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence parallel optimization technology, and in particular relates to a batch matrix multiplication parallel optimization method and system for CNN on the Shenwei architecture. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] In the field of high-performance computing (HPC), large-scale computational problems have long been at the core of scientific and engineering computing research. Empirical studies have shown that a wide range of computational science applications, such as machine learning, data mining, image and signal processing, computational fluid dynamics, astrophysics, and quantum chemistry, involve massive, fine-grained, small-scale matrix operations. Embedding these small operations into large-scale computational tasks has become a common paradigm. This trend has driven the widespread application and optimization of batch matrix operation techniques. Parallel optimization methods for batch matrix multiplication on domestic heterogeneous processors have demonstrated significant application potential in the field of machine learning. For example, image recognition in computer vision tasks involves feature extraction and classification of massive amounts of image data, typically implemented using convolutional neural networks (CNNs). During the training phase of CNNs, dense matrix multiplications constitute a core computational bottleneck, creating an urgent need for efficient batch matrix multiplication.

[0004] However, on the domestically produced Shenwei supercomputer platform (especially the SW26010-pro many-core architecture), existing technologies face the following key issues:

[0005] (1) There is a lack of batch matrix optimization solutions that are suitable for the Shenwei architecture. Although the Shenwei platform has powerful parallel computing capabilities, its heterogeneous many-core architecture is significantly different from traditional CPUs / GPUs. In addition, existing optimization solutions (such as MKL / cuBLAS) cannot be directly ported, resulting in batch matrix multiplication being difficult to fully utilize the hardware potential on the Shenwei platform, especially in high-concurrency, fine-grained computing scenarios where performance is limited.

[0006] (2) Communication bottlenecks and scalability limitations under traditional scheduling models. In existing technologies, task scheduling relies on the master core (MPE) to distribute parameters to the slave cores (CPE) one by one, resulting in frequent master-slave communication. As the batch size of tasks increases, the number of communication rounds increases linearly, resulting in significant synchronization delays, which seriously restricts the scalability of the system.

[0007] (3) Load imbalance and poor data locality. Static task allocation strategies are difficult to adapt to the dynamic changes in matrix dimensions, which can easily cause some slave cores to be idle while others are overloaded. At the same time, the local data memory (LDM) of the slave cores is not fully utilized, resulting in high-latency access to main memory, which becomes a performance bottleneck. In multi-dimensional iterative calculations, data loading relies on synchronous DMA operations, forcing the slave cores to wait for each data load, resulting in insufficient computing resource utilization. Moreover, data transmission delays cannot be hidden. Summary of the Invention

[0008] In order to overcome the deficiencies of the above-mentioned prior art, the present invention provides a parallel optimization method and system for batch matrix multiplication for CNN on the Shenwei architecture, which can improve the parallel processing efficiency of batch matrix multiplication between the master and slave cores of the Shenwei processor and optimize the algorithm performance.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0010] The first aspect of the present invention provides a parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture.

[0011] A batch matrix multiplication parallel optimization method for CNN on the Shenwei architecture includes: the parallel optimization method is implemented by a master core and multiple slave cores;

[0012] The implementation process of the main core includes:

[0013] The input feature map and convolution kernel in the CNN convolution layer are converted into input matrix and weight matrix respectively, and processed in batches to form multiple groups of independent matrix multiplication tasks;

[0014] Encapsulate the parameters in the matrix multiplication task into a parameter structure array in continuous memory, transfer it to the slave core via a single DMA, and trigger the slave core program start signal;

[0015] The implementation process of the plurality of slave cores includes:

[0016] Asynchronously read the parameter structure array and autonomously deduce the task scope; based on the total number of threads and the matrix height, a dynamic row block partitioning algorithm is used to divide the rows of the input matrix into row block tasks, and each slave core thread is allocated an independent row block according to the thread ID;

[0017] Sub-matrix multiplication calculations are performed on the allocated independent row blocks, and double-buffered DMA is used to asynchronously prefetch matrix sub-blocks of the input matrix and weight matrix, perform matrix multiplication and accumulation calculations, and return the results to the main memory after completion.

[0018] Furthermore, the input feature map and convolution kernel in the CNN convolution layer are converted into input matrix and weight matrix respectively, including: for a single input feature map, its local receptive field elements are expanded into column vectors, which are combined to form an input matrix; the convolution kernel is flattened into row vectors, which are combined to form a weight matrix; and the input matrix and weight matrix conversion are performed on multiple input samples respectively to generate multiple sets of independent matrix pairs.

[0019] Furthermore, the parameter structure array includes a batch size, a matrix dimension parameter, a source matrix memory address, and a target matrix memory address; wherein the batch size corresponds to the number of groups of independent matrix multiplication tasks.

[0020] Furthermore, a dynamic row block partitioning algorithm is used to divide the rows of the input matrix into row block tasks, and each slave core thread is allocated an independent row block according to the thread ID, including: dividing the row stripe blocks along the first dimension of the matrix, dividing the column stripe blocks along the second dimension, and decomposing the sub-blocks along the third dimension to form a matrix sub-block structure mapped to the slave core computing unit.

[0021] Furthermore, the execution process of the dynamic row block partitioning algorithm includes: calculating the basic row number and the residual row number of the input matrix; if the thread ID of the slave core thread is less than the residual row number, the number of allocated rows is increased by 1 on the basis of the basic row number, and at this time, the starting row is equal to the product of the thread ID and the allocated number of rows; if the thread ID of the slave core thread is not less than the residual row number, the number of allocated rows is consistent with the basic row number, and at this time, the starting row is equal to the product of the thread ID and the allocated number of rows plus the residual row number.

[0022] Furthermore, the execution process of the double-buffered DMA includes: allocating two LDM buffers for the sub-blocks of the input matrix and the weight matrix respectively; asynchronously prefetching the next sub-block to the free buffer when calculating the current sub-block; ensuring that the current calculation data is ready through DMA synchronization instructions, and alternately switching the calculation buffer and the prefetch buffer.

[0023] Furthermore, the performing of matrix multiplication and accumulation calculations includes: in an inner accumulation and summation loop, performing instruction-level parallel processing on continuous floating-point multiplication and accumulation operations.

[0024] The second aspect of the present invention provides a batch matrix multiplication parallel optimization system for CNN on the Shenwei architecture.

[0025] The batch matrix multiplication parallel optimization system for CNN based on the Shenwei architecture includes: a master core and multiple slave cores; wherein the master core includes a batch matrix construction module and a batch task scheduling module, and the slave core includes a matrix allocation module and a slave core calculation module;

[0026] The batch matrix building module is configured to convert the input feature maps and convolution kernels in the CNN convolution layer into input matrices and weight matrices respectively, and process them in batches to form multiple groups of independent matrix multiplication tasks;

[0027] The batch task scheduling module is configured to: encapsulate the parameters in the matrix multiplication task into a parameter structure array in continuous memory, transfer it to the slave core through a single DMA, and trigger the slave core program start signal;

[0028] The matrix allocation module is configured to: asynchronously read the parameter structure array and autonomously derive the task range; divide the rows of the input matrix into row block tasks using a dynamic row block partitioning algorithm based on the total number of threads and the matrix height, and allocate an independent row block to each slave core thread according to the thread ID;

[0029] The slave core computing module is configured to: perform sub-matrix multiplication calculations on the allocated independent row blocks, and use double-buffered DMA to asynchronously prefetch matrix sub-blocks of the input matrix and weight matrix, perform matrix multiplication and accumulation calculations, and return the results to the main memory after completion.

[0030] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a master core and multiple slave cores, implements the steps of the parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture as described in the first aspect of the present invention.

[0031] The fourth aspect of the present invention provides an electronic device, comprising a memory, a master core, multiple slave cores, and a program stored in the memory and executable on the master core and the multiple slave cores. When the master core and the multiple slave cores execute the program, they implement the steps of the parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture as described in the first aspect of the present invention.

[0032] One or more of the above technical solutions have the following beneficial effects:

[0033] (1) The present invention converts the input feature map and convolution kernel in the CNN convolution layer into input matrix and weight matrix respectively, and forms multiple independent matrix multiplication tasks in batch processing; by mapping the CNN convolution operation into batch matrix multiplication tasks, and combining the LDM local storage and DMA high-speed transmission characteristics of the Shenwei architecture, it breaks through the porting limitations of MKL / cuBLAS and other solutions in the existing technology; compared with the serial benchmark, it can achieve efficient acceleration and fully release the parallel computing potential of the Shenwei platform, especially in high-concurrency and fine-grained computing. It can significantly improve hardware utilization.

[0034] (2) The present invention addresses the communication bottleneck of parameter distribution by the main core (MPE) by adopting a parameter structure array single DMA transmission mechanism, that is, the main core encapsulates batch task parameters (matrix dimensions, memory addresses, etc.) into continuous memory blocks, and only one DMA operation is required to complete the full data transmission; at the same time, combined with the non-blocking trigger mode, it can significantly reduce the number of communication rounds and eliminate synchronization delays.

[0035] (3) The present invention addresses the problems of uneven load and underutilized LDM caused by static allocation, and designs a dynamic row block partitioning algorithm. Specifically, by accurately calculating the number of rows and the starting row index of each thread, it can ensure that the load deviation does not exceed one row, effectively avoiding core idleness or overload. At the same time, the DMA mechanism is used to preload data into the slave core LDM, converting main memory access into high-speed local storage access, which can effectively reduce data access latency. At the same time, the present invention addresses the problem of idle computing resources caused by synchronous DMA, and innovatively adopts a double-buffered DMA asynchronous pipeline. Specifically, by allocating double buffers to the sub-blocks of the input matrix and the weight matrix, a "prefetch-calculate" pipeline operation is implemented. When the slave core calculates the current sub-block, the background DMA asynchronously pre-fetches the next sub-block data, and ensures that the data is ready through synchronization instructions, so that data transmission and calculation are executed in parallel. In this way, data transmission latency can be hidden while avoiding the slave core waiting for data loading.

[0036] Advantages of additional aspects of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0038] Figure 1 This is a flowchart of the parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture in Example 1 of the present invention.

[0039] Figure 2 Schematic diagram of matrix data partitioning for matrix multiplication in the first embodiment of the present invention.

[0040] Figure 3 This is a structural diagram of the batch matrix multiplication parallel optimization system for CNN on the Shenwei architecture in Example 2 of the present invention. DETAILED DESCRIPTION

[0041] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present invention belongs.

[0042] It should be noted that the terms used herein are for describing particular embodiments only and are not intended to limit the exemplary embodiments according to the present invention.

[0043] In the absence of conflict, the embodiments of the present invention and the features thereof may be combined with each other.

[0044] Example 1

[0045] This embodiment discloses a parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture.

[0046] like Figure 1 As shown, a batch matrix multiplication parallel optimization method for CNN on the Shenwei architecture includes: the parallel optimization method is implemented by a master core and multiple slave cores;

[0047] The implementation process of the main core includes:

[0048] The input feature map and convolution kernel in the CNN convolution layer are converted into input matrix and weight matrix respectively, and processed in batches to form multiple groups of independent matrix multiplication tasks;

[0049] Encapsulate the parameters in the matrix multiplication task into a parameter structure array in continuous memory, transfer it to the slave core via a single DMA, and trigger the slave core program start signal;

[0050] The implementation process of the plurality of slave cores includes:

[0051] Asynchronously read the parameter structure array and autonomously deduce the task scope;

[0052] According to the total number of threads and matrix height, a dynamic row block partitioning algorithm is used to divide the rows of the input matrix into row block tasks. Each slave core thread is assigned an independent row block according to the thread ID.

[0053] Sub-matrix multiplication calculations are performed on the allocated independent row blocks, and double-buffered DMA is used to asynchronously prefetch matrix sub-blocks of the input matrix and weight matrix, perform matrix multiplication and accumulation calculations, and return the results to the main memory after completion.

[0054] Based on the above process, the present invention can improve the parallel processing efficiency of batch matrix multiplication between the master and slave cores of the Shenwei processor and optimize the algorithm performance. To facilitate understanding of the technical solution of the present invention, the specific implementation method of the technical solution of the present invention is further explained and illustrated below.

[0055] Methods for solving batch matrix problems within the domestically produced Sunway supercomputer environment have yet to be fully researched and applied. For example, despite the powerful parallel computing capabilities of the domestically produced Sunway Blue Light II supercomputer, which boasts extremely high floating-point performance, a systematic, high-performance optimization solution for batch dense matrix multiplication is still lacking. To fully exploit the parallel potential of the SW26010-pro many-core architecture, this paper presents a heterogeneous parallel optimization algorithm adapted to this platform: a parallel optimization method for batch matrix multiplication for CNNs on the Sunway architecture.

[0056] The batch matrix multiplication that this invention focuses on can be viewed as a set of independent dense matrix multiplications, which can be expressed as:

[0057] ;

[0058] in, 、 is a scalar, 、 and Respectively represent the number of rows and columns 、 and A dense matrix of Indicates the total number of samples. When the matrix multiplication is large, it is of great significance to organize the calculation and memory access, improve parallelism and computing throughput while keeping the matrix multiplications independent of each other. There are many possible variants of batch matrix multiplication, such as whether the matrix is ​​in transposed format or non-transposed format, whether the matrix shape in the same batch is fixed or variable, and what kind of data precision is supported. In this study, the present invention focuses on the case where the batch matrix multiplication is double precision, the matrix dimensions in all batches are fixed, and it is stored in a non-transposed format. This convention not only covers a large number of real application scenarios in high-performance numerical computing, but also makes performance optimization strategies such as batch scheduling, memory alignment optimization, vectorization and DMA prefetching feasible and adaptable.

[0059] The present invention is built on the SW26010-pro many-core processor and adopts a distributed computing architecture. Specifically, it deploys 512 computing nodes and integrates 1024 processors, accommodating a total of 399,360 computing cores. The core frequency is 2.25GHz, and the system peak performance reaches 14.3PFLOPS. Each processor integrates six core groups (CGs), each of which has one master computing control core (MPE) and 64 slave computing cores (CPEs). Each slave CPE is equipped with a 256KB local data memory (LDM) as an on-chip cache to reduce data access latency. The system uses direct memory access (DMA) technology to achieve efficient data transfer between main memory and LDM, with bandwidth performance significantly higher than the main memory direct access mode used in existing technologies.

[0060] In the implementation of the main kernel, the input feature map and convolution kernel in the CNN convolution layer are converted into input matrices and weight matrices respectively, and processed in batches to form multiple groups of independent matrix multiplication tasks. This can be achieved through the following methods:

[0061] A convolution kernel slides over a local region of the input feature map and performs a dot product operation to extract features. This process is mathematically equivalent to or converted to matrix multiplication. Specifically, the im2col operation is used to rearrange the elements of each local region (receptive field) on the input feature map of the convolution layer into a column vector. The expanded column vectors of all these local regions are combined to form a large matrix called the input matrix A. In the same way, all the convolution kernels in the convolution layer are rearranged into a matrix called the weight matrix B. Each row of this matrix represents a complete convolution kernel, and the number of columns matches the number of rows of the input matrix A.

[0062] By multiplying the converted input matrix A with the weight matrix B, the output matrix C can be obtained; it can be understood that the output matrix C can also be restored to the spatial structure of the output feature map through the col2im operation.

[0063] During the training and reasoning process of CNN, small batch processing is used to improve computational efficiency and model generalization ability. input samples (such as images). For each input sample in the batch ( =1,2,..., ), will independently perform the above-mentioned convolution operation to matrix multiplication conversion. This means that The group-independent matrix multiplication problem is to generate multiple groups of independent matrix pairs. These independent matrix multiplication operations can be regarded as a batch matrix multiplication operation, which has the same form as the one concerned by the present invention. Match; A and B correspond to The input matrix and weight matrix after the sample conversion. Through the above conversion mechanism, the most computationally intensive convolutional layer operation in CNN is effectively mapped into a series of independent, regular matrix multiplication operations. This enables the batch matrix multiplication heterogeneous parallel optimization method for the Shenwei architecture proposed in this patent to be applied to accelerate the training and inference process of CNN.

[0064] In the implementation of the master core, the parameters in the matrix multiplication task are encapsulated as a parameter structure array in continuous memory, transferred to the slave core through a single DMA, and the slave core program start signal is triggered. This can be achieved through the following methods:

[0065] After the batch matrix is ​​built, the master core MPE is responsible for efficient scheduling and parameter distribution of the entire batch task, thereby preparing for parallel processing by the slave core CPE. Specific task scheduling and parameter preloading can be achieved through the following methods:

[0066] 1) The master core MPE uniformly encapsulates batch matrix parameter information and constructs a parameter structure array to pass to the slave core CPE. The parameter structure array contains the batch size, matrix dimension parameters, source matrix memory address, and destination matrix memory address. The batch size corresponds to the number of independent matrix multiplication tasks.

[0067] 2) Parameter structure array preloading and transmission. These structures (parameter structure arrays) are batched into a contiguous memory block and written to the slave core's accessible memory via a one-time direct memory access (DMA) operation for retrieval. This parameter preprocessing method avoids the inefficient prior art method of sending individual tasks from the master core one by one, significantly reducing communication rounds between the master and slave cores and lowering communication latency.

[0068] 3) Start the slave core program. Figure 1 As shown in the figure, after parameter preparation is complete, the master core initiates the slave core program by initiating a signal trigger. Upon receiving the scheduling instruction, each slave core in the array immediately preloads the parameters by asynchronously reading the corresponding parameter structure array and automatically inferring its task scope based on the thread number. Each slave core only processes its own matrix multiplication task, without sharing data or locking synchronization with other slave cores. This reduces inter-thread interference and improves scalability. This reduces the number of communications between the master and slave cores, avoids frequent communication overhead, and thus improves computational efficiency, achieving a master-slave computation flow that combines non-blocking triggering with synchronous waiting.

[0069] By accelerating the parallel task scheduling and parameter preloading mechanism through the collaboration of the master and slave cores, the number of communications and waiting time between the master and slave cores can be effectively reduced, thereby improving performance and efficiency.

[0070] During the implementation of the slave core, the parameter structure array is read asynchronously and the task scope is independently derived. Based on the total number of threads and the matrix height, a dynamic row block partitioning algorithm is used to partition the rows of the input matrix into row block tasks. Each slave core thread is assigned an independent row block according to the thread ID. This can be achieved by the following method:

[0071] Based on the task information distributed by the master core and the preset strategy, specific computing tasks (such as matrix row blocks) are reasonably allocated to each slave core thread to achieve load balancing. Specifically:

[0072] 1) In the Sunway multi-core processor architecture, the DMA mechanism enables data transfer between main memory and LDM, cleverly converting direct access to main memory into access to the local LDM, improving data access efficiency. Based on this feature, in subsequent computations, data is preferentially loaded into the local data memory (LDM) before performing the corresponding computations, fully leveraging the performance advantages of the LDM.

[0073] In order to give full play to the performance advantages of LDM, the present invention adopts Figure 2 The matrix data partitioning method shown is to use a dynamic row block partitioning algorithm to divide the rows of the input matrix into row block tasks. Each slave core thread is allocated an independent row block according to the thread ID, including: dividing the row stripe blocks along the first dimension of the matrix (i.e., M dimension), dividing the column stripe blocks along the second dimension (i.e., N dimension), and decomposing the sub-blocks along the third dimension (i.e., the common dimension K dimension) to form a matrix sub-block structure mapped to the slave core computing unit. The scale of the input matrix A is M x K, i.e., M rows and K columns; the scale of the weight matrix B is K x N, i.e., K rows and N columns; therefore, the scale of the C matrix is ​​M x N, i.e., M rows and N columns. On this basis, partitioning along the M dimension means partitioning the input matrix A by rows, and partitioning along the N dimension means partitioning the weight matrix B by columns. Specifically: First, partition along the M and N dimensions of the matrix to divide the rows of the input matrix A into sub-blocks of size The row strip blocks bA, the columns of the weight matrix B are divided into blocks of size The column stripe block bB of the output matrix C is multiplied to generate a sub-block bC of the output matrix C. The partitioning scheme is then further divided along the K-dimensional plane. Based on the above division, the input matrix A is further divided along the columns and the weight matrix B along the rows. The row stripe block bA is decomposed into bM × bK sub-blocks and the column stripe block bB is decomposed into bK × bN sub-blocks, which are then multiplied. Finally, these blocks are mapped to the corresponding slave core computing units, achieving efficient block matrix computation. This partitioning scheme optimizes computing resource allocation and data locality through a multi-dimensional hierarchical partitioning strategy.

[0074] 2) Dynamic row block partitioning to achieve load balancing. In order to ensure that the load of each slave core is relatively balanced and avoid the situation where some slave cores are idle and some are overloaded, this paper designs a dynamic load balancing strategy, that is, to divide the row block tasks according to the total number of slave cores and the matrix height to ensure load balancing. The goal of dynamic row block partitioning is to evenly distribute the matrix height to the slave core threads (such as =64), aiming to minimize the load imbalance between threads. Its mathematical formula can be expressed as:

[0075] ;

[0076] in, Indicates the number of rows of matrix multiplication tasks that each thread needs to process, Indicates the thread ID, Indicates the number of slave core threads allocated, Indicates the height of the matrix.

[0077] When the thread ID is less than If it is, it means that the thread needs to share an extra line of tasks, otherwise, the thread only needs to handle basic tasks. In the slave core, row block tasks are assigned by thread ID to ensure that each thread starts processing from the correct row, avoiding overlap or omission between row blocks. The starting index of each row block is This can be confirmed by:

[0078] ;

[0079] From this, we can see that the execution process of the dynamic row block partitioning algorithm can actually be summarized as follows: calculate the base number of rows and the residual number of rows of the input matrix. If the thread ID of the slave thread is less than the residual number of rows, the number of rows allocated is based on the base number of rows plus 1. In this case, the starting row is equal to the product of the thread ID and the number of rows allocated. If the thread ID of the slave thread is not less than the residual number of rows, the number of rows allocated is consistent with the base number of rows. In this case, the starting row is equal to the product of the thread ID and the number of rows allocated plus the residual number of rows. Through the above dynamic row block partitioning method, row block tasks can be reasonably allocated according to the matrix height and the number of slave threads, effectively avoiding the situation where some slave cores are idle while others are overloaded, thereby ensuring a relatively balanced load on each core and improving the performance and efficiency of the entire system.

[0080] The dynamic row block partitioning algorithm ensures load balancing among slave cores. That is, row block tasks are dynamically divided according to the total number of slave cores and the matrix height. The algorithm calculates the number of rows and the starting row index for each thread, effectively avoiding idleness or overload of some slave cores.

[0081] In the implementation of the slave core, sub-matrix multiplication calculations are performed on the allocated independent row blocks, and double-buffered DMA is used to asynchronously pre-fetch matrix sub-blocks of the input matrix and weight matrix, perform matrix multiplication and accumulation calculations, and return the results to the main memory after completion. This can be achieved by the following methods:

[0082] 1) Design of a Direct Memory Access (DMA) asynchronous double buffering and computation overlap strategy. To fully utilize the computational resources and data transfer bandwidth of the slave core, an optimized data transfer and computation overlap strategy is adopted. The core of this strategy is that when the slave core calculates the matrix sub-blocks of the output matrix C it is responsible for, the K dimension involved in the innermost loop (i.e., the width of the input matrix A) is used. and the height of the weight matrix B ), a double buffer mechanism and asynchronous DMA transmission are used to load the data of input matrix A and weight matrix B, thereby realizing the pipeline operation of data prefetching and calculation.

[0083] 2) The execution process of double-buffered DMA includes: allocating two LDM buffers for the sub-blocks of the input matrix and the weight matrix respectively; asynchronously prefetching the next sub-block into the free buffer while calculating the current sub-block; ensuring the readiness of the current calculation data through DMA synchronization instructions, and alternating between the calculation buffer and the prefetch buffer.

[0084] In the specific implementation process, first, the double buffer needs to be initialized, that is, the slave core allocates a single buffer for the matrix sub-block of the output matrix C currently calculated in the direct memory access LDM. , which is used to accumulate intermediate results during the calculation of all K-dimensional sub-blocks. Two LDM buffers are allocated for the K-dimensional sub-blocks of the input matrix A ( 、 ) as a double buffer. Two LDM buffers are also allocated for the K-dimensional sub-blocks of the weight matrix B ( 、 ) as double buffering.

[0085] Before starting the iterative calculation along the K dimension, the C sub-blocks in the LDM are accumulated in the buffer is cleared to zero. Then, the first K-tile is preloaded, and the first A data tile and the first B data tile of the K dimension are loaded asynchronously into a set of available buffers in the double buffer. A K-tile refers to a slice divided along the common dimension K of matrix multiplication, that is, the K-dimensional sub-block currently being processed. A K-tile can be understood as a working unit for a K-dimensional iteration. Processing a K-tile means the accumulation of a local result of the C sub-block has been completed. Only when all K-tiles have been iterated is the final result of the C sub-block considered calculated. An A data tile refers to a sub-block of matrix A. It is the specific data extracted from matrix A required to process a K-tile step. It can be understood as the current computational slice of matrix A. Similarly, a B data tile refers to a sub-block of matrix B. It is the specific data extracted from matrix B required to process a K-tile step. It can be understood as the current computational slice of matrix B.

[0086] 3) Use K-dimensional iteration and pipeline methods to traverse the loop.

[0087] First, determine the data buffer that the current calculation should use (for example, 、 Existence 、 Sub-block); at the same time, determine the target buffer for the next DMA operation.

[0088] Then, asynchronously prefetch the next K-tile, if the next K-tile exists ( 、 ), an asynchronous DMA request is immediately initiated to load the data into the target buffer determined in the previous step. DMA operations are handled by the background DMA engine and do not block the main computation flow. A DMA synchronization instruction (CRTS_dma_wait_value) is used to ensure that the data block in the buffer required for the current computation has been transferred.

[0089] Finally, use the ready and The data block performs local matrix multiplication and accumulation operations, that is, in the inner accumulation and summation loop, the continuous floating-point multiplication and accumulation operations are parallelized at the instruction level. For the matrix multiplication and accumulation operation of the nested triple loop (that is, the inner accumulation and summation loop), the inner loop at this stage adopts loop unrolling optimization to reduce control overhead and improve computational efficiency, so as to give full play to the floating-point computing capability of the Shenwei processor. The calculation results are accumulated to a single buffer. After the calculation is completed, the current calculation buffer and the next DMA buffer are switched to prepare for the next iteration. When all tiles along the K dimension are calculated and accumulated, The final result of the current C sub-block is in the middle. At this time, asynchronous DMA is used to Write the contents back to the corresponding position of the global C matrix in the main memory and wait for the DMA operation to complete.

[0090] This method introduces double buffering and asynchronous DMA prefetching for the data blocks of input matrix A and weight matrix B in K-dimensional iteration, making data loading ( 、 ) can be combined with the calculation of the current data block ( 、 ) occur in parallel, forming an efficient "prefetch-wait-compute" pipeline. This effectively hides some data transmission delays, allowing the slave cores to be in a more continuous computing state, thereby improving overall computing throughput.

[0091] Based on the parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture provided by the present invention, the parallel potential of the SW26010-pro many-core architecture can be fully utilized.

[0092] Example 2

[0093] This embodiment discloses a batch matrix multiplication parallel optimization system for CNN on the Shenwei architecture.

[0094] like Figure 3 As shown, the batch matrix multiplication parallel optimization system for CNN on the Shenwei architecture includes: a master core and multiple slave cores; wherein the master core includes a batch matrix construction module and a batch task scheduling module, and the slave core includes a matrix allocation module and a slave core calculation module;

[0095] The batch matrix building module is configured to convert the input feature maps and convolution kernels in the CNN convolution layer into input matrices and weight matrices respectively, and process them in batches to form multiple groups of independent matrix multiplication tasks;

[0096] The batch task scheduling module is configured to: encapsulate the parameters in the matrix multiplication task into a parameter structure array in continuous memory, transfer it to the slave core through a single DMA, and trigger the slave core program start signal;

[0097] The matrix allocation module is configured to: asynchronously read the parameter structure array and autonomously derive the task range; divide the rows of the input matrix into row block tasks using a dynamic row block partitioning algorithm based on the total number of threads and the matrix height, and allocate an independent row block to each slave core thread according to the thread ID;

[0098] The slave core computing module is configured to: perform sub-matrix multiplication calculations on the allocated independent row blocks, and use double-buffered DMA to asynchronously prefetch matrix sub-blocks of the input matrix and weight matrix, perform matrix multiplication and accumulation calculations, and return the results to the main memory after completion.

[0099] Example 3

[0100] The purpose of this embodiment is to provide a computer-readable storage medium.

[0101] A computer-readable storage medium having a computer program stored thereon, which, when executed by a master core and multiple slave cores, implements the steps of the parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture as described in the first embodiment of the present disclosure.

[0102] Example 4

[0103] The purpose of this embodiment is to provide an electronic device.

[0104] An electronic device includes a memory, a master core, multiple slave cores, and a program stored in the memory and executable on the master core and the multiple slave cores. When the master core and the multiple slave cores execute the program, the steps of the parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture as described in the first embodiment of the present disclosure are implemented.

[0105] The steps involved in the apparatuses of Examples 2, 3, and 4 above correspond to those of Method Example 1. For detailed implementations, please refer to the relevant description of Example 1. The term "computer-readable storage medium" should be understood to mean a single medium or multiple media containing one or more instruction sets; it should also be understood to include any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and causing the processor to perform any method of the present invention.

[0106] Those skilled in the art will appreciate that the modules or steps of the present invention described above can be implemented using a general-purpose computer device. Alternatively, they can be implemented using program code executable by a computing device, which can then be stored in a storage device and executed by the computing device. Alternatively, they can be fabricated into separate integrated circuit modules, or multiple modules or steps can be fabricated into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0107] Although the above describes the specific embodiments of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without any creative work are still within the scope of protection of the present invention.

Claims

1. A parallel optimization method for batch matrix multiplication for CNN on the Shenwei architecture, characterized by: include: The parallel optimization method is implemented by a master core and multiple slave cores; The implementation process of the main core includes: The input feature map and convolution kernel in the CNN convolution layer are converted into input matrix and weight matrix respectively, and processed in batches to form multiple groups of independent matrix multiplication tasks; Encapsulate the parameters in the matrix multiplication task into a parameter structure array in continuous memory, transfer it to the slave core via a single DMA, and trigger the slave core program start signal; The parameter structure array includes a batch size, a matrix dimension parameter, a source matrix memory address, and a destination matrix memory address; wherein the batch size corresponds to the number of groups of independent matrix multiplication tasks; The implementation process of the plurality of slave cores includes: Asynchronously read the parameter structure array and autonomously deduce the task scope; based on the total number of threads and the matrix height, a dynamic row block partitioning algorithm is used to divide the rows of the input matrix into row block tasks, and each slave core thread is allocated an independent row block according to the thread ID; The mathematical formula for dynamic row block partitioning is expressed as: ； in, Indicates the number of rows of matrix multiplication tasks that each thread needs to process, Indicates the thread ID, Indicates the number of slave core threads allocated, Indicates the height of the matrix; The execution process of the dynamic row block partitioning algorithm includes: calculating the basic row number and the residual row number of the input matrix; if the thread ID of the slave thread is less than the residual row number, then adding 1 to the basic row number, and in this case, the starting row is equal to the product of the thread ID and the allocated row number; if the thread ID of the slave thread is not less than the residual row number, then the allocated row number is consistent with the basic row number, and in this case, the starting row is equal to the product of the thread ID and the allocated row number plus the residual row number; Perform sub-matrix multiplication calculations on the allocated independent row blocks, and use double-buffered DMA to asynchronously prefetch matrix sub-blocks of the input matrix and weight matrix, perform matrix multiplication and accumulation calculations, and return the results to main memory after completion; The execution process of the double-buffered DMA includes: allocating two LDM buffers for the sub-blocks of the input matrix and the weight matrix respectively; asynchronously prefetching the next sub-block to the idle buffer when calculating the current sub-block; ensuring the readiness of the current calculation data through DMA synchronization instructions, and alternately switching the calculation buffer and the prefetch buffer.

2. The batch matrix multiplication parallel optimization method for CNN on the Shenwei architecture according to claim 1, characterized in that: The input feature map and convolution kernel in the CNN convolution layer are converted into input matrix and weight matrix respectively, including: for a single input feature map, its local receptive field elements are expanded into column vectors and combined to form an input matrix; the convolution kernel is flattened into row vectors and combined to form a weight matrix; the input matrix and weight matrix conversion are performed on multiple input samples respectively to generate multiple independent matrix pairs.

3. The batch matrix multiplication parallel optimization method for CNN on the Shenwei architecture according to claim 1, characterized in that: A dynamic row block partitioning algorithm is used to divide the rows of the input matrix into row block tasks. Each slave core thread is allocated an independent row block according to the thread ID, including: dividing the row stripe blocks along the first dimension of the matrix, dividing the column stripe blocks along the second dimension, and decomposing the sub-blocks along the third dimension to form a matrix sub-block structure mapped to the slave core computing unit.

4. The batch matrix multiplication parallel optimization method for CNN on the Shenwei architecture according to claim 1, characterized in that: The execution of matrix multiplication and accumulation calculations includes: in an inner accumulation and summation loop, performing instruction-level parallel processing on continuous floating-point multiplication and accumulation operations.

5. A batch matrix multiplication parallel optimization system for CNN based on the Shenwei architecture, characterized by: include: A master core and multiple slave cores; wherein the master core includes a batch matrix construction module and a batch task scheduling module, and the slave core includes a matrix allocation module and a slave core calculation module; The batch matrix building module is configured to convert the input feature maps and convolution kernels in the CNN convolution layer into input matrices and weight matrices respectively, and process them in batches to form multiple groups of independent matrix multiplication tasks; The batch task scheduling module is configured to: encapsulate the parameters in the matrix multiplication task into a parameter structure array in continuous memory, transfer it to the slave core through a single DMA, and trigger the slave core program start signal; The parameter structure array includes a batch size, a matrix dimension parameter, a source matrix memory address, and a destination matrix memory address; wherein the batch size corresponds to the number of groups of independent matrix multiplication tasks; The matrix allocation module is configured to: asynchronously read the parameter structure array and autonomously derive the task range; divide the rows of the input matrix into row block tasks using a dynamic row block partitioning algorithm based on the total number of threads and the matrix height, and allocate an independent row block to each slave core thread according to the thread ID; The mathematical formula for dynamic row block partitioning is expressed as: ； in, Indicates the number of rows of matrix multiplication tasks that each thread needs to process, Indicates the thread ID, Indicates the number of slave core threads allocated, Indicates the height of the matrix; The execution process of the dynamic row block partitioning algorithm includes: calculating the basic row number and the residual row number of the input matrix; if the thread ID of the slave thread is less than the residual row number, then adding 1 to the basic row number, and in this case, the starting row is equal to the product of the thread ID and the allocated row number; if the thread ID of the slave thread is not less than the residual row number, then the allocated row number is consistent with the basic row number, and in this case, the starting row is equal to the product of the thread ID and the allocated row number plus the residual row number; The slave core computing module is configured to: perform sub-matrix multiplication calculations on the assigned independent row blocks, use double-buffered DMA to asynchronously prefetch matrix sub-blocks of the input matrix and weight matrix, perform matrix multiplication and accumulation calculations, and return the results to the main memory after completion; The execution process of the double-buffered DMA includes: allocating two LDM buffers for the sub-blocks of the input matrix and the weight matrix respectively; asynchronously prefetching the next sub-block to the idle buffer when calculating the current sub-block; ensuring the readiness of the current calculation data through DMA synchronization instructions, and alternately switching the calculation buffer and the prefetch buffer.

6. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a master core and multiple slave cores, the steps of the batch matrix multiplication parallel optimization method for CNN on the Shenwei architecture as described in any one of claims 1 to 4 are implemented.

7. An electronic device comprising a memory, a master core, a plurality of slave cores, and a program stored in the memory and executable on the master core and the plurality of slave cores, wherein: When the master core and multiple slave cores execute the program, the steps of the batch matrix multiplication parallel optimization method for CNN on the Shenwei architecture as described in any one of claims 1 to 4 are implemented.

Images