Data processing method, device, equipment, medium and program product of processor

Abstract

The application provides a data processing method, device, equipment, medium and program product of a processor. The method comprises the following steps: loading, by a first thread in a thread group of the processor, an mth data block in a first matrix to a first cache location of an internal memory and loading an nth data block in a second matrix to a second cache location of the internal memory; loading, by the first thread, an (n+1)th data block in the second matrix from an external memory to a third cache location of the internal memory, and performing, by a second thread, a multiplication operation on the data block in the first cache location and the data block in the second cache location; loading, by the first thread, the (n+1)th data block in the second matrix from the internal memory to the second cache location of the internal memory, and performing, by the second thread, the multiplication operation on the data block in the first cache location and the data block in the third cache location. According to the application, the calculation speed of the multiplication operation in the processor can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to processor hardware acceleration technology, and in particular to a data processing method and device of a processor, an apparatus, a medium and a program product. BACKGROUND

[0002] With the popularity of mobile computing and artificial intelligence technology, there is an increasing demand for efficient and low-power multiplication methods that can be performed within a processor. These multiplication methods form the basis of various computing cores and are widely used in real-time signal demodulation, lightweight neural network inference, and sensor data fusion scenarios.

[0003] In related technologies, multiplication methods within a processor are usually loaded one data block at a time, and multiplication operations are performed on the loaded data blocks. When the same data block is needed again, the data block is reloaded into the internal memory for multiplication operation processing, resulting in slow multiplication operation speed of the processor. SUMMARY

[0004] The embodiments of the present application provide a data processing method, device, apparatus, medium and program product of a processor, which can reduce the number of repeated loads of the same data block in the processor and improve the multiplication operation speed.

[0005] The technical solutions of the embodiments of the present application are implemented as follows:

[0006] The embodiments of the present application provide a data processing method of a processor, the processor comprising an internal memory and at least one thread group, each thread group comprising at least two threads, the method comprising:

[0007] loading, by a first thread in the thread group of the processor, an mth data block in a first matrix from an external memory to a first cache location of the internal memory, and loading an nth data block in a second matrix from the external memory to a second cache location of the internal memory, wherein the first matrix and the second matrix are matrices to be subjected to multiplication operation, m is a count of the data blocks in the first matrix, the data blocks in the first matrix are arranged in row-major order, n is a count of the data blocks in the second matrix, and the data blocks in the second matrix are arranged in column-major order;

[0008] in response to the first cache location and the second cache location of the internal memory being loaded with the data blocks, loading, by the first thread, an (n+1)th data block in the second matrix from the external memory to a third cache location of the internal memory, and performing, by a second thread in the thread group, the multiplication operation on the data block in the first cache location and the data block in the second cache location, to obtain a first multiplication operation result;

[0009] in response to the first cache location and the second cache location of the internal memory being loaded with the data blocks, loading, by the first thread, an (n+1)th data block in the second matrix from the external memory to a third cache location of the internal memory, and performing, by a second thread in the thread group, the multiplication operation on the data block in the first cache location and the data block in the second cache location, to obtain a first multiplication operation result;

[0010] combining the first multiplication operation result and the second multiplication operation result as a data processing result of the processor.

[0011] Embodiments of the present application provide a data processing device of a processor, the processor comprising an internal memory and at least one thread group, each thread group comprising at least two threads, and the device comprising:

[0012] a first calculation module, applied to load, by a first thread in a thread group of the processor, an mth data block in a first matrix from an external memory to a first cache location of the internal memory, and load an nth data block in a second matrix from the external memory to a second cache location of the internal memory, wherein the first matrix and the second matrix are matrices to be subjected to a multiplication operation, m is a count of the data blocks in the first matrix, the data blocks in the first matrix being arranged in row-major order, n is a count of the data blocks in the second matrix, the data blocks in the second matrix being arranged in column-major order; in response to the first cache location and the second cache location of the internal memory being loaded with the data blocks, loading, by the first thread, an (n+1)th data block in the second matrix from the external memory to a third cache location of the internal memory, and performing, by a second thread in the thread group, the multiplication operation on the data block in the first cache location and the data block in the second cache location, to obtain a first multiplication operation result;

[0013] a second calculation module, configured to load, by the first thread, an (n+1)th data block in the second matrix from the external memory to the second cache location of the internal memory, and perform, by the second thread, the multiplication operation on the data block in the first cache location and the data block in the third cache location to obtain a second multiplication operation result;

[0014] a result combination module, configured to combine the first multiplication operation result and the second multiplication operation result as a data processing result of the processor.

[0015] In the above scheme, the result combination module is further configured to, in a case where the thread group includes one first thread and a plurality of second threads, perform, by a first second thread in the thread group, the multiplication operation on the data block in the first cache location and the data block in the second cache location.

[0016] and,

[0017] perform, by a second second thread in the thread group, the multiplication operation on the data block in the first cache location and the data block in the second cache location.

[0018] In the above scheme, the result combination module is further configured to, before combining the first multiplication operation result and the second multiplication operation result as the data processing result of the processor, perform an iteration process, wherein the iteration process includes alternately performing the process of obtaining the first multiplication operation result and the process of obtaining the second multiplication operation result.

[0019] combining the first multiplication operation result and the second multiplication operation result as the data processing result of the processor includes:

[0020] in response to a sum of quantities of the first multiplication operation result and the second multiplication operation result being less than a set data block quantity of the data processing result, continuing the iteration process;

[0021] in response to the sum of quantities of the first multiplication operation result and the second multiplication operation result being equal to the set data block quantity of the data processing result, ending the iteration process, and combining the first multiplication operation result and the second multiplication operation result obtained in each iteration as the data processing result of the processor.

[0022] In the above scheme, the result combination module is further configured to, in a case where the second matrix includes K×N data blocks, wherein K is a row number of data blocks in the second matrix, and N is a count of data blocks included in each row of data blocks, perform:

[0023] in response to the nth data block used for calculating the first multiplication result being the k1xNth data block, taking the first data block in the k1+1th row as an (n+1)th data block, and calculating the second multiplication result based on the (n+1)th data block and the mth data block, where k1 is a count of data block rows in the second matrix;

[0024] in response to the (n+1)th data block used for calculating the second multiplication result being the k1xNth data block, taking the first data block in the k1+1th row as an (n+2)th data block, and calculating the first multiplication result in a next iteration based on the (n+2)th data block and the mth data block.

[0025] In the foregoing solution, the result combination module is further configured to, in response to the nth data block used for calculating the first multiplication result being the KxNth data block, reset n to 0, and calculate the second multiplication result based on the (m+1)th data block and the nth data block;

[0026] in response to the (n+1)th data block used for calculating the second multiplication result being the KxNth data block, reset n to 0, and calculate the first multiplication result in a next iteration based on the (m+1)th data block and the nth data block.

[0027] In the foregoing solution, the result combination module is further configured to, in the first matrix including MxK data blocks, where M is a count of data blocks included in each row of data blocks, perform:

[0028] in response to the mth data block used for calculating the first multiplication result being the Mxk2th data block, and the nth data block being the k1xNth data block, taking the first data block in the k2+1th column as an (m+1)th data block, and resetting n to 0, calculating the second multiplication result based on the (m+1)th data block and the nth data block;

[0029] in response to the mth data block used for calculating the second multiplication result being the Mxk2th data block, and the nth data block being the k1xNth data block, taking the first data block in the k2+1th column as an (m+1)th data block, and resetting n to 0, calculating the first multiplication result in a next iteration based on the (m+1)th data block and the nth data block.

[0030] Embodiments of the present application provide an electronic device, which comprises:

[0031] a memory for storing computer executable instructions or computer programs;

[0032] a processor for implementing the data processing method of the processor provided by the embodiments of the present application when executing the computer executable instructions or computer programs stored in the memory.

[0033] The embodiments of the present application provide a computer readable storage medium storing computer programs or computer executable instructions for implementing the data processing method of the processor provided by the embodiments of the present application when executed by a processor.

[0034] The embodiments of the present application provide a computer program product comprising computer programs or computer executable instructions, which, when executed by a processor, implement the data processing method of the processor provided by the embodiments of the present application.

[0035] The embodiments of the present application have the following beneficial effects:

[0036] The embodiments of the present application load the data blocks of the first matrix and the data blocks of the second matrix into the first cache location and the second cache location of the internal memory by the first thread in the processor from the external memory, and load the next data block of the second matrix into the third cache location by the first thread when the second thread executes the calculation of the data blocks in the first cache location and the second cache location, which takes advantage of the characteristic that the elements in the first matrix are reused multiple times when performing matrix multiplication, reduces the reading frequency of the data blocks in the first matrix by the processor, and improves the speed of performing multiplication in the processor. In addition, the double cache technology is used to process the data blocks in the second matrix, which further improves the calculation speed of multiplication in the processor. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a structural schematic diagram of an electronic device provided by the embodiments of the present application;

[0038] Figure 2 is an architectural schematic diagram of a processor provided by the embodiments of the present application;

[0039] Figure 3 is a flow schematic diagram of the data processing method of the processor provided by the embodiments of the present application Figure 1 ;

[0040] Figure 4 is a flow schematic diagram of the data processing method of the processor provided by the embodiments of the present application Figure 2 ;

[0041] Figure 5 is a flow schematic diagram in one application scenario provided by the embodiments of the present application Figure 1 ;

[0042] Figure 6 is a schematic diagram of reading data from a GEMM to an SRAM according to an embodiment of the present application;

[0043] Figure 7 is a schematic diagram of a position of multiplication operation on a data block of a matrix in an SRAM according to an embodiment of the present application Figure 1 ;

[0044] Figure 8 is a time distribution of operation performed by each thread group in a processor according to an embodiment of the present application Figure 1 ;

[0045] Figure 9 is a flowchart in an application scenario according to an embodiment of the present application Figure 2 ;

[0046] Figure 10 is a schematic diagram of a position of multiplication operation on a data block of a matrix in an SRAM according to an embodiment of the present application Figure 2 ;

[0047] Figure 11 is a time distribution of operation performed by each thread group in a processor according to an embodiment of the present application Figure 2 .

[0048] It should be noted that the above-mentioned "first", "second" are only used to distinguish different schemes, and do not represent the advantages or disadvantages of the schemes or the priority in the implementation process. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be described in further detail below with reference to the drawings, and the described embodiments should not be regarded as limiting the present application. All other embodiments obtained by a person of ordinary skill in the art without making creative labor fall within the scope of protection of the present application.

[0050] In the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict.

[0051] In the following description, the terms "first\second\third" are only used to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0052] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined target, and can be implemented entirely or partially by using software, hardware (such as a processing circuit or a memory) or a combination thereof. Similarly, one processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of an integral module or unit that includes the functions of the module or unit.

[0053] Unless otherwise defined, all technical and scientific terms used in the embodiments of the present application have the same meanings as commonly understood by one of ordinary skill in the art. The terms used in the embodiments of the present application are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0054] The relevant data collection and processing in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and within the scope of authorization of laws and regulations and the personal information subject, carry out subsequent data use and processing.

[0055] Before the embodiments of the present application are further described in detail, the terms and phrases involved in the embodiments of the present application are explained, and the terms and phrases involved in the embodiments of the present application are applicable to the following explanations.

[0056] 1) Domain Specific Accelerator (DSA) is a processor designed for specific computing fields (such as artificial intelligence, graphics processing or encryption computing), which is characterized by customized hardware logic (such as special instruction set, data path or parallel computing unit) to directly match the computing mode of tasks in this field, so as to achieve significant performance improvement and energy efficiency optimization when executing specific tasks compared with general-purpose processors; for example, Tensor Processing Unit (TPU) is dedicated to accelerating neural network operations, and GPU is originally designed for typical DSA for parallel graphics rendering.

[0057] 2) General Matrix Multiplication (GEMM) is a core routine in linear algebra operations, specifically referring to the general, optimized implementation of multiplying two dense matrices, whose standard form is C = aA x B + bC, where A and B are input matrices, C is an input / output matrix, and a and b are scalar coefficients. As a fundamental computing kernel in many fields such as scientific computing, machine learning, and graphics processing, its performance is crucial, so it is usually deeply optimized at the hardware level (such as the special instruction set of CPU, parallel architecture of GPU, or dedicated accelerator DSA) through techniques such as blocking, vectorization, parallelization, and memory layout optimization to achieve extremely high computing throughput and energy efficiency. For example, the fully connected layer calculation in neural network forward propagation, the convolution operation in image processing (which can be converted into GEMM), and the transformation matrix solving in physical simulation all rely on efficient GEMM implementation to achieve real-time performance.

[0058] 3) Thread specialization is a strategy used in parallel computing, which means that when assigning work to multiple threads, instead of having all threads perform the same general task, threads are assigned specific, different roles and tasks based on their identification, location, or group membership, to achieve higher execution efficiency and resource utilization. For example, in graphics rendering, some threads may be responsible for processing vertex data, while others are responsible for processing pixel shading; in the producer-consumer model, one group of threads is specialized as "producers" to generate data (i.e. the first thread in the subsequent example), and another group of threads is specialized as "consumers" to process data (i.e. the second thread in the subsequent example), through such division of labor to reduce resource competition and synchronization overhead between threads.

[0059] 4) Intra-thread group synchronization is a key operation in parallel programming, used to ensure that all threads within the same thread group (such as CUDA thread blocks or OpenCL workgroups) must have completed all their designated operations before a certain point in the program, thus coordinating the execution progress of group threads, ensuring data consistency, and laying the foundation for subsequent collective communication. This mechanism is usually implemented through barrier operations or memory fence instructions; for example, after the thread group cooperates to write to a certain shared array, it must ensure that all write operations have been completed through intra-thread group synchronization, and any thread in the group can safely start reading other parts of the array or performing collective calculations such as reduction sum.

[0060] 5) Asynchronous pipeline, a design paradigm of computer architecture, its core feature is that there is no unified global clock signal to synchronize between the stages of the pipeline, but through the point-to-point handshake protocol (such as request-response signal) between stages to control the pace of data transfer and processing. After the current stage completes its operation, it sends data and ready signal to the next stage, and the next stage only sends back the "response" signal and accepts data when it is idle and can receive. This mechanism allows each stage to run at its own fastest speed independently, without waiting for other slow stages, effectively solving the performance bottleneck problem of synchronous pipeline caused by the clock period being subject to the slowest stage, and better adapting to different operations with variable delay. For example, in network packet processing, asynchronous arithmetic logic unit or complex on-chip network communication, asynchronous pipeline structure is used to achieve higher energy efficiency and lower inherent delay.

[0061] 6) Dynamic random access memory (DRAM), a semiconductor memory that uses capacitor charge to store data bits, each storage unit is composed of only one transistor and one capacitor. This high-density structure makes DRAM have the core advantages of low cost and large capacity, which is very suitable for computer main memory. However, due to the inevitable leakage of capacitor, the charge will be lost, so DRAM must be equipped with periodic "refresh" circuit to rewrite data regularly, which not only brings additional power consumption, but also makes its access speed lower than SRAM.

[0062] 7) Static random access memory (SRAM), a semiconductor memory that uses bistable flip-flop circuit (usually composed of 6 transistors) to store data bits. As long as the power supply is maintained, its state can always be stable and does not need to be refreshed. This transistor interlocking structure has very high access speed, but its unit structure is complex and occupies a large chip area, resulting in high cost and small capacity, so it is mainly used as a cache in the processor to make up for the speed gap between the processor core and the low-speed main memory.

[0063] 8) Load, a fundamental memory operation performed by the processor, whose core purpose is to transfer a block of data starting at a specific address from a larger but slower external memory (e.g. main memory / DRAM) to a smaller but extremely fast internal memory of the processor (e.g. register or cache) for subsequent instructions to directly and efficiently access and compute. This operation is initiated by the processor issuing a load instruction and specifying the target address, usually involving the cache controller checking whether the required data already exists in the cache (i.e. cache hit or miss), and if not, initiating a read transaction over the memory bus, after obtaining the data, not only sending it to the target register, but also leaving a copy in the cache for future use; for example, after executing an instruction that loads data from a memory address to a register, all subsequent accesses to this address will directly act on the register or cache copy, greatly improving the execution speed of the program, which is one of the key steps to achieve computational efficiency.

[0064] 9) Matrix multiplication, especially its implementation as General Matrix Multiplication (GEMM), exhibits several core characteristics that are crucial for computational optimization: first, it is compute-intensive, requiring O(N³) multiply-add operations for two N x N matrices, with the computation volume growing rapidly with the scale, providing a huge space for performance improvement through parallel computation; second, it has significant data reuse, i.e. each element in the input matrix is reused when calculating multiple elements of the output matrix, for example, a row of matrix A is multiplied with multiple columns of matrix B, which makes it crucial to design a carefully designed memory hierarchy (such as cache) to reduce the delay of accessing external memory; third, its regular and parallelizable computation pattern allows the computation process to be systematically decomposed into a large number of independent multiply-add operations, which is very suitable for efficient execution through thread-level parallelism on hardware with a large number of computing cores (such as GPU or tensor core); finally, its decomposability allows large-scale matrices to be divided into smaller data blocks (block technology), better matching the cache capacity of the processor, and optimizing the layout of data in memory (such as row-major or column-major) to ensure continuous memory access patterns, maximizing the utilization of memory bandwidth. These characteristics collectively form the theoretical basis for the deep hardware and software co-optimization of GEMM, making it a key benchmark for measuring the performance of a computing system.

[0065] In the related art, when performing general matrix multiplication operation by a general processor such as a CPU, it is usually necessary to traverse each element in the input matrix and calculate each element of the output matrix C one by one. For each element C[i][j] in the matrix C, the processor needs to load the i-th row of the matrix A and the j-th column of the matrix B from the external memory, and then multiply and accumulate the corresponding elements in the pair of rows and columns one by one. When the processor then calculates the next element C[i][j+1] in the same row, it has to extract the same row of the matrix A again (the data just used) and extract the next column of the matrix B from the memory again. Similarly, when calculating the elements in the next row, the same column of the matrix B is repeatedly extracted. This access mode results in a large number of repeated data loading operations on the rows of the matrix A and the columns of the matrix B, especially.

[0066] However, by performing matrix multiplication in the above manner, although the same row of the matrix A and the same column of the matrix B are used multiple times to calculate different output elements, the processor may need to reload these data from the limited cache capacity each time a new multiplication and accumulation calculation is performed, or even repeatedly read from the external memory with extremely slow speed. This results in two serious problems: first, the memory bandwidth is used to transmit a large amount of repeated data instead of supporting new calculations; second, the powerful computing unit of the processor is in an idle state most of the time, "waiting" for data to arrive from the external memory. Therefore, the main reason for the slow speed of matrix multiplication is not the speed of the multiplication and accumulation operation itself, but the huge memory access overhead caused by repeated extraction of the same data, which makes the actual performance of the computing system much lower than the theoretical peak value.

[0067] Based on the problems in the related art, the embodiments of the present application provide a data processing method and device of a processor, an electronic device, a computer readable storage medium and a computer program product, which can reduce the number of times of repeatedly reading data blocks from the external memory by the processor and improve the speed of performing multiplication operation in the processor. The exemplary application of the data processing device of the processor provided by the embodiments of the present application is described below.

[0068] Referring to Figure 1 , Figure 1 is a structural schematic diagram of an electronic device 100 provided by the embodiments of the present application, Figure 1 The electronic device 100 shown in FIG. 1 includes at least one processor 10, a volatile memory 120 and a non-volatile memory 130. The various components in the electronic device 100 are coupled together through a bus system 140. It can be understood that the bus system 140 is used to realize the connection and communication between the components. In addition to the data bus, the bus system 140 also includes a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, only the data bus is shown in the figure.Figure 1 Various buses are labeled as bus system 140.

[0069] Processor 10, having signal processing capability, such as a general purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc., where the general purpose processor can be a microprocessor or any conventional processor, etc. The processor in the embodiments of the present application can be used as a domain specific accelerator (DSA) for matrix multiplication.

[0070] The volatile memory 120, an exemplary hardware device includes a random access memory (RAM). The volatile memory 120 can store data to support various operations, examples of these data include programs, modules and data structures or subsets or supersets thereof, the memory mentioned below is the volatile memory. The internal memory SRAM in the subsequent embodiments belongs to the volatile memory 120.

[0071] In some embodiments, the data processing device of the processor provided by the embodiments of the present application can be realized in a software manner, for example, the processor 10 realizes the data processing device by running Figure 1 The processor's data processing device 131 in the non-volatile memory 130 shown is realized, which can be software in the form of programs and plug-ins, including the following software modules: first calculation module 1311, second calculation module 1312 and result combination module 1313, these modules are logical, so they can be combined or further split according to the realized function. The functions of each module will be described below.

[0072] Referring to Figure 2 , Figure 2 The processor 10 provided by the embodiments of the present application is shown in the architecture diagram, as Figure 2 The processor 10 can include internal memory 111, first thread 112, second thread 113, multiplication operation unit 114 and write thread 115, the first thread 112 and the second thread 113 can be combined into a thread group, wherein:

[0073] The first thread 112 is used to read the data in the external memory to load the data in the external memory into the internal memory 111.

[0074] The internal memory 111 is used to temporarily store data blocks for the second thread 113 and the write thread 115 to read.

[0075] The second thread 113 is configured to read data from the internal memory 111 and transmit the read data to the multiplication operation unit 114.

[0076] The multiplication operation unit 114 is configured to perform multiplication operation on the data read from the internal memory 111 by the second thread 113, and store the multiplication result in the internal memory 111.

[0077] The write thread 115 is configured to transmit the multiplication result in the internal memory 111 to the external memory.

[0078] In the embodiment, the multiplication operation unit 114 can perform parallel multiplication operation on the data blocks read from the external memory by the plurality of thread groups in the processor 10 in parallel to obtain the corresponding multiplication result.

[0079] Referring to Figure 2 When the processor 10 receives a request of performing multiplication operation on a first matrix and a second matrix in the external memory, the processor 10 loads the mth data block in the first matrix from the external memory to the first cache location of the internal memory 111 and loads the nth data block in the second matrix from the external memory to the second cache location of the internal memory by the first thread 112 in the thread group, where the first matrix and the second matrix are the matrices to be operated, m is the count of the data blocks in the first matrix, the data blocks in the first matrix are arranged in row-major order, n is the count of the data blocks in the second matrix, and the data blocks in the second matrix are arranged in column-major order. The processor 10 loads the (n+1)th data block in the second matrix from the external memory to the third cache location of the internal memory 111 by the first thread 112 and performs multiplication operation on the data block in the first cache location and the data block in the second cache location by the second thread 113 in the thread group to obtain a first multiplication result. The processor 10 loads the (n+1)th data block in the second matrix from the external memory to the second cache location of the internal memory by the first thread 112 and performs multiplication operation on the data block in the first cache location and the data block in the third cache location by the second thread to obtain a second multiplication result. The processor 10 combines the first multiplication result and the second multiplication result as the data processing result of the processor.

[0080] As an example, in the field of computational photography, medical imaging or security monitoring, real-time filtering, edge detection or style transfer processing is required for high-resolution images (such as 4K or even 8K pictures). The core algorithm of such processing, such as convolution calculation, can be essentially converted into intensive general matrix multiplication. By loading the data to be processed into the internal memory of the processor through the processor, and performing multiplication operation on the data in the internal memory, the multiplication-addition operation can be performed with extremely low power consumption and delay, and the instantaneous convolution operation of each frame of image pixel and the filter kernel can be realized. This enables high-end smartphones or drones to render video streams with complex artistic effects in real time, or enables medical imaging devices to complete the sharpening and enhancement of three-dimensional images in a very short time without consuming a large amount of resources and power of general-purpose processors.

[0081] As an example, in the field of video live streaming, cloud gaming and large-scale multimedia data center, video encoding (such as H.265 / HEVC) and decoding are core tasks, in which key steps such as discrete cosine transform, quantization and motion compensation all rely on massive multiplication operations. By loading the data to be processed into the internal memory of the processor through the processor, and performing multiplication operation on the data in the internal memory, the multiplication-addition operation can be performed with extremely low power consumption and delay, and the throughput of video transcoding can be increased by several times, while the overall energy consumption of the server is significantly reduced. This enables streaming service providers to provide users with more high-definition and smoother 4K / 8K video on-demand and live streaming services at a lower operating cost.

[0082] Referring to Figure 3 , Figure 3 is a flowchart of a data processing method of a processor provided by an embodiment of the present application Figure 1 will be described in conjunction with the steps shown in Figure 3 As shown in Figure 3 , a processor is taken as an example to illustrate the execution subject of the data processing method of the processor, and the method includes the following steps 101 and 104.

[0083] The processor includes an internal memory and at least one thread group, and each thread group includes at least two threads.

[0084] In step 101, by the first thread in the thread group of the processor, the mth data block in the first matrix is loaded from the external memory to the first cache location of the internal memory, and the nth data block in the second matrix is loaded from the external memory to the second cache location of the internal memory.

[0085] Wherein, the first matrix and the second matrix are matrices to be executed multiplication operation, m is the count of data blocks in the first matrix, the data blocks in the first matrix are arranged in row-major order, n is the count of data blocks in the second matrix, and the data blocks in the second matrix are arranged in column-major order.

[0086] Here, the first matrix and the second matrix are operand matrices to be subjected to multiplication operation, which usually represent input data, weight parameters or transformation basis matrices. In matrix multiplication, the first matrix acts as the multiplicand, whose column number must match the row number of the second matrix to ensure the effectiveness of the multiplication operation. The first matrix and the second matrix can include numerical data, feature representation or model parameters, which are usually derived from initialization, training or original data. In machine learning embodiments, the first matrix and the second matrix are commonly found in the fully connected layer, convolutional layer or attention mechanism of neural networks, as weight matrices or input feature matrices.

[0087] For example, in the self-attention mechanism of the Transformer model, the Query (Q) matrix in the Transformer model is taken as the first matrix, and the transpose of the Key (K) matrix is taken as the second matrix. The multiplication of the Q matrix and the K matrix can calculate a set of attention scores, which represent the strength of the correlation between different positions in the sequence.

[0088] Here, the external memory is a storage medium located outside the chip where the processor is located, which is specially used to store data that is too large to be loaded into the internal memory of the processor for operation, such as the first matrix and the second matrix. The external memory acts as a persistent repository for large amounts of data, characterized by a storage capacity much larger than the internal memory, but with relatively slow data access speed. When performing large-scale matrix multiplication, the computing system needs to use a specific data scheduling strategy to block the matrices stored in the external memory into the internal memory for operation, thereby supporting the entire computing task.

[0089] For example, when training a super large-scale neural network, the weight matrix of its fully connected layer (as the first matrix, dimension 50,000 x 50,000) and the input feature matrix (as the second matrix, dimension 50,000 x 10,000) may have a total volume of tens of GB, far exceeding the memory capacity of the server. At this time, these two matrices are completely saved in the external memory of the solid state disk. The computing process is divided into multiple steps: first, read several row blocks of the first matrix and several column blocks of the second matrix from the external memory to the internal memory, then call these data blocks from the internal memory to the multiplication operation unit, get the multiplication operation results of each data block, and finally integrate all the multiplication operation results of the data blocks.

[0090] ​Here, the first thread is a thread specialized for a thread as a "producer". The first thread is specialized for the producer means that the first thread is not responsible for the logical calculation of the business in the concurrent programming architecture, but focuses on efficiently obtaining the data block from the data source (external memory) for other threads of "consumers". The system can dynamically create and manage multiple first threads, which can read data from different areas of the external memory concurrently, thereby achieving high-bandwidth data loading. For example, the first matrix is divided into four areas, and each first thread reads the data block of one area.

[0091] Here, the first cache location and the second cache location are different cache locations in the internal memory. The different cache locations are used to store data blocks originating from different matrices.

[0092] Here, the data block loading according to the column-major order means that the elements of a column are loaded first, and then the elements of the next column are loaded. The data block loading according to the row-major order means that the elements of a row are loaded first, and then the elements of the next row are loaded. When multiplying matrices, in order to efficiently utilize the cache, the data reading method must match the storage order: if the matrix is stored in row-major order, the inner loop should traverse the columns of the right matrix (i.e., read by row) to ensure that contiguous memory is accessed; if stored in column-major order, the inner loop should traverse the rows of the left matrix (i.e., read by column). The core principle is to let the innermost loop operation on contiguous memory blocks to maximize performance.

[0093] In step 102, in response to the first cache location and the second cache location of the internal memory loading the data blocks, the first thread loads the n+1th data block in the second matrix from the external memory to the third cache location of the internal memory, and the second thread in the thread group performs multiplication operation on the data block in the first cache location and the data block in the second cache location to obtain the first multiplication result.

[0094] Here, when the first cache location and the second cache location of the internal memory load the data blocks, it means that the calculation of the elements in the data blocks is ready. Taking the first data block of the first matrix stored in the first cache location and the first data of the second matrix stored in the second cache location as an example, a step 0 synchronization instruction group.sync(0) can be issued as a ready signal to advance the processor system to the next operation in an asynchronous pipeline manner within the thread group. In this way, it can be ensured that all write operations within the group are completed before the next calculation or reading is started, to ensure the integrity of the data involved in the operation.

[0095] In the embodiment of the present application, after confirming that the data is ready by parsing the 0th step synchronization instruction group.sync(0), the second thread can call the data in the internal memory and perform multiplication operation in the multiplication operation unit. In succession to the above embodiment, the first thread loads the data block at the 2nd position in the second matrix to the third cache position from the external memory, and issues the 1st step synchronization signal group.sync(1). In the embodiment of the present application, the second cache position and the third cache position are the ping cache position and the pong cache position of the data block in the second matrix.

[0096] The ping cache position and the pong cache position refer to a pair of double-buffered storage regions commonly used to realize efficient parallel processing of data. Specifically, when the second cache position and the third cache position are the ping and pong cache positions, they form a ping-pong buffer structure: one cache position (such as ping) is used for current data reading or processing, while the other cache position (such as pong) is used for writing or loading the next batch of data at the same time. This design allows the system to work continuously without waiting for the data transmission to complete, thereby improving throughput and performance.

[0097] For example, in the multiplication operation of graphics processing or data stream processing, the second cache position serves as the ping buffer for the GPU to read the matrix representing texture data from the external memory; at the same time, the third cache position serves as the pong buffer for storing the next frame of data to be processed. When the data in the ping buffer is processed, the roles are switched - the third cache position becomes ping for reading, and the second cache position becomes pong for writing, and so on. This ping-pong mechanism ensures the continuity and efficiency of data access, avoiding resource conflicts.

[0098] It should be noted that since the elements in the first matrix need to participate in repeated calculations when performing multiplication, and the time-consuming of loading data blocks from the external memory by the first thread is high, the embodiment of the present application reserves the data blocks (from the first matrix) that need to be repeatedly calculated in the internal memory when performing multiplication of the data blocks, and preloads the data blocks in the second matrix that need to be calculated in the next round, thereby reducing the number of times the data blocks in the first matrix are repeatedly called, and improving the speed of multiplication operation in the processor.

[0099] In step 103, the first thread loads the n+1th data block in the second matrix to the second cache position in the internal memory from the internal memory, and the second thread performs multiplication operation on the data block in the first cache position and the data block in the third cache position to obtain a second multiplication operation result.

[0100] Here, after obtaining the first multiplication result, the first thread has loaded the data block required for the next round of multiplication into the internal memory, at this time, the multiplication can be performed on the data loaded into the internal memory by the second thread, and the first thread loads the data block required for the next round of multiplication into the internal memory for the second thread to perform multiplication.

[0101] In some embodiments, further comprising the case that the thread group includes one first thread and a plurality of second threads, in which case, the multiplication can be performed by performing the following steps.

[0102] First, the multiplication is performed on the data block in the first cache location and the data block in the second cache location by the first second thread in the thread group.

[0103] Then, the multiplication is performed on the data block in the first cache location and the data block in the second cache location by the second second thread in the thread group.

[0104] The core reason that the embodiment of the application can fully utilize the SRAM resource by setting one producer and two consumers in the thread group is that this structure realizes efficient pipeline parallelism, bandwidth optimization and dynamic load balancing: the producer can continuously write data into a specific area of the SRAM, while the two consumers can alternately or in parallel read and process data from other areas of the SRAM; this design makes the data supply, calculation and output stages overlap, the read-write ports and storage units of the SRAM are continuously occupied, avoiding resource idling or blocking waiting caused by the mismatch of the processing speed of a single consumer, thereby converting the high-bandwidth characteristics of the SRAM into actual data throughput improvement, and maximizing its performance potential.

[0105] In step 104, the first multiplication result and the second multiplication result are combined into the data processing result of the processor.

[0106] The embodiment of the application reduces the number of times of calling data from the external memory by repeatedly using the data block loaded into the internal memory in the process of performing multiplication by the processor, thereby bringing three-fold benefits of performance, power consumption and system complexity: it enables the computing unit to work continuously instead of waiting for data by multiplexing data, thereby shifting the system performance bottleneck from low-speed storage bandwidth to high-speed computing unit, improving the data throughput rate; it also reduces the energy consumption required to drive the high-power off-chip memory interface, optimizes the energy efficiency ratio, reduces the dependence on high-speed storage interface and complex control logic, and simplifies the overall system design. In addition, by alternately calculating the data blocks in the second cache location and the third cache location, the probability of the thread used to perform data calculation waiting for data preparation is reduced, thereby improving the efficiency of multiplication.

[0107] In some embodiments, referring to Figure 4 , Figure 4 It is shown that before step 104, the first multiplication result and the second multiplication result can also be obtained by performing steps 105 to 107.

[0108] In step 105, an iteration process is performed.

[0109] Wherein, the iteration process includes alternately performing the first multiplication result and the second multiplication result.

[0110] In some embodiments, also including the case that the second matrix includes KxN data blocks, in this case, the second multiplication result and the first multiplication result can be obtained by performing the following steps 1051 and 1052.

[0111] Wherein, K is the number of rows of data blocks in the second matrix, and N is the data block count of each row containing data blocks in the second matrix.

[0112] In response to the nth data block used to calculate the first multiplication result being the k1xN data block, the (k1+1)th row first data block is taken as the (n+1)th data block, and the second multiplication result is calculated based on the (n+1)th data block and the mth data block, wherein k1 is the count of the number of rows of data blocks in the second matrix.

[0113] Here, the (n+1)th data block and the mth data block read into the internal memory can be read into the multiplication operation unit (i.e. GEMM) by the second thread for multiplication operation to obtain the second multiplication result.

[0114] Or,

[0115] In response to the (n+1)th data block used to calculate the second multiplication result being the k1xN data block, the (k1+1)th row first data block is taken as the (n+2)th data block, and the first multiplication result of the next round of iteration process is calculated based on the (n+2)th data block and the mth data block.

[0116] Here, the second thread reads the (k1+1)th row first data block and the mth data block read into the internal memory into the multiplication operation unit (i.e. GEMM) for multiplication operation to obtain the first multiplication result and complete the line switching to retrieve elements in the second matrix.

[0117] The embodiment of the application realizes that, for each data in the first matrix, only one loading is needed to complete the multiplication operation with multiple data blocks in the second matrix, reduces the calling of the data blocks in the first matrix, alleviates the bandwidth pressure of the internal memory, and improves the calculation efficiency, compared with the multiplication operation method of traversing the positions of each output matrix.

[0118] In some embodiments, the second multiplication operation result and the first multiplication operation result can also be obtained by performing the following steps.

[0119] In response to the nth data block used for calculating the first multiplication operation result being the KxN data block, n is reset to 0, and the second multiplication operation result is calculated based on the m+1 data block and the nth data block.

[0120] Here, the second thread reads the m+1 data block in the internal memory and the nth data block reset to 0 into a multiplication operation unit (i.e., GEMM) to perform multiplication operation, obtains the first multiplication operation result, and realizes that, after completing the multiplication operation of one element in the first matrix with all elements in the second matrix, the next element in the first matrix is called and the second multiplication operation of the next element with all elements in the second matrix is performed.

[0121] Or,

[0122] In response to the n+1 data block used for calculating the second multiplication operation result being the KxN data block, n is reset to 0, and the first multiplication operation result in the next round of iteration processing is calculated based on the m+1 data block and the nth data block.

[0123] Here, the second thread reads the m+1 data block in the internal memory and the nth data block reset to 0 into a multiplication operation unit (i.e., GEMM) to perform multiplication operation, obtains the second multiplication operation result, and realizes that, after completing the multiplication operation of one element in the first matrix with all elements in the second matrix, the next element in the first matrix is called and the first multiplication operation of the next element with all elements in the second matrix is performed.

[0124] The embodiment of the present application realizes that, for each data in the first matrix, the multiplication operation with the data blocks in the multiple rows of the second matrix can be completed through only one loading, by multiplexing the data blocks in the first matrix, and performing the multiplication operation between one data block in the first matrix and all data blocks in one row of the second matrix, and then performing the multiplication operation between the data block in the next row of the second matrix and all data blocks in the next row of the second matrix, compared with the multiplication operation method of traversing the positions of each output matrix, the calling of the data blocks in the first matrix is reduced, the bandwidth pressure of the internal memory is alleviated, and the calculation efficiency is improved.

[0125] In some embodiments, the first matrix includes MxK data blocks, and the second multiplication operation result and the first multiplication operation result can be obtained by performing the following steps.

[0126] M is the data block count of each column containing data blocks in the first matrix.

[0127] In response to the mth data block used for calculating the first multiplication operation result being the Mxk2 data block, and the nth data block being the k1xN data block, the first data block in the k2+1 column is taken as the m+1th data block, n is reset to 0, and the second multiplication operation result is calculated based on the m+1th data block and the nth data block.

[0128] Here, the second thread reads the first data block in the k2+1 column and the nth data block with n reset to 0 in the internal memory to the multiplication operation unit (i.e., GEMM) to perform the multiplication operation, to obtain the second multiplication operation result, and further to realize the column change reading of the first matrix.

[0129] Or,

[0130] In response to the mth data block used for calculating the second multiplication operation result being the Mxk2 data block, and the nth data block being the k1xN data block, the first data block in the k2+1 column is taken as the m+1th data block, n is reset to 0, and the first multiplication operation result of the next iteration round is calculated based on the m+1th data block and the nth data block.

[0131] Here, the second thread reads the first data block in the k2+1 column and the nth data block with n reset to 0 in the internal memory to the multiplication operation unit (i.e., GEMM) to perform the multiplication operation, to obtain the first multiplication operation result, and further to realize the column change reading of the first matrix.

[0132] The embodiment of the present application realizes that each data block in the first matrix is loaded only once in the whole matrix calculation process, reduces the calling of the data blocks in the first matrix, relieves the bandwidth pressure of the internal memory, and improves the calculation efficiency by multiplexing the data blocks in the first matrix, and performing multiplication operation between one data block in the first matrix and all data blocks in the second matrix, and then performing multiplication operation between the next data block in the first matrix and all data blocks in the second matrix.

[0133] In step 106, in response to the sum of the number of the first multiplication operation result and the second multiplication operation result being less than the set data block number of the data processing result, the iteration processing is continued.

[0134] In step 107, in response to the sum of the number of the first multiplication operation result and the second multiplication operation result being equal to the set data block number of the data processing result, the iteration processing is ended, and the first multiplication operation result and the second multiplication operation result obtained in each iteration are combined to form the data processing result of the processor.

[0135] The embodiment of the present application converts the large-scale matrix calculation into a cache-friendly local operation, and reduces the number of data loading required by the local operation. The method divides the huge input matrix (the first matrix and the second matrix) into data blocks with controllable size, so that when the calculation is performed, each element in the first matrix is loaded only once, and each time the calculation is performed, only the data blocks in the second matrix are loaded, and the next data block in the first matrix is loaded only when the multiplication operation between the data block in the first matrix and all data blocks in the second matrix is completed. In this way, once the data block in the first matrix is loaded, it can be repeatedly and efficiently accessed by the calculation unit in the cache for calculating the corresponding part of the output matrix, thereby reducing the number of external memory access and greatly relieving the memory bandwidth pressure. At the same time, the block structure naturally decouples the data dependency between different calculation units, so that the calculation of multiple blocks can be carried out in parallel, fully exploiting the parallel computing capability of multi-core processors or special accelerators, and through the combination of pipeline technologies such as double buffering, the perfect overlap of data loading and calculation is realized, and finally the calculation throughput and energy efficiency are improved by orders of magnitude.

[0136] In the following, an exemplary application of the embodiment of the present application in an actual application scenario will be described.

[0137] In the related art, when a general-purpose processor such as a CPU performs a general matrix multiplication operation, it is usually necessary to traverse each element in the input matrix and calculate each element of the output matrix C one by one. For each element C[i][j] in the matrix C, the processor needs to load the ith row of the matrix A and the jth column of the matrix B from the external memory, and then multiply and accumulate the corresponding elements in the pair of rows and columns one by one. When the processor then calculates the next element C[i][j+1] in the same row, it must again extract the same row of the matrix A (the data just used) and re-extract the next column of the matrix B from the memory. Similarly, when calculating the elements of the next row, the same column of the matrix B is repeatedly extracted. This access mode results in a large number of repeated data loading operations on the rows of A and the columns of B, especially for the input matrix.

[0138] Figure 5 is a flowchart of a process in an application scenario provided by an embodiment of the present application Figure 1 , see Figure 5 , comprising the following steps 201 to 204.

[0139] In step 201, the thread Thread0 (i.e., the first thread in the above embodiment) loads the first data block A00 in the left matrix (i.e., the first matrix in the above embodiment) from the DRAM (i.e., the external memory in the above embodiment) to the ping buffer of the SRAM (i.e., the internal memory in the above embodiment) (i.e., the first cache location in the above embodiment), loads the first data block B00 in the right matrix (i.e., the second matrix in the above embodiment) to the SRAM, and issues a step 0 ready signal group.sync(0).

[0140] In the present embodiment, see Figure 6 , Figure 6 illustrates the principle of reading data from GEMM to SRAM. In Figure 6 , the left matrix is a matrix of M rows and K columns, and the right matrix is a matrix of K rows and N columns. Each thread will slide the reading window in the left matrix and the right matrix, and read the data block passed by the window to the SRAM at the same position as the row where the left matrix is located and the same position as the column where the matrix is located. By

[0141] In step 202, after detecting the step 0 ready signal, the thread Thread1 (i.e., the second thread in the above embodiment) performs a multiplication operation on A00 and B00 in the SRAM, completing a ping operation.

[0142] In step 203, the thread Thread0 loads the 2nd data block B01 in the right matrix from the DRAM onto the pong buffer of the SRAM (i.e. the third buffer position in the above embodiment), and sends the 1st step ready signal.

[0143] In step 204, after detecting the 1st step ready signal, the thread Thread1 performs multiplication operation on A00 and B01 in the SRAM, completing one pong operation.

[0144] Figure 7 is the position principle diagram of multiplication operation on data blocks in the matrix in the SRAM provided by the embodiment of the present application Figure 1 , see Figure 7 , by loading the data block of the mth row in the left matrix (including the elements of ki columns) and the data block of the nth column in the right matrix (including the elements of ki rows), the method of the embodiment of the present application can obtain the data block of the mth row and the nth column in the output matrix (including the elements of mi columns and ni rows, mi and ni can be any positive integer). Figure 8 is the time distribution of operation performed by each thread group in the processor provided by the embodiment of the present application Figure 1 , see Figure 8 , after Thread0 writes the data required for the next round of GEMM into the SRAM, it starts to write the data required for the next round of GEMM into the next position of the SRAM. After the data in the SRAM is ready, Thread1 performs GEMM calculation, and after completing the GEMM calculation of the current round, the data required for the next round of GEMM has also been loaded into the SRAM. In addition, only one loading of the data block A of the left matrix is required to complete the multiplication operation of the data block and multiple data blocks of the right matrix.

[0145] Figure 9 is the flow diagram in one application scenario provided by the embodiment of the present application Figure 2 , see Figure 9 , comprising the following steps 301 to 304.

[0146] In step 301, the thread Thread0 loads the 1st data block A00 in the left matrix from the DRAM onto the ping buffer of the SRAM, loads the 1st data block B00 in the right matrix onto the SRAM, and sends the 0th step ready signal group.sync(0).

[0147] In step 302, after detecting the 0th step ready signal, the thread Thread1 performs multiplication operation on A00 and B00 in the SRAM, completing one ping operation.

[0148] In step 303, thread Thread0 loads the second data block in the left matrix from DRAM into the pong buffer in SRAM and issues the first step ready signal group.sync(1).

[0149] In step 304, after detecting the ready signal of step 1, thread Thread2 performs a multiplication operation on A00 and B01 in SRAM to complete a pong operation.

[0150] Figure 10 This is a schematic diagram illustrating the positional principle of performing multiplication operations on data blocks of a matrix in SRAM, as provided in the embodiments of this application. Figure 2 See Figure 10 By performing a pingpong operation on the data block in the m-th row of the left matrix and the data blocks in the n-th and n+1-th columns of the right matrix using the method of this application embodiment, the data block in the m-th row and n-th column of the output matrix and the data block in the m-th row and n+1-th column of the output matrix can be obtained. Figure 11 The time distribution of operations performed by each thread group in the processor provided in the embodiments of this application is as follows: Figure 2 See Figure 11 After Thread0 writes the data needed for the next round of GEMM into SRAM, it begins writing the data needed for the next round of GEMM into the next location in SRAM. Once the data in SRAM is ready, Thread1 executes the GEMM calculation. After the data needed for the next round of GEMM is written into SRAM, Thread1 has not yet completed the multiplication operation; at this point, Thread2 executes the GEMM operation. After Thread1 and Thread2 each complete their current round of GEMM calculation, the data needed for their respective next round of GEMM has also been loaded into SRAM. Furthermore, only one loading of the data block A of the left matrix is ​​needed to complete the multiplication operation between this data block and multiple data blocks of the right matrix.

[0151] In some embodiments, the data processing apparatus of the processor provided in this application can be implemented in software. For example, the processor 10 operates by running... Figure 1 The data processing device 131 of the processor in the shown non-volatile memory 130 can be software in the form of programs and plug-ins, including the following software modules: a first calculation module 1311, a second calculation module 1312, and a result combination module 1313. These modules are logically related and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0152] The first calculation module 1311 is applied to loading, by a first thread in a thread group of a processor, an mth data block in a first matrix from an external memory to a first cache location of an internal memory and loading an nth data block in a second matrix from the external memory to a second cache location of the internal memory, wherein the first matrix and the second matrix are matrices to be executed multiplication operation, m is a count of data blocks in the first matrix, the data blocks in the first matrix are arranged in row-major order, n is a count of data blocks in the second matrix, and the data blocks in the second matrix are arranged in column-major order; in response to the first cache location and the second cache location of the internal memory loading the data blocks, loading, by the first thread, an (n+1)th data block in the second matrix from the external memory to a third cache location of the internal memory, and executing, by a second thread in the thread group, multiplication operation on the data block in the first cache location and the data block in the second cache location to obtain a first multiplication operation result;

[0153] The second calculation module 1312 is applied to loading, by the first thread, the (n+1)th data block in the second matrix from the external memory to the second cache location of the internal memory, and executing, by the second thread, multiplication operation on the data block in the first cache location and the data block in the third cache location to obtain a second multiplication operation result.

[0154] The result combination module 1313 is applied to combining the first multiplication operation result and the second multiplication operation result as a data processing result of the processor.

[0155] In some embodiments, the result combination module 1313 is further applied to, in the above scheme, in a case that the thread group includes one first thread and a plurality of second threads, executing, by a first second thread in the thread group, multiplication operation on the data block in the first cache location and the data block in the second cache location.

[0156] And,

[0157] Executing, by a second second thread in the thread group, multiplication operation on the data block in the first cache location and the data block in the second cache location.

[0158] In some embodiments, the result combination module 1313 is further applied to, in the above scheme, before combining the first multiplication operation result and the second multiplication operation result as the data processing result of the processor, executing iteration processing, wherein the iteration processing includes alternately executing processes of obtaining the first multiplication operation result and obtaining the second multiplication operation result.

[0159] Combining the first multiplication operation result and the second multiplication operation result as the data processing result of the processor includes:

[0160] In response to the sum of the number of the first multiplication operation result and the second multiplication operation result being less than the set data block number of the data processing result, the iteration processing is continued;

[0161] In response to the sum of the number of the first multiplication operation result and the second multiplication operation result being equal to the set data block number of the data processing result, the iteration processing is ended, and the first multiplication operation result and the second multiplication operation result obtained in each round of iteration are combined to form the data processing result of the processor.

[0162] In the above scheme, the result combination module 1313 is further configured to include K×N data blocks in the second matrix, where K is the number of rows of data blocks in the second matrix, and N is the number of data blocks contained in each row of data blocks, and perform:

[0163] In response to the nth data block used to calculate the first multiplication operation result being the k1×N data block, the (k1+1)th row first data block is taken as the (n+1)th data block, and the second multiplication operation result is calculated based on the (n+1)th data block and the mth data block, where k1 is the count of the number of rows of data blocks in the second matrix;

[0164] In response to the (n+1)th data block used to calculate the second multiplication operation result being the k1×N data block, the (k1+1)th row first data block is taken as the (n+2)th data block, and the first multiplication operation result in the next round of iteration processing is calculated based on the (n+2)th data block and the mth data block.

[0165] In some embodiments, the result combination module 1313 is further configured to, in response to the nth data block used to calculate the first multiplication operation result being the K×N data block, reset n to 0, and calculate the second multiplication operation result based on the (m+1)th data block and the nth data block;

[0166] In response to the (n+1)th data block used to calculate the second multiplication operation result being the K×N data block, reset n to 0, and calculate the first multiplication operation result in the next round of iteration processing based on the (m+1)th data block and the nth data block.

[0167] In some embodiments, the result combination module 1313 is further configured to include M×K data blocks in the first matrix, where M is the number of data blocks contained in each row of data blocks, and perform:

[0168] In response to the mth data block used to calculate the first multiplication operation result being the M×k2 data block, and the nth data block being the k1×N data block, the (k2+1)th column first data block is taken as the (m+1)th data block, and n is reset to 0, and the second multiplication operation result is calculated based on the (m+1)th data block and the nth data block;

[0169] in response to the mth data block used to compute the first multiplication result being the Mxk2data block and the nth data block being the k1xN data block, the first data block in the k2+1column is taken as the m+1th data block and n is reset to 0, and the first multiplication result of the next iteration round is computed based on the m+1th data block and the nth data block.

[0170] In some embodiments, the result combination module 1313 is further configured to, in response to the nth data block used to compute the first multiplication result being the KxN data block, reset n to 0, and compute the second multiplication result based on the m+1th data block and the nth data block;

[0171] In response to the mth data block used to compute the second multiplication result being the Mxk2data block and the nth data block being the k1xN data block, the first data block in the k2+1column is taken as the m+1th data block and n is reset to 0, and the first multiplication result of the next iteration round is computed based on the m+1th data block and the nth data block.

[0172] In some embodiments, the result combination module 1313 is further configured to include MxK data blocks in the first matrix, where M is the count of data blocks included in each row of data blocks, by performing:

[0173] in response to the mth data block used to compute the first multiplication result being the Mxk2data block and the nth data block being the k1xN data block, the first data block in the k2+1column is taken as the m+1th data block and n is reset to 0, and the second multiplication result is computed based on the m+1th data block and the nth data block;

[0174] in response to the mth data block used to compute the second multiplication result being the Mxk2data block and the nth data block being the k1xN data block, the first data block in the k2+1column is taken as the m+1th data block and n is reset to 0, and the first multiplication result of the next iteration round is computed based on the m+1th data block and the nth data block.

[0175] In some embodiments, the result combination module is further configured to include MxK data blocks in the first matrix, where M is the count of data blocks included in each row of data blocks, by performing:

[0176] in response to the mth data block used to compute the first multiplication result being the Mxk2data block and the nth data block being the k1xN data block, the first data block in the k2+1column is taken as the m+1th data block and n is reset to 0, and the second multiplication result is computed based on the m+1th data block and the nth data block;

[0177] In response to the mth data block used to calculate the first multiplication operation result being the Mxk2 data block and the nth data block being the k1xN data block, the first data block in the k2+1 column is taken as an (m+1)th data block, n is reset to 0, and the first multiplication operation result in the next iteration round is calculated based on the (m+1)th data block and the nth data block.

[0178] In some embodiments, the result combination module 1313 is further configured to, in response to the nth data block used to calculate the first multiplication operation result being the KxN data block, reset n to 0, and calculate the second multiplication operation result based on the (m+1)th data block and the nth data block.

[0179] In response to the (n+1)th data block used to calculate the second multiplication operation result being the KxN data block, n is reset to 0, and the first multiplication operation result in the next iteration round is calculated based on the (m+1)th data block and the nth data block.

[0180] In some embodiments, the result combination module 1313 is further configured to include MxK data blocks in the first matrix, where M is the count of data blocks included in each row of data blocks, and perform:

[0181] In response to the mth data block used to calculate the first multiplication operation result being the Mxk2 data block and the nth data block being the k1xN data block, the first data block in the k2+1 column is taken as an (m+1)th data block, n is reset to 0, and the second multiplication operation result is calculated based on the (m+1)th data block and the nth data block.

[0182] In response to the mth data block used to calculate the first multiplication operation result being the Mxk2 data block and the nth data block being the k1xN data block, the first data block in the k2+1 column is taken as an (m+1)th data block, n is reset to 0, and the first multiplication operation result in the next iteration round is calculated based on the (m+1)th data block and the nth data block.

[0183] The embodiment of the present application provides a computer program product, which comprises a computer program or computer executable instructions stored in a computer readable storage medium. A processor of an electronic device reads the computer executable instructions from the computer readable storage medium, and the processor executes the computer executable instructions, so that the electronic device executes the data processing method of the processor described above.

[0184] The embodiment of the present application provides a computer readable storage medium, wherein computer executable instructions or computer programs are stored, and when the computer executable instructions or computer programs are executed by a processor, the processor executes the data processing method of the processor provided by the embodiment of the present application, for example, as shown in the following. Figure 3 The data processing method of the processor is shown.

[0185] In some embodiments, the computer readable storage medium can be RAM, ROM, flash memory, magnetic surface memory, optical disc, or CD-ROM memory, etc.; and can also be various devices including one or any combination of the above storage.

[0186] In some embodiments, the computer executable instructions can be in the form of programs, software, software modules, scripts or codes, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and can be deployed in any form, including being deployed as independent programs or being deployed as modules, components, subroutines or other units suitable for use in a computing environment.

[0187] As an example, the computer executable instructions can but not necessarily correspond to files in a file system, can be stored in a part of a file storing other programs or data, for example, stored in one or more scripts in a Hyper Text Markup Language (HTML) document, stored in a single file dedicated to the program in question, or stored in multiple cooperative files (for example, files storing one or more modules, subroutines or code parts).

[0188] As an example, the computer executable instructions can be deployed to be executed on one electronic device, or executed on multiple electronic devices located in one place, or executed on multiple electronic devices distributed in multiple places and interconnected through a communication network.

[0189] In summary, by the first thread in the processor, the first matrix data block is loaded from the external memory to the first cache location of the internal memory, and the second matrix data block is loaded to the second cache location of the internal memory, and when the second thread performs the calculation of the data block in the first cache location and the second cache location, the first thread loads the next data block of the second matrix to the third cache location, which takes advantage of the characteristics that the elements in the first matrix are reused multiple times when performing matrix multiplication, reduces the reading frequency of the data block in the first matrix by the processor, and improves the speed of performing multiplication operation in the processor. In addition, the double cache technology is also used to process the data block in the second matrix, which further improves the calculation speed of the multiplication operation in the processor. By repeatedly using the data block loaded into the internal memory during the multiplication operation in the processor, the number of data calls from the external memory is reduced, which brings three benefits of performance, power consumption and system complexity: it uses data reuse to make the calculation unit work continuously instead of waiting for data, thereby transferring the system performance bottleneck from the low-speed storage bandwidth to the high-speed calculation unit, improving the data throughput rate; it also reduces the energy consumption required to drive the high-power off-chip memory interface, optimizes the energy efficiency ratio, reduces the dependence on high-speed storage interfaces and complex control logic, and simplifies the overall system design. In addition, by alternately calculating the data blocks in the second cache location and the third cache location, the probability of the thread waiting for data preparation for performing data calculation is reduced, thereby improving the efficiency of the multiplication operation.

[0190] The above merely describes the embodiments of the present application, but is not used to limit the protection scope of the present application. Any modification, equivalent replacement and improvement within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. A data processing method for a processor, characterized in that, The processor includes internal memory and at least one thread group, each thread group including at least two threads, and the method includes: The first thread in the processor's thread group loads the m-th data block from the first matrix into the first cache location of the internal memory from the external memory, and loads the n-th data block from the second matrix into the second cache location of the internal memory from the external memory. The first matrix and the second matrix are matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). In response to the data blocks being loaded in the first and second cache locations of the internal memory, the first thread loads the (n+1)th data block in the second matrix from the external memory into the third cache location of the internal memory, and the second thread in the thread group performs the multiplication operation on the data blocks in the first and second cache locations to obtain the first multiplication result. The first thread loads the (n+1)th data block in the second matrix from the internal memory into the second cache location of the internal memory, and the second thread performs the multiplication operation on the data block in the first cache location and the data block in the third cache location to obtain the second multiplication result. The result of the first multiplication operation and the result of the second multiplication operation are combined to form the data processing result of the processor.

2. The method according to claim 1, characterized in that, The thread group includes one first thread and multiple second threads; The step of performing the multiplication operation on the data block in the first cache location and the data block in the second cache location through the second thread in the thread group includes: The multiplication operation is performed on the data block in the first cache location and the data block in the second cache location by the first second thread in the thread group; The step of performing the multiplication operation on the data block in the first cache location and the data block in the third cache location through the second thread includes: The second thread in the thread group performs the multiplication operation on the data block in the first cache location and the data block in the second cache location.

3. The method according to claim 1, characterized in that, The method further includes: Before combining the first multiplication result and the second multiplication result into the data processing result of the processor, an iterative process is performed, wherein the iterative process includes alternately executing the process of obtaining the first multiplication result and obtaining the second multiplication result; The step of combining the first multiplication result and the second multiplication result to form the data processing result of the processor includes: If the sum of the number of the first multiplication result and the second multiplication result is less than the set number of data blocks of the data processing result, the iterative processing continues; In response to the sum of the number of the first multiplication operation results and the second multiplication operation results being equal to the set number of data blocks of the data processing result, the iterative processing ends, and the first multiplication operation results and the second multiplication operation results obtained in each iteration are combined to form the data processing result of the processor.

4. The method according to claim 3, characterized in that, The second matrix includes K×N data blocks, where K is the number of rows of the data blocks in the second matrix, and N is the count of data blocks contained in each row of the data blocks; The method further includes: In response to the fact that the nth data block used to calculate the result of the first multiplication operation is the k1×Nth data block, the first data block in the k1+1th row is taken as the n+1th data block, and the second multiplication operation result is calculated based on the n+1th data block and the mth data block, where k1 is the count of the number of data block rows in the second matrix; In response to the fact that the (n+1)th data block used to calculate the result of the second multiplication operation is the (k1×N)th data block, the first data block in the (k1+1)th row is taken as the (n+2)th data block, and the result of the first multiplication operation in the next round of iteration is calculated based on the (n+2)th data block and the (m)th data block.

5. The method according to claim 4, characterized in that, The method further includes: In response to the fact that the nth data block used to calculate the result of the first multiplication operation is the K×Nth data block, n is reset to 0, and the second multiplication operation result is calculated based on the (m+1)th data block and the nth data block; In response that the (n+1)th data block used to calculate the result of the second multiplication operation is the K×Nth data block, n is reset to 0, and the result of the first multiplication operation in the next round of iteration is calculated based on the (m+1)th data block and the nth data block.

6. The method according to claim 4, characterized in that, The first matrix includes M×K data blocks, where M is the count of data blocks contained in each row of data blocks; The method further includes: In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block in column k2+1 is taken as the (m+1)-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the (m+1)-th data block and the n-th data block. In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block in the k2+1-th column is taken as the (m+1)-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the (m+1)-th data block and the n-th data block.

7. A data processing apparatus for a processor, characterized in that, The processor includes internal memory and at least one thread group, each thread group including at least two threads, and the device includes: A first computing module is configured to load the m-th data block of a first matrix from external memory into a first cache location of internal memory via a first thread in the thread group of the processor, and load the n-th data block of a second matrix from external memory into a second cache location of internal memory. The first and second matrices are matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). In response to the data blocks being loaded into the first and second cache locations of internal memory, the first thread loads the (n+1)-th data block of the second matrix from external memory into a third cache location of internal memory, and the second thread in the thread group performs the multiplication operation on the data blocks in the first and second cache locations to obtain a first multiplication result. The second calculation module is used to load the (n+1)th data block in the second matrix from the external memory into the second cache location of the internal memory through the first thread, and to perform the multiplication operation on the data block in the first cache location and the data block in the third cache location through the second thread to obtain the second multiplication operation result; The result combination module is used to combine the first multiplication result and the second multiplication result into the data processing result of the processor.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method of the processor according to any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, they implement the data processing method of the processor according to any one of claims 1 to 6.

10. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, they implement the data processing method of the processor according to any one of claims 1 to 6.

Images