Data processing method and device of processor, equipment, medium and program product

Abstract

The invention provides a data processing method and device of a processor, equipment, a medium and a program product. The method comprises the steps of loading an mth data block in a first matrix to a first cache position of an internal memory through a first thread in a thread group of a processor, and loading an nth data block in a second matrix to a second cache position of the internal memory; loading the (n + 1) th data block in the second matrix from the external memory to a third cache position of the internal memory through the first thread, and performing multiplication operation on the data blocks in the first cache position and the data blocks in the second cache position through the second thread; and loading the (n + 1) th data block in the second matrix from the internal memory to a second cache position of the internal memory through the first thread, and performing multiplication operation on the data block in the first cache position and the data block in the third cache position through the second thread. According to the invention, the calculation speed of multiplication in the processor can be improved.

Description

Technical Field

[0001] This application relates to processor hardware acceleration technology, and more particularly to a processor data processing method, apparatus, device, medium, and program product. Background Technology

[0002] With the popularization of mobile computing and artificial intelligence technologies, the demand for high-efficiency, low-power multiplication methods that can be executed within processors is becoming increasingly urgent. These multiplication methods form the basis of various computing cores and are widely used in scenarios such as real-time signal demodulation, lightweight neural network inference, and sensor data fusion.

[0003] In related technologies, the multiplication operation method in the processor usually involves loading data blocks one by one, performing multiplication operations on the loaded data blocks, and then reloading the data blocks into internal memory for multiplication operations when the same data blocks are needed again, which results in a slow speed for the processor to perform multiplication operations. Summary of the Invention

[0004] This application provides a data processing method, apparatus, device, medium, and program product for a processor, which can reduce the number of times the same data block is repeatedly loaded in the processor and improve the speed of multiplication operations.

[0005] The technical solution of this application embodiment is implemented as follows: This application provides a data processing method for a processor, the processor including internal memory and at least one thread group, each thread group including at least two threads, the method including: The first thread in the processor's thread group loads the m-th data block from the first matrix into the first cache location of the internal memory from the external memory, and loads the n-th data block from the second matrix into the second cache location of the internal memory from the external memory. The first matrix and the second matrix are matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). In response to the data blocks being loaded in the first and second cache locations of the internal memory, the first thread loads the (n+1)th data block in the second matrix from the external memory into the third cache location of the internal memory, and the second thread in the thread group performs the multiplication operation on the data blocks in the first and second cache locations to obtain the first multiplication result. The first thread loads the (n+1)th data block in the second matrix from the external memory into the second cache location of the internal memory, and the second thread performs the multiplication operation on the data block in the first cache location and the data block in the third cache location to obtain the second multiplication result. The result of the first multiplication operation and the result of the second multiplication operation are combined to form the data processing result of the processor.

[0006] This application provides a data processing apparatus for a processor, the processor including internal memory and at least one thread group, each thread group including at least two threads, the apparatus comprising: A first computing module is configured to load the m-th data block of a first matrix from external memory into a first cache location of internal memory via a first thread in the thread group of the processor, and load the n-th data block of a second matrix from external memory into a second cache location of internal memory. The first and second matrices are matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). In response to the data blocks being loaded into the first and second cache locations of internal memory, the first thread loads the (n+1)-th data block of the second matrix from external memory into a third cache location of internal memory, and the second thread in the thread group performs the multiplication operation on the data blocks in the first and second cache locations to obtain a first multiplication result. The second calculation module is used to load the (n+1)th data block in the second matrix from the external memory into the second cache location of the internal memory through the first thread, and to perform the multiplication operation on the data block in the first cache location and the data block in the third cache location through the second thread to obtain the second multiplication operation result; The result combination module is used to combine the first multiplication result and the second multiplication result into the data processing result of the processor.

[0007] In the above scheme, the result combination module is further configured to perform the multiplication operation on the data block in the first cache location and the data block in the second cache location by the first second thread in the thread group when the thread group includes one first thread and multiple second threads; as well as, The second thread in the thread group performs the multiplication operation on the data block in the first cache location and the data block in the second cache location.

[0008] In the above scheme, the result combination module is further configured to perform iterative processing before combining the first multiplication result and the second multiplication result into the data processing result of the processor, wherein the iterative processing includes alternatingly executing the process of obtaining the first multiplication result and obtaining the second multiplication result; The step of combining the first multiplication result and the second multiplication result to form the data processing result of the processor includes: If the sum of the number of the first multiplication result and the second multiplication result is less than the set number of data blocks of the data processing result, the iterative processing continues; In response to the sum of the number of the first multiplication operation results and the second multiplication operation results being equal to the set number of data blocks of the data processing result, the iterative processing ends, and the first multiplication operation results and the second multiplication operation results obtained in each iteration are combined to form the data processing result of the processor.

[0009] In the above scheme, the result combination module is further configured to perform the operation when the second matrix includes K×N data blocks, where K is the number of rows in the second matrix and N is the count of data blocks contained in each row: In response to the fact that the nth data block used to calculate the result of the first multiplication operation is the k1×Nth data block, the first data block in the k1+1th row is taken as the n+1th data block, and the second multiplication operation result is calculated based on the n+1th data block and the mth data block, where k1 is the count of the number of data block rows in the second matrix; In response to the fact that the (n+1)th data block used to calculate the result of the second multiplication operation is the (k1×N)th data block, the first data block in the (k1+1)th row is taken as the (n+2)th data block, and the result of the first multiplication operation in the next round of iteration is calculated based on the (n+2)th data block and the (m)th data block.

[0010] In the above scheme, the result combination module is further configured to, in response to the fact that the nth data block used to calculate the result of the first multiplication operation is the K×Nth data block, reset n to 0, and calculate the second multiplication operation result based on the (m+1)th data block and the nth data block; In response that the (n+1)th data block used to calculate the result of the second multiplication operation is the K×Nth data block, n is reset to 0, and the result of the first multiplication operation in the next round of iteration is calculated based on the (m+1)th data block and the nth data block.

[0011] In the above scheme, the result combination module is further configured to perform the following when the first matrix includes M×K data blocks, where M is the count of data blocks contained in each row of data blocks: In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block in column k2+1 is taken as the (m+1)-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the (m+1)-th data block and the n-th data block. In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block in the k2+1-th column is taken as the (m+1)-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the (m+1)-th data block and the n-th data block.

[0012] This application provides an electronic device, the electronic device comprising: Memory is used to store executable instructions or computer programs. The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method of the processor provided in the embodiments of this application.

[0013] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions, which, when executed by a processor, implements the data processing method of the processor provided in this application.

[0014] This application provides a computer program product, including a computer program or computer executable instructions. When the computer program or computer executable instructions are executed by a processor, they implement the data processing method of the processor provided in this application.

[0015] The embodiments of this application have the following beneficial effects: This embodiment of the application loads data blocks of a first matrix from external memory into a first cache location of internal memory, and loads data blocks of a second matrix into a second cache location of internal memory, using a first thread in the processor. While the second thread performs calculations on the data blocks in the first and second cache locations, the first thread loads the next data block of the second matrix into a third cache location. This leverages the characteristic that elements in the first matrix are reused multiple times during matrix multiplication, reducing the number of times the processor reads data blocks from the first matrix and improving the speed of multiplication operations in the processor. Furthermore, by using dual-caching technology to process data blocks in the second matrix, the computational speed of multiplication operations in the processor is further improved. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application; Figure 2 This is a schematic diagram of the processor architecture provided in an embodiment of this application; Figure 3 This is a flowchart illustrating the data processing method of the processor provided in the embodiments of this application. Figure 1 ; Figure 4 This is a flowchart illustrating the data processing method of the processor provided in the embodiments of this application. Figure 2 ; Figure 5 This is a flowchart illustrating an application scenario provided in an embodiment of this application. Figure 1 ; Figure 6 This is a schematic diagram illustrating the principle of reading data from GEMM to SRAM provided in an embodiment of this application; Figure 7 This is a schematic diagram illustrating the positional principle of performing multiplication operations on data blocks of a matrix in SRAM, as provided in the embodiments of this application. Figure 1 ; Figure 8 This is the time distribution of operations performed by each thread group in the processor provided in the embodiments of this application. Figure 1 ; Figure 9 This is a flowchart illustrating an application scenario provided in an embodiment of this application. Figure 2 ; Figure 10 This is a schematic diagram illustrating the positional principle of performing multiplication operations on data blocks of a matrix in SRAM, as provided in the embodiments of this application. Figure 2 ; Figure 11 The time distribution of operations performed by each thread group in the processor provided in the embodiments of this application is as follows: Figure 2 .

[0017] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0019] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0020] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0021] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0022] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0023] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0024] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0025] 1) A Domain Specific Accelerator (DSA) is a processor optimized for a specific computing domain (such as artificial intelligence, graphics processing, or cryptographic computing). Its core lies in directly matching the computing mode of the task in that domain through customized hardware logic (such as dedicated instruction sets, data paths, or parallel computing units). This allows it to achieve significant performance improvements and energy efficiency optimizations compared to general-purpose processors when performing specific tasks. For example, the Tensor Processing Unit (TPU) is dedicated to accelerating neural network operations, while GPUs were originally designed as typical DSAs for parallel graphics rendering.

[0026] 2) The General Matrix Multiplication (GEMM) is a core routine in linear algebra operations, specifically referring to a general, optimized implementation for multiplying two dense matrices. Its standard form is C = αA × B + βC, where A and B are input matrices, C is the input / output matrix, and α and β are scalar coefficients. As the foundational computing kernel in many fields such as scientific computing, machine learning, and graphics processing, its performance is crucial. Therefore, it is typically deeply optimized at the hardware level (such as dedicated instruction sets for CPUs, parallel architectures for GPUs, or dedicated DSA accelerators) through techniques like block partitioning, vectorization, parallelization, and memory layout optimization to achieve extremely high computational throughput and energy efficiency. For example, the computation of fully connected layers in the forward propagation of neural networks, convolution operations in image processing (which can be converted to GEMM), and the solution of transformation matrices in physics simulations all rely on efficient GEMM implementations to achieve real-time performance.

[0027] 3) Thread specialization is a strategy used in parallel computing. It refers to assigning specific, distinct roles and tasks to multiple threads instead of having them perform the same general tasks. This aims to achieve higher execution efficiency and resource utilization. For example, in graphics rendering, some threads might specialize in processing vertex data, while others might specialize in pixel shading. In a producer-consumer model, one group of threads is specialized as "producers" to generate data (i.e., the first thread in the following embodiment), while another group is specialized as "consumers" to process data (i.e., the second thread in the following embodiment). This division of labor reduces resource contention and synchronization overhead between threads.

[0028] 4) Thread group synchronization is a crucial operation in parallel programming. It ensures that within the same thread group (such as a thread block in CUDA or a workgroup in OpenCL), all threads must have completed all their specified operations before reaching a specific point in the program. This coordinates the execution progress of threads within the group, guarantees data consistency, and lays the foundation for subsequent collective communication. This mechanism is typically implemented through barrier operations or memory fence instructions. For example, after a thread group collaboratively completes a write to a shared array, synchronization within the group must ensure that all write operations are complete before any thread in the group can safely begin reading other parts of the array or performing collective calculations such as reduction summation.

[0029] 5) Asynchronous pipelining is a pipeline design paradigm in computer architecture. Its core feature is that there is no unified global clock signal for synchronization between the various stages of the pipeline. Instead, the transmission and processing rhythm of data is controlled by a point-to-point handshake protocol (such as request-response signals) between stages. After the previous stage completes its operation, it sends data and a ready signal to the next stage. The next stage only sends back a "response" signal and accepts data when it is idle and ready to receive. This mechanism allows each stage to run independently at its fastest speed without waiting for other slower stages. This effectively solves the performance bottleneck problem of synchronous pipelines, where the clock cycle is limited by the slowest stage, and can better adapt to scenarios with variable latency in different operations. For example, in network packet processing, asynchronous arithmetic logic units, or complex on-chip network communication, asynchronous pipeline structures are used to achieve higher energy efficiency and lower inherent latency.

[0030] 6) Dynamic Random Access Memory (DRAM) is a semiconductor memory that uses capacitor charge to store data bits. Each memory cell consists of only one transistor and one capacitor. This high-density structure gives DRAM the core advantages of low cost and large capacity, making it very suitable as the main memory of a computer. However, since capacitors inevitably leak current and lose charge, DRAM must be equipped with periodic "refresh" circuits to rewrite data periodically. This not only brings additional power consumption, but also makes its access speed lower than SRAM.

[0031] 7) Static Random Access Memory (SRAM) is a semiconductor memory that uses bistable flip-flop circuits (usually composed of 6 transistors) to store data bits. As long as it is powered, its state can be kept stable without refresh operation. This transistor interlocking structure gives it extremely high access speed, but its cell structure is complex and occupies a large chip area, resulting in high cost and small capacity. Therefore, it is mainly used as a high-speed cache inside the processor to make up for the speed gap between the processor core and the low-speed main memory.

[0032] 8) Loading is a fundamental memory operation performed by the processor. Its core purpose is to transfer a block of data starting at a specific address from a larger but slower external memory (such as main memory / DRAM) to a smaller but much faster internal processor memory (such as registers or cache), so that subsequent instructions can directly and efficiently access and compute it. This operation is initiated by the processor issuing a load instruction specifying the target address. It typically involves the cache controller checking whether the required data already exists in the cache (i.e., cache hit). If it does not exist, a read transaction is initiated through the memory bus. After the data is retrieved, it is not only sent to the target register, but a copy is also kept in the cache for future use. For example, after executing an instruction to load data from a memory address into a register, all subsequent accesses to that address will directly operate on the register or cache copy, thereby greatly improving the execution speed of the program. This is one of the key steps in achieving computational performance.

[0033] 9) Matrix multiplication, especially its implementation as Generalized Matrix Multiplication (GEMM), exhibits several core characteristics crucial for computational optimization: First, it is computationally intensive; multiplying two N×N matrices requires O(N³) multiplication-addition operations, with the computational load increasing dramatically with scale. This provides significant potential for performance improvement through parallel computing. Second, it exhibits significant data reusability; each element in the input matrix is ​​reused multiple times when calculating multiple elements of the output matrix. For example, a row of matrix A may be multiplied by multiple columns of matrix B. This characteristic allows for performance optimization through carefully designed memory hierarchies (such as high-resolution memory). The use of fast caching to reduce latency in accessing external memory becomes crucial; furthermore, its regular and parallelizable computational pattern allows the computation process to be systematically decomposed into a large number of independent multiply-accumulate operations, making it ideal for efficient execution through thread-level parallelism on hardware with a large number of computing cores (such as GPUs or Tensor Cores); finally, its decomposability allows large-scale matrices to be divided into smaller data blocks (blocking techniques), thereby better matching the processor's cache capacity and ensuring continuous memory access patterns by optimizing the layout of data in memory (such as row-major or column-major), maximizing memory bandwidth utilization. These characteristics together form the theoretical basis for deep hardware and software co-optimization of GEMM, making it a key benchmark for measuring the performance of computing systems.

[0034] In related technologies, when performing general matrix multiplication operations using a general-purpose processor such as a CPU, it is typically necessary to traverse every element of the input matrix and calculate each element to obtain each element of the output matrix C. For each element C[i][j] in matrix C, the processor needs to load the i-th row of matrix A and the j-th column of matrix B from external memory, then multiply and accumulate the corresponding elements in this row and column. When the processor then calculates the next element C[i][j+1] in the same row, it must again retrieve the same row of matrix A (the data just used) and retrieve the next column of matrix B from memory. Similarly, when calculating the elements of the next row, the same column of matrix B is repeatedly retrieved. This access pattern results in a large number of repetitive data loading operations on the input matrix, especially the rows of A and the columns of B.

[0035] However, while performing matrix multiplication in this way allows the same row of matrix A and the same column of matrix B to be used multiple times to calculate different output elements, the processor may need to reload this data from the limited cache each time it performs a new multiplication-accumulation operation, or even repeatedly read it from extremely slow external memory. This leads to two serious problems: first, memory bandwidth is used to transfer large amounts of repetitive data instead of supporting new calculations; second, the processor's powerful computing units spend most of their time "waiting" for data to arrive from external memory, remaining idle. Therefore, the main reason for the slow speed of matrix multiplication is not the speed of the multiplication-accumulation operation itself, but rather the huge memory access overhead caused by repeatedly fetching the same data, which makes the actual performance of the computing system far lower than its theoretical peak.

[0036] In view of the problems existing in related technologies, embodiments of this application provide a processor data processing method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can reduce the number of times the processor repeatedly reads data blocks from external memory and improve the speed of multiplication operations performed in the processor. The exemplary application of the processor data processing apparatus provided in the embodiments of this application is described below.

[0037] See Figure 1 , Figure 1 This is a schematic diagram of the structure of the electronic device 100 provided in the embodiments of this application. Figure 1 The illustrated electronic device 100 includes at least one processor 10, volatile memory 120, and non-volatile memory 130. Various components in the electronic device 100 are coupled together via a bus system 140. It is understood that the bus system 140 is used to implement communication between these components. In addition to a data bus, the bus system 140 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 1 The general labeled all buses as Bus System 140.

[0038] The processor 10 has signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor. The processor in this embodiment can be used as a domain-specific accelerator (DSA) for matrix multiplication.

[0039] Volatile memory 120, an exemplary hardware device, includes random access memory (RAM). Volatile memory 120 is capable of storing data to support various operations; examples of this data include programs, modules, and data structures, or subsets or supersets thereof. The memory mentioned below is volatile memory. Internal memory SRAM in subsequent embodiments is a type of volatile memory 120.

[0040] In some embodiments, the data processing apparatus of the processor provided in this application can be implemented in software. For example, the processor 10 operates by running... Figure 1 The data processing device 131 of the processor in the shown non-volatile memory 130 can be software in the form of programs and plug-ins, including the following software modules: a first calculation module 1311, a second calculation module 1312, and a result combination module 1313. These modules are logically related and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0041] See Figure 2 , Figure 2 This is a schematic diagram of the architecture of the processor 10 provided in an embodiment of this application, as shown below. Figure 2 As shown, the processor 10 may include internal memory 111, a first thread 112, a second thread 113, a multiplication unit 114, and a write thread 115. The first thread 112 and the second thread 113 can be combined into a thread group, wherein: The first thread 112 is used to read data from external memory and load the data from external memory into internal memory 111.

[0042] Internal memory 111 is used to temporarily store data blocks for reading by the second thread 113 and the write thread 115.

[0043] The second thread 113 is used to read data from the internal memory 111 and transfer the read data to the multiplication unit 114.

[0044] The multiplication unit 114 is used to perform multiplication operations on the data read from the internal memory 111 by the second thread 113, and the result of the multiplication operation will be stored in the internal memory 111.

[0045] Write thread 115 is used to transfer the multiplication result in internal memory 111 to external memory.

[0046] In this embodiment of the application, the multiplication unit 114 can perform parallel multiplication operations on data blocks read from external memory through multiple thread groups in the processor 10 to obtain the corresponding multiplication results.

[0047] See Figure 2 When processor 10 receives a request to perform a multiplication operation on the first matrix and the second matrix in external memory, processor 10, through the first thread 112 in the thread group, loads the m-th data block of the first matrix from external memory into the first cache location of internal memory 111, and loads the n-th data block of the second matrix from external memory into the second cache location of internal memory. Here, the first matrix and the second matrix are the matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). Processor 10 responds to the first cache location and the second cache location of internal memory 111. The processor 10 loads a data block, and through the first thread 112, loads the (n+1)th data block in the second matrix from external memory into the third cache location of internal memory 111. Then, through the second thread 113 in the thread group, it performs a multiplication operation on the data block in the first cache location and the data block in the second cache location to obtain a first multiplication operation result. The processor 10, through the first thread 112, loads the (n+1)th data block in the second matrix from external memory into the second cache location of internal memory. Then, through the second thread, it performs a multiplication operation on the data block in the first cache location and the data block in the third cache location to obtain a second multiplication operation result. The processor 10 combines the first multiplication operation result and the second multiplication operation result into the processor's data processing result.

[0048] As an example, in computational photography, medical imaging, or security surveillance, there is a need for real-time filtering, edge detection, or style transfer processing of high-resolution images (such as 4K or even 8K footage). The core algorithms for this type of processing, such as convolution computation, can essentially be transformed into intensive general-purpose matrix multiplication. By loading the data to be processed into the processor's internal memory and performing multiplication operations on the data in internal memory, multiplication-addition operations can be performed with extremely low power consumption and latency, achieving instantaneous convolution operations between each frame's image pixels and the filtering kernel. This enables high-end smartphones or drones to render video streams with complex artistic effects in real time, or allows medical imaging equipment to sharpen and enhance 3D images in a very short time, without consuming large amounts of resources and power from a general-purpose processor.

[0049] As an example, in live video streaming, cloud gaming, and large-scale multimedia data centers, video encoding (such as H.265 / HEVC) and decoding are core tasks. Key steps such as discrete cosine transform, quantization, and motion compensation all rely on massive multiplication operations. By loading the data to be processed into the processor's internal memory and performing multiplication operations on the data in internal memory, multiply-accumulate operations can be performed with extremely low power consumption and latency. This can increase video transcoding throughput several times over while significantly reducing the overall power consumption of the server. This allows streaming media service providers to offer users higher-definition and smoother 4K / 8K video-on-demand and live streaming services at lower operating costs.

[0050] See Figure 3 , Figure 3 This is a flowchart illustrating the data processing method of the processor provided in the embodiments of this application. Figure 1 , will combine Figure 3 The steps shown are explained as follows: Figure 3 As shown, the data processing method of the processor is described using the processor as the execution subject. The method includes the following steps 101 to 104.

[0051] The processor includes internal memory and at least one thread group, each thread group including at least two threads.

[0052] In step 101, the first thread in the processor's thread group loads the m-th data block in the first matrix from external memory into the first cache location of the internal memory, and loads the n-th data block in the second matrix from external memory into the second cache location of the internal memory.

[0053] Wherein, the first matrix and the second matrix are the matrices to be multiplied, m is the count of the data blocks in the first matrix, which are arranged in row-major order, and n is the count of the data blocks in the second matrix, which are arranged in column-major order.

[0054] Here, the first and second matrices are operand matrices for the multiplication operation, typically representing input data, weight parameters, or transformation basis matrices. In matrix multiplication, the first matrix acts as the multiplicand, and its column count must match the row count of the second matrix to ensure the validity of the multiplication operation. The first and second matrices can include numerical data, feature representations, or model parameters, typically derived from the original data through initialization, training, or training. In machine learning implementations, the first and second matrices are commonly found in fully connected layers, convolutional layers, or attention mechanisms of neural networks, serving as weight matrices or input feature matrices.

[0055] For example, let's take the self-attention mechanism in the Transformer model as an example. In the Transformer model, the query matrix Q is used as the first matrix, and the transpose of the key matrix K is... Considered as a second matrix. Receiving Q and the matrix. Multiplication can calculate a set of attention scores, which can then represent the correlation strength between different positions in the sequence.

[0056] Here, external memory is a storage medium located outside the processor chip. It is specifically used to store data that is too large to be fully loaded into the processor's internal memory for computation, such as the first and second matrices. External memory acts as a persistent repository for massive amounts of data. Its characteristics include a storage capacity much larger than that of main memory, but a relatively slow data access speed. When performing large-scale matrix multiplication, the computing system needs to use a specific data scheduling strategy to load the matrices stored in external memory into main memory in blocks for computation, thereby supporting the entire computational task.

[0057] For example, when training a very large-scale neural network, the total size of its fully connected layer weight matrix (as the first matrix, dimension 50,000 × 50,000) and input feature matrix (as the second matrix, dimension 50,000 × 10,000) can reach tens of gigabytes, far exceeding the memory capacity of a server. In this case, these two matrices are completely stored in an external storage device called a solid-state drive. The computation process is broken down into multiple steps: first, several row blocks of the first matrix and several column blocks of the second matrix are read from the external storage into the internal storage; then, these data blocks are retrieved from the internal storage and used in the multiplication unit to obtain the multiplication results of each data block; finally, the multiplication results of all data blocks are integrated.

[0058] Here, the first thread is a thread specialized specifically for acting as a "producer." Specializing the first thread as a producer means that in a concurrent programming architecture, it is not responsible for business logic calculations, but rather focuses on efficiently retrieving data blocks from the data source (external memory) for use by other "consumer" threads. The system can dynamically create and manage multiple first threads, which can concurrently and simultaneously read data from different areas of external memory, thereby achieving high-bandwidth data loading. For example, the first matrix can be divided into four regions, with each first thread reading data blocks from one region.

[0059] Here, the first cache location and the second cache location are different cache locations in the internal memory. Different cache locations are used to store data blocks from different matrices.

[0060] Here, loading data blocks in column-major order means loading the elements of one column before loading the next column, while loading data blocks in row-major order means loading the elements of one row before loading the next row. When performing matrix multiplication, to efficiently utilize the cache, the data reading method must match the storage order: if the matrix is ​​stored in row-major order, the inner loop should traverse the columns of the right matrix (i.e., read by row) to ensure access to contiguous memory; if stored in column-major order, the inner loop should traverse the rows of the left matrix (i.e., read by column). The core principle is to maximize performance by having the innermost loop operate on contiguous memory blocks.

[0061] In step 102, in response to the loading of data blocks in the first and second cache locations of the internal memory, the (n+1)th data block in the second matrix is ​​loaded from the external memory into the third cache location of the internal memory through the first thread, and the second thread in the thread group performs a multiplication operation on the data blocks in the first and second cache locations to obtain the first multiplication result.

[0062] Here, when data blocks are loaded into the first and second cache locations of the internal memory, it indicates that the computation of the elements in the data blocks is ready. Taking storing the first data block of the first matrix in the first cache location and storing the first data of the second matrix in the second cache location as an example, the processor system can proceed to the next operation by issuing the 0th step synchronization instruction group.sync(0) as a ready signal, through asynchronous pipelined synchronization within the thread group. This method ensures that all write operations within the group are completed before proceeding to the next computation or read operation, thus ensuring the integrity of the data involved in the operation.

[0063] In this embodiment, the second thread, after parsing the synchronization instruction group.sync(0) in step 0 and confirming that the data is ready, can call the data in the internal memory and perform multiplication operations in the multiplication unit. Continuing with the above embodiment, the first thread loads the data block at the second position of the second matrix from the external memory into the third cache position and issues the synchronization signal group.sync(1) in step 1. In this embodiment, the second cache position and the third cache position are the ping cache position and pong cache position of the data block in the second matrix, respectively.

[0064] Ping and pong cache locations refer to a pair of mutually buffered storage areas, often used to achieve efficient parallel data processing. Specifically, when the second and third cache locations are mutually ping and pong cache locations, they form a ping-pong buffer structure: one cache location (e.g., ping) is used for current data reading or processing, while the other cache location (e.g., pong) is simultaneously used for writing or loading the next batch of data. This design allows the system to work continuously without waiting for data transfer to complete, thereby improving throughput and performance.

[0065] For example, in multiplication operations during graphics processing or data stream processing, the second cache location acts as a ping buffer, allowing the GPU to read matrices representing texture data from external memory; simultaneously, the third cache location acts as a pong buffer, storing the data for the next frame to be processed. After processing the data in the ping buffer, the roles switch—the third cache location becomes a ping buffer for reading, and the second cache location becomes a pong buffer for writing, and so on. This ping-pong mechanism ensures the continuity and efficiency of data access, avoiding resource conflicts.

[0066] It should be noted that, since the elements in the first matrix need to be repeatedly involved in the calculation during multiplication, and the first thread takes a long time to load data blocks from external memory, this embodiment of the application retains the data blocks that need to be repeatedly calculated (data blocks from the first matrix) in the internal memory when performing multiplication operations on data blocks, and preloads the data blocks in the second matrix that need to be calculated in the next round, thereby reducing the number of times the data blocks of the first matrix are repeatedly called, thereby improving the speed of multiplication operations in the processor.

[0067] In step 103, the first thread loads the (n+1)th data block in the second matrix from the internal memory into the second cache location of the internal memory, and the second thread performs a multiplication operation on the data block in the first cache location and the data block in the third cache location to obtain the second multiplication result.

[0068] Here, after obtaining the result of the first multiplication operation, the first thread has loaded the data block that needs to be multiplied in the next round into the internal memory. At this time, the second thread can perform multiplication on the data that has been loaded into the internal memory, and the first thread can load the data block required for the next round of multiplication into the internal memory so that the second thread can perform the multiplication operation.

[0069] In some embodiments, the thread group may include a first thread and multiple second threads, in which case the multiplication operation can be performed by executing the following steps.

[0070] First, the first thread in the thread group performs a multiplication operation on the data blocks in the first cache location and the data blocks in the second cache location.

[0071] Then, the second thread in the thread group performs a multiplication operation on the data blocks in the first cache location and the data blocks in the second cache location.

[0072] The core reason why this embodiment of the application can make full use of SRAM resources by setting up one producer and two consumers in the thread group is that this structure realizes efficient pipeline parallelism, bandwidth optimization and dynamic load balancing: the producer can continuously write data to a specific area of ​​SRAM, while the two consumers can alternately or in parallel read and process data from other areas of SRAM; this design allows the data supply, calculation and output stages to overlap, and the read and write ports and storage units of SRAM can be continuously occupied, avoiding resource idleness or blocking waiting caused by the mismatch of the processing speed of a single consumer, thereby transforming the high bandwidth characteristics of SRAM into an actual increase in data throughput and maximizing its performance potential.

[0073] In step 104, the results of the first multiplication operation and the second multiplication operation are combined into the data processing result of the processor.

[0074] This application embodiment reduces the number of times data is retrieved from external memory by reusing data blocks loaded into internal memory during processor multiplication operations, resulting in triple benefits in performance, power consumption, and system complexity. By reusing data, the computing unit operates continuously instead of waiting for data, shifting the system performance bottleneck from low-speed storage bandwidth to high-speed computing units, thus improving data throughput. It also reduces the energy consumption required to drive high-power off-chip memory interfaces, optimizing energy efficiency, reducing reliance on high-speed storage interfaces and complex control logic, and simplifying the overall system design. Furthermore, by alternating the computation of data blocks in the second and third cache locations, the probability of threads performing data computation waiting for data preparation is reduced, thereby improving the efficiency of multiplication operations.

[0075] In some embodiments, see Figure 4 , Figure 4 It is shown that before step 104, the first multiplication result and the second multiplication result can also be obtained by performing steps 105 to 107.

[0076] In step 105, iterative processing is performed.

[0077] The iterative process involves alternating between the first and second multiplication operations to obtain the results.

[0078] In some embodiments, the second matrix includes K×N data blocks, in which case the second multiplication result and the first multiplication result can be obtained by performing steps 1051 and 1052.

[0079] Where K is the number of rows of the data block in the second matrix, and N is the count of data blocks in each row of the second matrix.

[0080] In response to the fact that the nth data block used to calculate the result of the first multiplication operation is the k1×Nth data block, the first data block in the k1+1th row is taken as the n+1th data block, and the result of the second multiplication operation is calculated based on the n+1th data block and the mth data block, where k1 is the count of the number of data block rows in the second matrix.

[0081] Here, the second thread can read the (n+1)th data block and the mth data block from the internal memory into the multiplication unit (GEMM) for multiplication to obtain the second multiplication result.

[0082] or, In response to the fact that the (n+1)th data block used to calculate the result of the second multiplication operation is the (k1×N)th data block, the first data block of the (k1+1)th row is taken as the (n+2)th data block, and the result of the first multiplication operation in the next round of iteration is calculated based on the (n+2)th data block and the (m)th data block.

[0083] Here, the second thread reads the first data block of the k1+1th row and the mth data block from the internal memory and performs multiplication in the multiplication unit (GEMM) to obtain the first multiplication result and complete the line break to retrieve the elements in the second matrix.

[0084] This application embodiment reuses data blocks in the first matrix. After a data block in the first matrix is ​​multiplied by a data block in the second matrix, it is multiplied by the next data block in the second matrix. This allows for multiplication with multiple data blocks in the second matrix to be completed with only one loading for each data block in the first matrix. Compared to the multiplication method that traverses the position of each output matrix, this reduces the calls to data blocks in the first matrix, alleviates the bandwidth pressure on the internal memory, and improves computational efficiency.

[0085] In some embodiments, the results of the second multiplication operation and the first multiplication operation can also be obtained by performing the following steps.

[0086] In response that the nth data block used to calculate the result of the first multiplication operation is the K×Nth data block, n is reset to 0, and the result of the second multiplication operation is calculated based on the (m+1)th data block and the nth data block.

[0087] Here, the second thread reads the (m+1)th data block from the internal memory and the nth data block, which is reset to 0, and reads them into the multiplication unit (GEMM) to perform multiplication, obtain the first multiplication result, and implements the second multiplication operation by calling the next element in the first matrix and performing the second multiplication operation between the next element and all elements in the second matrix after completing the multiplication operation between one element in the first matrix and all elements in the second matrix.

[0088] or, In response that the (n+1)th data block used to calculate the result of the second multiplication operation is the K×Nth data block, n is reset to 0, and the result of the first multiplication operation in the next iteration is calculated based on the (m+1)th data block and the nth data block.

[0089] Here, the second thread reads the (m+1)th data block from the internal memory and the nth data block, which is reset to 0, and reads them into the multiplication unit (GEMM) to perform multiplication, obtain the second multiplication result, and implements the following after completing the multiplication operation of one element of the first matrix with all elements of the second matrix, calling the next element in the first matrix and performing the first multiplication operation of the next element with all elements of the second matrix.

[0090] This application embodiment reuses data blocks in the first matrix. After a data block in the first matrix is ​​multiplied with all data blocks in a row of the second matrix, it is then multiplied with all data blocks in the next row of the second matrix. This enables the multiplication of data in the first matrix with data blocks in multiple rows of the second matrix to be completed with only one loading. Compared with the multiplication method that traverses the position of each output matrix, this reduces the calls to data blocks in the first matrix, alleviates the bandwidth pressure on the internal memory, and improves computational efficiency.

[0091] In some embodiments, the first matrix may include M×K data blocks. In this case, the second multiplication result and the first multiplication result can be obtained by performing the following steps.

[0092] Where M is the number of data blocks in each column of the first matrix.

[0093] In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the m+1-th data block and the n-th data block.

[0094] Here, the second thread reads the first data block of column k2+1 from the internal memory and the nth data block with n reset to 0, and reads them into the multiplication unit (GEMM) for multiplication to obtain the second multiplication result, thereby realizing the column swapping of the first matrix.

[0095] or, In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the m+1-th data block and the n-th data block.

[0096] Here, the second thread reads the first data block of column k2+1 and the nth data block with n reset to 0 from the internal memory and reads them into the multiplication unit (GEMM) to perform multiplication operations, obtain the first multiplication result, and then realize the column swapping of the first matrix.

[0097] This embodiment of the application reuses data blocks in the first matrix. After a data block in the first matrix is ​​multiplied with all data blocks in the second matrix, the next data block in the first matrix is ​​multiplied with all data blocks in the second matrix. This achieves that each data block in the first matrix is ​​loaded only once during the entire matrix calculation process, reducing the calls to data blocks in the first matrix, alleviating the bandwidth pressure on the internal memory, and improving the calculation efficiency.

[0098] In step 106, in response to the fact that the sum of the number of the first multiplication result and the second multiplication result is less than the set number of data blocks of the data processing result, the iterative processing continues.

[0099] In step 107, in response to the sum of the number of the first multiplication operation results and the second multiplication operation results being equal to the set number of data blocks of the data processing result, the iterative processing ends, and the first multiplication operation results and the second multiplication operation results obtained in each iteration are combined to form the processor's data processing result.

[0100] This application's embodiments transform large-scale matrix computations into cache-friendly local operations, thereby reducing the number of data loads required for these local operations. The method divides the massive input matrix (a first matrix and a second matrix) into manageable-size data blocks. During computation, each element in the first matrix is ​​loaded only once, and each computation only requires loading data blocks from the second matrix. The next data block from the first matrix is ​​loaded only after the data blocks in the first matrix have been multiplied by all data blocks in the second matrix. In this way, once a data block in the first matrix is ​​loaded, it can be repeatedly and efficiently accessed by the computation unit in the cache for calculating the corresponding part of the output matrix, thus reducing the number of external memory accesses and greatly alleviating memory bandwidth pressure. Simultaneously, this block structure naturally decouples data dependencies between different computation units, allowing the computation of multiple blocks to be performed in parallel. This fully leverages the parallel computing capabilities of multi-core processors or dedicated accelerators. Furthermore, by combining this with pipeline techniques such as double buffering, a perfect overlap between data loading and computation is achieved, ultimately resulting in an order-of-magnitude improvement in overall computational throughput and energy efficiency.

[0101] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0102] In related technologies, when performing general matrix multiplication operations using a general-purpose processor such as a CPU, it is typically necessary to traverse every element of the input matrix and calculate each element to obtain each element of the output matrix C. For each element C[i][j] in matrix C, the processor needs to load the i-th row of matrix A and the j-th column of matrix B from external memory, then multiply and accumulate the corresponding elements in this row and column. When the processor then calculates the next element C[i][j+1] in the same row, it must again retrieve the same row of matrix A (the data just used) and retrieve the next column of matrix B from memory. Similarly, when calculating the elements of the next row, the same column of matrix B is repeatedly retrieved. This access pattern results in a large number of repetitive data loading operations on the input matrix, especially the rows of A and the columns of B.

[0103] Figure 5 This is a flowchart illustrating an application scenario provided in an embodiment of this application. Figure 1 See Figure 5 This includes the following steps 201 to 204.

[0104] In step 201, thread Thread0 (i.e., the first thread in the above embodiment) loads the first data block A00 in the left matrix (i.e., the first matrix in the above embodiment) from DRAM (i.e., the external memory in the above embodiment) to the ping buffer (i.e., the first cache location in the above embodiment) of SRAM (i.e., the internal memory in the above embodiment), loads the first data block B00 in the right matrix (i.e., the second matrix in the above embodiment) to SRAM, and issues the 0th step ready signal group.sync(0).

[0105] In the embodiments of this application, see Figure 6 , Figure 6 This illustrates the principle of reading data from GEMM into SRAM. Figure 6 In this matrix, the left matrix is ​​an M-row, K-column matrix, and the right matrix is ​​a K-row, N-column matrix. Each thread slides its read window between the left and right matrices, reading the data blocks that the window passes through into the SRAM at the same row and column as the left matrix. In step 202, after detecting the ready signal of step 0, thread Thread1 (i.e. the second thread in the above embodiment) performs a multiplication operation on A00 and B00 in SRAM to complete a ping operation.

[0106] In step 203, the second data block B01 in the right matrix is ​​loaded from DRAM to the pong buffer of SRAM (i.e., the third buffer position in the above embodiment) by thread Thread0, and the first step ready signal is issued.

[0107] In step 204, after detecting the ready signal of step 1, thread Thread1 performs a multiplication operation on A00 and B01 in SRAM to complete a pong operation.

[0108] Figure 7 This is a schematic diagram illustrating the positional principle of performing multiplication operations on data blocks of a matrix in SRAM, as provided in the embodiments of this application. Figure 1 See Figure 7 By performing an operation on the data block in the m-th row of the left matrix (including the elements in column ki) and the data block in the n-th column of the right matrix (including the elements in row ki) using the method of the embodiments of this application, the data block in the m-th row and n-th column of the output matrix (including the elements in column mi and row ni, where mi and ni can be any positive integers) can be obtained. Figure 8 This is the time distribution of operations performed by each thread group in the processor provided in the embodiments of this application. Figure 1 See Figure 8 After Thread0 writes the data needed for the next round of GEMM into SRAM, it begins writing the data needed for the next round of GEMM into the next location in SRAM. Once the data in SRAM is ready, Thread1 executes the GEMM calculation. After completing the GEMM calculation for the current round, the data needed for the next round of GEMM has also been loaded into SRAM. Furthermore, only one loading of data block A from the left matrix is ​​needed to complete the multiplication operation between this data block and multiple data blocks from the right matrix.

[0109] Figure 9 This is a flowchart illustrating an application scenario provided in an embodiment of this application. Figure 2 See Figure 9 This includes the following steps 301 to 304.

[0110] In step 301, the first data block A00 in the left matrix is ​​loaded from DRAM to the ping buffer of SRAM by thread Thread0, the first data block B00 in the right matrix is ​​loaded into SRAM, and the 0th step ready signal group.sync(0) is issued.

[0111] In step 302, after detecting the ready signal of step 0, thread Thread1 performs a multiplication operation on A00 and B00 in SRAM to complete a ping operation.

[0112] In step 303, thread Thread0 loads the second data block in the left matrix from DRAM into the pong buffer in SRAM and issues the first step ready signal group.sync(1).

[0113] In step 304, after detecting the ready signal of step 1, thread Thread2 performs a multiplication operation on A00 and B01 in SRAM to complete a pong operation.

[0114] Figure 10 This is a schematic diagram illustrating the positional principle of performing multiplication operations on data blocks of a matrix in SRAM, as provided in the embodiments of this application. Figure 2 See Figure 10 By performing a ping-pong operation on the data block in the m-th row of the left matrix and the data blocks in the n-th and (n+1)-th columns of the right matrix using the method of this application embodiment, the data block in the m-th row and (n+1)-th column of the output matrix can be obtained. Figure 11 The time distribution of operations performed by each thread group in the processor provided in the embodiments of this application is as follows: Figure 2 See Figure 11 After Thread0 writes the data needed for the next round of GEMM into SRAM, it begins writing the data needed for the next round of GEMM into the next location in SRAM. Once the data in SRAM is ready, Thread1 executes the GEMM calculation. After the data needed for the next round of GEMM is written into SRAM, Thread1 has not yet completed the multiplication operation; at this point, Thread2 executes the GEMM operation. After Thread1 and Thread2 each complete their current round of GEMM calculation, the data needed for their respective next round of GEMM has also been loaded into SRAM. Furthermore, only one loading of the data block A of the left matrix is ​​needed to complete the multiplication operation between this data block and multiple data blocks of the right matrix.

[0115] In some embodiments, the data processing apparatus of the processor provided in this application can be implemented in software. For example, the processor 10 operates by running... Figure 1 The data processing device 131 of the processor in the shown non-volatile memory 130 can be software in the form of programs and plug-ins, including the following software modules: a first calculation module 1311, a second calculation module 1312, and a result combination module 1313. These modules are logically related and can therefore be arbitrarily combined or further divided according to the functions they implement. The functions of each module will be described below.

[0116] The first computing module 1311 is used to load the m-th data block in the first matrix from external memory into the first cache location of the internal memory, and load the n-th data block in the second matrix from external memory into the second cache location of the internal memory, via the first thread in the thread group of the processor. The first and second matrices are matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). In response to the data blocks being loaded into the first and second cache locations of the internal memory, the first thread loads the (n+1)-th data block in the second matrix from external memory into the third cache location of the internal memory, and the second thread in the thread group performs a multiplication operation on the data blocks in the first and second cache locations to obtain the first multiplication result. The second calculation module 1312 is used to load the (n+1)th data block in the second matrix from the external memory into the second cache location of the internal memory through the first thread, and to perform a multiplication operation on the data block in the first cache location and the data block in the third cache location through the second thread to obtain the second multiplication operation result; The result combination module 1313 is used to combine the results of the first multiplication operation and the second multiplication operation into the data processing result of the processor.

[0117] In some embodiments, the result combination module 1313 is further configured to perform a multiplication operation on the data block in the first cache location and the data block in the second cache location by the first second thread in the thread group when the thread group includes a first thread and multiple second threads. as well as, The second thread in the thread group performs a multiplication operation on the data blocks in the first cache location and the data blocks in the second cache location.

[0118] In some embodiments, the result combination module 1313 is further configured to perform iterative processing before combining the first multiplication result and the second multiplication result into the processor's data processing result, wherein the iterative processing includes alternatingly executing the process of obtaining the first multiplication result and obtaining the second multiplication result; The processor's data processing result, which combines the results of the first and second multiplication operations, includes: If the sum of the results of the first and second multiplication operations is less than the set number of data blocks for the data processing result, the iterative processing continues. In response to the sum of the first and second multiplication results being equal to the set number of data blocks for the data processing result, the iterative processing ends, and the first and second multiplication results obtained in each iteration are combined to form the processor's data processing result.

[0119] In the above scheme, the result combination module 1313 is also used to perform the following operation when the second matrix includes K×N data blocks, where K is the number of rows of the data blocks in the second matrix and N is the count of data blocks contained in each row of the data blocks: In response to the fact that the nth data block used to calculate the result of the first multiplication operation is the k1×Nth data block, the first data block in the k1+1th row is taken as the n+1th data block, and the result of the second multiplication operation is calculated based on the n+1th data block and the mth data block, where k1 is the count of the number of data block rows in the second matrix; In response to the fact that the (n+1)th data block used to calculate the result of the second multiplication operation is the (k1×N)th data block, the first data block of the (k1+1)th row is taken as the (n+2)th data block, and the result of the first multiplication operation in the next round of iteration is calculated based on the (n+2)th data block and the (m)th data block.

[0120] In some embodiments, the result combination module 1313 is further configured to, in response to the fact that the nth data block used to calculate the result of the first multiplication operation is the K×Nth data block, reset n to 0, and calculate the result of the second multiplication operation based on the (m+1)th data block and the nth data block; In response that the (n+1)th data block used to calculate the result of the second multiplication operation is the K×Nth data block, n is reset to 0, and the result of the first multiplication operation in the next iteration is calculated based on the (m+1)th data block and the nth data block.

[0121] In some embodiments, the result combination module 1313 is further configured to perform the following when the first matrix includes M×K data blocks, where M is the count of data blocks contained in each row of data blocks: In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the m+1-th data block and the n-th data block. In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the m+1-th data block and the n-th data block.

[0122] In some embodiments, the result combination module 1313 is further configured to, in response to the fact that the nth data block used to calculate the result of the first multiplication operation is the K×Nth data block, reset n to 0, and calculate the result of the second multiplication operation based on the (m+1)th data block and the nth data block; In response that the (n+1)th data block used to calculate the result of the second multiplication operation is the K×Nth data block, n is reset to 0, and the result of the first multiplication operation in the next iteration is calculated based on the (m+1)th data block and the nth data block.

[0123] In some embodiments, the result combination module 1313 is further configured to perform the following when the first matrix includes M×K data blocks, where M is the count of data blocks contained in each row of data blocks: In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the m+1-th data block and the n-th data block. In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the m+1-th data block and the n-th data block.

[0124] In some embodiments, the result combination module is further configured to perform the following when the first matrix includes M×K data blocks, where M is the count of data blocks contained in each row of data blocks: In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the m+1-th data block and the n-th data block. In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the m+1-th data block and the n-th data block.

[0125] In some embodiments, the result combination module 1313 is further configured to, in response to the fact that the nth data block used to calculate the result of the first multiplication operation is the K×Nth data block, reset n to 0 and calculate the result of the second multiplication operation based on the (m+1)th data block and the nth data block; In response that the (n+1)th data block used to calculate the result of the second multiplication operation is the K×Nth data block, n is reset to 0, and the result of the first multiplication operation in the next iteration is calculated based on the (m+1)th data block and the nth data block.

[0126] In some embodiments, the result combination module 1313 is further configured to perform the following when the first matrix includes M×K data blocks, where M is the count of data blocks contained in each row of data blocks: In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the m+1-th data block and the n-th data block. In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block of the k2+1-th column is taken as the m+1-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the m+1-th data block and the n-th data block.

[0127] This application provides a computer program product, which includes a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the data processing method of the processor described in this application embodiment.

[0128] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will perform the data processing method provided in this application. For example, ... Figure 3 The processor's data processing method is shown.

[0129] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0130] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0131] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0132] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0133] In summary, through the embodiments of this application, the first thread in the processor loads the data block of the first matrix from external memory into the first cache location of internal memory, and loads the data block of the second matrix into the second cache location of internal memory. While the second thread performs calculations on the data blocks in the first and second cache locations, the first thread loads the next data block of the second matrix into the third cache location. This utilizes the characteristic that elements in the first matrix are reused multiple times during matrix multiplication, reducing the number of times the processor reads the data blocks of the first matrix and improving the speed of multiplication operations in the processor. Furthermore, by using dual-caching technology to process the data blocks of the second matrix, the calculation speed of multiplication operations in the processor is further improved. By reusing data blocks loaded into internal memory during processor multiplication operations, the number of times data is retrieved from external memory is reduced, resulting in a triple benefit in performance, power consumption, and system complexity. Firstly, by reusing data, the computation unit operates continuously instead of waiting for data, shifting the system performance bottleneck from low-speed memory bandwidth to high-speed computation units, thus improving data throughput. Secondly, it reduces the energy consumption required to drive high-power off-chip memory interfaces, optimizing energy efficiency, reducing reliance on high-speed memory interfaces and complex control logic, and simplifying the overall system design. Furthermore, by alternating computations on data blocks in the second and third cache locations, the likelihood of threads performing data computations waiting for data preparation is reduced, thereby improving the efficiency of multiplication operations.

[0134] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. A data processing method for a processor, characterized in that, The processor includes internal memory and at least one thread group, each thread group including at least two threads, and the method includes: The first thread in the processor's thread group loads the m-th data block from the first matrix into the first cache location of the internal memory from the external memory, and loads the n-th data block from the second matrix into the second cache location of the internal memory from the external memory. The first matrix and the second matrix are matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). In response to the data blocks being loaded in the first and second cache locations of the internal memory, the first thread loads the (n+1)th data block in the second matrix from the external memory into the third cache location of the internal memory, and the second thread in the thread group performs the multiplication operation on the data blocks in the first and second cache locations to obtain the first multiplication result. The first thread loads the (n+1)th data block in the second matrix from the internal memory into the second cache location of the internal memory, and the second thread performs the multiplication operation on the data block in the first cache location and the data block in the third cache location to obtain the second multiplication result. The result of the first multiplication operation and the result of the second multiplication operation are combined to form the data processing result of the processor.

2. The method according to claim 1, characterized in that, The thread group includes one first thread and multiple second threads; The step of performing the multiplication operation on the data block in the first cache location and the data block in the second cache location through the second thread in the thread group includes: The multiplication operation is performed on the data block in the first cache location and the data block in the second cache location by the first second thread in the thread group; The step of performing the multiplication operation on the data block in the first cache location and the data block in the third cache location through the second thread includes: The second thread in the thread group performs the multiplication operation on the data block in the first cache location and the data block in the second cache location.

3. The method according to claim 1, characterized in that, The method further includes: Before combining the first multiplication result and the second multiplication result into the data processing result of the processor, an iterative process is performed, wherein the iterative process includes alternately executing the process of obtaining the first multiplication result and obtaining the second multiplication result; The step of combining the first multiplication result and the second multiplication result to form the data processing result of the processor includes: If the sum of the number of the first multiplication result and the second multiplication result is less than the set number of data blocks of the data processing result, the iterative processing continues; In response to the sum of the number of the first multiplication operation results and the second multiplication operation results being equal to the set number of data blocks of the data processing result, the iterative processing ends, and the first multiplication operation results and the second multiplication operation results obtained in each iteration are combined to form the data processing result of the processor.

4. The method according to claim 3, characterized in that, The second matrix includes K×N data blocks, where K is the number of rows of the data blocks in the second matrix, and N is the count of data blocks contained in each row of the data blocks; The method further includes: In response to the fact that the nth data block used to calculate the result of the first multiplication operation is the k1×Nth data block, the first data block in the k1+1th row is taken as the n+1th data block, and the second multiplication operation result is calculated based on the n+1th data block and the mth data block, where k1 is the count of the number of data block rows in the second matrix; In response to the fact that the (n+1)th data block used to calculate the result of the second multiplication operation is the (k1×N)th data block, the first data block in the (k1+1)th row is taken as the (n+2)th data block, and the result of the first multiplication operation in the next round of iteration is calculated based on the (n+2)th data block and the (m)th data block.

5. The method according to claim 4, characterized in that, The method further includes: In response to the fact that the nth data block used to calculate the result of the first multiplication operation is the K×Nth data block, n is reset to 0, and the second multiplication operation result is calculated based on the (m+1)th data block and the nth data block; In response that the (n+1)th data block used to calculate the result of the second multiplication operation is the K×Nth data block, n is reset to 0, and the result of the first multiplication operation in the next round of iteration is calculated based on the (m+1)th data block and the nth data block.

6. The method according to claim 4, characterized in that, The first matrix includes M×K data blocks, where M is the count of data blocks contained in each row of data blocks; The method further includes: In response to the fact that the m-th data block used to calculate the result of the first multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block in column k2+1 is taken as the (m+1)-th data block, and n is reset to 0. The result of the second multiplication operation is calculated based on the (m+1)-th data block and the n-th data block. In response to the fact that the m-th data block used to calculate the result of the second multiplication operation is the M×k2-th data block and the n-th data block is the k1×N-th data block, the first data block in the k2+1-th column is taken as the (m+1)-th data block, and n is reset to 0. The result of the first multiplication operation in the next iteration round is calculated based on the (m+1)-th data block and the n-th data block.

7. A data processing apparatus for a processor, characterized in that, The processor includes internal memory and at least one thread group, each thread group including at least two threads, and the device includes: A first computing module is configured to load the m-th data block of a first matrix from external memory into a first cache location of internal memory via a first thread in the thread group of the processor, and load the n-th data block of a second matrix from external memory into a second cache location of internal memory. The first and second matrices are matrices to be multiplied, m is the count of the data blocks in the first matrix (arranged in row-major order), and n is the count of the data blocks in the second matrix (arranged in column-major order). In response to the data blocks being loaded into the first and second cache locations of internal memory, the first thread loads the (n+1)-th data block of the second matrix from external memory into a third cache location of internal memory, and the second thread in the thread group performs the multiplication operation on the data blocks in the first and second cache locations to obtain a first multiplication result. The second calculation module is used to load the (n+1)th data block in the second matrix from the external memory into the second cache location of the internal memory through the first thread, and to perform the multiplication operation on the data block in the first cache location and the data block in the third cache location through the second thread to obtain the second multiplication operation result; The result combination module is used to combine the first multiplication result and the second multiplication result into the data processing result of the processor.

8. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions or computer programs. A processor, when executing computer-executable instructions or computer programs stored in the memory, implements the data processing method of the processor according to any one of claims 1 to 6.

9. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, they implement the data processing method of the processor according to any one of claims 1 to 6.

10. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by the processor, they implement the data processing method of the processor according to any one of claims 1 to 6.

Images