Matrix computation apparatus and matrix computation method

By employing memory affinity and cache alignment allocation methods under the NUMA architecture, the problem of low computational efficiency of sparse matrix vector multiplication is solved, achieving more efficient matrix computation.

WO2026233122A1PCT designated stage Publication Date: 2026-11-12HUAWEI TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2026/092504
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2025-05-08
Filing Date
2026-04-23
Publication Date
2026-11-12

AI Technical Summary

Technical Problem

The computational efficiency of sparse matrix-vector multiplication in existing technologies is low, resulting in low overall efficiency of iterative solution algorithms, especially with performance degradation issues in multi-threaded computation under NUMA architecture.

Method used

By adopting a memory-friendly allocation method, the block matrix is ​​allocated to the local memory near the processor core, and combined with a cache-aligned allocation method, the access efficiency of the matrix computing device is improved.

Benefits of technology

It improves the computational efficiency of sparse matrix-vector multiplication, reduces data movement across NUMA nodes, and enhances the performance of multi-threaded computation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2026092504_12112026_PF_FP_ABST
    Figure CN2026092504_12112026_PF_FP_ABST
Patent Text Reader

Abstract

Embodiments of the present application relate to the technical field of chips, and provide a matrix computation apparatus and a matrix computation method. The efficiency of matrix computation is improved by improving the access efficiency of matrix computation. The specific solution is as follows: the matrix computation apparatus comprises a first processor and a plurality of second processors. The first processor is configured to acquire an input vector and a coefficient matrix, and perform block processing on the input vector and the coefficient matrix to obtain a plurality of block matrices. The first processor is further configured to respectively store the plurality of block matrices in a plurality of local memories on the basis of architecture information of a plurality of NUMA nodes and a matrix allocation strategy. The matrix allocation strategy comprises: each block matrix corresponds to one thread, a processor core is bound to at least one thread, and the address of each block matrix is located within an address range of the local memory corresponding to the processor core. The processor core is configured to run the thread and load the block matrix in the local memory to perform matrix computation. The embodiments of the present application are directed to a matrix computation process.
Need to check novelty before this filing date? Find Prior Art

Description

Matrix calculation device and matrix calculation method

[0001] The present application claims priority to the Chinese patent application No. 202510596304.7, filed on May 8, 2025, and entitled "Matrix calculation device and matrix calculation method", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the chip technology field, and in particular to a matrix calculation device and a matrix calculation method. BACKGROUND

[0003] Sparse matrix vector multiplication (SPMV) is a basic operation in matrix calculation, and is also the most basic and critical operation in image processing, mathematical optimization and numerical calculation. In many fields, sparse matrix vector multiplication is one of the most critical operations at the bottom, such as in the iterative method of numerical calculation solver or the primal-dual hybrid gradient (PDHG) algorithm of the first-order algorithm of mathematical optimization, sparse matrix vector multiplication may account for 60% to 80% of the total time of the iterative solution algorithm end to end. Therefore, the calculation efficiency of sparse matrix vector multiplication determines the overall efficiency of the iterative solution algorithm, and how to improve the calculation efficiency of sparse matrix vector multiplication has become a problem to be solved. SUMMARY

[0004] Embodiments of the present application provide a matrix calculation device and a matrix calculation method, which improve the efficiency of matrix calculation by improving the access efficiency of matrix calculation.

[0005] To achieve the above object, the embodiments of the present application adopt the following technical solutions.

[0006] In a first aspect, the embodiments of the present application provide a matrix calculation device, the matrix calculation device comprising a first processor and a plurality of second processors, each second processor comprising a plurality of non-uniform memory access (NUMA) nodes, each NUMA node comprising a local memory and a plurality of processor cores. The first processor is configured to obtain an input vector and a coefficient matrix, and perform block processing on the input vector and the coefficient matrix to obtain a plurality of block matrices. The first processor is further configured to store the plurality of block matrices in the plurality of local memories based on architecture information of the plurality of NUMA nodes and a matrix allocation strategy, the matrix allocation strategy comprising: each block matrix corresponding to a thread, a processor core being bound to at least one thread, and an address of each block matrix being within an address range of a local memory corresponding to the processor core. The processor core is configured to run the thread to load the block matrix in the local memory for matrix calculation.

[0007] Compared with the allocation manner of allocating multiple block matrices to the local memory of a single processor or the allocation manner of randomly allocating to the memory, the first processor in the matrix computing device provided by the embodiment of the application uses the memory affinity allocation manner when allocating the addresses of the block matrices. That is, the first processor allocates the addresses of the block matrices according to the architecture information of the multiple NUMA nodes and the matrix allocation strategy. The first processor can perceive the architecture information of the underlying hardware through the architecture information of the multiple NUMA nodes, so as to allocate the block matrices to the local memory close to the processor core, thereby avoiding the problem that the processor core needs to read the block matrices from the remote memory when running the thread, improving the access efficiency of the processor core in executing the matrix computing operation, and thus improving the matrix computing efficiency of the matrix computing device, for example, improving the computing efficiency of the sparse matrix vector multiplication.

[0008] In a possible implementation, each processor core further includes a multi-level cache, and the matrix allocation strategy further includes a starting address of each block matrix in the local memory and a cache line alignment of the multi-level cache.

[0009] In this implementation, the first processor can further use the cache alignment allocation manner to further reduce the number of times of reading the block matrices from the memory, improve the access efficiency of the processor core, and improve the matrix computing efficiency of the matrix computing device.

[0010] In a possible implementation, the first processor further includes an allocation interface, and the first processor is further configured to, in a case where the allocation interface obtains a preset allocation condition, store the multiple block matrices to the multiple local memories based on the architecture information of the multiple NUMA nodes, the matrix allocation strategy, and the preset allocation condition.

[0011] In this implementation, the address computing device can further provide a user-defined strategy (i.e., the preset allocation condition), support the user to limit the strategy through an environment variable, and formulate a matrix allocation strategy scheme, thereby providing high flexibility for the allocation of the block matrices and further improving the matrix computing efficiency of the matrix computing device.

[0012] In a possible implementation, the architecture information of the multiple NUMA nodes includes at least one of the following: the number of the multiple second processors, the number of NUMA nodes of each second processor, the number of processor cores in each NUMA node, and the size of each level of cache of each processor core.

[0013] In this implementation, the architecture information of the multiple NUMA nodes is the architecture information of the underlying hardware. The first processor perceives the architecture information of the underlying hardware to provide support of the underlying hardware for the subsequent allocation of the block matrices, avoids performance degradation when executing multiple threads, and fully utilizes the performance of the underlying hardware.

[0014] In one possible implementation, the first processor is also used to acquire architectural information about multiple NUMA nodes.

[0015] In one possible implementation, the first processor is also used to acquire and summarize the results of matrix calculations by multiple processor cores to obtain an output vector.

[0016] Secondly, embodiments of this application provide a matrix calculation method applied to a matrix calculation device. The matrix calculation device includes a first processor and multiple second processors. Each second processor includes multiple Non-Uniform Memory Access Model (NUMA) nodes, and each NUMA node includes local memory and multiple processor cores. The matrix calculation method includes: the first processor acquiring an input vector and a coefficient matrix, and performing block processing on the input vector and coefficient matrix to obtain multiple block matrices. The first processor stores the multiple block matrices into multiple local memories based on the architecture information of the multiple NUMA nodes and a matrix allocation strategy. The matrix allocation strategy includes: each block matrix corresponds to a thread, a processor core is bound to at least one thread, and the address of each block matrix is ​​located within the address range of the local memory corresponding to the processor core. The processor core runs the thread and loads the block matrices in the local memory for matrix calculation.

[0017] In one possible implementation, each processor core also includes a multi-level cache, and the matrix allocation strategy further includes aligning the starting address of each block matrix in local memory with the cache line of the multi-level cache.

[0018] In one possible implementation, the first processor further includes an allocation interface, and the address calculation method further includes: when the allocation interface obtains preset allocation conditions, the first processor stores multiple block matrices into multiple local memories based on the architecture information of multiple NUMA nodes, matrix allocation strategy and preset allocation conditions.

[0019] In one possible implementation, the architectural information of the multiple NUMA nodes includes at least one of the following: the number of multiple second processors, the number of NUMA nodes for each second processor, the number of processor cores in each NUMA node, and the size of the cache per level for each processor core.

[0020] In one possible implementation, before the first processor acquires the input vector and coefficient matrix, the method further includes: the first processor acquiring architectural information of multiple NUMA nodes.

[0021] In one possible implementation, the first processor acquires and aggregates the results of matrix calculations by multiple processor cores to obtain an output vector.

[0022] Thirdly, embodiments of this application provide an electronic device including a circuit board and a matrix computing device as described in any of the first aspects, the matrix computing device being disposed on the circuit board.

[0023] Fourthly, embodiments of this application provide an electronic device including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the matrix calculation method in any of the above aspects and any possible implementations.

[0024] Fifthly, embodiments of this application provide a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the matrix calculation method in any of the above aspects and any possible implementations.

[0025] In one possible implementation, the computer-readable storage medium is a non-transitory computer-readable storage medium.

[0026] Sixthly, embodiments of this application provide a computer program product that, when run on a computer or processor, causes the computer or processor to execute the matrix calculation method in any of the above aspects and any possible implementations.

[0027] It is understood that any of the electronic devices, computer-readable storage media, or computer program products provided above can be applied to the address calculation device provided above. Therefore, the beneficial effects that can be achieved can be referred to the beneficial effects in the corresponding address calculation device, which will not be repeated here.

[0028] These or other aspects of this application will become more readily apparent in the following description. Attached Figure Description

[0029] Figure 1 is a schematic diagram of a CSR data format provided in an embodiment of this application;

[0030] Figure 2 is a flowchart of matrix data processing provided in an embodiment of this application;

[0031] Figure 3 is an optimized flowchart of SPMV calculation provided in an embodiment of this application;

[0032] Figure 4 is a performance graph of multiple threads provided in an embodiment of this application;

[0033] Figure 5 is a schematic diagram of a NUMA architecture provided in an embodiment of this application;

[0034] Figure 6 is a schematic diagram of matrix calculation allocation provided in an embodiment of this application;

[0035] Figure 7 is a schematic diagram of an address calculation device provided in an embodiment of this application;

[0036] Figure 8 is a schematic diagram of a matrix partitioning method provided in an embodiment of this application;

[0037] Figure 9 is a schematic diagram of a matrix allocation strategy provided in an embodiment of this application;

[0038] Figure 10 is a flowchart of a matrix allocation strategy provided in an embodiment of this application;

[0039] Figure 11 is a flowchart of a matrix calculation provided in an embodiment of this application;

[0040] Figure 12 is a performance graph of various algorithms provided in an embodiment of this application;

[0041] Figure 13 is a flowchart of a matrix calculation method provided in an embodiment of this application. Detailed Implementation

[0042] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the embodiments, unless otherwise stated, "multiple" means two or more.

[0043] Furthermore, the term "coupling" is used to refer to electrical connections, including direct connections via wires or terminals or indirect connections via other devices. Therefore, "coupling" should be considered a broad type of electronic communication connection.

[0044] It should be noted that, in this application, the terms "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or illustration. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or design solutions. Specifically, the use of terms such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0045] A sparse matrix is ​​a matrix in which the number of zero elements far exceeds the number of non-zero elements, and the distribution of non-zero elements is irregular. In contrast, a dense matrix is ​​a matrix in which non-zero elements constitute the majority.

[0046] The sparse matrix-vector multiplication operation is essentially a mathematical operation: y = Ax, where x and y are n-dimensional vectors, x is the input vector, and y is the output vector. In practical applications, x and y are generally dense vectors and stored using dense arrays. A is the coefficient matrix, which is generally a sparse matrix and uses different data structures in various applications. One of the most common data structures is the compressed sparse row (CSR) data format.

[0047] The CSR data format can be illustrated in Figure 1. Figure 1(a) shows an example matrix, and Figure 1(b) shows the storage format of the CSR data. The CSR data format can be characterized using three arrays: values, column indices, and row offsets. The column indices array and the values ​​array store the column indices and values ​​of the non-zero elements in the matrix, respectively. The storage order is traversed row by row, and the length is the number of non-zero elements in the matrix.

[0048] Referring to Figure 1(a), the first row contains two non-zero elements {1, 1}, the second row contains no non-zero elements, the third row contains two non-zero elements {3, 3}, and the fourth row contains four non-zero elements {4, 4, 4, 4}. Corresponding to Figure 1(b), the value array is represented as {1, 1, 3, 3, 4, 4, 4, 4}.

[0049] The row offset array is used to store the position of each row, with a length of m+1, where m is the number of rows in the matrix. Here, [row_offsets[i], row_offsets[i+1]) represents the storage position of the i-th row of the matrix in the value array and column index array. Corresponding to (b) in Figure 1, [row_offsets[0], row_offsets[1]) = [0, 2) indicates that the 0th row of the matrix includes two non-zero elements, the 0th and 1st elements in the value array / column index array. [row_offsets[1], row_offsets[2]) = [2, 2) indicates that the 1st row of the matrix does not include any non-zero elements, and [row_offsets[2], row_offsets[3]) = [2, 4) indicates that the 2nd row of the matrix includes two non-zero elements, the 2nd and 3rd elements in the value array / column index array. Similarly, the column index array is represented as {0, 2, 2, 3, 0, 1, 2, 3}, and the row offset array is represented as {0, 2, 2, 4, 8}.

[0050] After obtaining the matrix data, the processor needs to analyze it, as shown in Figure 2, which illustrates a flowchart of matrix data processing. This process can include matrix import and matrix partitioning. The processor can partition the matrix according to load balancing requirements to ensure a balance between computational load and data read / write volume for each thread, and to guarantee similar computation time for each thread.

[0051] This processing flow can also include SPMV computation, with multiple threads (e.g., thread 1 to thread N) each performing SPMV computation. During the SPMV computation phase, hardware computing power is fully utilized, and numerous optimizations are performed to adapt to hardware characteristics. For example, as shown in Figure 3, the optimization flow can include: algorithm design, instruction set optimization, and just-in-time (JIT) compilation optimization. Algorithm design can include cache-awareness, single instruction multiple data (SIMD) friendliness, and parallelism friendliness. Instruction set optimization can include SIMD vectorization, masking, branchless programming, and prefetching. JIT optimization can include adaptive optimization, dynamic recompilation, and microarchitecture acceleration.

[0052] Referring to Figure 2, the process can also include: summarizing and writing the results calculated by multiple threads, and outputting them to the next stage of calculation.

[0053] Although the above processing flow has been optimized to a high degree for hardware architecture in terms of load balancing, vectorization, and data prefetching, performance degradation will occur in actual use if the number of threads is large. Figure 4 illustrates the performance of multiple threads. The horizontal axis represents the number of threads, and the vertical axis represents the speedup ratio, i.e., the ratio of the speed of multiple threads to the speed of a single thread. Figure 4 specifically shows the performance of multiple threads for various algorithms, such as Algorithm 1, Algorithm 2, Algorithm 3, Algorithm 4, and Algorithm 5.

[0054] As can be seen from Figure 4, when the number of threads is large, especially when the number of threads is greater than 16, the speedup ratio of some algorithms even begins to decrease, showing a deterioration in performance.

[0055] This is because the processing flow lacks awareness of the hardware architecture. Currently, most modern computer architectures or hardware architectures generally employ a non-uniform memory access (NUMA) architecture. In a NUMA architecture, the processor cores are not symmetrical; all processor cores are divided into several NUMA nodes. Figure 5 illustrates a schematic diagram of a NUMA architecture. Figure 5 specifically shows NUMA node 1 and NUMA node 2, which communicate with each other via a bus, such as a quick path interconnect (QPI) bus.

[0056] Taking NUMA node 1 as an example, NUMA node 1 includes a socket and multiple processor cores installed on the socket, such as processor core 1, processor core 2, processor core 3, and processor core 4. NUMA node 1 also includes directly connected dynamic random access memory (DRAM), which is the local memory of NUMA node 1 and also the remote memory of NUMA node 2.

[0057] Processor cores within the same NUMA node use directly connected DRAM to transfer data, resulting in faster transfer speeds. If a processor core accesses DRAM across NUMA nodes, it needs to use the QPI bus to transfer data, which is slower.

[0058] Because the processing flow in Figure 2 lacks awareness of the hardware architecture, it may be limited to a single NUMA node when allocating threads and processor cores, resulting in other NUMA nodes being idle and computational efficiency being low. Alternatively, the processing flow may allocate threads and processor cores randomly, leading to a lot of data movement across NUMA nodes during matrix computation. Figure 6 illustrates a matrix computation allocation diagram. Figure 6 specifically shows multiple threads (e.g., thread 1, thread 2, thread 3, and thread 4), multiple processor cores (e.g., processor core 1, processor core 2, processor core 3, and processor core 4), and multiple memory blocks (e.g., memory block 1, memory block 2, memory block 3, and memory block 4). Processor core 1 and processor core 2 belong to the same NUMA node, as do processor core 3 and processor core 4. Furthermore, memory block 1 is the local memory of processor core 1, memory block 2 is the local memory of processor core 2, memory block 3 is the local memory of processor core 3, and memory block 4 is the local memory of processor core 4.

[0059] In a possible example, processor core 1 executes thread 3, and the data corresponding to thread 3 is stored in memory block 1. In this case, processor core 1 can access the data from the nearest memory block. However, processor core 2 processes the data corresponding to thread 1, which is stored in memory block 4. In this case, processor core 2 needs to access the data across NUMA nodes. Similarly, processor cores 3 and 4 face the same problem as processor core 2. Therefore, if the number of threads executing simultaneously is large, performance degradation will occur, preventing the processor from fully utilizing the underlying hardware's computing power.

[0060] Therefore, this application provides a matrix computing device. When allocating the storage address of the block matrix, the first processor in the matrix computing device uses a memory-friendly allocation method to allocate the block matrix to the local memory near the processor core. This avoids the problem that the processor core needs to read the block matrix from remote memory when running threads, which can improve the access efficiency of the processor core in performing matrix computing operations, and thus improve the matrix computing efficiency of the matrix computing device, such as improving the computing efficiency of sparse matrix vector multiplication.

[0061] In the above scenarios, the address calculation device provided in this application embodiment can be applied to terminal devices, such as servers, mobile terminals, tablet computers, laptops, augmented reality (AR) devices, virtual reality (VR) devices, and in-vehicle terminals. The address calculation device provided in this application can be applied to scenarios involving matrix calculations related to central processing units (CPUs), high-performance computing (HPC), and artificial intelligence (AI), such as scalar calculation units, vector calculation units, matrix calculation units, and tensor calculation units.

[0062] In one possible implementation, the address calculation device provided in this application embodiment can be a chip, such as a system-on-a-chip (SoC), or a chip such as a CPU, graphics processing unit (GPU), tensor processing unit (TPU), and neural network processing unit (NPU).

[0063] The address calculation device provided in the embodiments of this application will be further described below with reference to the accompanying drawings.

[0064] This application provides an address calculation device, which includes a first processor and a plurality of second processors. Each second processor includes a plurality of NUMA nodes, and each NUMA node includes local memory and a plurality of processor cores.

[0065] Figure 7 shows a schematic diagram of an address calculation device. Specifically, Figure 7 shows N second processors, namely second processor 1, second processor 2, ..., second processor N. Taking second processor 1 as an example, it shows M NUMA nodes, namely NUMA node 1, NUMA node 2, ..., NUMA node M. Taking NUMA node 1 as an example, it shows local memory 1 and P processor cores, namely processor core 1, processor core 2, ..., processor core P. Where N, M, and P are all positive integers.

[0066] For example, the first or second processor may be a SoC or include multiple chips. Optionally, the first or second processor may include one or more of a CPU or microcontroller, and may also include other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gates, transistor logic devices, or discrete hardware components.

[0067] For example, the first processor can be the same processor as the second processor, and the first processor can also include multiple NUMA nodes.

[0068] The first processor is used to obtain the input vector and coefficient matrix, and to perform block processing on the input vector and coefficient matrix to obtain multiple block matrices.

[0069] For example, the address calculation device provided in this application embodiment can be applied to matrix calculations, such as matrix multiplication, that is, performing matrix multiplication on an input vector and a coefficient matrix. In a possible example, the address calculation device provided in this application embodiment can be applied to sparse matrix-vector multiplication, where the input vector is x mentioned above, and the coefficient matrix is ​​A mentioned above.

[0070] For example, as shown in Figure 8, a schematic diagram of matrix partitioning is illustrated. Figure 8(a) shows a schematic diagram of matrix partitioning, i.e., matrix partitioning can be uniform partitioning. The matrix shown in Figure 8(a) can be an 8x4 matrix, which can be partitioned into 2x4 blocks, such as A1, A2, A3, and A4.

[0071] Figure 8(b) illustrates another type of matrix partitioning, namely, non-uniform matrix partitioning. The matrix shown in Figure 8(b) can also be an 8x4 matrix, which can be divided into four partition matrices, such as B1, B2, B3, and B4. B1 can be a 1x4 partition matrix, B2 can be a 3x4 partition matrix, B3 can be a 2x4 partition matrix, and B4 can be a 2x4 partition matrix.

[0072] In addition, the first processor can rearrange the input vector or coefficient matrix before performing block processing. For example, when the input vector or coefficient matrix is ​​a sparse matrix, the first processor rearranges the input vector or coefficient matrix, concentrating non-zero elements in certain regions, while the remaining regions contain only zero elements. The first processor then performs block processing on the transformed input vector or coefficient matrix, thereby obtaining multiple block matrices containing only zero elements. Block matrices containing only zero elements reduce the amount of matrix computation, thus further improving the efficiency of matrix computation.

[0073] Understandably, the first processor can also perform block processing according to load balancing requirements to ensure that the computational load and data read / write load of each thread are balanced, and that each thread has similar computation time. This application does not limit the block processing method.

[0074] The first processor is also used to store multiple block matrices into multiple local memories based on the architecture information and matrix allocation strategy of multiple NUMA nodes. The matrix allocation strategy includes: each block matrix corresponds to one thread, the processor core is bound to at least one thread, and the address of each block matrix is ​​located within the address range of the local memory corresponding to the processor core.

[0075] For example, the architecture information of multiple NUMA nodes is also called hardware topology. The architecture information of multiple NUMA nodes may include at least one of the following: the number of multiple second processors, the number of NUMA nodes for each second processor, the number of multiple processor cores in each NUMA, and the size of the cache per level for each processor core.

[0076] In addition, the architecture information for multiple NUMA nodes can also include the size of their local memory.

[0077] For example, when allocating the block matrix, the first processor first uses thread binding technology to bind processor cores to at least one thread. Specifically, thread binding technology fixes a thread to execute on one or more processor cores, which can be done by adjusting the CPU affinity mask of the thread. The CPU affinity mask is a bitmap structure, where each bit represents an available processor core. If a bit is set to 1, it indicates that the corresponding processor core is available; if a bit is set to 0, it indicates that the corresponding processor core is unavailable.

[0078] In this application, a thread can be bound to multiple processor cores, and multiple threads can be bound to a single processor core. This application does not limit this.

[0079] For example, local memory can be a contiguous storage space, which can have a starting address and a size. When the first processor allocates the address of the block matrix, the address allocated to the block matrix is ​​within the address range of local memory, that is, the block matrix corresponding to the thread is allocated to the processor core corresponding to the corresponding local memory.

[0080] For example, the matrix allocation strategy can be determined by the first processor based on the architecture information and block division method of multiple NUMA nodes. This matrix allocation strategy can also be called a memory affinity allocation method.

[0081] In one possible example, the matrix allocation strategy could be uniform distribution within a single NUMA node, where multiple partitioned matrices are distributed across processor cores on a single NUMA node for computation. This reduces data interaction between multiple NUMA nodes, freeing up other NUMA nodes to execute other threads.

[0082] In another possible example, the matrix allocation strategy could be uniform distribution across multiple NUMA nodes, where multiple block matrices are evenly distributed across processor cores in multiple NUMA nodes for computation. This allows for matrix computation to be performed by a larger number of processor cores, thus reducing computation time.

[0083] In another possible example, the matrix allocation strategy could be to prioritize NUMA allocation, that is, to preferentially allocate multiple block matrices to the same NUMA node for computation. Data interaction between the same NUMA node is faster, which can reduce the time required for matrix computation.

[0084] In yet another possible example, the matrix allocation strategy could also be a complex strategy of uneven partitioning or other methods, which are not limited in this application.

[0085] The processor core is used to run threads and load block matrices from local memory for matrix calculations.

[0086] Based on Figure 6, Figure 9 illustrates a matrix allocation strategy. Compared to the random allocation in Figure 6, the allocation in Figure 9 employs a memory affinity strategy.

[0087] In this process, processor core 1 executes thread 1, and the corresponding block matrix for thread 1 is stored in memory block 1, which is the local memory of processor core 1. Processor core 1 can access and load the block matrix from memory block 1 for matrix calculations. Similarly, processor core 2 executes thread 2, and the corresponding block matrix for thread 2 is stored in memory block 2, which is the local memory of processor core 2. Processor core 2 can access and load the block matrix from memory block 2 for matrix calculations. Processor cores 3 and 4 can also achieve local access, which will not be elaborated upon here.

[0088] For example, before the processor core runs a thread, the block matrix corresponding to the thread has already been allocated to the memory block corresponding to the corresponding processor core. No additional handling is required when the corresponding thread is calculated, thereby avoiding the problem that the processor core needs to read the block matrix from remote memory when running the thread. This can improve the access efficiency of the processor core in performing matrix calculation operations, and also improve the matrix calculation efficiency of the matrix calculation device.

[0089] Optionally, each processor core also includes a multi-level cache, and the matrix allocation strategy also includes aligning the starting address of each block matrix in local memory with the cache line of the multi-level cache.

[0090] For example, a multi-level cache is a temporary storage space between the processor core and main memory. Multi-level caches have a smaller capacity than main memory but offer faster data exchange. A multi-level cache can include a Level 1 cache, a Level 2 cache, and a Level 3 cache. The Level 1 and Level 2 caches can be private caches for each processor core, while the Level 3 cache (LLC) can be a shared cache for multiple processor cores. When a processor core needs to read data, it searches sequentially in the Level 1, Level 2, and Level 3 caches. If the data is not found in a higher-level cache, it searches in a lower-level cache. If the data is not found in any of the multi-level caches, it searches in main memory. When the target data is found in main memory, it is also loaded into the cache for temporary storage.

[0091] Because the operation unit of a multi-level cache is a data block (i.e., a cache line) rather than a single byte, when a processor core accesses data destined for a specific location, the entire cache line is loaded into the cache. To improve cache utilization, the first processor core allocates the addresses of the block matrix according to the cache line boundaries to achieve cache alignment.

[0092] Therefore, in allocating the addresses of the block matrix, in addition to using the memory-friendly allocation method, a cache-aligned allocation method is further adopted, which can further reduce the number of times the block matrix is ​​read from memory, improve the access efficiency of the processor core, and thus improve the matrix computing efficiency of the matrix computing device.

[0093] Optionally, the first processor also includes an allocation interface, which is further configured to store multiple block matrices into multiple local memories based on the architecture information of multiple NUMA nodes, matrix allocation strategy and preset allocation conditions, when the allocation interface obtains preset allocation conditions.

[0094] For example, the matrix computing device provided in this application embodiment can provide an allocation interface, which can provide a user-defined strategy (i.e., preset allocation conditions) to support users in restricting strategies through environment variables and formulating matrix allocation strategy schemes.

[0095] As exemplified, Figure 10 illustrates a flowchart for determining a matrix allocation strategy. Figure 10 specifically shows a strategy determiner, which can be a module or unit of a first processor.

[0096] The strategy determiner can identify the matrix allocation strategy based on the architecture information, partitioning method, and preset allocation conditions of multiple NUMA nodes. The architecture information of the multiple NUMA nodes can include the number of secondary processors, the number of NUMA nodes in each secondary processor, the number of processor cores in each NUMA node, and the size of the cache per level for each processor core. The matrix allocation strategy can include: uniform allocation within a single NUMA node, uniform allocation across multiple NUMA nodes, a strategy prioritizing the same NUMA node, and complex strategies involving uneven partitioning.

[0097] Therefore, the strategy determiner can provide a memory-friendly allocation strategy for the allocation of block matrices, and can also provide high flexibility, thereby improving the matrix computing efficiency of the matrix computing device.

[0098] Optionally, the first processor is also used to obtain architectural information about multiple NUMA nodes.

[0099] For example, the first processor can obtain the architecture information of multiple NUAM nodes upon power-on or reset to perceive the architecture information of the underlying hardware, provide underlying hardware support for the subsequent allocation of the block matrix, avoid performance degradation when executing multi-threading, and fully utilize the performance of the underlying hardware.

[0100] Optionally, the first processor is also used to acquire and summarize the structure of matrix calculations by multiple processor cores to obtain an output vector.

[0101] For example, as shown in Figure 11, a flowchart of matrix computation is presented. Figure 11 illustrates a matrix computation process that, based on Figure 2, adds steps for acquiring the architecture information of multiple NUMA nodes and reallocating the block matrix. The block matrix reallocation is performed by the first processor based on the architecture information of the multiple NUMA nodes and the matrix allocation strategy. After performing the block matrix reallocation step, multiple memory blocks are obtained, such as memory block 1, memory block 2, ..., memory block N. The processor core can run threads to load the block matrix from the memory blocks for SPMV computation. After the processor core's computation is complete, the first processor can summarize the results of the multiple threads' computations and write them into an output vector, which is then output to the next stage of computation.

[0102] Therefore, applying the matrix calculation process in Figure 11 to the difficult test dataset of sparse matrix vector multiplication using the iterative data calculation method can improve the computational efficiency of sparse matrix vector multiplication, especially for sparse matrices with tens of millions or more non-zero elements. Figure 12 shows the performance graphs of various algorithms. The horizontal axis represents the number of particles for different algorithms, and the vertical axis represents the speedup ratio. Figure 12 specifically shows the performance graphs of multiple particles corresponding to various algorithms, such as Algorithm 1, Algorithm 2, Algorithm 3, Algorithm 4, and Algorithm 5. As can be seen from Figure 12, these algorithms can achieve an average speedup of 1.64 times or 2.68 times compared to the baseline version, with 32 or 64 threads.

[0103] Applying to the aforementioned matrix calculation device, this application also provides a matrix calculation method, as shown in Figure 13, which illustrates a flowchart of a matrix calculation method. The matrix calculation method includes the following steps.

[0104] S1301, The first processor obtains the input vector and coefficient matrix, and performs block processing on the input vector and coefficient matrix to obtain multiple block matrices.

[0105] S1302, the first processor stores multiple block matrices into multiple local memories based on the architecture information and matrix allocation strategy of multiple NUMA nodes.

[0106] The matrix allocation strategy includes: each block matrix corresponds to one thread, the processor core is bound to at least one thread, and the address of each block matrix is ​​located in the address range of the local memory corresponding to the processor core.

[0107] S1303, the processor core runs a thread that loads a block matrix from local memory for matrix calculation.

[0108] Therefore, compared to allocation methods that distribute multiple block matrices to the local memory of a single processor or random allocation to memory, the first processor in the matrix calculation method provided in this application uses a memory-affinity allocation method when allocating the storage addresses of block matrices. That is, the first processor allocates block matrices based on the architecture information of multiple NUMA nodes and the matrix allocation strategy. The first processor can perceive the underlying hardware architecture information through the architecture information of the multiple NUMA nodes, allowing it to allocate block matrices to the local memory nearest to the processor core. This avoids the problem of the processor core needing to read block matrices from remote memory when running threads, improving the access efficiency of the processor core performing matrix calculation operations, and thus improving the matrix calculation efficiency of the matrix calculation device, such as improving the calculation efficiency of sparse matrix vector multiplication.

[0109] Optionally, each processor core also includes a multi-level cache, and the matrix allocation strategy also includes: aligning the starting address of each block matrix in local memory with the cache lines of the multi-level cache.

[0110] For example, the first processor may also use a cache-aligned allocation method to further reduce the number of times the block matrix is ​​read from memory, improve the access efficiency of the processor core, and thus improve the matrix computing efficiency of the matrix computing device.

[0111] Optionally, the matrix calculation method further includes: when the allocation interface obtains the preset allocation conditions, the first processor stores the multiple block matrices into multiple local memories based on the architecture information of multiple NUMA nodes, the matrix allocation strategy and the preset allocation conditions.

[0112] For example, the matrix calculation method can also provide a user-defined strategy (i.e., preset allocation conditions), allowing users to restrict strategies through environment variables and formulate matrix allocation strategy schemes, providing high flexibility for the allocation of block matrices and further improving the matrix calculation efficiency of the matrix calculation device.

[0113] Optionally, the architecture information of the multiple NUMA nodes includes at least one of the following: the number of multiple second processors, the number of NUMA nodes for each second processor, the number of multiple processor cores in each NUMA, and the size of the cache per level for each processor core.

[0114] For example, the architecture information of multiple NUMA nodes is the architecture information of the underlying hardware. By perceiving the architecture information of the underlying hardware, the first processor provides underlying hardware support for the subsequent allocation of the block matrix, avoiding performance degradation when executing multi-threaded operations, so as to give full play to the performance of the underlying hardware.

[0115] Optionally, prior to S1301, the matrix calculation method further includes: the first processor acquiring the architecture information of multiple NUMA nodes.

[0116] Optionally, the matrix calculation method further includes: the first processor acquiring and summarizing the results of matrix calculations from multiple processor cores to obtain an output vector.

[0117] This application also provides an electronic device, which includes a circuit board and a matrix computing device disposed on the circuit board.

[0118] This application also provides an electronic device, including one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program code, including computer instructions. When the one or more processors execute the computer instructions, the electronic device performs the aforementioned method steps to implement the matrix calculation method in the above embodiments.

[0119] Embodiments of this application also provide a computer-readable storage medium storing computer instructions. When the computer instructions are executed on an electronic device, the electronic device performs the aforementioned method steps to implement the matrix calculation method in the above embodiments.

[0120] For example, the computer-readable storage medium in the above embodiments can be a non-transitory computer-readable storage medium. The non-transitory readable storage medium can include non-volatile media such as read-only memory (ROM), or some volatile media such as some random access memory (RAM).

[0121] The embodiments of this application also provide a computer program product that, when run on a computer, causes the computer to perform the aforementioned related steps to implement the matrix calculation method executed by the electronic device in the above embodiments.

[0122] In addition, embodiments of this application also provide an apparatus, which may specifically be a chip, component, or module. The apparatus may include a connected processor and a memory. The memory is used to store computer execution instructions. When the apparatus is running, the processor can execute the computer execution instructions stored in the memory to cause the chip to execute the matrix calculation method executed by the electronic device in the above-described method embodiments.

[0123] The electronic devices, computer-readable storage media, computer program products or chips provided in this embodiment all relate to the matrix computing device provided above. Therefore, the beneficial effects they can achieve can be referred to the beneficial effects in the matrix computing device provided above, and will not be repeated here.

[0124] Through the above description of the embodiments, those skilled in the art will understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0125] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0126] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0127] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0128] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0129] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A matrix calculation device, characterized in that, include: A first processor and multiple second processors, each second processor comprising multiple non-uniform memory access model NUMA nodes, each NUMA node comprising local memory and multiple processor cores; The first processor is configured to acquire an input vector and a coefficient matrix, and to perform block processing on the input vector and the coefficient matrix to obtain multiple block matrices; The first processor is further configured to store the multiple block matrices into multiple local memories based on the architecture information of the multiple NUMA nodes and the matrix allocation strategy. The matrix allocation strategy includes: each block matrix corresponds to a thread, the processor core is bound to at least one thread, and the address of each block matrix is ​​located within the address range of the local memory corresponding to the processor core. The processor core is used to run the thread and load the block matrix in the local memory for matrix calculation.

2. The apparatus according to claim 1, characterized in that, Each processor core also includes a multi-level cache, and the matrix allocation strategy further includes aligning the starting address of each block matrix in the local memory with the cache line of the multi-level cache.

3. The apparatus according to claim 1 or 2, characterized in that, The first processor also includes an allocation interface; The first processor is further configured to, when the allocation interface obtains preset allocation conditions, store the multiple block matrices into the multiple local memories respectively based on the architecture information of the multiple NUMA nodes, the matrix allocation strategy and the preset allocation conditions.

4. The apparatus according to any one of claims 1-3, characterized in that, The architecture information of the plurality of NUMA nodes includes at least one of the following: the number of the plurality of second processors, the number of NUMA nodes for each second processor, the number of processor cores in each NUMA node, and the size of each level of cache for each processor core.

5. The apparatus according to any one of claims 1-4, characterized in that, The first processor is also used to obtain the architecture information of the plurality of NUMA nodes.

6. The apparatus according to any one of claims 1-5, characterized in that, The first processor is also used to acquire and summarize the results of the calculation of the multiple processor core matrices to obtain an output vector.

7. A matrix calculation method, characterized in that, The method is applied to a matrix computing device, the matrix computing device comprising: a first processor and a plurality of second processors, each second processor comprising a plurality of non-uniform memory access model (NUMA) nodes, each NUMA node comprising local memory and a plurality of processor cores; the method comprising: The first processor acquires an input vector and a coefficient matrix, and performs block processing on the input vector and the coefficient matrix to obtain multiple block matrices; The first processor stores the multiple block matrices into multiple local memories based on the architecture information of the multiple NUMA nodes and the matrix allocation strategy. The matrix allocation strategy includes: each block matrix corresponds to a thread, the processor core is bound to at least one thread, and the address of each block matrix is ​​located within the address range of the local memory corresponding to the processor core. The processor core runs the thread and loads the block matrix in the local memory to perform matrix calculations.

8. The method according to claim 7, characterized in that, Each processor core also includes a multi-level cache, and the matrix allocation strategy further includes aligning the starting address of each block matrix in the local memory with the cache line of the multi-level cache.

9. The method according to claim 7 or 8, characterized in that, The first processor further includes an allocation interface, and the method further includes: When the allocation interface obtains the preset allocation conditions, the first processor stores the multiple block matrices into the multiple local memories based on the architecture information of the multiple NUMA nodes, the matrix allocation strategy, and the preset allocation conditions.

10. The method according to any one of claims 7-9, characterized in that, The architecture information of the plurality of NUMA nodes includes at least one of the following: the number of the plurality of second processors, the number of NUMA nodes for each second processor, the number of processor cores in each NUMA node, and the size of each level of cache for each processor core.

11. The method according to any one of claims 7-10, characterized in that, Before the first processor acquires the input vector and coefficient matrix, the method further includes: The first processor acquires the architecture information of the plurality of NUMA nodes.

12. The method according to any one of claims 7-11, characterized in that, The method further includes: The first processor acquires and summarizes the results of the calculations of the multiple processor core matrices to obtain an output vector.

13. An electronic device, characterized in that, include: The circuit board and the matrix computing device as described in any one of claims 1-6, wherein the matrix computing device is disposed on the circuit board.

14. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in any one of claims 7-12.