Assignment of processing threads for matrix-matrix multiplication

Abstract

Non-transitory storage medium (800) that stores machine-readable instructions (804) which, when executed by a machine, cause the machine to: Access to data representing a first dimension of a first matrix (210), a second dimension of a second matrix (214), and a third dimension shared by the first matrix (210) and the second matrix (214), Determining a multitude of candidate decompositions for the first matrix (210), wherein the candidate decompositions vary in size relative to each other along the first dimension and the third dimension; Determining an associated fitness value for each candidate decomposition from the multitude of candidate decompositions; Selecting a candidate decomposition from the multitude of candidate decompositions based on fitness scores to provide a selected candidate decomposition; and Providing data representing the selected candidate decomposition to effect the assignment of processing threads (112) to sub-matrices of the first matrix (210), wherein the processing threads (112) determine a third matrix (220) based on a multiplication of the first matrix (210) and the second matrix (214); wherein the instructions (804), when executed by the machine, further cause the machine to determine a block size for the plurality of decomposition candidates, such that each decomposition candidate of the plurality of decomposition candidates is decomposed along the third dimension according to the block size, and wherein the instructions (804), when executed by the machine, further cause the machine to determine the block size based on a property of a cache memory used by the processing threads (112) in determining the third matrix (220).

Description

BACKGROUND

[0001] A given computer system may contain a specialized mathematical library that provides programs to support arithmetic operations in a variety of applications in engineering, data mining, numerical processing, data analysis, and machine learning. Such a program may implement a version of a generalized matrix-matrix multiplication (GEMM) algorithm to perform a matrix-matrix multiplication. In some applications, the matrices involved in the matrix-matrix multiplications may be relatively large (e.g., a given matrix may have thousands or hundreds of thousands of rows and columns, if not more), resulting in a relatively large number of floating-point multiplication operations for each matrix-matrix multiplication. YINAN, Li; DONGARRA, Jack; TOMOV, Stanimire: A note on auto-tuning GEMM for GPUs.In: Computational Science - ICCS 2009: 9th International Conference, Baton Rouge, LA, USA, May 25-27, 2009. Proceedings, Part I. Berlin: Springer, 2009 (Lecture Notes in Computer Science; 5544). pp. 884-892. - ISBN 978-3-642-01969-2 describes an auto-tuning approach for optimizing the performance of General Matrix Multiplication (GEMM) on graphics processing units (GPUs). This approach automatically generates various code variants, evaluates their performance, and selects the most suitable variant to achieve high computational efficiency. A multithreaded matrix operation method is known from US 2018 / 0 341 517 A1, which divides a matrix into first submatrices (whose dimensions are multiples of a block size) and second submatrices (for remainders). The operations for both types of submatrices are then assigned to different threads to equalize processing time and reduce bottlenecks.CN 111 158 874 A discloses a data processing method and a device for matrix multiplication that defines a scheduling method for multiplication tasks based on the dimensions of two matrices. It includes the assignment of partial multiplication tasks to parallel threads to optimize memory efficiency and increase computing speed. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a schematic representation of a computer system according to an example implementation. Fig. Figure 2 is a flowchart representing a process performed by a thread decomposition engine and a contention mitigation engine of the computer system. Fig. 1 is used according to an example implementation. Fig. Figure 3 is an illustration of nested processing loops of a generalized matrix-matrix multiplication (GEMM) algorithm according to an example implementation. Fig. Figure 4 is an illustration of matrix-matrix multiplication processing by a K-group of processing threads according to an example implementation. Fig. 5A and Fig. Figure 5B shows a flowchart of a process for performing adaptive threading of dimension K according to an example implementation. Fig. Figure 6 is a flowchart showing a process for determining subblock sizes for processing thread subcycles according to an example implementation. Fig. Figure 7 is a flowchart that represents a process that determines whether or not the use of a local temporary buffer is recommended to mitigate the conflict between processing threads according to a sample implementation. Fig. Figure 8 is an illustration of a non-transitory storage medium that stores machine-executable instructions which, when executed by the machine, cause the machine to provide data that constitutes a decomposition for processing the thread assignment according to a sample implementation. Fig. Figure 9 is a schematic diagram of a device containing a processor to provide data representing block sizes for threading by a multiplication algorithm according to an example implementation. Fig. Figure 10 is a flowchart showing a process for multiplying matrices according to an example implementation. DETAILED DESCRIPTION

[0002] The multiplication of two matrices (referred to here as "matrix-matrix multiplication" or "matrix multiplication") can be performed in a computer system that has one or more multicore central processing unit (CPU) semiconductor packages (or "chips"). The CPU semiconductor package can contain multiple CPU processing cores that have access to local on-chip memory. Furthermore, the CPU semiconductor package can use a non-uniform memory access (NUMA) architecture. In general, a NUMA architecture recognizes that processing nodes have faster access times to local memory than to non-local memory. Accordingly, in a NUMA architecture, processing cores can be grouped into NUMA domains, so that the processing cores of each NUMA domain perform most of their computations using local memory access.The NUMA domain can have one or more cache levels, making access to the local on-chip memory more efficient by storing recently accessed data in a faster cache memory.

[0003] For matrix-matrix calculations, the computer system can use a generalized matrix-matrix multiplication (GEMM) algorithm, which relies on different processing threads (on corresponding NUMA nodes and sockets) to perform different parts of the multiplication. Matrix-matrix multiplication can involve multiplying considerably large matrices containing thousands, if not hundreds of thousands (if not more), of rows and columns.

[0004] To accommodate such computationally intensive operations, a technique called "threading" can be used to distribute the various parts of the matrix-matrix multiplication workload across processing threads of the computer system. In this context, a "processing thread" (or "thread") refers to a unit of machine-executable instructions associated with a processing kernel of the computer system. The processing threads can be executed in parallel (i.e., simultaneously) by the processing kernels associated with them.

[0005] In the context of this revelation, "threading" refers to the allocation of the processing load of matrix-matrix multiplication to different processing threads of the computer system. Threading can, for example, partition the input matrices involved in the matrix-matrix multiplication and assign the resulting partitions to corresponding processing threads.

[0006] One type of threading is "M / N" threading, where matrix partitioning is performed along the M and N dimensions to form corresponding MX N submatrices or blocks that are assigned to different processing threads. Here, the "M dimension" refers to a row dimension of an input matrix A, and the "N dimension" refers to a column dimension of an input matrix B in a matrix-matrix multiplication defined as follows: AXB = C, where "C" represents the output matrix, or the result of the matrix-matrix multiplication. Accordingly, the output matrix C has a dimension of MXN, or M rows × N columns. The input matrix A and the input matrix B share a common dimension, referred to here as the "K dimension." More precisely, the K dimension is the column dimension of the input matrix A and the row dimension of the input matrix B.

[0007] One challenge with M / N threading is that the resulting data units corresponding to the sub-matrices may be too small to effectively utilize the cache (e.g., cache row boundaries of a CPU's last-level cache (LLC)). Consequently, due to insufficient cache utilization, there may not be enough data in the cache to offset the cost of moving data from main system memory, resulting in slower matrix-matrix multiplication processing.

[0008] In accordance with the example implementations described here, threading is performed across the K-dimension, that is, the dimension that the input matrices have in common. More precisely, in accordance with the example implementations, a thread decomposition with an adaptive K-dimension blocking engine (referred to here as the "thread decomposition engine") provides processing thread assignments for a GEMM algorithm. The corresponding sub-matrices or sub-blocks are dimensioned to improve cache memory utilization and, in general, reduce the processing times for matrix-matrix multiplication. According to the example implementations, the thread decomposition engine assigns processing threads to the sub-matrices or blocks based on a two-dimensional matrix partitioning, where one dimension is the K-dimension and the other dimension is either the M-dimension or the N-dimension.

[0009] The K-dimension-based threading described here can take one of two forms: "M / K threading," which refers to threading where processing threads are assigned to the respective matrix blocks derived by partitioning along the M and K dimensions, and "N / K threading," which refers to threading where processing threads are assigned to the matrix blocks derived by matrix partitioning along the N and K dimensions. The M / K or N / K processing thread assignment results in block sizes that are closer to or equal to optimal cache block sizes; that is, block sizes that lead to more cache hits and better cache memory utilization when performing matrix-matrix multiplication than is the case with M / N threading.Furthermore, according to example implementations, the thread decomposition engine performs partitioning along the K-dimension in an adaptive manner that takes optimal cache block sizes into account.

[0010] As also described here, a contention mitigation engine provides the GEMM algorithm with additional information that reduces, if not eliminates, memory conflicts between processing threads during matrix-matrix multiplication. Specifically, according to example implementations, the contention mitigation engine subdivides or partitions along the M and / or N dimensions to create subblocks of each MX N block of the output matrix C. Each processing thread can then subcycle over the corresponding subblocks. As a result of the subblocks and subcycles, the competition for memory resources between a "K group" of processing threads can be reduced. In this context, a "K group" of processing threads refers to all processing threads contributing to a particular MX N block of the output matrix C.

[0011] Furthermore, in accordance with example implementations, the Contention Mitigation Engine recommends, as a further measure to reduce memory resource conflicts, whether a K-group of processing threads should each use a local temporary buffer (i.e., a "scratchpad" buffer) to determine results for a corresponding MX N block of the output matrix C. If the K-group of processing threads uses local temporary buffers, then each processing thread in the K-group uses the local temporary buffer to determine a contribution to the MX N block of the output matrix C, and the contributions from the local temporary buffers can then be transferred to the final output buffer for the output matrix C via additive updates.

[0012] Referring to Fig. 1. As a more specific example, a computer system 100 may, in accordance with some implementations, include a GEMM engine 110. In this context, the “GEMM engine 110” represents a software-controlled engine in which several parallel processing threads 112 jointly apply a GEMM algorithm to perform matrix-matrix multiplication. Referring to Fig. 2 in conjunction with Fig. For example implementations described here, matrix-matrix multiplication refers to the multiplication of an input matrix A 210 and an input matrix B 214 to determine an output matrix C 220. In other words, the GEMM engine 110 determines the product described by the following matrix-matrix multiplication: AXB = C.

[0013] The input matrix A 210 has rows along a row dimension M (i.e., M rows indexed by an M-index) and columns along a K dimension (i.e., K columns indexed by a K-index). The input matrix B 214 has rows along the K dimension (i.e., K rows indexed by the K-index) and columns along the N dimension (i.e., N columns indexed by an N-index). Thus, the input matrix A 210 and the input matrix B 214 share a common dimension K. The output matrix C 220 has rows along the dimension M (i.e., M rows indexed by the M-index) and columns along the dimension N (i.e., N columns indexed by the N-index).

[0014] To use the GEMM engine 110 ( Fig. 1) To configure the GEMM engine 110 for matrix-matrix multiplication, it receives data 250 which, in accordance with example implementations, represents the allocation of processing thread assignments and may represent other parameters for further improving matrix-matrix multiplication processing, as further described herein. In accordance with example implementations, at least some of the data 250 is provided by a thread decomposition engine with an adaptive K-threading engine 114 (referred to here as the "Thread Decomposition Engine 114"). The Thread Decomposition Engine 114 analyzes the row and column sizes of the matrices 210, 214, and 220 against certain criteria, as described herein, to provide parameters (referred to here as "BSM," "BSN," and "BSK") that represent a matrix partitioning for the allocation of processing threads 112.In accordance with example implementations, partitioning is performed along the K-dimension and either along the N- or M-dimension.

[0015] The BSM parameter (referred to here as "BSM block size") represents the number of rows in dimension M for each matrix partition or block. The BSN parameter (referred to here as "BSN block size") represents the number of columns in dimension N for each matrix block. The BSK parameter (referred to here as "BSK block size") represents the number of rows / columns (depending on whether the input matrix is ​​A 210 or B 214) along the K dimension for a matrix partition or block.

[0016] According to example implementations, for a given matrix-matrix multiplication, the thread decomposition engine 114 performs either M / K threading (corresponding to the matrix partitioning along the M and K dimensions) or N / K threading (corresponding to the matrix partitioning along the N and K dimensions) to determine the matrix blocks to be allocated to the processing threads 112. In N / K threading, the thread decomposition engine 114 partitions the input matrix B 214 into blocks of size BSK x BSN (i.e., each block has a size of BSK rows by BSN columns); and each BSK x BSN block of the input matrix B 214 is allocated to a different corresponding processing thread 112.More precisely, as a result of N / K threading, the thread decomposition engine 114 provides the following block sizes: a BSN block size representing the number of columns along the N dimension per matrix block; a BSK block size representing the number of rows / columns along the K dimension per matrix block; and the default BSM block size, since partitioning for work-sharing in the M dimension does not occur for N / K threading. Each processing thread 112 processes matrix-matrix multiplication operations belonging to its associated BSK x BSN block of input matrix B 214 to derive a corresponding BSM x BSN block of output matrix 220.

[0017] For M / K threading, the Thread Decomposition Engine 114 divides the input matrix A 210 into blocks, and each BSM x BSK block of the input matrix A 210 is assigned to a different corresponding processing thread 112. More precisely, the Thread Decomposition Engine 114 provides the following block sizes for M / K threading: a BSM block size representing the number of rows along the M dimension per matrix block; a BSK block size representing the number of rows / columns along the K dimension per matrix block; and the default BSN block size, since partitioning for N-dimension work sharing does not occur for M / K threading. Each processing thread 112 processes matrix-matrix multiplication operations relating to its assigned BSM X BSK block of input matrix A 210 to derive a corresponding BSM X BSN block of output matrix 220.

[0018] As a more specific example, the Thread Decomposition Engine 114 can apply N / K threading to determine assignments for sixteen processing threads 112. For this example, the matrices can have the following dimensions: the input matrix A 210 can have 96 rows along the M dimension and 3,840 columns along the K dimension, represented by the MX K dimensions of "(96 × 3,840)"; the input matrix B 214 can have KX N dimensions of (3,840 × 192); and the output matrix C 220 can have the corresponding MX N dimensions of (96 × 192). The thread decomposition engine 114 can evaluate these dimensions, as described further below, and determine that the BSM, BSN, and BSK block sizes are 96, 48, and 960, respectively. Thus, the sixteen processing threads 112 are assigned to the sixteen 48 x 960 blocks of the input matrix B in this example.

[0019] Referring to Fig. 3 in conjunction with Fig. The processing of matrix-matrix multiplication by the GEMM engine 110 follows, according to some implementations of nested processing loops. In this context, it shows Fig. 3 an outer mc loop 304, which relates to processing in the M dimension. In M / K threading, a given processing thread 112 processes operations for its assigned BSM X BSK block, while processing thread 112 indexes across the K dimension and N dimension in the corresponding inner kc loop 308 and inner nc loop 312, respectively. In N / K threading, a given processing thread 112 processes operations for its assigned BSN X BSK block, since processing thread 112 indexes across the M dimension and the K dimension in the corresponding mc loop 304 and kc loop 308, respectively.

[0020] In Fig. Figure 3 also shows the following three nested innermost loops, which are part of the macro kernel of the GEMM engine 110: an mr loop 324, an nr loop 328, and a kr loop 332. In loops 324, 328, and 332, the processing thread 112 iterates over the M, N, and K dimensions, respectively, for a given BSM X BSN block of the output matrix C 220 ( Fig. 2) The subcycle introduces two processing loops outside the macrokernel of the GEMM engine 110: an m_sub loop 316, corresponding to the subcycle across the M dimension, and an n_sub loop 320, corresponding to the subcycle across the N dimension. More precisely, with reference to Fig. 2 in conjunction with Fig. 1. A Contention Mitigation Engine 118 of Computer System 100, in accordance with example implementations, subdivides the BSM and BSN block sizes into corresponding submatrices or subblocks with dimensions BSMsub and BSNsub, respectively. The GEMM Engine 110 contains the following nested loops to perform the sub-cycle: an m_sub loop 316 to perform a sub-cycle over the BSMsub-sized subblocks of each matrix block, and an n_sub loop 320 to perform a sub-cycle over the BSNsub-sized subblocks of each matrix block. In accordance with example implementations, BSMsub and BSNsub could be BSMR and BSNR, respectively. As described below, this sub-cycle procedure, in accordance with example implementations, can prevent memory congestion by a K group of processing threads 112 when the processing threads 112 jointly process a BSM x BSN block of the output matrix C 220.

[0021] In addition to providing the BSMsub and BSNsub sizes as part of the data 250, the Contention Mitigation Engine 118, in accordance with example implementations, can recommend to the GEMM Engine 110 whether to use a temporary local output buffer. In this context, the "temporary output buffer" refers to a "scratchpad" buffer that can be used by each processing thread in a K group of processing threads that collectively process a particular BSM x BSN block of the output matrix C 220. Instead of each processing thread 112 in the K group of processing threads 112 making its updates to the BSM x BSN block in a final result buffer, the processing thread 112 instead updates the temporary output buffer. Furthermore, in accordance with example implementations, the temporary local output buffer is optimally aligned with cache boundaries.When the K group of processing threads 112 has completed processing the BSM X BSN block of output matrix C 220 and the temporary local output buffer stores the complete result for the subblock of the output matrix, the contents of the temporary buffer can be transferred (e.g. via an update routine of the GEMM engine 110 and additive updates) to a final output buffer for matrix C 220.

[0022] Back to Fig. 1: The GEMM engine 110, the thread decomposition engine 114, and the contention mitigation engine 118 can be deployed according to the many possible implementations by any number of different computer architectures. In the example implementation of Fig. 1 The computer system 100 contains one or more central processing units (CPUs) 102 (e.g., CPU semiconductor packages or “chips”), each CPU 102 containing a set of processing cores 101. In accordance with some implementations, a number of processing cores 101 on a given CPU 102 can form a corresponding NUMA domain; and there can be multiple NUMA domains per CPU 102.

[0023] In accordance with other example implementations, processing cores other than CPU processing cores can also be used. For example, processing core 101, in accordance with other implementations, can be a graphics processing unit (GPU) core, a field programmable gate array (FPGA), a node accelerator core, etc.

[0024] In accordance with example implementations, the computer system 100 can be formed from one or more physical machines, each physical machine consisting of or being formed from actual software and actual hardware. In this way, in accordance with example implementations, each physical machine can include one or more CPUs 102 and memory. The total memory of the physical machine(s) is given in Fig. 2 represented by a memory 106. For example, the memory 106 can store machine-executable instructions 108 which, when executed by the CPU(s) 102, form one or more of the software components described herein, such as the GEMM engine 110, the thread decomposition engine 114, and the contention mitigation engine 118.

[0025] According to example implementations, memory 106 can store data such as data representing an input matrix A; data representing an input matrix B; data representing intermediate and final results of an output matrix C; data representing temporary local output buffers; data representing parameters described here, such as block sizes, subblock sizes, etc.; data representing block allocations for the processing threads 112, etc.

[0026] Memory 106 is generally a non-transient storage medium that can be composed of semiconductor memory devices, memristor-based memory devices, magnetic memory devices, phase-change memory devices, a combination of memory devices corresponding to one or more of these memory technologies, etc. Furthermore, Memory 106 can be volatile memory, non-volatile memory, or a combination of different memory types, such as volatile and / or non-volatile memory.

[0027] The physical machine(s) of Computer System 100 can take many different forms, such as one or more rack modules, one or more server blades, a desktop computer, a laptop computer, a tablet computer, a smartphone, a wearable computer, and so on. Depending on the specific implementation, the GEMM Engine 110, the Thread Decomposition Engine 114, and the Contention Mitigation Engine 118 can be formed from an entire physical machine, multiple physical machines, or parts thereof. Furthermore, according to some implementations, the GEMM Engine 110, the Thread Decomposition Engine 114, and the Contention Mitigation Engine 118 can contain and / or correspond to one or more virtual components or one or more virtual environments of an actual physical machine, such as one or more virtual machines, one or more containers, etc.

[0028] In accordance with example implementations, the Thread Decomposition Engine 114 executes a Process 500, which runs in the Fig. 5A and Fig. Figure 5B illustrates the process for determining either the BSM and BSK block sizes (for M / K threading) or the BSN and BSK block sizes (for N / K threading). In general, determining the block sizes consists of two parts: a first part in which the Thread Decomposition Engine 114 determines candidate matrix decompositions (Block 502) and the BSK block size (Block 504); and a second part (i.e., the remaining part of Process 500) in which the Thread Decomposition Engine 114 performs an iterative process that evaluates the candidate matrix decompositions for the purpose of determining the remaining BSM block size (for M / K threading) or the BSN block size (for N / K threading).

[0029] To take a specific example, the Thread Decomposition Engine 114, in accordance with some implementations, can determine the BSK block size (according to block 504 of Fig. 5A) as follows. Initially, the Thread Decomposition Engine 114 defines the BSM, BSN, and BSK block sizes as follows (where the subscript "init" denotes the initial state): BSMinit=min(BSMopt, M), BSNinit=min(BSNopt, N), and BSKinit=min(BSKopt, K).

[0030] In these equations, "BSMopt", "BSNopt", and "BSKopt" represent optimal values ​​for the BSM, BSN, and BSK block sizes, respectively; and "min()" represents the minimum function, that is, a function that selects the minimum value of a tuple of multiple values. In this context, an optimal block size refers to a number of rows or columns corresponding to a unit of data that is aligned to, or nearly aligned with, the cache row boundaries. Therefore, according to example implementations, the optimal block size can be, for example, a function of the cache architecture (e.g., a last-level cache (LLC) architecture) and the sizes of the matrix entries.

[0031] For M / K threading, the Thread Decomposition Engine 114 can determine several parameters used in deriving the BSK block size. These parameters include a number (called "nblk") of BSN-sized blocks in the N dimension; the number (called "mblk_max") of BSMR-sized blocks in the M dimension, where "BSMR" represents the smallest possible sub-block size (i.e., M rows) of the output matrix C; a newly determined BSM block size; and a number (called "mblk") of BSM-sized blocks in the M dimension. nblk=ceil(N / BSN), mblk_max=ceil(M / BSMR), BSM=min(BSMinit. ceil(mblk_max / div_m)*BSMR), and mblk=ceil(M / BSM).

[0032] In these equations, "ceil ( ) represents the application of a ceiling or boundary function; "div_m" represents the number of threads in dimension M; "N" represents the number of columns of the input matrix B; and "M" represents the number of rows of the input matrix A.

[0033] According to example implementations, there is a preference for the number of M blocks to be an integer multiple of the number of threads in dimension M; and according to this preference, the thread decomposition engine 114 evaluates the following equality: if (mblk <div_m)mblk=ceil(mblk / div_m)*div_m.

[0034] The thread decomposition engine then determines the corresponding number (here called "div_k") of threads in dimension K as follows: div_k=maxth / div_m, where "maxth" represents the total number of processing threads.

[0035] For N / K threading, the Thread Decomposition Engine 114 determines the mblk count as follows, in accordance with example implementations: mblk=ceil(M / BSM).

[0036] Furthermore, in accordance with example implementations for N / K threading, the Thread Decomposition Engine 114 determines the number (here called "nblk_max") of BSNR-sized blocks in the N dimension, where "BSNR" represents the smallest possible subblock size along the N dimension of the output matrix C; the BSN block size; and the number (here called "nblk") of BSN-sized blocks in the N dimension as follows: nblk_max=ceil(N / BSNR), BSN=min(BSNinit, ceil(nblk_max / div_n)*BSNR, and nblk=ceil(N / BSN).

[0037] In accordance with example implementations, there is a preference for the number of N blocks being an integer multiple of the number of threads in the N dimension; and accordingly, the thread decomposition engine 114 evaluates the following equality in accordance with example implementations: If (nblk>div_n)nblk=ceil(nblk / div_n)*div_n.

[0038] Then, in accordance with example implementations, the Thread Decomposition Engine 114 determines the div_k number of threads in dimension K as follows: div_k=maxth / div_n.

[0039] In accordance with example implementations, the Thread Decomposition Engine 114 dynamically adjusts the BSK block size to optimize efficiency, regardless of whether M / K or N / K threading is used. For an out-of-place / copy GEMM kernel (i.e., a GEMM kernel that copies input matrices A and B into buffers for contiguous memory access), Thread Decomposition Engine 114 determines an optimal resident pack buffer size for input matrices A and B as follows: Size of packing buffer = BSKopt*(BSMopt+BSNopt).

[0040] Using the optimal resident size of the pack buffers, the Thread Decomposition Engine 114 can determine the maximum possible BSK block size as follows: BSK=BSKopt*(BSMopt+BSNopt) / (BSM+BSN), BSK=ceil(BSK / BSKR)*BSKR, BSK=min(BSK,K),and kblk=ceil(K / BSK), where “BSKR” is the roll-off factor of the innermost K-loop 324 ( Fig. 3) in the macrokernel. Rolling is a loop transformation technique for improving performance by completing multiple iterations per loop, and the "rolling factor" refers to the number of iterations per loop.

[0041] In accordance with example implementations, Thread Decomposition Engine 114 can then adjust the BSK block size as follows. For M / K threading, Thread Decomposition Engine 114 evaluates the following equality: If(div_k>kblk)AND if(nblk*kblk % div_k !=0)kblk=div_k. If the above equality is TRUE, then according to example implementations, the Thread Decomposition Engine 114 uses the optimal BSK, which was determined as described above, because there are enough N blocks for an even distribution of work.

[0042] For N / K thread cutting, the following equality is determined: If (div_k>kblk) AND if (mblk*kblk % div_k !=0)kblk=div_k. If equality is TRUE, then Thread Decomposition Engine 114 calculates a different BSK block size, as described below. Otherwise, the optimal BSK block size can be used because there are enough M blocks for an even distribution of workload.

[0043] If the Thread Decomposition Engine 114 determines that the optimal BSK block size cannot be used, then, in accordance with example implementations, the Thread Decomposition Engine 114 recalculates the BSK block size as follows: BSK=min(BSK,ceil(ceil(K / BSKR) / kblk)*BSKR).

[0044] The Thread Decomposition Engine 114 can then determine the number of threads in dimension K as follows: kblk=ceil(K / BSK).

[0045] As in Fig. As shown in Figure 5A, (Block 502) the Thread Decomposition Engine 114 determines candidate matrix decompositions according to example implementations, using all possible divisors of the number of threads divided between dimensions M and K (for M / K threading) or in dimensions N and K (for N / K threading). The Thread Decomposition Engine 114 determines (Block 508) condition values ​​for each candidate matrix decomposition based on the BSM, BSN, and BSK block sizes. It should be noted that, in accordance with example implementations, the default BSN block size is used for M / K threading and the default BSM block size is used for N / K threading.

[0046] According to Block 512, Thread Decomposition Engine 114 normalizes the condition values. In this context, "normalizing" a given condition value refers to determining a maximum of the condition value for each of the candidate matrix decompositions and normalizing the condition values ​​based on this maximum. This effectively weights the condition values ​​for a given condition value category. Thread Decomposition Engine 114 then determines the fitness score for each candidate matrix decomposition based on the normalized condition values ​​(e.g., Thread Decomposition Engine 114 adds the normalized condition values ​​together), as described in Block 516.

[0047] As in Fig. As shown in Figure 5B, the Thread Decomposition Engine 114 (Block 520) sorts or orders the candidate matrix decompositions based on their corresponding individual fitness values. For example, according to some implementations, the Thread Decomposition Engine 114 orders the candidate matrix decompositions in ascending order according to their fitness values, so that the candidate matrix decomposition with the lowest corresponding fitness value is at the top of the list, i.e., it is the highest-ranked candidate matrix decomposition.

[0048] Thread Decomposition Engine 114 then performs an iterative process (e.g., the iterative process outlined in Blocks 524 and 528) using the ranked candidate matrix decompositions to select the specific candidate matrix decomposition based on the ranking. More precisely, in accordance with example implementations, Thread Decomposition Engine 114 (Block 524) selects the next candidate matrix decomposition based on the ranking. For example, Thread Decomposition Engine 114 selects the highest-ranked candidate matrix decomposition for the initial selection. Thread Decomposition Engine 114 then determines (Decision Block 528) whether this candidate matrix decomposition is acceptable based on one or more selection criteria, as further described here.If the candidate matrix decomposition is not acceptable, the Thread Decomposition Engine 114, according to example implementations, returns to block 524 to select the next candidate matrix decomposition, which is the matrix decomposition with the next highest rank (relative to the last selection), and proceeds to decision block 528 to determine whether this candidate matrix decomposition is acceptable.

[0049] Based on the selection criterion or criteria, the Thread Decomposition Engine 114 eventually selects a specific candidate matrix decomposition that is acceptable and then, according to Block 532, transmits data representing the block sizes of the selected matrix decomposition to the GEMM Engine 110.

[0050] Examples of condition values ​​that the Thread Decomposition Engine 114 can use to determine the corresponding fitness values ​​(according to blocks 508, 512 and 516 of Fig. 5A), according to example implementations for M / K threading, the Thread Decomposition Engine 114 can determine one or more (or all) of the following seven condition values ​​for a candidate matrix decomposition for the purpose of determining the fitness value: Condition value one: absolute difference between the number of subblocks in dimension K and the minimum such value (preferably the number of blocks as specified by BSK). Condition value two: absolute difference between BSN and BSK block sizes. Condition value Three: normalized upper bound calculation of the ratio between the number of threads in dimension M and the maximum number of decomposable blocks of the input matrix B available for joint packing. Condition value four: remainder of the local size in dimension M divided by the BSM block size. Condition value five: Difference between the BSMopt and BSM block sizes. Condition value six: Difference between the BSM and BSMinit block sizes, if the BSM block size is necessarily smaller than the BSMinit block size. Condition value seven: add one if the BSK block size is not equal to the BSKinit block size, and also add one if the BSM block size is not equal to the BSMinit block size (strict inequality).

[0051] According to example implementations, the Thread Decomposition Engine 114 for N / K threading can determine one or more (or all) of the following eight condition values ​​for a candidate matrix decomposition for the purpose of determining the fitness value: Condition value one: absolute difference between the number of sub-blocks in dimension N and the minimum such value (preferably the number of blocks as set by BSN). Condition value two: absolute difference between the BSN and BSK block sizes. Condition value Three: Zero if the number of threads in dimension N is an integer multiple of the number of processor cores per NUMA node, otherwise One. Condition value Four: Zero if the number of threads in dimension N is less than or equal to the number of processing cores per NUMA node, otherwise one. Condition value five: absolute difference between the number of threads in dimension N and the number of processing cores per LLC. Condition value six: normalized upper bound calculation of the ratio between the number of threads in dimension N and the maximum number of decomposable blocks of the input matrix A available for joint packing. Condition value seven: Difference between the BSN and BSNinit block sizes, if the difference is strictly smaller than the BSNinit block size. Condition value eight: Add one if the BSK block size is not equal to the BSKinit block size, and also add one if the BSN block size is not equal to the BSNinit block size (strict inequality).

[0052] According to example implementations, the Thread Decomposition Engine 114 then normalizes (with respect to the largest value) each condition value across all possible candidate matrix decompositions to provide appropriate weighted contributions; and then, according to example implementations, the Thread Decomposition Engine 114 sums the condition values ​​for each candidate matrix decomposition to determine an appropriate single fitness score for the candidate matrix decomposition. Next, the Thread Decomposition Engine 114 sorts or orders the candidate matrix decompositions in ascending order (e.g., it orders according to Block 520 of Fig. 5B) and selects the candidate matrix decomposition that has the minimum corresponding fitness value if the following conditions are met.

[0053] According to example implementations, the Thread Decomposition Engine 114 for M / K Threading determines whether a particular candidate decomposition (identified by the ranking) is acceptable (e.g., it executes decision block 528 of Fig. 5B), based on whether both of the following conditions are met: Condition one: Number of threads in dimension K > 1; and Condition two: The BSK block size is less than or equal to the BSM block size, or ceil(BSN / BSNR) is less than the total number of threads.

[0054] According to example implementations, the thread decomposition engine 114 for N / K threading selects that the candidate matrix decomposition (identified by the ranking) is acceptable (e.g., it executes decision block 528 of Fig. 5B), if both of the following condition values ​​are met: Condition one: Number of threads in dimension K > 1; and Condition value two: The BSK block size is less than or equal to the BSN block size, or ceil(BSM / BSMR) is less than the total number of threads.

[0055] If the condition values ​​for a given candidate matrix decomposition identified by the ranking are not met, then the next higher candidate matrix decomposition is selected and the corresponding condition values ​​are then re-evaluated, as described above in connection with the Fig. 5A and Fig. 5B described.

[0056] Fig. Figure 4 is an example representation 400 of M / K threading and shows in particular the partitioning of the input matrix A 210 into corresponding blocks 410, where each block BSM X BSK 410 is assigned to a specific processing thread 112 ( Fig. 1) is assigned. It is noted that, although in example representation 400 the BSN block size is shown as equal to N (i.e., the number of columns of the input matrix B is 214 and the number of columns of the output matrix C is 220), according to further example implementations the BSN block size may be smaller than N. It is noted that, although block 410 in Fig. While block 4 is represented as square, block 410 may be rectangular because the BSM and BSK block sizes can differ. A specific K-group of processing threads 112 (e.g., a K-group of processing threads assigned to blocks 410 of a series 412 of blocks 410 in Fig. 4 corresponds) jointly determines a corresponding BSM x BSN block 420 of the output matrix C 220. In other words, the processing thread corresponding to each block 410 of row 412 computes a portion of the BSM x BSN block 420 of the output matrix C 220. For example, processing thread 112, which is assigned, say, to block 410-1 of row 412, performs matrix multiplication operations based on block 410-1 and block 414-1 of the input matrix B 214 to generate a corresponding contribution to block 420 of the output matrix C 220. As another example, processing thread 112, which is assigned to block 410-2, performs matrix multiplication operations based on block 410-2 and block 414-2 of the input matrix B 214 to generate another contribution to block 420 of the output matrix C 220.Since the processing threads 112 in a given K group update the same common BSM X BSN block of the output matrix C, there may be a potential memory resource competition between these processing threads 112.

[0057] According to example implementations, the Contention Mitigation Engine 118 ( Fig. 1) An optimal subdivision of the blocks of the output matrix 220 into the M and / or N dimensions in a way that allows the processing threads 112 in a given K-group to perform matrix-matrix multiplication operations more efficiently by accessing smaller data blocks. Further subdivision along the M and / or N dimensions allows each processing thread 112 to perform a "sub-cycle" over the corresponding sub-blocks. As part of the sub-cycle, each processing thread 112 processes its associated sub-blocks sequentially according to loops 316 and 320 of Fig. 3.

[0058] Fig. Figure 6 illustrates a process 600 used by the Contention Mitigation Engine 118 to further decompose the M and / or N dimensions according to example implementations. Referring to Fig. 6 in conjunction with Fig. According to example implementations (Block 630), Contention Mitigation Engine 118 determines candidate submatrix decompositions by considering all divisors of the number of threads in a given K-group. More specifically, according to example implementations, Contention Mitigation Engine 118 can determine a set of condition values ​​for each candidate submatrix decomposition (Block 634) and a fitness score for each set of condition values. For example, according to example implementations, Contention Mitigation Engine 118 can determine a fitness score for each candidate submatrix decomposition based on the following condition values: Condition value one: Add one if the BSMsub subblock size is greater than or equal to the BSNsub block size (i.e., there is a preference for the M dimension to be larger than the N dimension). Condition value two: add one if the BSNsub subblock size is equal to the BSNopt block size (i.e., there is a preference for using the optimal cache block size for dimension N).

[0059] Dispute Resolution Engine 118 can sort or rank the eligible submatrix decompositions based on their associated condition values ​​according to Block 638 (e.g., rank the submatrix decompositions based on the fitness values ​​determined from the condition values). Dispute Resolution Engine 118 can then select the submatrix decomposition based on this ranking (Block 642). In accordance with some implementations, for example, Dispute Resolution Engine 118 can rank the candidate submatrix decompositions in descending order based on their corresponding fitness values, and Dispute Resolution Engine 118 can then select the candidate submatrix decomposition with the highest corresponding fitness value, as determined from the two condition values ​​above.

[0060] The Contention Mitigation Engine 118 can then determine (decision block 650) whether the specified matrix decomposition performs threading only in the K dimension, and if so, make a decision (decision block 654) as to whether M / K threading is used. If M / K threading is used (i.e., if "yes" is determined in decision block 654), the Contention Mitigation Engine 118 transmits data to the GEMM Engine 110 (block 662), representing a subcycle in the M dimension only, where the M dimension is subdivided according to a sub-block size of the M dimension (BSMsub) equal to the BSMR block size.Otherwise, if threading occurs only in the K dimension and N / K threading is used (the "no" part of decision block 662), then the contention mitigation engine 118 communicates data to the GEMM engine 110 according to block 656, representing a subcycle in the N dimension only, where the N dimension is subdivided according to an N-dimension sub-block size (BSNsub) equal to the BSNR block size. If threading occurs in a dimension other than the K dimension (the "no" part of decision block 650), then the contention mitigation engine 118 communicates data to the GEMM engine 110 according to process 600, representing the BSNsub and BSMsub values ​​(block 668).

[0061] In accordance with some implementations, the Contention Mitigation Engine 118 can make a recommendation as to whether each K processing thread group should use a temporary local buffer to derive the corresponding BSM X BSN block of output matrix C, so that the contents of the temporary local buffer can be transferred to the output buffer for output matrix C when the K processing thread group completes processing for that block. In accordance with some implementations, the local temporary buffer is aligned to cache boundaries, and updates to the temporary local buffer are additive.

[0062] The Contention Mitigation Engine 118 can execute a Process 700, which is in Fig. Figure 7 is shown to determine whether or not the use of the temporary local output buffer should be recommended. In accordance with example implementations, Process 700 implies that the Contention Mitigation Engine 118 recommends the use of the temporary local output buffer if two supports of a two-stage test are satisfied. Otherwise, the Contention Mitigation Engine 118 does not recommend the use of the temporary local output buffer. Referring to Fig. 7 in connection with Fig. In the first condition (decision block 704), the Contention Mitigation Engine 118, in accordance with example implementations, determines whether any of the BSM block sizes (decision block 704), BSN block sizes (decision block 708), or BSK block sizes (decision block 712) are equal to or greater than the optimal block sizes BSMopt, BSNopt, or BSKopt, respectively. If so, the first condition is met, and the Contention Mitigation Engine 118 determines, for the second condition (decision block 720), whether the number of threads in the K-dimension is less than a predetermined threshold (e.g., a threshold of "8"). If the number of threads in the K-dimension is greater than this predetermined threshold, then according to example implementations, both prongs are satisfied, and the Contention Mitigation Engine 118 recommends using the local temporary buffer and communicates data (block 724) with this recommendation.Otherwise, if either of the two conditions is not met, the Contention Mitigation Engine 118 communicates data to the GEMM Engine 110, as shown in example implementations, representing the non-use of a local temporary output buffer, as shown in block 716.

[0063] Back to Fig. 1: In accordance with some implementations, the GEMM Engine 110 may have one or more of the following load balancing features. The GEMM Engine 110 adjusts the number of threads cooperatively packing a common block of either matrix A and / or B using a fair-sharing principle to improve group synchronization. If the GEMM Engine 110 uses a local temporary output buffer with a sub-cycle, it computes the GEMM solution for the specified sub-block and uses a non-blocking loop over the update function. If the GEMM Engine 110 uses a local temporary output buffer without a sub-cycle, it computes the GEMM solution and then immediately calls the update function for the specified sub-block.If the GEMM engine 110 uses a sub-cycle without a local temporary output buffer, then within a loop, the GEMM engine 110 locally packs the input matrix A or B and waits for a lock to compute the GEMM solution that updates the output matrix C. If the GEMM engine 110 uses neither sub-cycling nor a local temporary output buffer, then the GEMM engine 110 waits for a lock to compute the GEMM solution that updates the output matrix C.

[0064] As a specific example of the outputs of Thread Decomposition Engine 114 and Contention Mitigation Engine 118, Engines 114 and 118 can determine block sizes, subblock sizes, and other parameters for the following matrices for sixteen processing threads: Matrix A (192 x 3840), Matrix B (3840 x 96), and Matrix C (192 x 96). For this example, it is assumed that CPU 102 has sixty-four processor cores, four NUMA domains, and four processor cores sharing a last-level cache (LLC). It is also assumed that N / K threading is used.

[0065] For this example, the Thread Decomposition Engine 114 evaluates the following M × N × K candidate decompositions: 1×1×16, 1×2×8, 1×4×4, 1×8×2, and 1×16×1, where 1×16×1 is invalid (because there is no partitioning in dimension N). Based on the selection criteria, the Thread Decomposition Engine 114 selects the 1×1×16 decomposition, so threading occurs only over the K dimension. The Thread Decomposition Engine 114 selects the BSM, BSN, and BSK block sizes for this example to be 192 / 96 / 240.

[0066] In this example, Engine 118 specifies, for conflict mitigation purposes, that the M-dimension be subdivided into blocks of BSMR size. Additionally, Engine 118 specifies for this example that the use of a local temporary output buffer is not recommended.

[0067] As a further example for the sample computer system described above, the Thread Decomposition Engine 114 and the Contention Mitigation Engine 118 can process the following matrices: Matrix A (96 x 3840), Matrix B (3840 x 192), and Matrix C (96 x 192). This example assumes the use of N / K threading.

[0068] For this example, the Thread Decomposition Engine 114 evaluates the following M x N x K decomposition candidates: 1x1x16, 1x2x8, 1x4x4, 1x8x2, and 1x16x1, where 1x16x1 is invalid (because no threading or decomposition occurs along the N dimension). Based on the selection criteria, the Thread Decomposition Engine 114 selects the 1x4x4 decomposition, so that threading occurs across the N and K dimensions. For this example, the Thread Decomposition Engine 114 selects the BSM / BSN / BSK blocks to 96 / 48 / 960.

[0069] For this second example, Contention Mitigation Engine 118 iterates through subcycles across dimensions M and N. Additionally, Contention Mitigation Engine 118 recommends using a local temporary output buffer for this second example to improve efficiency.

[0070] Referring to Fig. 8 stores a non-transitory storage medium 800 according to example implementations machine-readable instructions 804 which, when executed by a machine, cause the machine to access data representing a first dimension of a first matrix, a second dimension of a second matrix, and a third dimension shared by the first and second matrices. The instructions 804, when executed by the machine, further cause the machine to determine a plurality of partition candidates for the first matrix, the partition candidates varying in size relative to each other along the first and third dimensions.When executed by the machine, instructions 804 further cause the machine to determine an associated suitability value for each candidate decomposition of the plurality of candidate decompositions; and to select one candidate decomposition from the plurality of candidate decompositions based on the suitability values, in order to provide a selected candidate decomposition. When executed by the machine, instructions 804 further cause the machine to provide data representing the selected candidate decomposition, in order to assign processing threads to submatrices of the first matrix. The processing threads determine a third matrix based on a multiplication of the first matrix and the second matrix.

[0071] Referring to Fig. Device 900, according to example implementations, contains a processor 910 and a memory 904 for storing instructions 908. When executed by the processor 910, the instructions 908 cause the processor 910 to perform threading of a first matrix along a first dimension and a second dimension. The threading represents block sizes of the first matrix, which are allocated to the process threads of a multiplication algorithm to determine a third matrix, which is a product of the first matrix and a second matrix. The block sizes comprise a first block size along the first dimension and a second block size along the second dimension. The second matrix shares the second dimension with the first matrix.When executed by processor 910, instructions 908 cause processor 910 to supply data to the multiplication algorithm representing the first block size and the second block size.

[0072] Referring to Fig.Section 10 comprises a procedure 1000 according to example implementations, comprising at least one hardware processor that accesses data (block 1002) representing a first matrix, a second matrix, a first block size, and a second block size. The first block size represents a sub-matrix decomposition size of the first matrix along a first dimension; and the second block size represents a sub-matrix decomposition size of the first matrix along the second dimension. The second matrix and the first matrix share the second dimension. The procedure 1000 involves the hardware processor(s) multiplying the first matrix and the second matrix (block 1004) to determine a third matrix. The multiplication includes assigning processing threads to sub-matrices of the first matrix based on the first block size and the second block size.

[0073] In accordance with example implementations, a block size is determined for the multiple decomposition candidates along the third dimension, so that the multiple decomposition candidates are each decomposed along the third dimension according to the block size. A particular advantage is that the processing efficiency of matrix-matrix multiplication can be increased.

[0074] According to example implementations, the block size can be determined based on a property of a cache memory used by the processing threads to determine the third matrix. A particular advantage is that the processing efficiency of matrix-matrix multiplication can be increased.

[0075] According to example implementations, for each fitness value, a multitude of conditions for the corresponding candidate decomposition are determined; and each fitness value is determined based on this multitude of conditions. A particular advantage is that the processing efficiency of matrix-matrix multiplication can be improved.

[0076] In accordance with example implementations, for each fitness value, every condition from the set of conditions is normalized to provide a set of normalized condition values. These multiple normalized condition values ​​are then combined to determine the fitness value. A particular advantage is the improved processing efficiency of matrix-matrix multiplication.

[0077] In accordance with example implementations, the fitness values ​​are ranked to create a corresponding ranking of the multiple candidate decompositions. The selected candidate decomposition is then chosen based on this ranking. A particular advantage is the improved processing efficiency of matrix-matrix multiplication.

[0078] In accordance with example implementations, a first candidate decomposition is selected based on the ranking; however, it is determined that the first candidate decomposition does not meet a selection criterion. A second candidate decomposition, different from the first, is selected based on the ranking; however, it is determined that the second candidate decomposition meets the selection criterion. The second candidate decomposition is then determined as the selected candidate decomposition. A particular advantage is that the processing efficiency of matrix-matrix multiplication can be improved.

[0079] According to example implementations, the selected candidate decomposition corresponds to the subdivision of the third matrix into a multitude of submatrices. Each submatrix is ​​assigned to a group of processes, and a sub-matrix decomposition is determined for the multitude of submatrices. Data representing the sub-matrix decomposition is provided to instruct each processing thread to execute a sub-cycle based on the sub-matrix decomposition. A particular advantage is that the processing efficiency of matrix-matrix multiplication can be improved.

[0080] In accordance with example implementations, the system determines, based on the characteristic(s) of the selected candidate decomposition, whether the use of a buffer to store preliminary multiplication results is recommended. A routine updates the third matrix using additive updates of the contents stored in the buffer. A particular advantage is that the processing efficiency of the matrix-matrix multiplication can be increased.

[0081] While the present disclosure has been described with respect to a limited number of implementations, the person skilled in the art, aware of the advantages of this disclosure, will appreciate numerous modifications and variations thereof. It is intended that the appended claims cover all such modifications and variations.

Claims

[1] Non-transitory storage medium (800) that stores machine-readable instructions (804) which, when executed by a machine, cause the machine to: Access to data representing a first dimension of a first matrix (210), a second dimension of a second matrix (214), and a third dimension shared by the first matrix (210) and the second matrix (214), Determining a multitude of candidate decompositions for the first matrix (210), wherein the candidate decompositions vary in size relative to each other along the first dimension and the third dimension; Determining an associated fitness value for each candidate decomposition from the multitude of candidate decompositions; Selecting a candidate decomposition from the multitude of candidate decompositions based on fitness scores to provide a selected candidate decomposition; and Providing data representing the selected candidate decomposition to effect the assignment of processing threads (112) to sub-matrices of the first matrix (210), wherein the processing threads (112) determine a third matrix (220) based on a multiplication of the first matrix (210) and the second matrix (214); wherein the instructions (804), when executed by the machine, further cause the machine to determine a block size for the plurality of decomposition candidates, such that each decomposition candidate of the plurality of decomposition candidates is decomposed along the third dimension according to the block size, and wherein the instructions (804), when executed by the machine, further cause the machine to determine the block size based on a property of a cache memory used by the processing threads (112) in determining the third matrix (220). [2] Storage medium (800) according to claim 1, wherein the instructions (804), when executed by the machine, also cause the machine to: for each fitness value of the fitness values: to determine a multitude of conditions for the associated candidate decomposition; and to determine the respective fitness value based on the multitude of conditions. [3] Storage medium (800) according to claim 2, wherein the instructions (804), when executed by the machine, also cause the machine to: for the respective fitness value: to normalize each condition of the multitude of conditions in order to provide a multitude of normalized condition values; and Combining several normalized state values ​​to determine the respective fitness value. [4] Storage medium (800) according to claim 1, wherein the instructions (804), when executed by the machine, also cause the machine to: to rank the fitness values ​​in order to provide a corresponding ranking of the multiple candidate decompositions; and to select the chosen candidate breakdown based on the ranking. [5] Storage medium (800) according to claim 4, wherein the instructions (804), when executed by the machine, also cause the machine to: to select an initial candidate breakdown based on the ranking; to determine that the first decomposition candidate does not meet a selection criterion; to select a second candidate decomposition that differs from the first candidate decomposition, based on the ranking; to determine that the second candidate breakdown meets the selection criterion; and to determine the second candidate decomposition as the selected candidate decomposition. [6] Storage medium (800) according to claim 1, wherein the selected candidate decomposition corresponds to the third matrix (220) which is subdivided into a plurality of sub-matrices, each sub-matrix of the plurality of sub-matrices being assigned to a group of processes of the plurality of processes, and the instructions (804), when executed by the machine, further cause the machine to: to determine a large number of candidate submatrix decompositions, where the candidate submatrix decompositions vary in size relative to each other along the first and second dimensions; to determine an associated second fitness value for each candidate sub-matrix decomposition from the multitude of candidate sub-matrix decompositions; to select a candidate submatrix decomposition from the multitude of candidate submatrix decompositions based on the second fitness values ​​in order to provide a selected candidate submatrix decomposition; and To provide data representing the selected candidate sub-matrix decomposition to cause each processing thread (112) of the processing threads (112) to perform a sub-cycle based on the selected sub-matrix decomposition. [7] Storage medium (800) according to claim 1, wherein the selected candidate decomposition corresponds to the third matrix (220) which is subdivided into a plurality of sub-matrices, each sub-matrix of the plurality of sub-matrices being assigned to a group of processes of the plurality of processes, and the instructions (804), when executed by the machine, further cause the machine to: to determine a sub-matrix decomposition for the multitude of sub-matrices; and To provide data representing the sub-matrix decomposition in order to cause each processing thread (112) of the processing threads (112) to perform a sub-cycle based on the sub-matrix decomposition. [8] Storage medium (800) according to claim 1, wherein the instructions (804), when executed by the machine, further cause the machine to determine, on the basis of at least one property of the selected candidate decomposition, whether the use of a buffer to store preliminary results of the multiplication should be recommended, wherein a routine updates the third matrix (220) using additive updates of contents stored in the buffer. [9] Storage medium (800) according to claim 8, wherein the instructions (804), when executed by the machine, further cause the machine to determine whether the use of the buffer should be recommended, based on at least one of the following: a comparison of a first block size of the candidate decomposition along the first dimension with a first cache optimum block size along the first dimension, a comparison of a second block size of the candidate decomposition along the second dimension with a second cache optimum block size along the second dimension, or a comparison of a third block size of the candidate decomposition along the third dimension with a third cache optimum block size along the third dimension. [10] Storage medium (800) according to claim 1, wherein: the first matrix (210) comprises an MX K-matrix, the second matrix (214) comprises a KX N-matrix; the first dimension corresponds to an M-dimension of the MX K-matrix; the second dimension of an N dimension of the KX N-matrix corresponds to; the third dimension corresponds to the K-dimension of the MX K-matrix and the K-dimension of the KX N-matrix; the multiplication includes a (MXK)(KXN) matrix multiplication; and Determining the multitude of candidate decompositions for the first matrix (210) includes M / K threading. [11] Storage medium (800) according to claim 1, wherein: the second matrix (214) comprises an MX K-matrix, the first matrix (210) comprises a KX N-matrix; the first dimension of an N dimension of the KX N-matrix corresponds to; the second dimension corresponds to an M-dimension of the MX K-matrix; the third dimension corresponds to the K-dimension of the MX K-matrix and the K-dimension of the KX N-matrix; the multiplication includes a (MXK)(KXN) matrix multiplication; and Determining the multitude of candidate decompositions for the first matrix (210) includes N / K threading. [12] Device (900) comprising the following: a processor (910); and a memory (904) for storing instructions (908) which, when executed by the processor (910), cause the processor (910) to: to perform threading of a first matrix (210) along a first dimension of the first matrix (210) and a second dimension of the matrix, wherein the threading represents block sizes of the first matrix (210) to allocate them to process threads of a multiplication algorithm to determine a third matrix (220) that is a product of the first matrix (210) and a second matrix (214), wherein the block sizes comprise a first block size along the first dimension and a second block size along the second dimension, and the second matrix (214) shares the second dimension with the first matrix (210); and Provide the multiplication algorithm with data representing the first block size and the second block size; wherein a multitude of candidate decompositions are determined for the first matrix (210), the candidate decompositions varying in size relative to each other along the first and third dimensions; a corresponding fitness value is determined for each candidate decomposition from the multitude of candidate decompositions; a candidate decomposition is selected from the multitude of candidate decompositions based on fitness values ​​to provide a selected candidate decomposition; Data are provided representing the selected candidate decomposition to effect the assignment of processing threads (112) to sub-matrices of the first matrix (210), wherein the processing threads (112) determine a third matrix (220) based on a multiplication of the first matrix (210) and the second matrix (214); wherein a block size is determined for the plurality of decomposition candidates such that each decomposition candidate of the plurality of decomposition candidates is decomposed along the third dimension according to the block size; and where the block size is determined based on a property of a cache memory used by the processing threads (112) in determining the third matrix (220). [13] Device (900) according to claim 12, wherein the multiplication algorithm comprises a generalized matrix-matrix multiplication algorithm (GEMM). [14] Device (900) according to claim 12, wherein the instructions (908), when executed by the processor (910), also cause the processor (910) to: Determining the second block size; and Evaluating candidate matrix decompositions, each decomposed along the second dimension according to the second block size, using the evaluation to determine the second block size. [15] Device (900) according to claim 12, wherein the threading corresponds to the subdivision of the third matrix (220) into a plurality of sub-matrices, wherein each sub-matrix of the plurality of sub-matrices is assigned to a group of threads of the plurality of threads, and the instructions (908), when executed by the processor (910), further cause the processor (910) to: Determining a sub-matrix decomposition for the plurality of sub-matrices; and Providing data representing the sub-matrix decomposition to cause each thread of the multitude of threads to perform a sub-cycle based on the sub-matrix decomposition. [16] Device (900) according to claim 12, wherein the instructions (908), when executed by the processor (910), further cause the processor (910) to determine, on the basis of at least one of the block sizes, whether the use of a buffer to store preliminary results of the multiplication should be recommended, wherein a routine updates the third matrix (220) using additive updates of contents stored in the buffer. [17] Method (1000) comprising the following: at least one hardware processor accessing data (1002) representing a first matrix (210), a second matrix (214), a first block size and a second block size, wherein the first block size represents a sub-matrix decomposition size of the first matrix (210) along a first dimension, the second block size represents a sub-matrix decomposition size of the first matrix (210) along the second dimension, and the second matrix (214) and the first matrix (210) have the second dimension in common; wherein the at least one hardware processor multiplies the first matrix (210) and the second matrix (214) (1004) to determine a third matrix (220), wherein the multiplication (1004) includes the allocation of processing threads (112) to submatrices of the first matrix (210) based on the first block size and the second block size; wherein a multitude of candidate decompositions are determined for the first matrix (210), the candidate decompositions varying in size relative to each other along the first and third dimensions; a corresponding fitness value is determined for each candidate decomposition from the multitude of candidate decompositions; a candidate decomposition is selected from the multitude of candidate decompositions based on fitness values ​​to provide a selected candidate decomposition; Data are provided representing the selected candidate decomposition to effect the assignment of processing threads (112) to sub-matrices of the first matrix (210), wherein the processing threads (112) determine a third matrix (220) based on a multiplication of the first matrix (210) and the second matrix (214); wherein a block size is determined for the plurality of decomposition candidates such that each decomposition candidate of the plurality of decomposition candidates is decomposed along the third dimension according to the block size; and where the block size is determined based on a property of a cache memory used by the processing threads (112) in determining the third matrix (220). [18] Method (1000) according to claim 17, wherein the multiplication (1004) comprises that each processing thread (112) of the processing threads (112) subcycles over subblocks of the sub-matrix assigned to the processing thread (112).

Images