Matrix multiplier with submatrix sequencing

Abstract

A graphics processing unit (GPU) [100] of a processor sequences application of sub-matrices at a matrix multiplier [110] to reduce the number of input changes at input registers [106, 107] of the matrix multiplier. The matrix multiplier is configured to perform matrix multiplication on relatively small matrices. To multiply two larger matrices [220, 222], the GPU decomposes the larger matrices into smaller sub-matrices [221] and stores the sub-matrices at the input registers of the matrix multiplier in a sequence to compute each column of a result matrix. The GPU sequences the storage of the sub-matrices at the input registers to maintain input data at one of the input registers over multiple computation cycles of the matrix multiplier, thereby reducing power consumption at the GPU.

Description

Background Technology

[0001] Modern processor applications typically require relatively complex manipulations of vectors, matrices, and similar structures. For example, vector and matrix manipulation is used in graphics operations, digital signal processing applications, neural network applications, and so on. To improve the processing efficiency of these applications and operations, processors may include graphics processing units (GPUs). GPUs include dedicated hardware to perform parallel processing on relatively large blocks of data. Therefore, GPUs can support graphics applications as well as other applications that require vector and matrix manipulation. To further improve processing efficiency, GPUs may include dedicated hardware to perform specific types of matrix operations, including matrix multiplication. However, this conventional approach using dedicated hardware can consume significant amounts of power and thus negatively impact processor performance. Attached Figure Description

[0002] This disclosure will be better understood by referring to the accompanying drawings, and many features and advantages of this disclosure will become apparent to those skilled in the art. The same reference numerals are used in different drawings to indicate similar or identical items.

[0003] Figure 1 This is a block diagram of a GPU for a processor according to some implementation schemes, the GPU being configured to perform matrix multiplication by ordering the application of submatrices to matrix multipliers.

[0004] Figure 2 This illustrates, according to some implementation schemes, for use in Figure 1 A diagram of an example matrix where multiplication is performed at the GPU.

[0005] Figure 3 This shows the pair matrix according to some implementation schemes. Figure 1 A diagram illustrating an example of the application of matrix multipliers for ordering.

[0006] Figure 4 It is based on the submatrix ordering of some implementation schemes. Figure 1 A block diagram of additional aspects of the GPU.

[0007] Figure 5 This is a flowchart illustrating a method for ordering the application of submatrices at the matrix multiplier of a GPU, based on some implementation schemes. Detailed Implementation

[0008] Figures 1 to 5This paper illustrates a technique for reducing power consumption at the processor's graphics processing unit (GPU) by ordering the application of submatrices at the matrix multiplier to reduce the number of input changes at the matrix multiplier's input register. The matrix multiplier is configured to perform matrix multiplication on relatively small matrices (e.g., 4×4 matrices). To multiply two larger matrices (e.g., 16×16 matrices), the GPU decomposes the larger matrix into smaller submatrices and stores them sequentially at the input register of the matrix multiplier, thereby computing each column of the resulting matrix. The GPU orders the storage of submatrices at the input register to maintain input data at one location in the input register across multiple computation cycles of the matrix multiplier, thereby reducing power consumption at the GPU.

[0009] Figure 1 This refers to a GPU 100 of a processor according to some embodiments, which is configured to perform matrix multiplication by ordering the application of submatrices. In at least one embodiment, GPU 100 is part of a processor typically configured to execute an instruction set to perform operations on behalf of an electronic device. Thus, in different embodiments, GPU 100 is part of an electronic device such as a desktop or laptop computer, a server, a handheld electronic device (such as a smartphone or tablet), a game console, etc. GPU 100 is typically configured to perform graphics and vector processing operations on behalf of a processor. For example, in some embodiments, the processor's central processing unit (CPU, ... Figure 1 (not shown) provides the GPU with a set of operations for execution, where the set of operations is associated with graphics or vector processing.

[0010] To facilitate the execution of the provided operations, GPU 100 includes multiple Single Instruction Multiple Data (SIMD) processing units (e.g., SIMD units 102 and 104). It should be understood that GPU 100 also includes additional modules to support the SIMD units, such as fetch and decode logic for fetching and decoding instructions for the SIMD units, register files for storing operands of the SIMD units, etc. To support matrix multiplication operations on a set of input matrices 105, each SIMD unit includes a matrix multiplier, a corresponding input register, and a corresponding output register. For example, SIMD unit 102 includes a matrix multiplier 110, input registers 106 and 107, and an output register 108. It should be understood that the term "register" refers to any memory module configured to store matrices (including submatrices).

[0011] Matrix multiplier 110 is configured to multiply the matrices stored in registers 106 and 107 and store the resulting product in register 108. The generation of a single product for the matrices in input registers 106 and 102 is referred to herein as a “multiplication cycle” of matrix multiplier 110. In some embodiments, SIMD unit 102 is timed by a clock signal (denoted as “CLK”), and the multiplication cycle of matrix multiplier 110 corresponds to a single clock cycle of the CLK clock signal. That is, within a single clock cycle of the CLK clock signal, matrix multiplier 110 is configured to generate a product in register 108 based on the input operands stored in input registers 106 and 107. In other embodiments, each multiplication cycle of matrix multiplier 110 requires multiple cycles of the CLK clock signal.

[0012] In some implementations, to save circuit area, matrix multiplier 110 is configured to generate a product of relatively small input matrices. For example, in some implementations, matrix multiplier 110 is a 4×4×4 multiplier, such that matrix multiplier 110 is configured to multiply a 4×4 matrix stored at input register 106 with a 4×4 matrix stored at input register 107 to generate a 4×4 product (result) matrix at output register 108. However, in some implementations, the CPU provides the GPU 100 with operations requiring multiplication of larger matrices (such as multiplication of 16×16 matrices). To support multiplication of larger matrices, SIMD is configured to decompose the larger matrix into multiple smaller submatrices, multiply the submatrices at matrix multiplier 110 to generate a set of intermediate structures, and combine the intermediate results to generate a product of the larger matrix.

[0013] For illustration, in some embodiments, matrix multiplier 110 multiplies the input matrices (represented as matrix A (M×K matrix) and matrix B (K×N matrix)) to compute the result matrix (M×N matrix R). Matrix A and B are stored in input registers 106 and 107, respectively, and the result matrix R is stored in output register 108. In some embodiments, matrix multiplier 110 computes the result matrix R by calculating the inner product (dot product) of the i-th row and j-th column, as illustrated by the following equation:

[0014] For i in [0..M-1] and j in [0..N-1], R i,j =Sum A i,k *B k,j k = 0..K-1

[0015] In other implementations, matrix multiplier 110 computes the result matrix R by summing the K outer products of columns k of matrix A and rows k of matrix B, as illustrated by the following equation:

[0016] R = Sum R k R k i,j =A i,k *B k,j

[0017] For a large matrix, SIMD 102 decomposes the input matrix into smaller submatrices of a specified input size for matrix multiplier 110, multiplies the submatrices at matrix multiplier 110 to generate a set of intermediate results, and combines the intermediate results to determine the final result matrix R. Therefore, in an implementation where the matrix multiplier computes the inner product, SIMD 102 decomposes the input matrix into smaller submatrices, determines the product of different groups based on the dot products of different groups of submatrices, and then computes the outer product of the resulting dot products to determine the final result matrix.

[0018] It should be understood that, as further described herein, computing different intermediate results (different submatrix products) requires changing the submatrix stored in input registers 106 and 107. However, each change to the data stored in the input registers consumes power at GPU 102. Additionally, each change to the data at the input of the corresponding arithmetic logic unit (ALU) or other module of matrix multiplier 110 consumes additional power compared to keeping the input data in a constant state. Therefore, to save power, SIMD 102 orders the storage of the submatrix in input registers 106 and 107 such that the submatrix remains in one of the input registers (e.g., register 107) for multiple consecutive multiplication cycles until the computation of the result matrix R no longer requires the submatrix. In other words, SIMD 102 orders the application of the input submatrix in input registers 106 and 107 to reduce the amount of input switching at one of the registers and therefore at one of the inputs of matrix multiplier 110, thereby saving power.

[0019] An example of ordering the input submatrix at GPU 100. Figure 2 and Figure 3 Describe it. Figure 2 Examples of two 16×16 matrices, 220 and 222, denoted as matrix A and matrix B, are shown. Each of matrices A and B comprises 16 4×4 submatrices (e.g., submatrix 221 of matrix A). Matrices A and B are multiplied at GPU 102 to produce a resulting matrix 224 (denoted as matrix R), which also comprises multiple 4×4 submatrices. Matrix R can be considered as a set of columns of the submatrices. For example, the first column of R consists of submatrices R0. 0,0 R 1,0 R 2,0 and R 3,0Composition. As further described below, in some implementations, GPU 100 computes matrix R by computing each column of a submatrix of R, and then concatenates the different columns to form matrix R. In some implementations, each column of a submatrix of R is computed simultaneously at different corresponding SIMDs of GPU 100, and then one of the SIMDs concatenates the submatrix columns to form R.

[0020] To compute the columns of a submatrix of R, the corresponding SIMD method uses its matrix multiplier to determine a set of inner products (dot products) of the corresponding submatrices of matrices A and B, and then concatenates the outer products on the inner product results. For example, to generate a submatrix R... 0,0 SIMD 102 performs the following calculations:

[0021] R 0,0 =A 0,0 *B 0,0

[0022] R 0,0 +=A 0,1 *B 1,0

[0023] R 0,0 +=A 0,2 *B 2,0

[0024] R 0,0 +=A 0,3 *B 3,0

[0025] The "+=" signifies adding the corresponding product to R. 0,0 The previous value. Similarly, in order to generate the submatrix R. 1,0 SIMD 102 performs the following calculations:

[0026] R 1,0 =A 1,0 *B 0,0

[0027] R 0,0 +=A 1,1 *B 1,0

[0028] R 0,0 +=A 1,2 *B 2,0

[0029] R 0,0 +=A 1,3 *B 3,0

[0030] SIMD 102 performs simulation calculations to generate submatrix R. 2,0 and R3,0 .

[0031] To perform each multiplication for calculating the corresponding submatrices, SIMD 102 loads the corresponding submatrices of matrices A and B into input registers 106 and 107, respectively, and matrix multiplier 110 performs the multiplication, storing the result in output register 108. However, as can be seen from the example above, submatrices of matrix B are reused to calculate different submatrices of matrix R. Therefore, SIMD 102 is configured to order the multiplications such that the submatrices of matrix B, such as those stored in input register 107, remain unchanged across multiple consecutive multiplication cycles of matrix multiplier 110. This reduces the number of times input to input register 107 is loaded and the changes to the input of matrix multiplier 110, thus reducing power consumption.

[0032] According to some implementation schemes, examples of a portion of the input sequence are shown in... Figure 3 It is shown here. Figure 3 The diagram shows a set of consecutive multiplication cycles 301 to 305 and the corresponding contents of each of the input registers 106 and 107. Therefore, in the depicted example, within multiplication cycle 301, SIMD 102 multiplies submatrix A... 0,0 and B 0,0 The results are loaded into input registers 106 and 107 respectively. As further described below, matrix multiplier 110 multiplies the submatrices to compute the intermediate result of the first column of the result matrix R. In the next multiplication cycle 302 (the multiplication cycle immediately following multiplication cycle 301), SIMD 102 multiplies the submatrix A 1,0 Submatrix B is loaded into input register 106, but remains in input register 107. 0,0 Matrix multiplier 110 multiplies the submatrices to compute another intermediate result in the first column of the resulting matrix R.

[0033] In the next multiplication cycle 303, SIMD 102 will submatrix A 2,0 Submatrix B is loaded into input register 106, but remains in input register 107. 0,0 Matrix multiplier 110 multiplies the submatrices to compute another intermediate result from the first column of the resulting matrix R. In the next multiplication cycle 304, SIMD 102 multiplies the submatrix A... 3,0 Submatrix B is loaded into input register 106, but remains in input register 107. 0,0 Matrix multiplier 110 multiplies the submatrices to compute another intermediate result in the first column of the resulting matrix R. Within the next multiplication cycle 305, the required submatrix B has been processed. 0,0All calculations. Therefore, within a multiplication period of 305, SIMD 102 will perform submatrix A 0,1 Load into input register 106, and set submatrix B 1,0 The submatrix B is loaded into input register 107. Therefore, in the depicted example, SIMD 102 maintains the submatrix B at input register 107 for four consecutive (continuous) multiplication cycles. 0,0 This saves the power that would otherwise be needed to change the input submatrix and compute the results of matrix multiplication.

[0034] SIMD 102 continues to perform multiplication operations at matrix multiplier 110 and combines the resulting products to compute the first column of the result matrix R. The sequence of multiplications (including the corresponding input matrices loaded and maintained at input registers 106 and 107) is as follows:

[0035] R 0,0 =A 0,0 *B 0,0

[0036] R 1,0 =A 1,0 *B 0,0 <= Maintain B input

[0037] R 2,0 =A 2,0 *B 0,0 <= Maintain B input

[0038] R 3,0 =A 3,0 *B 0,0 <= Maintain B input (using 4×)

[0039] R 0,0 +=A 0,1 *B 1,0 <= Update B input

[0040] R 1,0 +=A 1,1 *B 1,0 <= Maintain B input

[0041] R 2,0 +=A 2,1 *B 1,0 <= Maintain B input

[0042] R 3,0 +=A 3,1 *B 1,0 <= Maintain B input (using 4×)

[0043] R 0,0 +=A 0,2 *B 2,0<= Update B input

[0044] R 1,0 +=A 1,2 *B 2,0 <= Maintain B input

[0045] R 2,0 +=A 2,2 *B 2,0 <= Maintain B input

[0046] R 3,0 +=A 3,2 *B 2,0 <= Maintain B input (using 4×)

[0047] R 0,0 +=A 0,3 *B 3,0 <= Update B input

[0048] R 1,0 +=A 1,3 *B 3,0 <= Maintain B input

[0049] R 2,0 +=A 2,3 *B 3,0 <= Maintain B input

[0050] R 3,0 +=A 3,3 *B 3,0 <= Complete the calculation for the first column of R.

[0051] GPU 102 performs similar computations to compute the other columns of the resulting matrix R. In some implementations, GPU 102 employs different SIMDs to simultaneously compute the corresponding columns of the resulting matrix R, and uses one or more modules of the SIMDs to concatenate the different columns into the final resulting matrix R.

[0052] Figure 4 This illustrates, according to some embodiments, methods for supporting the ordering of input submatrices of matrix multiplier 110. Figure 1Additional aspects of SIMD 102. In the depicted example, SIMD 102 includes a data storage 435 connected to a sequencer 430. The data storage 435 is a buffer, cache, register file, or other memory structure configured to store submatrices (e.g., submatrix 433) for matrix multiplier 110. The sequencer 430 is a hardware module configured to decompose the input matrix 105 (matrix A and matrix B) into corresponding submatrices and store the submatrices at the data storage 435. The sequencer 430 is also configured to retrieve one or more submatrices from the data storage 435 during corresponding multiplication cycles and load each retrieved submatrix into corresponding input registers 106 and 107. Thus, the sequencer 430 controls the ordering of the input submatrices at matrix multiplier 110 to perform matrix multiplication on relatively large matrices.

[0053] Figure 5 This is a flowchart of a method 500 for ordering the application of submatrices at the matrix multiplier of a GPU, based on some implementation schemes. For descriptive purposes, regarding... Figure 1 The example implementation at GPU 100 is used to describe method 500. At box 502, sequencer 430 initializes the submatrix (e.g., submatrix A). 0,0 and B 0,0 The submatrices stored in input registers 106 and 107 are loaded from data storage 435, respectively. At block 504, matrix multiplier 110 multiplies the submatrices stored in input registers 106 and 107 to generate a product and adds the result to the intermediate result (if any) of the corresponding column of the result matrix R, as described above. The method flow moves to block 506 and sequencer 430 determines whether the input submatrix at input register 106 corresponds to the last row of matrix A. If not, the method flow moves to block 508 and sequencer 430 loads the submatrix of A corresponding to the current column (e.g., column 0) and the next row into input register 106. The submatrix of B stored in input register 107 is maintained to save power. The method flow returns to block 504 and matrix multiplier 110 performs the next multiplication operation, that is, performs the next multiplication cycle.

[0054] Returning to box 506, if the input submatrix at input register 106 corresponds to the last row of matrix A, the method flow moves to box 510 and sequencer 430 determines whether the input submatrix stored at input register 510 corresponds to the last row of matrix B. If not, the method flow moves to box 512 and sequencer 430 loads the submatrix of B corresponding to the columns of R being computed into input register 107. Additionally, sequencer 430 loads the submatrix of A corresponding to the initial row (e.g., row 0) and the next column into input register 106. The method flow returns to box 504 and matrix multiplier 110 performs the next multiplication operation.

[0055] Returning to box 510, if the input submatrix stored in input register 510 corresponds to the last row of matrix B, the calculation of the corresponding column of the result matrix R is completed. Therefore, the method flow moves to box 514 and SIMD 102 stores the final result of the columns of R. At box 516, GPU 100 combines each of the calculated columns to generate the result matrix R. In some embodiments, GPU 102 provides the result matrix R to the CPU for further processing. In other embodiments, GPU 100 uses the result matrix R to, for example, generate one or more objects in a display frame and provides the display frame to a frame buffer for display on a display device.

[0056] As disclosed herein, in some embodiments, a method includes: multiplying a first matrix and a second matrix within a first multiplication cycle of a matrix multiplier of a graphics processing unit (GPU): multiplying a first submatrix of the first matrix stored in a first input register with a first submatrix of the second matrix stored in a second input register; within a second multiplication cycle of the matrix multiplier, the second multiplication cycle following the first multiplication cycle: multiplying the first submatrix of the first matrix stored in the first input register with a second submatrix of the second matrix stored in the second input register; and maintaining the first submatrix at the first input register within the first and second multiplication cycles. In one aspect, the method includes: within a third multiplication cycle of the matrix multiplier, the third multiplication cycle following the second multiplication cycle: multiplying the first submatrix of the first matrix stored in the first input register with a second submatrix of the second matrix stored in the second input register; and maintaining the first submatrix at the first input register within the first, second, and third multiplication cycles. In another aspect, the first submatrix includes at least one non-zero element.

[0057] In one aspect, the method includes determining a product of a first matrix and a second matrix based on the results of a first multiplication cycle and a second multiplication cycle, the product comprising a result matrix. In another aspect, determining the product includes determining a submatrix of the result matrix based on the results of the first multiplication cycle and the second multiplication cycle. In yet another aspect, the submatrix of the result matrix comprises one of the columns and rows of the result matrix. In still another aspect, determining the product includes determining an outer product based on the results of the first multiplication cycle and the second multiplication cycle. In yet another aspect, the method includes: within a third multiplication cycle of the matrix multiplier, the third multiplication cycle following the second multiplication cycle: multiplying a second submatrix of the first matrix stored in a first input register with a second submatrix of the second matrix stored in the second input register; and within the third multiplication cycle, changing the first submatrix of the first matrix stored in the first input register to the second submatrix of the first matrix.

[0058] As disclosed herein, in some embodiments, a method includes multiplying a submatrix of a first matrix with a submatrix of a second matrix at a matrix multiplier of a graphics processing unit (GPU), wherein the multiplication includes maintaining the first submatrix at a first input register of the matrix multiplier for a first plurality of multiplication cycles. In one aspect, the multiplication further includes changing the submatrix at a second input register of the matrix multiplier for the first plurality of multiplication cycles. In another aspect, the multiplication further includes maintaining the second submatrix at a second input register of the matrix multiplier for a second plurality of multiplication cycles. In yet another aspect, at least one element of the first submatrix is ​​a non-zero element.

[0059] As disclosed herein, in some embodiments, a graphics processing unit (GPU) includes: a first input register; a second input register; a matrix multiplier for multiplying a submatrix stored in the first input register with a submatrix stored in the second input register; and a sequencer for controlling the submatrixes stored in the first and second input registers, the sequencer being configured to: during a first multiplication cycle of the matrix multiplier, store a first submatrix of a first matrix in the first input register and store a first submatrix of a second matrix in the second input register; during a second multiplication cycle of the matrix multiplier, maintain the first submatrix of the first matrix in the first input register and store a second submatrix of the second matrix in the second input register, the second multiplication cycle following the first multiplication cycle. In one aspect, the sequencer is configured to: during a third multiplication cycle of the matrix multiplier, the third multiplication cycle following the first multiplication cycle, maintain the first matrix stored in the first input register and store a second submatrix of the second matrix in the second input register. In another aspect, the first submatrix includes at least one non-zero element.

[0060] In one aspect, the GPU is configured to determine the product of the first matrix and the second matrix based on the results of the first multiplication cycle and the second multiplication cycle, the product comprising a result matrix. In another aspect, the GPU is configured to determine the product by determining a submatrix of the result matrix based on the results of the first multiplication cycle and the second multiplication cycle. In yet another aspect, the submatrix of the result matrix comprises one of the columns and rows of the result matrix. In yet another aspect, the GPU is configured to determine the product by determining an outer product based on the results of the first multiplication cycle and the second multiplication cycle. In yet another aspect, the sequencer is configured to: within a third multiplication cycle of the matrix multiplier, the third multiplication cycle following the second multiplication cycle: store a second submatrix of the first matrix at the first input register and store a second submatrix of the second matrix at the second input register.

[0061] Computer-readable storage media may include any non-transitory storage medium or a combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media may include, but are not limited to, optical media (e.g., compressed optical discs (CDs), digital versatile optical discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disk drives), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. Computer-readable media may be embedded in a computing system (e.g., system RAM or ROM), permanently attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or a USB-based flash memory), or coupled to a computer system via a wired or wireless network (e.g., network accessible storage (NAS)).

[0062] In some implementations, certain aspects of the techniques described above may be implemented by one or more processors of a processing system executing software. The software includes one or more sets of executable instructions stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software may include instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects described above. The non-transitory computer-readable storage medium may include, for example, magnetic or optical disk storage devices, solid-state storage devices (such as flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices), etc. The executable instructions stored on the non-transitory computer-readable storage medium may be implemented as source code, assembly language code, object code, or other instruction formats interpreted or otherwise executed by one or more processors.

[0063] It should be noted that not all activities or elements described in the general description above are required; a particular activity or device may not be required as part of a specific activity or device, and one or more other activities may be performed, or elements other than those described may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Additionally, the concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes may be made without departing from the scope of this disclosure as set forth in the appended claims. Therefore, this specification and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of this disclosure.

[0064] The benefits, other advantages, and problem-solving solutions described above with respect to specific embodiments have been presented. However, the benefits, advantages, problem-solving solutions, and any features that bring any benefit, advantage, or problem-solving solution to fruition or become more prominent should not be construed as key, essential, or necessary features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter can be modified and practiced in different but equivalent ways that would be apparent to a person skilled in the art benefiting from the teachings herein. No limitation is intended to be made on the details of the constructions or designs shown herein other than those described in the appended claims. Therefore, it is apparent that the specific embodiments disclosed above can be altered or modified, and all such changes are considered to be within the scope of the disclosed subject matter. Thus, the protection sought herein is as set forth in the appended claims.

Claims

1. A matrix multiplication method, the method comprising: Within the first multiplication cycle [301] of the matrix multiplier [110] of the graphics processing unit (GPU) [100], a first matrix [220] and a second matrix [222] are multiplied, wherein the matrix multiplier is configured to perform matrix multiplication on matrices of a specific size to generate the result matrix of the matrix multiplier: Multiply the first submatrix of the first matrix stored in the first input register [106] with the first submatrix of the second matrix stored in the second input register [107]; Within the second multiplication cycle [302] of the matrix multiplier, the second multiplication cycle follows the first multiplication cycle: Multiply the first submatrix of the first matrix stored in the first input register with the second submatrix of the second matrix stored in the second input register; as well as The first submatrix is ​​maintained at the first input register during the first multiplication cycle and the second multiplication cycle. Wherein, the size of each of the first submatrix of the first matrix, the first submatrix of the second matrix, and the second submatrix of the second matrix is ​​the specific size, and wherein the graphics processing unit decomposes the first matrix and the second matrix into submatrixes of the specific size and orders the storage of the submatrixes at the first input register and the second input register, and maintains the input data at one of the first input register and the second input register for multiple calculation cycles after the second multiplication cycle of the matrix multiplier.

2. The method of claim 1, further comprising: Within the third multiplication cycle [303] of the matrix multiplier, the third multiplication cycle follows the second multiplication cycle: Multiply the first submatrix of the first matrix stored in the first input register with the third submatrix of the second matrix stored in the second input register; as well as The first submatrix is ​​maintained at the first input register during the first multiplication cycle, the second multiplication cycle, and the third multiplication cycle.

3. The method of claim 1, wherein the first submatrix comprises at least one non-zero element.

4. The method of claim 1, further comprising: The product of the first matrix and the second matrix is ​​determined based on the results of the first multiplication period and the second multiplication period, and the product includes the final result matrix [224].

5. The method of claim 4, wherein determining the product comprises: The submatrices of the final result matrix are determined based on the results of the first multiplication cycle and the second multiplication cycle.

6. The method of claim 5, wherein the submatrix of the final result matrix includes one of the columns and rows of the final result matrix.

7. The method of claim 4, wherein determining the product comprises: The outer product is determined based on the results of the first multiplication period and the second multiplication period.

8. The method of claim 1, further comprising: In the third multiplication cycle of the matrix multiplier, the third multiplication cycle follows the second multiplication cycle: Multiply the second submatrix of the first matrix stored in the first input register with the second submatrix of the second matrix stored in the second input register; as well as During the third multiplication cycle, the first submatrix of the first matrix stored in the first input register is changed to the second submatrix of the first matrix.

9. A graphics processing unit (GPU) [100], comprising: First input register [106]; Second input register [107]; A matrix multiplier [110] is configured to multiply a submatrix stored in the first input register with a submatrix stored in the second input register, the matrix multiplier being configured to perform matrix multiplication on a matrix of a specific size to generate the result matrix of the matrix multiplier; as well as Sequencer [430], the sequencer being configured to control submatrices stored in the first input register and the second input register, the sequencer being configured to: During the first multiplication cycle [301] of the matrix multiplier, the first submatrix of the first matrix is ​​stored in the first input register and the first submatrix of the second matrix is ​​stored in the second input register; During the second multiplication cycle [302] of the matrix multiplier, the first submatrix of the first matrix is ​​maintained at the first input register and the second submatrix of the second matrix is ​​stored at the second input register. The second multiplication cycle follows the first multiplication cycle. Wherein, the size of each of the first submatrix of the first matrix, the first submatrix of the second matrix, and the second submatrix of the second matrix is ​​the specific size, and wherein the graphics processing unit decomposes the first matrix and the second matrix into submatrixes of the specific size and orders the storage of the submatrixes at the first input register and the second input register, and maintains the input data at one of the first input register and the second input register for multiple calculation cycles after the second multiplication cycle of the matrix multiplier.

10. The graphics processing unit (GPU) of claim 9, wherein the sequencer is configured to: In the third multiplication cycle of the matrix multiplier, the third multiplication cycle follows the second multiplication cycle: Maintain the first matrix stored in the first input register and store the third submatrix of the second matrix in the second input register.

11. The graphics processing unit (GPU) of claim 9, wherein the first submatrix includes at least one non-zero element.

12. The graphics processing unit (GPU) of claim 9, wherein the GPU is configured to: The product of the first matrix and the second matrix is ​​determined based on the results of the first multiplication period and the second multiplication period, and the product includes the final result matrix.

13. The graphics processing unit (GPU) of claim 12, wherein the GPU is configured to determine the product in the following manner: The submatrices of the final result matrix are determined based on the results of the first multiplication cycle and the second multiplication cycle.

14. The graphics processing unit (GPU) of claim 13, wherein the submatrix of the final result matrix includes one of the columns and rows of the final result matrix.

15. The graphics processing unit (GPU) of claim 12, wherein the GPU is configured to determine the product in the following manner: The outer product is determined based on the results of the first multiplication period and the second multiplication period.

Images