Matrix multiplication unit using flexible precision arithmetic

Abstract

The processing unit (140) includes a plurality of vector signal processors (VSPs) (151-154), each VSP including multiplication / accumulation elements (320-323). The processing unit also includes a plurality of registers associated with the plurality of VSPs. Before a first round comprising multiple iterations, first portions of the first and second matrices (205, 210) are extracted into the plurality of registers. Before extracting second portions of the first and second matrices into the plurality of registers for a second round, the multiplication / accumulation elements perform matrix multiplication and accumulation on different combinations of subsets of the first portions of the first and second matrices during the multiple iterations. In response to completion of the multiple iterations, the accumulated result of multiplying the first portions of the first and second matrices is written into output buffers (325-328).

Description

Background Technology

[0001] Data-intensive applications such as deep learning, high-performance computing (HPC), cloud computing, and graphics rendering are used to address challenges including large-scale simulations, climate change, computational biology, disease prevention, and financial modeling. For example, graphics processing units (GPUs) are designed to provide high floating-point performance and high memory bandwidth to support data-intensive applications. For instance, each single-instruction multiple-data (SIMD) element in a GPU includes four vector signal processors (VSPs) to perform parallel operations such as matrix multiplication. Corresponding software platforms allow engineers to leverage the resources of high-performance GPUs. In some cases, the software platform supports deep learning operations (dlop), which provide flexible mixed-precision capabilities to support dynamic workloads, such as training neural networks and running inference on trained neural networks. Implementing flexible mixed-precision capabilities requires combining complex multiplexers, cross-switching between VSPs in the GPU, and increasing the complexity of register layouts such as vector general-purpose registers (VGPRs). Attached Figure Description

[0002] This disclosure will be better understood by referring to the accompanying drawings, and its many features and advantages will become apparent to those skilled in the art. The same reference numerals are used in different drawings to indicate similar or identical items.

[0003] Figure 1 This is a block diagram of a processing system that supports flexible precision matrix operations based on some implementation schemes.

[0004] Figure 2 It is a flowchart of matrix multiplication of two matrices according to some implementation schemes.

[0005] Figure 3 It is a block diagram of a single instruction multiple data (SIMD) unit comprising four vector signal processors (VSPs) according to some implementation schemes, the VSPs collaboratively retrieving portions of a matrix from system memory to perform matrix multiplication.

[0006] Figure 4 This is a block diagram of a VSP buffer during the first and second iterations of an iterative sequence in a round of matrix multiplication, according to some implementation schemes.

[0007] Figure 5 This is a block diagram of a VSP buffer during the third and fourth iterations of an iterative sequence in a round of matrix multiplication, according to some implementation schemes.

[0008] Figure 6 It is a block diagram of matrix multiplication of two matrices according to some implementation schemes, wherein the matrix multiplication uses an alternating arrangement of submatrices in a matrix.

[0009] Figure 7 This is a block diagram of a method, according to some implementation schemes, to perform a round of matrix multiplication by cyclically passing operands through buffers in multiple VSPs of the processing unit. Detailed Implementation

[0010] Data-intensive applications consume significant processing resources, including memory, processing power, and bandwidth, to move data between memory and the processor. Therefore, these applications require hardware that delivers high computational density with low power consumption, and supports varying precision in floating-point and integer operations. GPU performance is limited by operand precision and the rate of deep learning operations (dlop) per area and per watt. The performance of 32-bit stream processors can be scaled by implementing extended cumulative register files to support higher throughput multi-precision dlop. However, the increase in dlop rate is limited by the architecture of separate two VGPR files and cannot make the logic of matrix pipelines usable for general computations, such as HPC applications requiring double precision.

[0011] Figures 1 to 7 A technique for reducing power consumption per unit area in a high-performance processing unit during matrix multiplication of first and second matrices is disclosed, while increasing data reuse and thus reducing bandwidth consumption in processing units comprising multiple vector signal processors (VSPs). The VSPs collaboratively fetch portions of the first and second matrices from memory into corresponding registers (e.g., vector general-purpose registers VGPR), which implement single-instruction multiple-data (SIMD) elements of the processing unit. In a first iteration, portions of the first and second matrices are written from the registers into corresponding first and second buffers in the multiple VSPs. Multiply-accumulate elements in the multiple VSPs combine the values ​​in the first and second buffers and store the result for accumulation with subsequent results. In subsequent iterations, portions of the first matrix are cyclically passed through the first buffers of the multiple VSPs, such that different combinations of portions of the first and second matrices are multiplied and accumulated by the multiple VSPs. For example, if four VSPs are implemented in the SIMD elements, four portions of the first matrix and four portions of the second matrix are written into four first buffers and four second buffers. Multiply-accumulate elements in the VSPs perform matrix multiplication operations, and then the portions of the first matrix are cyclically passed through different first buffers in different VSPs via cross switches. The multiply-accumulate element simultaneously performs another set of matrix multiplication operations, and the process is repeated four times. In response to performing multiplication / accumulation of each part of the first matrix with each part of the second matrix stored in registers associated with multiple VSPs, the accumulated result of each of the multiple VSPs is written back to the VGPR (or to the corresponding output buffer before the VGPR). The new parts of the first and second matrices are then fetched into registers for the next round of matrix multiplication operations.

[0012] Figure 1 This is a block diagram of a processing system 100 supporting flexible-precision matrix operations according to some embodiments. The processing system 100 includes or can access system memory 105 or other storage components implemented using non-transitory computer-readable media such as dynamic random access memory (DRAM). However, some embodiments of memory 105 are implemented using other types of memory including static random access memory (SRAM), non-volatile RAM, etc. The processing system 100 also includes a bus 110 to support communication between entities implemented in the processing system 100, such as memory 105. Some embodiments of the processing system 100 include other buses, bridges, switches, routers, etc., which are not shown for clarity. Figure 1 As shown in the image.

[0013] Processing system 100 includes a central processing unit (CPU) 115. Some embodiments of CPU 115 include multiple processing elements that execute instructions simultaneously or in parallel (not shown in the image for clarity). Figure 1 (As shown in the diagram). The processing element is referred to as a processor core, computing unit, or other terms. CPU 115 is connected to bus 110 and therefore communicates with memory 105 via bus 110. CPU 115 executes instructions, such as program code 120 stored in memory 105, and CPU 115 stores information, such as the results of the executed instructions, in memory 105. CPU 115 is also capable of initiating graphics processing by issuing a drawing call.

[0014] Input / output (I / O) engine 125 handles input or output operations associated with display 130 and other components of processing system 100 such as keyboard, mouse, printer, external disk, etc. I / O engine 125 is coupled to bus 110, enabling it to communicate with memory 105, CPU 115, or other entities connected to bus 110. In the illustrated embodiment, I / O engine 125 reads information stored on external storage component 135, which is implemented using a non-transitory computer-readable medium such as a CD or DVD. I / O engine 125 also writes information, for example, the results of processing performed by CPU 115, to external storage component 135.

[0015] Processing system 100 includes a graphics processing unit (GPU) 140 that renders images for presentation on a display 130. For example, GPU 140 renders objects to produce pixel values ​​provided to display 130, which uses these pixel values ​​to display an image representing the rendered object. Some embodiments of GPU 140 are used for general-purpose computing. In the illustrated embodiment, GPU 140 communicates with memory 105 (and other entities connected to bus 110) via bus 110. However, some embodiments of GPU 140 communicate with memory 105 via a direct connection or via other buses, bridges, switches, routers, etc. GPU 140 executes instructions stored in memory 105, and GPU 140 stores information such as the results of the executed instructions in memory 105. For example, memory 105 stores a copy 145 of instructions representing program code to be executed by GPU 140.

[0016] GPU 140 includes one or more Single Instruction Multiple Data (SIMD) units 150, but for clarity, Figure 1 Only one is shown. SIMD unit 150 includes multiple processing elements, such as vector signal processors (VSPs) 151, 152, 153, and 154, which are collectively referred to herein as "VSPs 151-154". In the illustrated embodiment, VSPs 151-154 are interconnected by a crossbar switch 155, which provides a communication path for transmitting information between VSPs 151-154. VSPs 151-154 perform vector operations on information retrieved from system memory 105 into registers such as vector general-purpose registers (VGPRs), which, for clarity, are... Figure 1 Not shown in the image.

[0017] To perform matrix multiplication, VSPs 151-154 collaboratively fetch information from system memory 105, perform matrix multiplication on subsets of the information, and then exchange subsets of information to allow VSPs 151-154 to perform matrix multiplication on different combinations of subsets. Some embodiments of VSPs 151-154 collaboratively fetch matrix portions from system memory 105 and then perform matrix multiplication on the fetched matrix portions. These portions are cyclically passed through VSPs 151-154, for example via crossbar switch 155, such that matrix multiplication of different portions of the matrix is ​​performed and accumulated before additional portions of the matrix are fetched from system memory 105. In some embodiments, first portions of the first and second matrices are fetched into the VGPR before a first round comprising multiple iterations. Before second portions of the first and second matrices are fetched into the VGPR for a second round, the multiplication / accumulation elements in VSPs 151-154 perform matrix multiplication and accumulation on different combinations of subsets of the first portions of the first and second matrices in multiple iterations.

[0018] Some implementations of the GPU 140 are designed as 32-bit stream processors that can operate flexibly with varying degrees of precision. For example, the GPU 140 uses single-precision operands, double-precision operands, FP16 operands, and 8-bit integer operands to perform regular mathematical and matrix mathematical operations.

[0019] Figure 2 This is a block diagram of matrix multiplication 200 of two matrices according to some implementation schemes. Matrix multiplication 200 is composed of... Figure 1 Some implementations of the GPU 140 shown are performed. Matrix multiplication 200 is used to multiply matrices 205 and 210 to generate an output matrix 215. In the illustrated implementation, matrix multiplication 200 is performed by multiplying submatrices (or subsets or parts) from matrices 205 and 210 and accumulating the appropriate results to generate corresponding submatrices of the output matrix 215. For example, submatrix A1 from matrix 205 and submatrix B1 from matrix 210 are multiplied and accumulated to generate submatrix 220 of the output matrix 215.

[0020] As discussed herein, the submatrix groups of matrices 205 and 210 are collaboratively retrieved from system memory by multiple VSPs in the SIMD, and then the VSPs share the retrieved data to perform matrix multiplication and result accumulation for different combinations of submatrices. In the illustrated embodiment, submatrices A1, A2, A3, A4 from matrix 205 and submatrices B1, B2, B3, B4 from matrix 210 are, for example... Figure 1 The four VSPs shown in VSPs 151-154 are extracted collaboratively. Then, four sets of matrix multiplications are performed and accumulated to generate the elements in part 225 of the output matrix 215.

[0021] Matrix multiplications are performed in each iteration. In the first iteration, submatrix A1 is accessed from a buffer in one VSP and via the cross switches of the other VSPs. Matrix multiplications are performed across the four VSPs:

[0022] A1×B1

[0023] A1×B2

[0024] A1×B3

[0025] A1×B4

[0026] Submatrices A1, A2, A3, and A4 from matrix 205 are cyclically passed through the VSP, while submatrices B1, B2, B3, and B4 from matrix 210 are kept in their original positions for the second iteration. As used herein, the term "cyclically" refers to changing submatrices A1, A2, A3, and A4 accessed by the VSP via its internal buffer or crossbar switch. In the illustrated embodiment, the cyclical passing of submatrices A1, A2, A3, and A4 after the first iteration results in the VSP accessing submatric A2 via its internal buffer or crossbar switch. However, in other embodiments, internal buffers and crossbar switches are used to implement other types or patterns of cyclical passing.

[0027] In the second iteration, the four VSPs perform matrix multiplication:

[0028] A2×B1

[0029] A2×B2

[0030] A2×B3

[0031] A2×B4

[0032] The submatrices A1, A2, A3, and A4 from matrix 205 are again looped through the VSP, and the submatrices B1, B2, B3, and B4 from matrix 210 are kept in their original positions for the third iteration. In the third iteration, matrix multiplication is performed across the four VSPs:

[0033] A3×B1

[0034] A3×B2

[0035] d3×B3

[0036] d3×B4

[0037] The submatrices A1, A2, A3, and A4 from matrix 205 are again looped through the VSP, and the submatrices B1, B2, B3, and B4 from matrix 210 are kept in their original positions for the fourth iteration. In the fourth iteration, matrix multiplication is performed across the four VSPs:

[0038] A4×B1

[0039] d4×B2

[0040] d4×B3

[0041] A4×B4

[0042] At this point, the VSP has collectively performed matrix multiplications of all combinations of submatrices A1, A2, A3, A4 and B1, B2, B3, B4 from matrix 210, which are needed to generate the corresponding contribution 225 to the output matrix 215. In response to the completion of the fourth iteration, the accumulated result is written in and the VSP collaboratively extracts the other submatrices of matrices 210 and 215 to perform another round of iterations.

[0043] Figure 3 This is a block diagram of a SIMD unit 300 comprising four VSPs 301, 302, 303, and 304, according to some implementation schemes. The VSPs collaboratively retrieve portions of a matrix from system memory to perform matrix multiplication. The SIMD unit 300 is used to implement... Figure 1 Some embodiments of the SIMD unit 145 are shown. VSPs 301-304 are associated with VGPRs 305, 306, 307, and 308 (collectively referred to herein as "VGPRs 305-308"), which store two matrices (e.g., Figure 2 The portions of matrices 205 and 210 shown are to be multiplied. As discussed herein, at the start of a round of matrix multiplication involving the extracted portions, the portions of the matrices are cooperatively extracted from system memory into VGPR 305-308.

[0044] VSPs 301-304 include first buffers 310, 311, 312, and 313 (collectively referred to herein as "first buffers 310-313") and second buffers 315, 316, 317, and 318 (collectively referred to herein as "second buffers 315-318"). First buffers 310-313 store subsets of a portion of a first matrix stored in VGPRs 305-308, and second buffers 315-318 store subsets of a portion of a second matrix stored in VGPRs 305-308. VSPs 301-304 also include matrix multiplication / accumulation elements 320, 321, 322, and 323 (collectively referred to herein as "multiplication / accumulation elements 320-323") that perform matrix multiplication on the subsets stored in the first buffers 310-313 and the second buffers 315-318. The results of the matrix multiplication are then accumulated by matrix multiplication / accumulation elements 320-323.

[0045] VSPs 301-304 are interconnected via a crossbar switch 330, which allows the contents of the first buffers 310-313 to be transferred or circulated between VSPs 301-304. In the illustrated embodiment, before initiating a round of matrix multiplication, portions A1, A2, A3, A4 of the first matrix and portions B1, B2, B3, B4 of the second matrix are fetched from system memory into VGPRs 305-308. Portions A1, A2, A3, A4 of the first matrix are copied from VGPRs 305-308 to their corresponding first buffers 310-313, and portions B1, B2, B3, B4 of the second matrix are copied from VGPRs 305-308 to their corresponding second buffers 315-318. During the first iteration of the round, multiplication / accumulation elements 320-323 perform matrix multiplication on the contents of the second buffers 315-318 and the contents of the first buffers 310-313, or values ​​transmitted from the first buffers 310-313 via crossbar switch 330. The contents of the first buffers 310-313 are then looped through, and another round of iteration of the multiplication / accumulation process is performed. This process is iterated until all combinations of portions A1, A2, A3, A4 of the first matrix and portions B1, B2, B3, B4 of the second matrix have been multiplied. Multiplication / accumulation elements 320-323 then write the accumulated result to the corresponding output buffers 325, 326, 327, 328 (collectively referred to herein as "output buffers 325-328"). In some embodiments, output buffers 325-328 are not used; instead, the accumulated result is written directly to VGPRs 305-308.

[0046] Figure 4 This is a block diagram of matrix multiplication performed during the first iteration 400 and the second iteration 401 of an iterative sequence in a round of matrix multiplication, according to some embodiments. Before starting a round of iteration, portions A1, A2, A3, A4 and portions B1, B2, B3, B4 are loaded into buffers in the corresponding VGPR and VSP. In the illustrated embodiment, respectively, the buffer in the first VSP includes portions A1 and B1 of the first and second matrices, the buffer in the second VSP includes portions A2 and B2 of the first and second matrices, the buffer in the third VSP includes portions A3 and B3 of the first and second matrices, and the buffer in the fourth VSP includes portions A4 and B4 of the first and second matrices.

[0047] During the first iteration 400, the arithmetic logic unit in the corresponding VSP forms submatrix pairs 405, 406, 407, and 408, which are collectively referred to as submatrix pairs 405-408 in this paper. The arithmetic logic unit multiplies 405-408 and sums the results. Some implementations of the arithmetic logic unit correspond to... Figure 3The multiplication / accumulation elements 320-323 in the VSPs 301-304 are shown. Depending on the iteration (or loop), the arithmetic logic units access the values ​​of submatrix pairs 405-408 from the buffer in the VSP or via a crossbar switch, as described below. In the illustrated embodiment, the arithmetic logic units in the first VSP access portions A1, B1 from the buffer to form pair 405 in the first iteration 400. The arithmetic logic units then perform matrix multiplication on the contents of the buffer and accumulate the results. The arithmetic logic units in the other VSPs access portion A1 via a crossbar switch during the first iteration 400. The arithmetic logic units then perform matrix multiplication on pairs 406-408. At the end of the first iteration 400, portions A1, A2, A3, A4 are looped and portions B1, B2, B3, B4 are not looped.

[0048] During the second iteration 401, the arithmetic logic units in the corresponding VSPs form submatrix pairs 410, 411, 412, and 413, which are collectively referred to herein as submatrix pairs 410-413. The arithmetic logic units multiply 410-413 and accumulate the results. In the illustrated embodiment, the arithmetic logic units in the second VSP access portions A2 and B2 from the buffer to form pair 411. The arithmetic logic units then perform matrix multiplication on the contents of the buffer and accumulate the results. The arithmetic logic units in the other VSPs access portion A2 via a crossbar switch. The arithmetic logic units then perform matrix multiplication on pairs 410, 412, and 413. At the end of the second iteration 401, the loop portions A1, A2, A3, and A4 and the non-loop portions B1, B2, B3, and B4 are completed.

[0049] Figure 5 This is a block diagram of matrix multiplication performed during the third iteration 500 and the fourth iteration 501 of an iterative sequence in a round of matrix multiplication, according to some implementation schemes. In some implementation schemes, the third iteration 500 and the fourth iteration 501 are performed during... Figure 4 Following the first iteration 400 and the second iteration 401, and within the same round of matrix multiplication operations as the first and second iterations, portions A1, A2, A3, A4 and portions B1, B2, B3, B4 are loaded into buffers in the corresponding VGPR and VSP. In the illustrated embodiment, respectively, the buffer in the first VSP includes portions A1 and B1 of the first and second matrices, the buffer in the second VSP includes portions A2 and B2 of the first and second matrices, the buffer in the third VSP includes portions A3 and B3 of the first and second matrices, and the buffer in the fourth VSP includes portions A4 and B4 of the first and second matrices.

[0050] During the third iteration 500, the arithmetic logic units in the corresponding VSP form submatrix pairs 505, 506, 507, and 508, which are collectively referred to herein as submatrix pairs 505-508. The arithmetic logic units multiply 505-508 and accumulate the results. In the illustrated implementation, the arithmetic logic units in the third VSP access portions A3 and B3 from the buffer to form pair 507. The arithmetic logic units then perform matrix multiplication on the contents of the buffer and accumulate the results. The arithmetic logic units in the other VSPs access portion A3 via a crossbar switch. The arithmetic logic units then perform matrix multiplication on pairs 505, 506, and 508. At the end of the third iteration 500, the loop portions A1, A2, A3, and A4 and the non-loop portions B1, B2, B3, and B4 are completed.

[0051] During the fourth iteration 501, the arithmetic logic units in the corresponding VSP form submatrix pairs 510, 511, 512, and 513, which are collectively referred to herein as submatrix pairs 510-513. The arithmetic logic units multiply 510-513 and accumulate the results. In the illustrated implementation, the arithmetic logic unit in the fourth VSP accesses portions A4 and B4 from the buffer to form pair 513. The arithmetic logic unit then performs matrix multiplication on the contents of the buffer and accumulates the results. The arithmetic logic units in the other VSPs access portion A4 via a crossbar switch. The arithmetic logic units then perform matrix multiplication on pairs 510-512. At the end of the fourth iteration 501, the arithmetic logic unit writes the accumulated result to the corresponding output buffer, for example... Figure 3 The output buffers 325-328 are shown. The new portions of the first and second matrices are then extracted into the VGPR associated with the VSP, and these portions are copied into the corresponding buffers to begin another round of matrix multiplication.

[0052] Figure 6 This is a block diagram of matrix multiplication 600 of two matrices according to some implementation schemes, wherein the matrix multiplication uses an alternating arrangement of submatrices within a matrix. Matrix multiplication 600 is composed of... Figure 1 Some implementations of the GPU 140 shown are performed. Matrix multiplication 600 is used to multiply matrices 605 and 610 to generate an output matrix 615. In the illustrated implementation, matrix multiplication 600 is performed by multiplying submatrices (or subsets or parts) from matrices 605 and 610 and summing the appropriate results to generate corresponding submatrices of the output matrix 615. For example, submatrix A1 from matrix 605 and submatrix B1 from matrix 610 are multiplied and summed to generate submatrix 620 of the output matrix 615. Matrix multiplication 600 differs from other implementations by arranging the submatrices of matrix 610 in a column-based orientation rather than a row-based orientation. Figure 2The matrix multiplication 200 is shown. Therefore, part 625 of the output matrix 615 generated by multiplying submatrices A1, A2, A3, A4 with submatrices B1, B2, B3, B4 is configured as a column-based structure, rather than... Figure 2 The output matrix 215 shown is a portion 225 of an array-based configuration.

[0053] Figure 7 This is a block diagram of method 700, which performs a round of matrix multiplication by cyclically passing operands through buffers in multiple VSPs of a processing unit, according to some implementation schemes. Method 700 in Figure 1 The GPU 140 shown and Figure 3 This is implemented in some embodiments of the SIMD unit 300 shown.

[0054] In box 705, portions of (first) matrix A and (second) matrix B are retrieved from system memory and stored in registers associated with the VSP, for example... Figure 3 The VGPR305-308 shown herein. The portions of matrix A and matrix B are also referred to herein as operands A and B for matrix multiplication operations performed by the multiplication / accumulation unit in the VSP.

[0055] In box 710, operands A and B are loaded from the registers into the buffer of the VSP, for example... Figure 3 The first buffer 310-313 and the second buffer 315-318 are shown.

[0056] In box 715, the iteration of the matrix multiplication operation begins and the multiplication / accumulation unit in the VSP performs multiplication and accumulation operations on the A and B operands stored in the corresponding buffers of the VSP. As discussed herein, one of the A operands is accessed from the corresponding buffer in the corresponding VSP, and the value of this A operand is accessed by other VSPs via a crossbar switch.

[0057] In decision block 720, the processing unit determines whether matrix multiplication / accumulation has been performed on all combinations of operands A and B. In some implementations, the number of iterations equals the number of VSPs, such that each operand in A is multiplied by each operand in B before a round is completed. If multiplication / accumulation has been performed on all combinations, method 700 proceeds to block 725 and the multiplication / accumulation unit writes the accumulated result to the corresponding output buffer, for example... Figure 3 The output buffers 325-328 are shown. If multiplication / accumulation has not yet been performed on all combinations of operands A and B, then method 700 reaches box 730.

[0058] In box 730, operand A is looped around VSP. In some implementations, this is achieved via, for example... Figure 3The cross switch 330 shown transmits information representing operand A to cycle through operand A. The cycling of operand A occurs in... Figure 4 and Figure 5 As shown in the diagram. Then method 700 returns to box 715 to begin subsequent iterations of the wheel.

[0059] The above-described implementation schemes can be better understood by considering the following example implementations:

[0060] Example 1. A device comprising:

[0061] Multiple vector signal processors (VSPs) including multiplying / accumulating elements; and

[0062] Multiple registers associated with the plurality of VSPs, wherein a first portion of the first and second matrices is extracted into the plurality of registers before a first round comprising multiple iterations, wherein a second portion of the first and second matrices is extracted into the plurality of registers for a second round, wherein the multiplication / accumulation element performs matrix multiplication and accumulation on different combinations of subsets of the first portions of the first and second matrices in the multiple iterations.

[0063] Example 2. The device as described in Example 1, wherein the plurality of VSPs further includes a first buffer, a second buffer, and an output buffer, and wherein, prior to initiating the plurality of iterations, the subset of the first portion of the first and second matrices is copied from the plurality of registers to the first and second buffers in the plurality of VSPs.

[0064] Example 3. The device as described in Example 2, wherein during the current iteration of the plurality of iterations, the multiplication / accumulation element performs matrix multiplication and accumulation on the subset of the first portion of the first and second matrices stored in the first and second buffers corresponding to the current iteration.

[0065] Example 4. The device as described in Example 3, wherein during the current iteration, after the multiplication / accumulation element performs the matrix multiplication and accumulation for the current iteration, the subset of the first portion of the first matrix is ​​looped to different VSPs.

[0066] Example 5. The device as described in Example 4 further includes:

[0067] A cross switch interconnects the plurality of VSPs, wherein the subset of the first portion of the first matrix is ​​cyclically routed to the different VSPs via the cross switch.

[0068] Example 6. The device as described in Example 1, wherein the multiple iterations correspond to the multiple VSPs, such that the multiplication / accumulation element performs the matrix multiplication and accumulation on all combinations of the subsets of the first portion of the first and second portions during the multiple iterations of the first round.

[0069] Example 7. The device as described in any one of Examples 1 to 6, wherein the plurality of VSPs further includes:

[0070] An output buffer, wherein after performing the matrix multiplication and accumulation on the multiple iterations in the first round and before starting the second round, the multiplication / accumulation element writes the accumulated result of the multiplication to the output buffer.

[0071] Example 8. The device as described in Example 7, wherein in response to the multiplication / accumulation element writing the accumulation result to the output buffer, the second portion of the first and second matrices is extracted into the plurality of registers.

[0072] Example 9. A method comprising:

[0073] Prior to the first round, which includes multiple iterations, the first portions of the first and second matrices are extracted into multiple registers associated with multiple vector signal processors (VSPs);

[0074] In the multiple iterations, matrix multiplication and accumulation are performed on different combinations of subsets of the first portion of the first and second matrices; and

[0075] In response to the completion of the multiple iterations in the first round, the second portions of the first and second matrices are extracted into the plurality of registers for a second round.

[0076] Example 10. The method described in Example 9 further includes:

[0077] Before initiating the multiple iterations, the subset of the first portion of the first and second matrices is copied from the multiple registers to the first and second buffers in the multiple VSPs.

[0078] Example 11. The method described in Example 10 further includes:

[0079] During the current iteration of the multiple iterations, matrix multiplication and accumulation are performed on the subset of the first portion of the first and second matrices.

[0080] Example 12. The method described in Example 11 further includes:

[0081] During the current iteration, after performing the matrix multiplication and accumulation for the current iteration, the subset of the first part of the first matrix is ​​looped to different VSPs.

[0082] Example 13. The method as described in Example 12, wherein cycling the subset of the first portion of the first matrix to different first buffers includes cycling the subset of the first portion of the first matrix via a cross switch that interconnects the plurality of VSPs.

[0083] Example 14. The method as described in Example 11, wherein the multiple iterations correspond to the multiple VSPs, and wherein performing the matrix multiplication and accumulation includes the matrix multiplication and accumulation for all combinations of the subsets of the first and second portions during the multiple iterations of the first round.

[0084] Example 15. The method as described in any one of Examples 9 to 14, further comprising:

[0085] After performing the matrix multiplication and accumulation on the multiple iterations in the first round and before starting the second round, the accumulated result of the multiplication is written to the output buffer; and

[0086] Extracting the second portion of the first and second matrices includes extracting the second portion of the first and second matrices into the plurality of registers in response to writing the accumulated result into the output buffer.

[0087] Computer-readable storage media include any non-transitory storage medium or combination of non-transitory storage media that can be accessed by a computer system during use to provide instructions and / or data to the computer system. Such storage media include, but are not limited to, optical media (e.g., compressed optical discs (CDs), digital versatile optical discs (DVDs), Blu-ray discs), magnetic media (e.g., floppy disks, magnetic tapes, or magnetic hard disks), volatile memory (e.g., random access memory (RAM) or cache), non-volatile memory (e.g., read-only memory (ROM) or flash memory), or microelectromechanical systems (MEMS) based storage media. Some embodiments of computer-readable storage media are embedded in a computing system (e.g., system RAM or ROM), fixedly attached to a computing system (e.g., a magnetic hard disk drive), removably attached to a computing system (e.g., an optical disc or a USB-based flash memory), or coupled to a computer system via a wired or wireless network (e.g., a network-accessible storage device (NAS)).

[0088] In some implementations, certain aspects of the techniques described above are implemented by one or more processors of a processing system executing the software. The software includes one or more executable instruction sets stored or otherwise tangibly embodied on a non-transitory computer-readable storage medium. The software includes instructions and certain data that, when executed by one or more processors, manipulate one or more processors to perform one or more aspects of the techniques described above. Non-transitory computer-readable storage media include, for example, disk or optical disk storage devices, solid-state storage devices (e.g., flash memory, cache, random access memory (RAM), or one or more other non-volatile memory devices). The executable instructions stored on the non-transitory computer-readable storage medium are in the form of source code, assembly language code, object code, or other instruction formats that are interpreted or otherwise executed by one or more processors.

[0089] It should be noted that not all activities or elements described in the general description above are required, nor is it necessary to include only a portion of a particular activity or apparatus, and one or more other activities are required, or elements other than those described may be included. Furthermore, the order in which the activities are listed is not necessarily the order in which they are performed. Additionally, the concepts have been described with reference to specific embodiments. However, those skilled in the art will understand that various modifications and changes may be made without departing from the scope of this disclosure as set forth in the appended claims. Therefore, this specification and drawings are to be regarded as illustrative rather than restrictive, and all such modifications are intended to be included within the scope of this disclosure.

[0090] The benefits, other advantages, and solutions to the problems described above with respect to specific embodiments have been presented. However, the benefits, advantages, and solutions to the problems, as well as any features that may bring about or make more prominent any benefit, advantage, or solution to the problem, should not be construed as being key, necessary, or essential features of any or all claims. Furthermore, the specific embodiments disclosed above are merely illustrative, as the disclosed subject matter can be modified and practiced in different but equivalent ways by those skilled in the art who have benefited from the teachings herein. There is no intention to limit oneself to the details of the construction or design shown herein, except as described in the appended claims. Therefore, it is apparent that the specific embodiments disclosed above may be altered or modified, and all such changes are considered to be within the scope of the disclosed subject matter. Therefore, the protection sought herein is as set forth in the appended claims.

Claims

1. An apparatus comprising: a plurality of vector signal processors (VSPs) including multiply and accumulate elements; and a plurality of registers associated with the plurality of VSPs, wherein a first portion of first and second matrices is fetched into the plurality of registers prior to a first round of a matrix operation including a plurality of iterations, wherein the multiply and accumulate elements perform matrix multiplication and accumulation on different combinations of a subset of the first portion of the first and second matrices in the plurality of iterations prior to a second portion of the first and second matrices being fetched into the plurality of registers for a second round of the matrix operation, wherein the subset accessed by the vector signal processors is changed during a current iteration of the plurality of iterations such that a different vector signal processor accesses the subset after the matrix multiplication and accumulation of the current iteration is performed by the multiply and accumulate elements.

2. The apparatus of claim 1, wherein the plurality of VSPs further comprise a first buffer, a second buffer, and an output buffer, and wherein the subset of the first portion of the first and second matrices is copied from the plurality of registers to the first and second buffers in the plurality of VSPs prior to initiating the plurality of iterations.

3. The apparatus of claim 2, wherein during a current iteration of the plurality of iterations, the multiply and accumulate elements perform matrix multiplication and accumulation on the subset of the first portion of the first and second matrices stored in the corresponding first and second buffers during the current iteration.

4. The apparatus of claim 3, wherein during the current iteration, the subset of the first portion of the first matrix comprises an operand that is circulated among different vector signal processors by a crossbar that interconnects the plurality of vector signal processors after the multiply and accumulate elements perform the matrix multiplication and accumulation for the current iteration.

5. The apparatus of claim 4, further comprising: a crossbar that interconnects the plurality of VSPs, wherein the subset of the first portion of the first matrix is circulated to the different VSPs via the crossbar.

6. The apparatus of claim 1, wherein the plurality of iterations correspond to the plurality of VSPs such that the multiply and accumulate elements perform the matrix multiplication and accumulation on all combinations of the subset of the first portion of the first and second portions during the plurality of iterations of the first round.

7. The apparatus of any one of claims 1 to 6, wherein the plurality of VSPs further comprise: an output buffer, wherein the multiply and accumulate elements write accumulation results of the multiplication to the output buffer after performing the matrix multiplication and accumulation for the plurality of iterations in the first round and prior to starting the second round.

8. The apparatus of claim 7, wherein the second portion of the first and second matrices is fetched into the plurality of registers in response to the multiply and accumulate elements writing the accumulation results to the output buffer.

9. A method comprising: extracting a first portion of the first and second matrices into a plurality of registers associated with a plurality of vector signal processors (VSPs) prior to a first round of matrix operations including a plurality of iterations; performing matrix multiplication and accumulation on different combinations of a subset of the first portion of the first and second matrices in the plurality of iterations; extracting a second portion of the first and second matrices into the plurality of registers for a second round of matrix operations in response to completing the plurality of iterations in the first round; and changing the subset accessed by the vector signal processors during a current iteration of the plurality of iterations such that different vector signal processors access the subset after the matrix multiplication and accumulation of the current iteration.

10. The method of claim 9, further comprising: copying the subset of the first portion of the first and second matrices from the plurality of registers to first and second buffers in the plurality of VSPs prior to initiating the plurality of iterations.

11. The method of claim 10, further comprising: performing matrix multiplication and accumulation on the subset of the first portion of the first and second matrices during a current iteration of the plurality of iterations.

12. The method of claim 11, further comprising: circulating the subset of the first portion of the first matrix to a different VSP after performing the matrix multiplication and accumulation for the current iteration during the current iteration.

13. The method of claim 12, wherein circulating the subset of the first portion of the first matrix to a different vector signal processor comprises circulating the subset of the first portion of the first matrix via a crossbar that interconnects the plurality of VSPs.

14. The method of claim 11, wherein the plurality of iterations corresponds to the plurality of VSPs, and wherein performing the matrix multiplication and accumulation comprises the matrix multiplication and accumulation for all combinations of the subset of the first and second portions during the plurality of iterations of the first round.

15. The method of any of claims 9 to 14, further comprising: writing accumulation results of the multiplication to an output buffer after performing the matrix multiplication and accumulation for the plurality of iterations in the first round and prior to starting the second round; and wherein extracting the second portion of the first and second matrices comprises extracting the second portion of the first and second matrices into the plurality of registers in response to writing the accumulation results to the output buffer.

Images