High performance matrix linear operation device and method

Abstract

The application relates to the computer technical field and provides a high-performance matrix linear operation device and method, the scale of a PE array is determined according to the column number of a B matrix, the whole reduces the pipeline operation delay by completely adapting the whole-row access characteristic of an SRAM; the data of an A matrix adopts a broadcast mode, the reading of the A matrix is completely pipelined, bubble-free and uninterrupted, and the theoretical utilization rate of the PE can approach 100%. The data of the B matrix adopts a whole-row parallel distribution mode, and the additional handshake overhead is avoided. A local memory does not need to be arranged for the C matrix in the PE, the calculation result is directly pipelined and output after accumulation, and the storage overhead and hardware area occupation are reduced. The outputs of all the PEs are directly combined into a whole row of data of the C matrix according to the PE index sequence, one row is output per clock cycle, and the data interaction efficiency with the local memory is greatly optimized.

Description

Technical Field

[0001] This invention belongs to the field of computer technology and relates to a high-performance matrix linear operation device and method. Background Technology

[0002] In computationally intensive fields such as radar signal processing and image processing, matrix multiplication is one of the core fundamental operations, and its computational efficiency directly determines the performance of the entire system. Therefore, hardware accelerators are crucial for improving performance. Considering both resource efficiency and control complexity, the matrix multiplication architecture of linear arrays offers high adaptability.

[0003] Linear arrays offer several advantages: First, the number of processing units (PEs) depends only on the number of rows in matrix A, not on the number of columns in matrix B, resulting in relatively controllable hardware resource overhead. Second, data flow is unidirectional, with rows of matrix B flowing sequentially through the PE array, simplifying control logic and facilitating pipelined computation. Third, through an iterative accumulation mechanism, they can support matrix multiplication of varying scales, providing good scalability. However, traditional linear array architectures suffer from unfriendly storage access, discontinuous output, low storage efficiency, and wasted storage resources, leading to insufficient processing performance. Therefore, a high-performance matrix linear computation scheme is urgently needed to address the combined problems of SRAM memory access mismatch, discontinuous output, and low storage efficiency. Summary of the Invention

[0004] To address the problems existing in the above-mentioned traditional methods, this invention proposes a high-performance matrix linear operation device and a high-performance matrix linear operation method. By changing the way the size of the PE array is determined, the data loading direction, and the output merging method, it can fundamentally adapt to the access characteristics of SRAM, realize data pipelined loading and continuous output, eliminate the dependence of the PE on local memory, and improve the overall efficiency of matrix multiplication operations.

[0005] To achieve the above objectives, the embodiments of the present invention adopt the following technical solutions: On the one hand, a high-performance matrix linear operation device is provided, comprising: Multiple processing units (PEs); the number of processing units (PEs) is determined based on the number of columns in the B matrix to be computed; The data broadcast loading module is used to read individual elements in column order of matrix A in each clock cycle and synchronously broadcast them to all processing units (PE) in the form of a data stream; The data row distribution and loading module is used to read the entire row of data of the B matrix at once within one clock cycle and distribute it to each processing unit (PE) according to the element position; each processing unit (PE) buffers one B matrix element received. The core computing unit is used to perform multiplication and addition operations with the currently cached elements of matrix B after receiving the broadcast elements of matrix A, and output the calculation results. The C matrix storage control module is used to merge the output data of all processing units (PE) into a single row of data in the C matrix according to the PE index order; and outputs one row of data in the C matrix every clock cycle.

[0006] In one embodiment, the data broadcast loading module adopts a pipelined broadcast loading mechanism, which continuously broadcasts each element of matrix A to all processing units PE in the form of a data stream.

[0007] In one embodiment, the data row distribution and loading module is configured with a collaborative mechanism that triggers a new row B when a new column A is added, including: When the data broadcast loading module starts inputting data for a new column of matrix A, the data row distribution loading module is automatically triggered to read the next row of data in matrix B.

[0008] In one embodiment, the B matrix elements cached inside each processing unit PE are reused repeatedly, and the number of reuses is equal to the number of rows in the A matrix. Each processing unit (PE) calculates and outputs the column component values ​​of matrix C in each clock cycle. The data outputs of all processing units (PE) are then combined into the data components of matrix C.

[0009] In one embodiment, the number of processing units PE is equal to the number of columns in matrix B; When the number of columns n in matrix B is less than or equal to the number of processing units PE, matrix multiplication is performed directly. When the number of columns n in matrix B is greater than the number of processing units PE, matrix B is divided into blocks by columns and then the block matrix multiplication operation is performed.

[0010] In one embodiment, the C matrix storage control module writes the merged C matrix row data directly back to SRAM, with the write address incrementing continuously.

[0011] In one embodiment, during the calculation process, when performing non-first component calculations of matrix C, each processing unit PE reads the old value of matrix C from SRAM, splits it according to element position, and distributes it to each processing unit PE. Each processing unit PE then adds the current calculation result to the corresponding old value and outputs it. The first component calculation of matrix C is calculated from the first column element of matrix A and the first row element of matrix B.

[0012] In one embodiment, the calculation results of each processing unit PE are directly output after accumulation, and then merged into a whole row of data in the C matrix by the C matrix storage control module before being written back to SRAM.

[0013] On the other hand, a high-performance matrix linear operation method is also provided, including the following steps: Initialize the configuration, set the matrix dimension parameters, and reset the internal calculation units; where matrix A is an m x k ​​matrix, matrix B is a k x n matrix, and matrix C is an m x n matrix, where m, k, and n are all positive integers; In each clock cycle, individual elements of matrix A are read in column order to form a data stream, which is then synchronously broadcast to all processing units (PEs). At the same time, an entire row of data from matrix B is read from SRAM and distributed in parallel to each processing unit (PE) according to element position. Each processing unit (PE) corresponds to one element of matrix B, and the number of processing units (PEs) is determined based on the number of columns n of matrix B. Each processing unit (PE) performs multiplication and addition operations on the received A matrix elements and the cached B matrix elements and outputs the calculation results. When calculating a non-first component of the C matrix, each processing unit (PE) adds the old value of the C matrix read from SRAM to the current calculation result and outputs it. When calculating the first component of the C matrix, each processing unit (PE) directly outputs the result. The output data of all processing units (PE) are merged into a single row of data in the C matrix according to the PE index order. One row is output every clock cycle and the merged row data is written directly back to SRAM.

[0014] In one embodiment, the data loading process employs a collaborative mechanism where a new column A triggers a new row B: When a new column of data in matrix A is entered, the reading of the next row of data in matrix B is automatically triggered; the number of times the elements of matrix B cached in each processing unit PE are reused is equal to the number of rows in matrix A.

[0015] One of the above technical solutions has the following advantages and beneficial effects: The aforementioned high-performance matrix linear operation device and method, by determining the size of the PE array according to the number of columns of matrix B rather than the number of rows of matrix A, enables the matrix calculation to start by accessing one row of data of matrix B at once in one clock cycle and distributing it to all PEs. This fully adapts to the SRAM row-by-row access characteristics and avoids the problems caused by traditional solutions that require reading one column of data of matrix A each time due to determining the PE according to the number of rows of matrix A, resulting in slow reading speed and non-compliance with the SRAM one-row-at-a-time access rule. This reduces the overall pipeline operation latency.

[0016] Data for matrix A is streamed, with one element per column in each clock cycle, broadcast to all processing units (PEs). This ensures a completely streamlined, bubble-free, and uninterrupted reading process for matrix A, bringing the theoretical utilization of PEs close to 100%. Data for matrix B is distributed in parallel row-by-row, with a mechanism that triggers a new row in matrix B when a new column in matrix A is input. This ensures strict alignment between columns in matrix A and rows in matrix B, avoiding additional handshake overhead. No local memory is needed for matrix C within each PE; calculation results are directly piped out after accumulation, reducing storage overhead and hardware footprint. All PE outputs are merged into a single row of matrix C in PE index order, one row per clock cycle. Writing to SRAM involves continuous address incrementing without masking or address jumps, achieving 100% write bandwidth utilization and significantly optimizing data interaction efficiency with local memory. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the conventional technology, the drawings used in the description of the embodiments or the conventional technology will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the operation of a traditional linear array architecture; Figure 2 This is a schematic diagram of the module architecture of a high-performance matrix linear operation device in one embodiment; Figure 3 This is a schematic diagram of the computing architecture of a high-performance matrix linear operation device in one embodiment; Figure 4 This is a schematic diagram illustrating the operational principle of a linear array architecture in one embodiment; Figure 5 This is a flowchart of a high-performance matrix linear operation method in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention.

[0020] It should be noted that, in this document, the reference to "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The presentation of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will understand that the embodiments described herein can be combined with other embodiments. The term "and / or" as used herein refers to any combination of one or more of the associated listed items, and all possible combinations, including such combinations.

[0021] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0022] like Figure 1 As shown, in a traditional linear array structure, before computation begins, one column of elements from matrix A and all elements from matrix C need to be preloaded into each processing unit (PE). The number of PEs in this architecture depends on the number of rows in matrix A. After computation starts, a row of data from matrix B flows sequentially through the linear array and performs multiplication and addition operations with the elements of column A cached within each PE, as shown below. Figure 1 The multiplication and addition units shown in the diagram are implemented using FPGA hardware. Intermediate calculation results are temporarily stored in local memory (matrix C). After K iterations, the calculation of the current sub-matrix block is completed, and the final result is moved to off-chip memory. Then, the next sub-matrix block is processed, and this process is repeated. Here, matrix A is the left matrix of the matrix multiplication, matrix B is the right matrix of the matrix multiplication, and matrix C is the result matrix of the matrix multiplication.

[0023] However, in the traditional linear array architecture described above, the data organization method is incompatible with the access characteristics of Static Random Access Memory (SRAM). Matrix A requires reading an entire column of data from SRAM at once, which does not conform to the pipelined operation of accessing one row of SRAM data per clock cycle, leading to memory access conflicts and pipeline interruptions, increasing computational latency. The output is discontinuous, resulting in low storage efficiency and wasted storage resources. Traditional linear arrays have two output methods: first, merging the outputs of all processing units (PEs) into one column of data in matrix C each clock cycle, which requires multiple transpose operations and additional storage; second, temporarily storing the calculation results of each clock cycle in local memory, and then outputting the results of each processing unit (PE) row by row after all calculations are completed, which results in high storage resource overhead, wasted storage resources, and low utilization.

[0024] In one embodiment, such as Figure 2As shown, a high-performance matrix linear operation device is provided, including multiple processing units (PEs). The number of processing units (PEs) is determined based on the number of columns of the B matrix to be calculated. A data broadcast loading module 11 reads individual elements of the A matrix in column order each clock cycle and synchronously broadcasts them to all processing units (PEs) in the form of a data stream. A data row distribution loading module 13 reads an entire row of data from the B matrix at once within one clock cycle and distributes it sequentially to each processing unit (PE) according to element position; each processing unit (PE) buffers one received B matrix element. A core calculation unit 15 performs multiplication and addition operations with the currently buffered B matrix element after receiving the broadcast A matrix element and outputs the calculation result. A C matrix storage control module 17 merges the output data of all processing units (PEs) into an entire row of data in the C matrix according to the PE index order; it outputs one row of data from the C matrix each clock cycle.

[0025] It is understandable that, from the perspective of existing technology, the number of processing units (PEs) in a traditional linear array architecture depends on the number of rows (m) of matrix A. Before computation begins, an entire column of elements from matrix A and all elements from matrix C need to be preloaded into each processing unit (PE). Matrix A needs to be read from SRAM in one go, which is severely incompatible with the pipelined operation of SRAM, where one row of data is accessed per clock cycle, leading to memory access conflicts and pipeline interruptions. Furthermore, the traditional architecture's output method involves merging the outputs of all processing units (PEs) into a single column of data in matrix C each clock cycle, requiring multiple transpose operations and additional storage, or temporarily storing intermediate results in local memory and outputting them row by row after all computations are complete, resulting in wasted storage resources and low utilization.

[0026] This embodiment addresses the aforementioned shortcomings with a fundamentally improved design. For example... Figure 3 As shown, the number of processing units (PEs) (such as PE_1 to PE_n) is determined based on the number of columns in the B matrix to be computed, rather than the number of rows in the A matrix. The technical significance of this change is that when the number of processing units (PEs) is determined by the number of columns in the B matrix, the number of elements in a single row of data in the B matrix is ​​the same as the number of processing units (PEs). This allows a single row of data in the B matrix to be read from the SRAM and distributed in parallel to all processing units (PEs) within one clock cycle. This operation fully complies with the rule of accessing one row of data per clock cycle in SRAM, fundamentally solving the memory access conflict problem of traditional solutions.

[0027] The data broadcast loading module 11 reads individual elements of matrix A sequentially by column in each clock cycle and broadcasts them synchronously to all processing units (PEs) in the form of a data stream. Traditional solutions require reading an entire column of data from matrix A at once. Since SRAM does not support continuous access to a column of data within a single cycle, pipeline bubbles and interruptions occur. This embodiment changes the reading method of matrix A to reading element-by-element sequentially by column, reading only one element per clock cycle and broadcasting it synchronously to all processing units (PEs) after forming a data stream. Because only one address of SRAM is accessed per clock cycle, this operation is perfectly adapted to the single-cycle, single-address access characteristic of SRAM, eliminating memory access conflicts and enabling fully pipelined data reading of matrix A.

[0028] The data row distribution and loading module 13 reads the entire row of data in the B matrix at once within one clock cycle and distributes it sequentially to each processing unit (PE) according to the element position. Each processing unit (PE) buffers one B matrix element received. In traditional schemes, a row of data in the B matrix must flow through each processing unit (PE) sequentially, making it impossible to load the entire row in parallel. This embodiment utilizes the architectural advantage that the number of processing units (PE) equals the number of columns in the B matrix, reading the entire row of n elements of the B matrix from SRAM at once and distributing them in parallel to n processing units (PE). Each processing unit (PE) corresponds to exactly one B matrix element, maximizing the parallel loading efficiency of the B matrix data.

[0029] The core computing unit 15, upon receiving a broadcast element of matrix A, performs a multiplication-addition operation with the currently cached element of matrix B and outputs the calculation result. This module's design simplifies the computational operations within each processing unit (PE) to multiplication-addition operations, eliminating the need to maintain complex local states or caches within the PE. The core computing unit 15 may include multiple processing units (PEs).

[0030] The C-matrix storage control module 17 is used to merge the output data of all processing units PE into a single row of data in the C-matrix according to the PE index order, so as to output one row of data in the C-matrix each clock cycle. Traditional solutions require subsequent transposition to output a column of data in the C-matrix, resulting in discontinuous output. In this embodiment, the outputs of the n processing units PE are directly concatenated and merged according to the PE index order to form a complete row of data in the C-matrix, with one row output each clock cycle. The output address is continuously incremented in the SRAM. During subsequent non-first component calculations, the C-matrix storage control module 17 can also read back the old value of the C-matrix written after the previous component calculation and send it to the core computing unit 15 for accumulation. Each matrix is ​​stored in its corresponding existing storage unit, facilitating data interaction with the high-performance matrix linear operation device.

[0031] The modules described above work in close collaboration: the number of processing units (PEs) is determined by the number of columns in matrix B, making row-by-row distribution of matrix B possible; the broadcast loading of matrix A and the row-by-row distribution of matrix B are executed in parallel within the same clock cycle; the column elements of matrix A and the row elements of matrix B meet precisely in time within the processing units (PEs) and complete multiplication and addition operations; the results output by each processing unit (PE) are concatenated into a complete row of matrix C by the C matrix storage control module 17 according to the index order, realizing a fully pipelined closed loop from data input to result output. The collaborative effect of the aforementioned three levels ensures that the data flow of the entire device is unblocked and without waiting, maximizing PE utilization, and ultimately achieving efficient hardware acceleration of matrix multiplication operations.

[0032] The aforementioned high-performance matrix linear operation device determines the size of the PE array based on the number of columns in matrix B rather than the number of rows in matrix A. This allows the matrix calculation to begin by accessing one row of data from matrix B in a single clock cycle and distributing it to all PEs. This fully adapts to the characteristics of SRAM row-by-row access and avoids the problems caused by traditional solutions that require reading one column of data from matrix A each time due to determining the PE based on the number of rows in matrix A. This results in slow read speeds and does not conform to the rule of accessing one row at a time in SRAM, thereby reducing overall pipeline operation latency.

[0033] Data for matrix A is streamed, with one element per column in each clock cycle, forming a data stream that is then broadcast to all processing units (PEs). This ensures a completely piped, bubble-free, and uninterrupted reading process for matrix A, bringing the theoretical utilization of PEs close to 100%. Data for matrix B is distributed in a row-by-row parallel manner, with a mechanism that allows a new column in matrix A to trigger a new row in matrix B. When a new column of data is input into matrix A, the next row of data in matrix B is automatically read, ensuring strict alignment between the columns of matrix A and the rows of matrix B and avoiding additional handshake overhead. No local memory is needed for matrix C within each PE; the calculation results are directly piped out after accumulation, reducing storage overhead and hardware footprint. All PE outputs are directly merged into a single row of data in matrix C, in PE index order, one row per clock cycle. When writing to SRAM, the address increments continuously without masking or address jumps, achieving 100% write bandwidth utilization and significantly optimizing data interaction efficiency with local memory.

[0034] In one embodiment, the data broadcast loading module adopts a pipelined broadcast loading mechanism, which continuously broadcasts each element of matrix A to all processing units (PE) in the form of a data stream. This makes the data reading operation of matrix A compatible with the single-cycle single-address access characteristics of static random access memory (SRAM), avoiding memory access conflicts and pipeline interruptions caused by reading the entire column of data at once, and reducing computational latency.

[0035] It's understandable that the traditional linear array architecture's method of reading A matrix data fundamentally contradicts the SRAM access rules. The physical structure of SRAM dictates that only one read / write operation can be performed on one address per clock cycle, i.e., single-cycle, single-address access. Traditional solutions require reading an entire column of data from matrix A at once. Since the elements in this column are distributed across different address rows in SRAM, it takes multiple clock cycles to complete the reading of one column of data. During this process, pipeline bubbles and interruptions occur, resulting in accumulated computational latency.

[0036] This embodiment changes the data reading method of matrix A from reading it column-by-column at once to reading it element-by-element sequentially. Each clock cycle, the data broadcast loading module 11 reads only one element of matrix A, which comes from a position in the currently processed column, and then broadcasts it to all processing units (PEs) in the form of a data stream. Since only one address of SRAM is accessed each clock cycle, the read operation is completely consistent with the physical access rules of SRAM, and there are no address conflicts. Once all elements of a column have been read sequentially, the reading of a new column of matrix A automatically begins, forming a continuous data stream.

[0037] Thus, at the memory access level, the reading of matrix A data achieves bubble-free pipelined processing. A new element of matrix A enters the PE array each clock cycle, eliminating PE idle cycles caused by waiting for data from matrix A. At the computation level, because the arrival timing of matrix A elements precisely matches the reuse cycle of matrix B elements, the core computing unit 15 can perform one effective multiplication or addition operation per clock cycle, and the theoretical utilization rate of the processing unit PE can approach 100%. The adaptation of the data broadcast loading module to the single-cycle, single-address access characteristic of SRAM is the fundamental guarantee for the entire device to achieve efficient pipelined computing.

[0038] The number of processing units (PEs) is determined by the number of columns in matrix B, ensuring that the entire row of matrix B can be loaded at once. Meanwhile, the element-by-element pipelined broadcasting of matrix A ensures that the data from matrix A continuously flows into the processing units (PEs). The two achieve a fast and stable combination, coordinating vertically and horizontally, and together constitute a complete matrix outer product calculation data flow system.

[0039] In one embodiment, the data whole row distribution and loading module is equipped with a collaborative mechanism that triggers a new column in matrix A to trigger a new row in matrix B. This mechanism includes: when the data broadcast loading module starts inputting data for a new column of matrix A, it automatically triggers the data whole row distribution and loading module to read the next row of data in matrix B, ensuring that the column data of matrix A and the row data of matrix B are strictly aligned, thus avoiding additional handshake overhead.

[0040] It's understandable that the core of matrix outer product calculation lies in multiplying each column element of matrix A with the corresponding row element of matrix B to produce a component of matrix C. If the column data of matrix A and the row data of matrix B cannot be precisely aligned in time, a calculation error due to data mismatch will occur within the processing unit PE, or an additional handshake signal mechanism will be needed to coordinate the arrival timing of the two, increasing control complexity and latency overhead.

[0041] This embodiment solves the above problem by setting a collaborative mechanism where a new column in A triggers a new row in B. When the data broadcasting loading module finishes reading the last element of the current column and is about to start inputting the first element of the new column, it automatically generates a trigger signal. This trigger signal directly drives the data whole row distribution loading module 13 to read the next row of data in matrix B from SRAM. The mechanism works as follows: when matrix calculation starts, the first row of data in matrix B is first loaded and distributed to each processing unit (PE); then, the first column of matrix A begins to be input, and the first element is broadcast to each processing unit (PE) element by element. The core calculation unit 15 multiplies the A elements broadcast to each processing unit (PE) with the corresponding elements of the first row of B that have been cached; when the last element of the first column of matrix A has been broadcast, before the data broadcasting loading module 11 starts inputting the first element of the second column, it automatically triggers the data whole row distribution loading module 13 to read the second row of data in matrix B and distribute it to each processing unit (PE); and so on, until all k columns of matrix A and all k rows of matrix B have been processed.

[0042] Thus, the alignment of column data in matrix A with row data in matrix B is entirely automated by hardware, requiring no software intervention or additional handshake protocols. The generation and response of the trigger signal are completed within the same clock cycle, with zero latency. Due to the precision of the alignment, no data validity verification logic needs to be set within the processing unit PE, further simplifying the structure of the processing unit PE.

[0043] The mechanism of triggering a new row in B by a new column in A depends on the data flow pattern of broadcasting elements of matrix A in column order, while the ability of matrix B to load an entire row at once depends on the architecture design where the number of PEs is equal to the number of columns in matrix B. The three form a complete causal chain: architecture determines capability, data flow provides the timing basis, and triggering mechanism achieves precise alignment.

[0044] In one embodiment, the elements of matrix B cached within each processing unit (PE) are reused repeatedly, the number of times equal to the number of rows m of matrix A. Each processing unit (PE) calculates and outputs the column component values ​​of matrix C in each clock cycle, and the data outputs of all processing units (PE) are merged into the data components of matrix C.

[0045] It is understandable that, in order to address the problem of low utilization of B matrix elements in traditional linear array architecture, in the traditional architecture, a row of data in the B matrix flows through each processing unit (PE) sequentially. Each processing unit (PE) uses the B matrix element only once after receiving it and then passes it to the next processing unit (PE). The total utilization of the B matrix elements is limited by the pipeline depth of the processing unit (PE).

[0046] This embodiment changes the data flow direction so that each processing unit (PE) caches an element of matrix B locally upon receiving it, and reuses that element repeatedly over the next m clock cycles. Similarly, if matrix A is an m-row, k-column matrix and matrix B is a k-row, n-column matrix, then the matrix outer product operation requires multiplying each column element of A with the corresponding row element of B. When the x-th row element of matrix B is distributed to each PE, each PE caches one B element. Over the next m clock cycles, the x-th column element of matrix A is broadcast to all processing units (PEs) sequentially, with one element of matrix A arriving each clock cycle. The core computing unit 15 multiplies the arriving element of matrix A with the cached element of matrix B, producing one column component element of matrix C each time. After m clock cycles, the core computing unit 15 has completed the calculation of all m component elements of the corresponding column in matrix C. Therefore, each element of matrix B is reused m times within the PE's cache cycle, fully utilizing data locality.

[0047] Thus, on the one hand, the elements of the B matrix only need to be read from SRAM once and can be reused m times within the processing unit PE, significantly reducing the memory access bandwidth requirements of the B matrix data. On the other hand, each processing unit PE continuously performs effective multiplication and addition operations within m consecutive clock cycles, resulting in high utilization of the computational resources of the processing unit PE. Furthermore, each processing unit PE synchronously outputs a column component value of the C matrix in each clock cycle, and the n column component values ​​of n processing units PE are combined to form a complete row of the C matrix data components, laying the foundation for subsequent continuous row-by-row output.

[0048] This works in conjunction with the pipelined broadcasting mechanism of matrix A mentioned above. That is, the arrival of elements of matrix A in each clock cycle provides a precise timing rhythm for the m multiplexing of elements of matrix B. The two form a stable pairing relationship between multiplier and multiplicand within PE, and generate an effective product once per clock cycle.

[0049] In one embodiment, the number of processing units (PEs) is equal to the number of columns in matrix B. When the number of columns n in matrix B is less than or equal to the number of processing units (PEs), matrix multiplication is performed directly. When the number of columns n in matrix B is greater than the number of processing units (PEs), matrix B is divided into blocks by columns, and then block matrix multiplication is performed. After block multiplication, only matrix multiplication is involved, and no additional matrix addition is required.

[0050] It is understandable that there is a hardware adaptation issue for matrix multiplication of different sizes. In practical applications, the size of the matrices to be calculated varies. If the number of processing units (PE) is fixed while the number of columns in matrix B varies, a general processing strategy is needed to adapt to matrix multiplication operations of various sizes.

[0051] In this embodiment, the number of processing units (PEs) is designed to be equal to the number of columns (n) of matrix B, meaning the number of PEs matches the width of the matrix B to be processed. When the number of columns (n) of matrix B is less than or equal to the number of PEs, all columns of B are handled by corresponding PEs, allowing for direct execution of complete matrix multiplication. When the number of columns (n) of matrix B is greater than the number of PEs, the width of matrix B exceeds the coverage of the PE array. In this case, matrix B is divided into blocks by columns. The block division can be achieved by dividing matrix B into several sub-blocks, where the number of columns in each sub-block does not exceed the number of PEs. Matrix multiplication is performed on each sub-block, with each operation using the outer product of the corresponding row range of matrix A and the corresponding column range of the sub-block in matrix B to generate a partial result for matrix C.

[0052] The key improvement of this embodiment lies in the fact that only matrix multiplication is involved after block partitioning, eliminating the need for additional matrix addition. In traditional matrix block computation, when matrix B is partitioned by column, the multiplication results of each sub-block belong to different regions of matrix C, and there is no overlap between the results of each sub-block. Therefore, there is no need to merge the block results through addition. This is completely different from the traditional scheme of partitioning by both rows and columns, which usually requires matrix addition after block multiplication to merge the results of each sub-block.

[0053] Ultimately, this greatly simplifies the processing logic for large matrices. For a B matrix of arbitrary size, it is only necessary to determine whether it needs to be divided into blocks based on whether its number of columns exceeds the number of processing units (PEs). Each sub-block independently performs matrix multiplication, and the results of each sub-block are naturally concatenated in the C matrix, without the need for additional addition hardware or addition steps. This makes the control logic extremely simple and the hardware implementation complexity low.

[0054] In one embodiment, the C matrix storage control module directly writes the merged C matrix row data back to SRAM, with the write address continuously increasing. No output buffering or transpose operation is required, and the write bandwidth utilization reaches 100%.

[0055] Understandably, to address the low output efficiency of traditional linear array architectures, conventional solutions combine the outputs of all processing units (PEs) into a single column of data in a C matrix each clock cycle. This column's storage address in SRAM is discontinuous, requiring address jumps for each write operation and failing to utilize SRAM's continuous write mode, resulting in low write bandwidth utilization. Another approach temporarily stores intermediate results in the local memory within the processing unit (PE) and outputs them row by row after all calculations are complete. This not only consumes significant storage resources but also leads to accumulated latency during the output phase.

[0056] This embodiment completely solves the aforementioned problem by changing the data flow direction. Since the number of processing units (PEs) is equal to the number of columns n in matrix B, and each PE is responsible for calculating all elements in one column of matrix C, each PE outputs one element from a different column of matrix C in each clock cycle. The C matrix storage control module 17 merges the outputs of the n processing units (PEs) in PE index order; for example, PE_0 outputs the element in column 0 of matrix C, PE_1 outputs the element in column 1, and so on, into a single row of data in matrix C. This row contains n elements, which exactly correspond to the entire contents of one row of matrix C.

[0057] Since the output is a complete row of data, the target address for writing to SRAM is the starting address of the row. The next n elements are then written sequentially to a contiguous storage space with increasing addresses. The SRAM write operation is completely sequential, requiring no address transitions, no mask control, no cache transfer, and no subsequent transpose. One row is output and one row is written each clock cycle, with the write address continuously increasing from the starting address of the first row of the C matrix to the last row.

[0058] Thus, each clock cycle's write operation writes all n elements of a complete row to SRAM without any idle time, waiting, or redundant operations. The m rows of data in matrix C are all written within m clock cycles, and the output latency depends only on the number of rows in the matrix. Furthermore, no post-processing is required after output. The ability to output rows continuously relies on an architecture where the number of processing units (PEs) equals the number of columns in matrix B, and the core computing unit 15 is responsible for calculating one column of elements per PE. Only when each PE outputs corresponding row elements from different columns of matrix C can they be merged using indexes to form a complete row.

[0059] In one embodiment, during the calculation process, when performing calculations on non-first components of matrix C, each processing unit (PE) reads the old values ​​of matrix C from SRAM, splits them according to element positions, and distributes them to each processing unit (PE). Each processing unit (PE) then accumulates the current calculation result with the corresponding old value and outputs the result. In the first component calculation of matrix C, the result is calculated from the first column element of matrix A and the first row element of matrix B. Since there are no accumulated old values ​​of matrix C, each processing unit (PE) directly outputs the result.

[0060] The question is understandable: how to correctly sum the results of each component calculation in a matrix outer product operation to obtain the final C matrix. For example... Figure 4 As shown, the principle of matrix outer product operation is as follows: Each column element of matrix A is multiplied by the corresponding row element of matrix B, and each calculation yields a component Cx (1 <= x <= k) of matrix C. All components are then summed to obtain the final matrix C. If matrix A is m rows and k columns, matrix B is k rows and n columns, and matrix C is m rows and n columns, then k component calculations are required. The first component calculation uses the first column of matrix A (e.g., ...). a 1) and the first row of matrix B (e.g. b 1) The second time using the second column of matrix A (e.g. a 2) With the second row of matrix B (e.g. b 2), and so on.

[0061] During the first component calculation, the previously calculated C matrix value does not yet exist. Therefore, the processing unit PE directly outputs the current calculation result as the first component of the C matrix. Starting from the second component calculation, the current calculation result needs to be accumulated with the previously calculated old value of the C matrix stored in SRAM. The specific accumulation process is as follows: Before starting the current component calculation, the current old value of the C matrix is ​​read from SRAM. The C matrix is ​​an m-row n-column matrix, read row by row, with n elements per row. The read old value of the C matrix is ​​split according to the element position and distributed in parallel to the corresponding processing unit PE, that is, the element of the j-th column of the C matrix is ​​distributed to the j-th processing unit PE. Each processing unit PE performs local addition and accumulation of the current calculation result and the received old value of the C matrix, and outputs the updated C matrix element value after accumulation. The C matrix storage control module 17 merges the accumulated results of each processing unit PE into a whole row of data of the C matrix and writes it back to SRAM to update the stored value of the C matrix.

[0062] This embodiment introduces a mechanism to distinguish between first-time and non-first-time components. Its technical significance lies in the following: first-time component calculation does not require SRAM read operations, saving a read cycle; during non-first-time component calculation, SRAM reads and PE calculations are performed in parallel, reading the old values ​​of the C matrix already written after the previous component calculation, ensuring the correctness of the accumulation. Thus, through component accumulation, the final result of the C matrix is ​​gradually accumulated in SRAM, without the need for additional accumulation hardware or post-processing steps. The direct output of the first-time component and the accumulated output of non-first-time components are unified into an output operation at the PE level, simplifying the control logic.

[0063] In this process, the old value of the C matrix read during non-first component calculations is the result of the previous component calculation being written to the SRAM row by row by the C matrix storage control module 17. The new result after accumulation is then written back to the SRAM row by row by the C matrix storage control module 17, forming a complete closed loop of read-accumulation-write-back. The old value of the C matrix is ​​stored in the SRAM rather than inside the processing unit PE. Each component calculation reads from the SRAM, accumulates the result, and then writes it back to the SRAM, maintaining the stateless design of the processing unit PE.

[0064] In one embodiment, the calculation results of each processing unit PE are directly output after accumulation, and then merged into a whole row of data in the C matrix by the C matrix storage control module 17 before being written back to SRAM, so as to reduce the storage overhead and hardware area occupation of the processing unit PE.

[0065] It's understandable that traditional linear array architectures suffer from low storage efficiency, wasted storage resources, and excessive hardware area due to the local memory within the processing unit (PE). In traditional solutions, each PE needs its own local memory to temporarily store intermediate calculation results for matrix C. Let matrix A be an m x k ​​matrix and matrix C be an m x n matrix. In traditional solutions, each PE needs to temporarily store multiple iterative intermediate values ​​of corresponding elements in matrix C, and the capacity requirement of the local memory is positively correlated with the matrix size. When the matrix size is large, the area ratio of the local memory within the PE increases significantly, and the local memory spends most of its time in a hold state rather than an active read / write state, resulting in low storage resource utilization.

[0066] This embodiment, based on the changed data flow direction, completely eliminates the need for local memory within the processing unit PE. Since the number of processing units PE equals the number of columns n of matrix B, each processing unit PE is responsible for calculating all m elements of one column in matrix C. In each clock cycle, the processing unit PE directly outputs the current calculation result or accumulated result without needing to temporarily store it internally. The old value of matrix C is stored in SRAM, read from SRAM when accumulation is needed, and immediately written back to SRAM after accumulation. The processing unit PE itself does not retain any historical state throughout the calculation process; its work in each clock cycle is independent: such as receiving element A, multiplying it with the cached element B, accumulating it with the old value of matrix C from SRAM, and outputting the result.

[0067] Thus, at the storage level, the internal memory for caching intermediate results of the C matrix is ​​no longer needed within the processing unit PE. The hardware structure of each processing unit PE is simplified to a multiplier, an adder, and a small number of registers, significantly reducing the hardware area of ​​the processing unit PE. The cumulative storage area saved by n processing units PE is significant, allowing more processing units PE or other functional modules to be integrated within the same chip area. In terms of computational efficiency, the computation results of the processing unit PE are directly piped out, without waiting for the write and read latency of local memory, minimizing the computation-output latency per clock cycle.

[0068] The above embodiments define a mechanism for reading the old value of matrix C from SRAM and distributing it to the processing unit PE for accumulation. This mechanism eliminates the need for the processing unit PE to store the old value of matrix C locally, thus providing a prerequisite for the absence of local memory within the processing unit PE in this embodiment. Furthermore, the mechanism of writing the accumulation result back to SRAM row by row ensures that the processing unit PE does not need to retain a local copy after outputting the result. Ultimately, these three elements together constitute a complete local memory-free computational closed loop: SRAM is the sole data storage location for matrix C, and the processing unit PE is a purely computational unit.

[0069] Each module in the aforementioned high-performance matrix linear operation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of a device with data processing capabilities, or stored in software within the memory of the aforementioned device, so that the processor can call and execute the operations corresponding to each module. The aforementioned device can be, but is not limited to, various types of computer devices already existing in the art.

[0070] In one embodiment, such as Figure 5 As shown, a high-performance matrix linear operation method is also provided, which may include the following steps S12 to S18: S12, Initialization Configuration: Set matrix dimension parameters and reset internal computing units. Here, matrix A is an m x k ​​matrix, matrix B is a k x n matrix, and matrix C is an m x n matrix, where m, k, and n are all positive integers. Resetting internal computing units (such as the core computing unit) clears any residual data from the previous calculation, ensuring that the current calculation starts from the initial state.

[0071] S14: In each clock cycle, individual elements of matrix A are read in column order to form a data stream, which is then synchronously broadcast to all processing units (PEs). Simultaneously, an entire row of data from matrix B is read from SRAM and distributed in parallel to each processing unit (PE) according to its element position. Each processing unit (PE) corresponds to one element of matrix B. The number of processing units (PEs) is determined based on the number of columns n of matrix B. The number of processing units (PEs) is determined during the initialization phase based on the number of columns n of matrix B, such that the number of processing units (PEs) is equal to n.

[0072] Data loading for matrix A and matrix B is performed in parallel within the same clock cycle. Matrix A is read element-wise column-by-column: first, the first element of the first column is read and broadcast to all processing units (PEs); in the next clock cycle, the second element of the first column is read and broadcast to all PEs; this continues until the m-th element of the first column has been read. Matrix B is read row-by-row at once: in the clock cycle following the input of a new column of data to matrix A, all n elements of the corresponding row of matrix B are read from SRAM at once. Based on the element's index within the row, the 0th element is assigned to the 0th processing unit (PE), the 1st element to the 1st processing unit (PE), and so on. Since the number of processing units (PEs) is n, each PE receives and buffers exactly one element of matrix B.

[0073] In step S16, each processing unit (PE) performs a multiplication-addition operation on the received elements of matrix A and the cached elements of matrix B, and outputs the calculation result. When calculating a non-first component of matrix C, each processing unit (PE) adds the old value of matrix C read from SRAM to the current calculation result and outputs the result. When calculating the first component of matrix C, each processing unit (PE) directly outputs the result.

[0074] PE calculations distinguish between two execution paths based on the order of component calculations. For the first component calculation, i.e., the outer product of the first column of matrix A and the first row of matrix B, since there are no historical values ​​of matrix C yet, each processing unit (PE) directly outputs the product after performing the operation. For subsequent component calculations, i.e., the outer product of the x-th column of matrix A and the x-th row of matrix B, each processing unit (PE) reads the current old value of matrix C from SRAM before starting the calculation. The old value of matrix C is distributed to each processing unit (PE) in the same way as the row data of matrix B. Each processing unit (PE) performs the calculation result and the received old value of matrix C locally before outputting it.

[0075] S18: The output data of all processing units PE are merged into a whole row of data in C matrix according to the PE index order. One row is output every clock cycle and the merged row data is written directly back to SRAM.

[0076] The n output values ​​of n processing units (PEs) are concatenated in ascending order of their PE indices to form the n elements of a complete row in matrix C. The output row data is directly written to the starting address of the corresponding row in SRAM, and the write addresses of subsequent elements increment sequentially from this starting address. Each clock cycle, one row is output and written simultaneously until all m rows of data in matrix C have been output and written. Finally, when all data from matrices A and B has flowed in, the pipeline within each processing unit (PE) is empty, and all m rows of data in matrix C have been written to SRAM, the entire matrix operation process ends.

[0077] The aforementioned high-performance matrix linear operation method determines the size of the PE array based on the number of columns in matrix B rather than the number of rows in matrix A. This allows the matrix calculation to begin by accessing one row of data from matrix B in a single clock cycle and distributing it to all PEs. This fully adapts to the characteristics of SRAM row-by-row access and avoids the problems caused by traditional solutions that require reading one column of data from matrix A each time due to determining the PE based on the number of rows in matrix A. This results in slow read speeds and does not conform to the rule of accessing one row at a time in SRAM, thereby reducing overall pipeline operation latency.

[0078] Data for matrix A is streamed, with one element per column in each clock cycle, broadcast to all processing units (PEs). This ensures a completely streamlined, bubble-free, and uninterrupted reading process for matrix A, bringing the theoretical utilization of PEs close to 100%. Data for matrix B is distributed in parallel row-by-row, with a mechanism that triggers a new row in matrix B when a new column in matrix A is input. This ensures strict alignment between columns in matrix A and rows in matrix B, avoiding additional handshake overhead. No local memory is needed for matrix C within each PE; calculation results are directly piped out after accumulation, reducing storage overhead and hardware footprint. All PE outputs are merged into a single row of matrix C in PE index order, one row per clock cycle. Writing to SRAM involves continuous address incrementing without masking or address jumps, achieving 100% write bandwidth utilization and significantly optimizing data interaction efficiency with local memory.

[0079] In one embodiment, the data loading process employs a collaborative mechanism where a new column in matrix A triggers a new row in matrix B: when data in a new column of matrix A is input, the reading of the next row of matrix B is automatically triggered. The number of times the elements of matrix B cached within each processing unit (PE) are reused is equal to the number of rows in matrix A.

[0080] It is understood that the specific limitations of the features in the various embodiments of the above-mentioned high-performance matrix linear operation method can be found in the explanation of the corresponding features in the various embodiments of the high-performance matrix linear operation device above, and will not be repeated here.

[0081] It should be understood that, although Figure 5 The steps are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order in which these steps are executed; they can be performed in other orders. Figure 5 At least some of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0082] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0083] The above embodiments merely illustrate several implementation methods of the present invention, and their descriptions are relatively specific and detailed, but they should not be construed as limiting the scope of protection of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and all such modifications and improvements fall within the scope of protection of the present invention.

Claims

1. A high-performance matrix linear operation device, characterized in that, include: Multiple processing units (PEs); The number of processing units (PEs) is determined based on the number of columns in the B matrix to be computed; The data broadcast loading module is used to read individual elements in column order of matrix A in each clock cycle and synchronously broadcast them to all processing units (PE) in the form of a data stream; The data row distribution and loading module is used to read the entire row of data of the B matrix at once within one clock cycle and distribute it to each processing unit (PE) according to the element position; each processing unit (PE) buffers one B matrix element received. The core computing unit is used to perform multiplication and addition operations with the currently cached elements of matrix B after receiving the broadcast elements of matrix A, and output the calculation results. The C matrix storage control module is used to merge the output data of all processing units (PE) into a single row of data in the C matrix according to the PE index order; and outputs one row of data in the C matrix every clock cycle.

2. The high-performance matrix linear operation device according to claim 1, characterized in that, The data broadcast loading module adopts a pipelined broadcast loading mechanism, which continuously broadcasts each element of matrix A to all processing units (PE) in the form of a data stream.

3. The high-performance matrix linear operation device according to claim 1, characterized in that, The data whole-row distribution and loading module is configured with a collaborative mechanism that triggers a new row in B when a new column in column A is added, including: When the data broadcast loading module starts inputting data for a new column of matrix A, the data row distribution loading module is automatically triggered to read the next row of data in matrix B.

4. The high-performance matrix linear operation device according to any one of claims 1 to 3, characterized in that, The elements of matrix B cached within each processing unit (PE) are reused repeatedly, and the number of reuses is equal to the number of rows in matrix A. Each processing unit (PE) calculates and outputs the column component values ​​of matrix C in each clock cycle. The data outputs of all processing units (PE) are then combined into the data components of matrix C.

5. The high-performance matrix linear operation device according to claim 4, characterized in that, The number of processing units (PEs) is equal to the number of columns in matrix B; When the number of columns n in matrix B is less than or equal to the number of processing units PE, matrix multiplication is performed directly. When the number of columns n in matrix B is greater than the number of processing units PE, matrix B is divided into blocks by columns and then the block matrix multiplication operation is performed.

6. The high-performance matrix linear operation device according to claim 4, characterized in that, The C matrix storage control module writes the merged C matrix row data directly back to SRAM, with the write address incrementing continuously.

7. The high-performance matrix linear operation device according to claim 4, characterized in that, During the calculation process, when performing non-first component calculations of matrix C, each processing unit (PE) reads the old value of matrix C from SRAM, splits it according to element position, and distributes it to each processing unit (PE). Each processing unit (PE) then accumulates the current calculation result with the corresponding old value and outputs it. The first component calculation of matrix C is obtained by calculating the first column element of matrix A and the first row element of matrix B.

8. The high-performance matrix linear operation device according to claim 7, characterized in that, The calculation results of each processing unit PE are directly output after accumulation. After being merged into a whole row of data in the C matrix by the C matrix storage control module, they are written back to SRAM.

9. A high-performance matrix linear operation method, characterized in that, Including the following steps: Initialize the configuration, set the matrix dimension parameters, and reset the internal calculation units; where matrix A is an m x k ​​matrix, matrix B is a k x n matrix, and matrix C is an m x n matrix, where m, k, and n are all positive integers; In each clock cycle, individual elements of matrix A are read in column order to form a data stream, which is then synchronously broadcast to all processing units (PEs). At the same time, an entire row of data from matrix B is read from SRAM and distributed in parallel to each processing unit (PE) according to element position. Each processing unit (PE) corresponds to one element of matrix B, and the number of processing units (PEs) is determined based on the number of columns n of matrix B. Each processing unit (PE) performs multiplication and addition operations on the received A matrix elements and the cached B matrix elements and outputs the calculation results. When calculating a non-first component of the C matrix, each processing unit (PE) adds the old value of the C matrix read from SRAM to the current calculation result and outputs it. When calculating the first component of the C matrix, each processing unit (PE) directly outputs the result. The output data of all processing units (PE) are merged into a single row of data in the C matrix according to the PE index order. One row is output every clock cycle and the merged row data is written directly back to SRAM.

10. The high-performance matrix linear operation method according to claim 9, characterized in that, The data loading process uses a collaborative mechanism where a new column in column A triggers a new row in column B: When a new column of data in matrix A is entered, the reading of the next row of data in matrix B is automatically triggered; the number of times the elements of matrix B cached in each processing unit PE are reused is equal to the number of rows in matrix A.

Images