Matrix computing device, method, chip, board and electronic equipment

Abstract

The embodiment of the application discloses a kind of matrix computing device, method, chip, board card and electronic equipment, wherein the matrix computing device includes: module matrix generation unit, for converting two input matrices using target floating-point data format into integer matrix, and based on a plurality of preset modulus is modulated, obtain the pair of module matrix corresponding to each preset modulus, module matrix element uses integer data format, its data bit width is less than the data bit width of target floating-point data format;Tensor calculation core, for performing matrix multiplication to each pair of module matrix, according to the modulus reduction of corresponding preset modulus to multiplication result, obtain module reduction result matrix;Remainder reconstruction unit, for each module reduction result matrix is carried out remainder reconstruction, obtains the matrix multiplication result using target floating-point data format.The application can utilize the low-precision integer matrix operation capacity of tensor calculation core to complete high-precision floating-point matrix operation, which is conducive to improving high-precision matrix operation efficiency.

Description

Technical Field

[0001] This application relates to the field of on-chip matrix computing technology, and more specifically, to a matrix computing device, method, chip, board, and electronic device. Background Technology

[0002] Matrix operations are fundamental operations in scientific computing, engineering simulation, and artificial intelligence. In applications such as climate modeling, materials computation, quantum simulation, financial risk analysis, and large-scale numerical simulations, high-precision floating-point data types, such as 64-bit floating-point data types (FP64), are typically required for matrix operations to meet the precision requirements of the calculation results.

[0003] Current mainstream computing hardware, especially graphics processing units (GPUs), increasingly focuses on meeting the computational demands of artificial intelligence training and inference. To meet the computational throughput requirements of large-scale matrix multiplication in AI, mainstream GPUs typically incorporate highly optimized tensor cores for performing matrix multiplication operations. Their performance optimizations primarily focus on low-precision data types such as INT8, BF16, and TF32. Compared to high-precision data types, tensor cores can theoretically achieve peak computing power tens or even hundreds of times greater for low-precision data types. Therefore, existing tensor cores are mainly optimized for low-precision data types, and when directly performing high-precision floating-point matrix operations, they struggle to leverage their computational performance advantages with low-precision data types.

[0004] Therefore, how to improve the efficiency of matrix operations on existing computing hardware with high-precision data types is a technical problem that needs to be solved in this field. Summary of the Invention

[0005] In order to solve at least one of the technical problems in the background art described above, this application proposes a matrix computing device, method, chip, board, and electronic device.

[0006] One aspect of this application provides a matrix computing device, the matrix computing device comprising: The modular matrix generation unit is used to convert the first input matrix and the second input matrix, which adopt the target floating-point data format, into integer matrices respectively, and to perform modular decomposition on the integer matrices based on multiple preset moduli to obtain a pair of modular matrices corresponding to each preset moduli. The matrix elements of the modular matrices adopt integer data format, and the data bit width of the integer data format is smaller than the data bit width of the target floating-point data format. The tensor computation core is used to perform matrix multiplication on a pair of modulo matrices corresponding to each preset modulo, and perform modulo reduction on the matrix multiplication result according to the preset modulo to obtain the modulo reduction result matrix corresponding to each preset modulo. The remainder reconstruction unit is used to perform remainder reconstruction based on each of the modulo reduction result matrices to obtain the matrix multiplication result of the first input matrix and the second input matrix, wherein the matrix elements of the matrix multiplication result adopt the target floating-point data format.

[0007] Optionally, the modular matrix generation unit is tightly coupled to the tensor computation core, and the modular matrix generation unit transmits the modular matrix to the tensor computation core through an on-chip interconnect network; The remainder reconstruction unit is tightly coupled to the tensor calculation core, and the tensor calculation core transmits the modulo reduction result matrix to the remainder reconstruction unit through an on-chip interconnect network.

[0008] Optionally, the modular matrix generation unit includes: a near-Mersen number modular operation unit array, the near-Mersen number modular operation unit array including multiple near-Mersen number modular operation units respectively corresponding to each of the preset moduli and arranged in parallel, wherein the multiple preset moduli are pairwise coprime, and each of the preset moduli is a near-Mersen number; Each of the near-Mason numerator modular arithmetic units is used to calculate the modulo remainder of each matrix element in the integer matrix with respect to its corresponding preset modulus.

[0009] Optionally, the modulus matrix generation unit further includes: A fixed-point encoder is used to determine a scaling offset based on the exponent and mantissa of each matrix element in the first input matrix and the second input matrix, and to perform shifting and sign processing on the matrix elements in the first input matrix and the second input matrix based on the scaling offset to obtain the integer matrix; The matrix assembly and distribution buffer unit is used to arrange the modulo remainders calculated by each of the near-Mason modulo operation units according to the arrangement order of the matrix elements corresponding to each modulo remainder in the first input matrix or the second input matrix, and assemble them according to the matrix dimensions of the first input matrix and the second input matrix respectively, to obtain a pair of modulo matrices corresponding to each preset modulo.

[0010] Optionally, the near-Mason digitization unit includes: The block extraction unit is used to split each matrix element in the integer matrix into multiple data blocks, using the data bit width of the integer data format as the unit; The parallel weighted operation unit is used to perform parallel weighted operation on the multiple data blocks using preset weight coefficients associated with the corresponding preset modulus, and to obtain multiple weighted operation results. The preset weight coefficients are predetermined based on the data bit width of the integer data format, the corresponding preset modulus, and the block order of the corresponding data blocks. An accumulation unit is used to compress and sum the multiple weighted operation results through an accumulation tree to obtain an accumulation result; The near-Massen number modulo reduction simple element is used to reduce the accumulated result moduloly without performing division operations based on the corresponding preset modulus, so as to obtain the modulo remainder.

[0011] Optionally, the tensor computation core includes a modular reduction element and an output interface. The output interface has a result truncation mode corresponding to the integer data format. The modular reduction element is used to perform modular reduction on the matrix multiplication result for each preset modulus, to obtain the modular reduction result matrix corresponding to the preset modulus. The output interface is used to output the modular reduction result matrix in the integer data format under the result truncation mode.

[0012] Optionally, the remainder reconstruction unit includes: The segmented parallel multiplication unit is used to split the multiplication operation between each matrix element in the modulus reduction result matrix corresponding to the preset modulus and the remainder reconstruction coefficient corresponding to the preset modulus into multiple sub-multiplication operations for each preset modulus, execute the multiple sub-multiplication operations in parallel, and merge the results of each sub-multiplication operation to obtain the product result corresponding to each matrix element. The addition merging unit is used to sum the product results corresponding to the matrix elements at the same position in each of the modulo reduction result matrices through the addition tree to obtain the merged result matrix; A global modulo reduction simple element is used to perform modulo reduction on the merged result matrix based on a global modulus determined by the product of the multiple preset moduli, to obtain a reconstructed integer matrix; The floating-point format conversion and descaling unit is used to perform floating-point format conversion and descaling on the reconstructed integer matrix to obtain the matrix multiplication result, wherein the plurality of preset moduli are pairwise coprime.

[0013] Optionally, the matrix calculation device further includes: A unified control unit is used to provide the same clock enable signal to the plurality of near-Mason digitization units, so as to control the plurality of near-Mason digitization units to synchronously calculate the modulo remainder of each matrix element in the integer matrix with respect to its corresponding preset modulus.

[0014] Optionally, there are multiple tensor computation cores, each corresponding to a different preset modulus when performing matrix multiplication. The multiple tensor computation cores are used to perform matrix multiplication on a pair of modulo matrices corresponding to their respective preset modulo in parallel, and to perform modulo reduction on the matrix multiplication result according to their respective preset modulo.

[0015] Another aspect of this application provides a matrix calculation method, which includes: The first and second input matrices, which adopt the target floating-point data format, are converted into integer matrices respectively. The integer matrices are then decomposed based on multiple preset moduli to obtain a pair of moduli matrices corresponding to each preset moduli. The matrix elements of the moduli matrices adopt integer data format, and the data bit width of the integer data format is smaller than the data bit width of the target floating-point data format. For each preset modulus, matrix multiplication is performed on the pair of modulo matrices corresponding to the preset modulus, and the result of the matrix multiplication is reduced by modulus according to the preset modulus to obtain the modulus reduction result matrix corresponding to each preset modulus; The remainders are reconstructed based on the modulo reduction result matrices to obtain the matrix multiplication result of the first input matrix and the second input matrix, wherein the matrix elements of the matrix multiplication result adopt the target floating-point data format.

[0016] In another aspect of this application, a chip is also provided, including the matrix computing device described above.

[0017] Another aspect of this application provides a board card including the aforementioned chip.

[0018] In another aspect of this application, an electronic device is also provided, including at least one of the above-described chip and the above-described circuit board.

[0019] The beneficial effects of this application are as follows: This application uses a modular matrix generation unit to convert a first and second input matrices in the target floating-point data format into integer matrices, and performs modular decomposition based on multiple preset moduli to obtain multiple pairs of modular matrices with matrix elements in the integer data format. Since the data bit width of the integer data format is smaller than that of the target floating-point data format, the tensor computation core can perform matrix multiplication and modular reduction on the modular matrices in the integer data format with the smaller data bit width. The remainder reconstruction unit then performs remainder reconstruction based on each modular reduction result matrix to obtain the matrix multiplication result in the target floating-point data format. Therefore, this application can utilize the matrix operation capabilities of the tensor computation core in the integer format with a smaller data bit width for matrix multiplication in the target floating-point data format, which is beneficial for improving the computational efficiency of matrix multiplication in the target floating-point data format. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings: Figure 1 This is a first schematic diagram of a matrix computing device according to an embodiment of this application; Figure 2 This is a second schematic diagram of the matrix computing device according to an embodiment of this application; Figure 3 This is a schematic diagram of the modulus matrix generation unit in an embodiment of this application; Figure 4 This is a schematic diagram of the modified tensor computation core of an embodiment of this application; Figure 5 This is a flowchart of the matrix calculation method according to an embodiment of this application. Detailed Implementation

[0021] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0022] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims and accompanying drawings of this application are intended to cover non-exclusive inclusion. For example, a system, product or device that includes a series of units is not necessarily limited to those units that are explicitly listed, but may include other units that are not explicitly listed or that are inherent to such systems, products or devices.

[0023] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0024] like Figure 1 As shown, in one embodiment of this application, the matrix calculation device of this application includes: a modular matrix generation unit, a tensor calculation core, and a remainder reconstruction unit.

[0025] The modulus matrix generation unit is used to convert the first input matrix and the second input matrix, which adopt the target floating-point data format, into integer matrices respectively, and to perform modulus decomposition on the integer matrices based on multiple preset moduli to obtain a pair of modulus matrices corresponding to each preset moduli. The matrix elements of the modulus matrix adopt integer data format, and the data bit width of the integer data format is smaller than the data bit width of the target floating-point data format.

[0026] The tensor computation core is used to perform matrix multiplication on a pair of modulo matrices corresponding to each preset modulo, and to perform modulo reduction on the matrix multiplication result according to the preset modulo to obtain the modulo reduction result matrix corresponding to each preset modulo.

[0027] The remainder reconstruction unit is used to perform remainder reconstruction based on each of the modulo reduction result matrices to obtain the matrix multiplication result of the first input matrix and the second input matrix, wherein the matrix elements of the matrix multiplication result adopt the target floating-point data format.

[0028] In this embodiment, the target floating-point data format is the floating-point data format used by the first input matrix, the second input matrix, and the matrix multiplication result. The target floating-point data format can be determined based on the numerical range and computational precision required for matrix operations, and can be, for example, FP32, FP64, FP128, or other floating-point data formats. The integer data format is the data format used by the matrix elements of the modulo matrix, and can be a signed integer data format or an unsigned integer data format, such as INT4, INT8, INT16, INT32, or the corresponding unsigned integer data format. The integer data format can be determined based on the data format supported by the tensor computation core, the preset modulus range, and the numerical range required for remainder reconstruction, and its data bit width is smaller than the data bit width of the target floating-point data format.

[0029] FP64 and INT8 are specific data format combinations in this application. For ease of explanation, the following mainly uses FP64 and INT8 as examples to introduce the technical solution of this application. However, this application is not limited to this. Other data formats that meet the above conditions can also be used for the target floating-point data format and integer data format.

[0030] It should be noted that the integer matrix obtained by the modular matrix generation unit is not the same concept as the integer data format used by the modular matrix. The integer matrix is ​​an intermediate matrix obtained by performing a fixed-point transformation on the first and second input matrices, and its matrix elements can use a larger integer bit width. The modular matrix, on the other hand, is the matrix obtained by performing modular decomposition on the integer matrix, and its matrix elements use an integer data format that the tensor computation core can process. For example, in specific embodiments of FP64 and INT8, the FP64 matrix can be first converted into a 64-bit signed integer matrix, and then this 64-bit signed integer matrix can be decomposed into multiple sets of INT8 modular matrices.

[0031] Existing tensor computation cores are typically optimized for integer data formats with smaller bit widths (e.g., INT8), where matrix operation throughput is generally higher than that for target floating-point data formats (e.g., FP64). However, integer data formats with smaller bit widths cannot directly represent the numerical range and computational precision required for matrix elements in target floating-point data formats, nor can they directly obtain the matrix multiplication results in target floating-point data formats. To address this issue, this application first uses a modular matrix generation unit (with modular decomposition and reduction functions) to convert the matrix in the target floating-point data format into multiple sets of modular matrices in integer data formats. Then, the tensor computation core performs matrix multiplication and reduction on the modular matrices. Finally, the remainder reconstruction unit restores the matrix multiplication results in the target floating-point data format.

[0032] In this embodiment, the modular matrix generation unit first converts the first and second input matrices, which adopt the target floating-point data format, into integer matrices, respectively. This conversion may include scaling and fixed-point encoding. Through scaling and fixed-point encoding, floating-point matrix elements can be converted into integer form, and a scaling relationship between floating-point matrix elements and integer matrix elements can be established so that inverse scaling can be performed after remainder reconstruction. The specific implementation methods of scaling and fixed-point encoding will be described in subsequent embodiments.

[0033] In this embodiment, the modulus matrix generation unit further calculates the modulo remainder of each matrix element in the two integer matrices with respect to multiple preset moduli. For each preset modulus, the two integer matrices form a modulus matrix, thereby obtaining a pair of modulus matrices corresponding to the preset modulus.

[0034] The modular decomposition referred to in this application is not matrix decomposition in the conventional sense, but rather represents the same integer matrix as multiple sets of modulo remainder matrices based on multiple preset moduli. Calculating the modulo remainder of an element of an integer matrix with respect to a preset modulo is equivalent to performing modular reduction on that matrix element. By performing modular reduction on each element of the integer matrix according to multiple preset moduli, multiple modulo matrices corresponding to the integer matrix can be obtained, thus completing the modular decomposition. Therefore, the above-mentioned modular reduction is the specific modular operation used by the modular matrix generation unit to generate the modular matrix.

[0035] In specific embodiments of FP64 and INT8, the modulus matrix generation unit can decompose two 64-bit integer matrices into 16 sets of INT8 modulus matrices, thereby obtaining 16 pairs of INT8 modulus matrices. Each pair of INT8 modulus matrices corresponds to a preset modulus, and each matrix element in the modulus matrix represents the remainder of the corresponding integer matrix element with respect to the preset modulus.

[0036] Modulo remainder can be represented using either unsigned or signed integers. When using the signed INT8 data format, the modulo remainder can be converted to a signed integer that is congruent to it, ensuring that the converted value falls within the range that INT8 can represent. The default modulo value should match the representation range of INT8.

[0037] Through the above processing, the FP64 matrix can be converted into multiple sets of modular matrices that can be processed by the INT8 tensor computation core, providing a data foundation for performing FP64 matrix multiplication using the INT8 matrix operation capability of the tensor computation core.

[0038] In this embodiment, the tensor computation core performs matrix multiplication on a pair of modular matrices corresponding to each preset modulus. The tensor computation core can process multiple pairs of modular matrices corresponding to different preset moduli sequentially, or multiple tensor computation cores can be set up to process multiple pairs of modular matrices corresponding to different preset moduli in parallel.

[0039] Conventional INT8 tensor computation cores typically use INT8 data as matrix multiplication input and store the accumulated results in a larger data bit width format such as INT32. This accumulated result cannot be directly used as the modulo reduction result required for remainder reconstruction. Therefore, the tensor computation core in this embodiment also performs modulo reduction on the matrix multiplication result according to a corresponding preset modulus, obtaining the modulo reduction result matrix corresponding to that preset modulus.

[0040] The aforementioned modular reduction calculates the modulo remainder of the matrix multiplication result based on a preset modulus, rather than directly extracting the low-order data of the matrix multiplication result. Each matrix element in the modular reduction result matrix is ​​congruent to the corresponding matrix multiplication result with respect to the preset modulus, thus preserving the modulo remainder information required for remainder reconstruction. The modular reduction result can be further encoded into a data format supported by the output interface of the tensor computation core.

[0041] In specific implementations of FP64 and INT8, the tensor computation core performs matrix multiplication on 16 pairs of INT8 modular matrices and performs modular reduction based on the preset modulus of each pair of modular matrices, resulting in 16 modular reduction result matrices. Thus, the main matrix multiplication operations originally performed using the FP64 data format can be converted into INT8 matrix multiplication operations, which are more suitable for the tensor computation core.

[0042] In this embodiment, the remainder reconstruction unit obtains the modulo remainder of each element position in the matrix multiplication result under each preset modulus, and performs remainder reconstruction based on each modulo remainder and its corresponding preset modulus. Remainder reconstruction does not directly sum multiple modulo-reduced result matrices, but rather uses the modulo remainders under different preset moduli to jointly recover the corresponding integer matrix multiplication result.

[0043] When multiple preset moduli satisfy the remainder reconstruction conditions, and the product of the multiple preset moduli can cover the numerical range required for the integer matrix multiplication result, the integer matrix multiplication result can be recovered based on the reduction result matrix of each moduli. After obtaining the integer matrix multiplication result, the remainder reconstruction unit performs inverse scaling according to the scaling relationship used in fixed-point encoding, and converts the inversely scaled matrix elements into the target floating-point data format.

[0044] In specific embodiments of FP64 and INT8, the remainder reconstruction unit can perform remainder reconstruction based on the Chinese Remainder Theorem (CRT) according to 16 INT8 modulus reduction result matrices. The product of the 16 preset moduli is approximately 2 to the power of 125, which can provide the corresponding integer reconstruction range. After completing integer reconstruction, inverse scaling, and floating-point format conversion, the remainder reconstruction unit can obtain the FP64 matrix multiplication operation result.

[0045] Through the cooperation of the above three units, this application converts matrix multiplication using the target floating-point data format into multiple sets of integer modular matrix multiplications with smaller data bit widths. The tensor computation core completes the matrix multiplication and modular reduction, and the remainder reconstruction unit restores the matrix multiplication operation results in the target floating-point data format.

[0046] In specific implementations of FP64 and INT8, the tensor computing cores in mainstream computing hardware are typically optimized for low-precision data types such as INT8, BF16, and TF32 to meet the throughput requirements of matrix operations in artificial intelligence training and inference. Due to hardware resource allocation and data path design, the theoretical peak computing power of a tensor computing core in FP64 mode is usually significantly lower than its theoretical peak computing power in low-precision modes such as INT8, with differences potentially reaching tens or even hundreds of times. Therefore, a significant "precision-performance gap" exists between the performance of low-precision matrix operations and high-precision floating-point matrix operations in existing computing hardware.

[0047] To address the aforementioned issues, this application does not directly employ the FP64 data path of the tensor computation core to perform all matrix multiplication operations. Instead, it first converts the FP64 input matrix into an integer matrix, and then decomposes the integer matrix into multiple sets of INT8 modular matrices based on multiple preset moduli. The tensor computation core utilizes its INT8 matrix operation capabilities to perform matrix multiplication and modular reduction on each set of INT8 modular matrices. The remainder reconstruction unit recovers the integer matrix multiplication results based on each modular reduction result matrix, and obtains the FP64 matrix multiplication operation results through inverse scaling and floating-point format conversion. Thus, computationally intensive matrix multiplication operations can be transferred to the INT8 data path of the tensor computation core for execution, while retaining the numerical information required for remainder reconstruction using the modulo remainders under different preset moduli.

[0048] The proposed solution can obtain the results of FP64 matrix multiplication while leveraging the matrix operation performance advantages of the tensor computing core in INT8 mode, which is beneficial to improving the computational efficiency of FP64 matrix multiplication and easing the gap between the performance of low-precision matrix operations and high-precision floating-point matrix operations.

[0049] In this embodiment, the preset modulus is an integer parameter pre-configured for modulo decomposition, modulo reduction, and remainder reconstruction, and can be denoted as m0, m1, ..., m... n-1 Where n is the number of preset moduli. For the same preset moduli, the modulo matrix generation unit calculates the modulo remainder of each matrix element in the two integer matrices based on the preset modulo, the tensor calculation core performs modulo reduction on the matrix multiplication result based on the preset modulo, and the remainder reconstruction unit then uses the modulo reduction result corresponding to the preset modulo to reconstruct the remainder.

[0050] By employing multiple preset moduloes, integers with large data bit widths can be represented by multiple modulo remainders. The number and values ​​of the preset moduloes can be determined based on the target floating-point data format, integer data format, and the numerical range of the multiplication result of the integer matrix to be reconstructed. Each preset modulo should ensure that its corresponding modulo remainder can be represented in integer data format, and the product of multiple preset moduloes should cover the numerical range required for remainder reconstruction. In an embodiment using the Chinese Remainder Theorem for remainder reconstruction, the multiple preset moduloes can be pairwise coprime to determine the integer to be reconstructed based on the modulo remainders corresponding to each preset modulo.

[0051] Taking an example with a target floating-point data format of FP64 and an integer data format of INT8, 16 preset moduli that are pairwise coprime can be set, and the modulo remainders corresponding to each preset moduli can be represented using 8-bit integer encoding. The modulo matrix generation unit thus obtains 16 pairs of INT8 modulo matrices, and the tensor computation core obtains 16 modulo reduction result matrices. The product of the 16 preset moduli is approximately 2... 125 The number of preset moduli is on the order of magnitude, which can provide a corresponding remainder reconstruction range. The number of preset moduli mentioned above is only one specific implementation method. The specific number and value of the preset moduli can be adjusted according to the target floating-point data format, integer data format, and remainder reconstruction range. The specific value of the preset moduli and its near-Merssen number characteristics will be described in subsequent embodiments.

[0052] In the embodiments of this application, the modular matrix generation unit, tensor calculation core, and remainder reconstruction unit are all implemented by hardware circuits. These three units have data paths and computational circuits for completing the corresponding operations, and are not implemented solely by a general-purpose processor executing software programs. The integer conversion, modular decomposition, matrix multiplication, modular reduction, and remainder reconstruction operations in this application are performed by the corresponding hardware circuits.

[0053] In this embodiment, the modulus matrix generation unit can be implemented using a custom application-specific integrated circuit (ASIC) module or a programmable logic circuit in a field-programmable gate array (FPGA). The modulus matrix generation unit may include hardware circuitry for performing fixed-point encoding, modulo operations, and data buffering, thereby completing the integer conversion and modulo decomposition of the input matrix in hardware and outputting multiple modulus matrices corresponding to preset moduli.

[0054] In this embodiment, the tensor computation core can be implemented using an extended integer tensor computation core. For example, a modular reduction circuit can be added to a conventional integer multiplication-addition circuit, enabling the tensor computation core to perform matrix multiplication on a modular matrix and perform modular reduction on the matrix multiplication result according to the corresponding preset modulus. The matrix computation device can have a single tensor computation core or an array of tensor computation cores.

[0055] In the embodiments of this application, the remainder reconstruction unit can be implemented using a custom application-specific integrated circuit (ASIC) pipeline or using dedicated arithmetic circuits in a field-programmable gate array (FPGA). The remainder reconstruction unit may include multiplication circuits, addition circuits, modulo reduction circuits, and floating-point format conversion circuits to perform remainder reconstruction, inverse scaling, and floating-point format conversion in hardware.

[0056] In this embodiment, the modular matrix generation unit, tensor calculation core, and remainder reconstruction unit can be integrated into the same computing chip, and the modular matrix and the modular reduction result matrix can be transmitted through the chip's internal data path. The matrix calculation device can also include a matrix buffer unit, a configuration register group, and a result buffer unit, used to store the input matrix, operation parameters, and matrix multiplication results, respectively. The specific connection methods of each unit and the on-chip data transmission methods will be described in subsequent embodiments.

[0057] By configuring the above calculation process as hardware functional units, the control and scheduling overhead caused by software calling different computing resources to perform each stage of the operation can be reduced. When the above three units are integrated into the same computing chip, the modular matrix and the modular reduction result matrix can be transferred through the on-chip data path, which also helps to reduce the data movement overhead caused by the transfer of intermediate data between different computing resources and storage levels.

[0058] In one embodiment of this application, the modular matrix generation unit is tightly coupled to the tensor computation core, and the modular matrix generation unit transmits the modular matrix to the tensor computation core through an on-chip interconnect network; the remainder reconstruction unit is tightly coupled to the tensor computation core, and the tensor computation core transmits the modular reduction result matrix to the remainder reconstruction unit through an on-chip interconnect network.

[0059] In this embodiment, the modular matrix generation unit, the tensor computation core, and the remainder reconstruction unit are housed within the same computing chip. The modular matrix generation unit and the remainder reconstruction unit are physically tightly coupled with the tensor computation core, sharing data paths and storage hierarchies. In this application, the tight coupling primarily manifests as a direct data transfer path between the corresponding hardware functional units. This allows the modular matrix output by the modular matrix generation unit and the modular reduction result matrix output by the tensor computation core to be transmitted to the corresponding downstream hardware functional units via the on-chip interconnect network, reducing the data movement overhead caused by multiple read / write operations of intermediate data through shared memory.

[0060] After completing the modular decomposition, the modular matrix generation unit transmits the modular matrices corresponding to each preset modulus to the tensor computation core via the on-chip interconnect network. After performing matrix multiplication and modular reduction, the tensor computation core transmits the modular reduction result matrices corresponding to each preset modulus to the remainder reconstruction unit via the on-chip interconnect network. Thus, the modular matrix and the modular reduction result matrix can be directly transmitted within the chip along their corresponding data paths, eliminating the need for multiple writes and reads via shared memory between different computation stages.

[0061] Taking specific implementations of FP64 and INT8 as examples, the modular matrix generation unit converts the FP64 input matrix into multiple pairs of INT8 modular matrices, and transmits each pair of INT8 modular matrices to the tensor computation core through the on-chip interconnect network. After performing matrix multiplication and modular reduction on each pair of INT8 modular matrices, the tensor computation core transmits the resulting multiple INT8 modular reduction result matrices to the remainder reconstruction unit through the on-chip interconnect network, and the remainder reconstruction unit recovers the FP64 matrix multiplication operation results.

[0062] In some high-precision matrix computation schemes based on multiple modulus and remainder reconstructions, the various processing stages are mainly scheduled and organized by software. Matrix multiplication and other calculations are typically performed using GPU acceleration libraries, and then the modular decomposition, matrix multiplication, modular reduction, and remainder reconstruction stages are sequentially executed via software control flow from the CPU or GPU. Modulus decomposition and remainder reconstruction are usually performed by general-purpose computing cores such as CUDA cores in the GPU, while matrix multiplication is performed by tensor computing cores. Because the various computation stages are distributed across different hardware units, task switching and synchronization waiting between these units are necessary, which can easily lead to fragmentation of the computation process and uneven load on hardware resources.

[0063] Meanwhile, in the aforementioned software implementation, the modular matrix generated by modular decomposition typically needs to be written to shared memory by the general-purpose computing core and then read by the tensor computing core. The matrix multiplication results output by the tensor computing core also need to be written back to shared memory, then read by the general-purpose computing core and subjected to modular reduction or remainder reconstruction. The multiple transfers of intermediate data between the general-purpose computing core, the tensor computing core, and shared memory generate significant data movement overhead. Furthermore, processes such as matrix partitioning, modular scheduling, data layout transformation, and task synchronization usually require explicit software control by the CPU or GPU, further increasing control and scheduling overhead.

[0064] This application configures modular decomposition, matrix multiplication, modular reduction, and remainder reconstruction as corresponding hardware functional units, and tightly couples these hardware functional units onto the same computing chip. The modular matrix and the modular reduction result matrix can be directly transferred between the hardware functional units via an on-chip interconnect network. This helps reduce task switching and synchronization waiting between different hardware units, reduces data movement overhead caused by transferring intermediate data between different computing resources and storage levels, and reduces the additional overhead caused by software control and scheduling, thereby improving the overall computing efficiency of the matrix computing device.

[0065] In one embodiment of this application, the modular matrix generation unit includes: a near-Mersen number modular arithmetic unit array, the near-Mersen number modular arithmetic unit array including a plurality of near-Mersen number modular arithmetic units respectively corresponding to each of the preset moduli and arranged in parallel, wherein the plurality of preset moduli are pairwise coprime, and each of the preset moduli is a near-Mersen number; Each of the near-Mason numerator modular arithmetic units is used to calculate the modulo remainder of each matrix element in the integer matrix with respect to its corresponding preset modulus.

[0066] In this embodiment, the modulus matrix generation unit includes a near-Massen modulus arithmetic unit array. The near-Massen modulus arithmetic unit array includes multiple near-Massen modulus arithmetic units arranged in parallel. Each near-Massen modulus arithmetic unit constitutes a modulus arithmetic channel and corresponds to a preset modulus. For ease of description, the modulus arithmetic channel will be simply referred to as a channel below.

[0067] The integer matrices are specifically two integer matrices obtained by transforming the first input matrix and the second input matrix, respectively. For any matrix element of the near-Mason numerator (RMM) modular arithmetic unit array, each MRM unit calculates the modulo remainder of that matrix element with respect to its corresponding preset modulus, thereby forming multiple parallel modulo remainder data streams. These modulo remainder data streams can then be assembled according to the matrix dimensions of the first and second input matrices to obtain a pair of modulo matrices corresponding to each preset modulus.

[0068] In this embodiment, multiple preset moduli are pairwise coprime, allowing the modulo remainders of the same integer across multiple preset moduli to be used for subsequent remainder reconstruction. Each preset modulo is a near-Merssen number. In this application, a near-Merssen number can be expressed as:

[0069] in, Indicates the first A preset modulus, The data bit width indicates the integer data format. For relative to Smaller non-negative integers. Because and Given the aforementioned relationship, the corresponding modulo operation can be converted into shift operation and addition / subtraction operation, reducing the reliance on a general-purpose divider.

[0070] In specific implementations of FP64 and INT8, The near-Mason digitization unit array includes 16 parallel near-Mason digitization units, and the 16 preset modules can be: {256, 255, 253, 251, 247, 239, 233, 229, 227, 223, 217, 211, 199,197, 193, 191} The 16 preset moduli mentioned above are all pairwise coprime and all satisfy the following conditions: ,in, The value range is 0 to 65. Each preset modulus is no greater than... Its modulo remainder can be encoded using an 8-bit integer. When using the signed INT8 data format, modulo remainders greater than the range of positive signed INT8 numbers can be converted into negative numbers that are congruent to it, so that the converted modulo remainders are within the range of values ​​that INT8 can represent.

[0071] Let the product of the above 16 preset moduli be the global moduli M, then: M=43, 736, 450, 490, 573, 631, 126, 655, 766, 116, 979, 429, 120; The global modulus, which is greater than 2 to the power of 125 and less than 2 to the power of 126, can provide the corresponding integer reconstruction range. The number and specific values ​​of the preset modulus are not limited to the examples above, and can be determined based on the numerical range required for the integer matrix multiplication results, the data bit width of the integer data format, and hardware resources.

[0072] In one embodiment of this application, the near-Mason digitization unit includes: The block extraction unit is used to split each matrix element in the integer matrix into multiple data blocks, using the data bit width of the integer data format as the unit; The parallel weighted operation unit is used to perform parallel weighted operation on the multiple data blocks using preset weight coefficients associated with the corresponding preset modulus, and to obtain multiple weighted operation results. The preset weight coefficients are predetermined based on the data bit width of the integer data format, the corresponding preset modulus, and the block order of the corresponding data blocks. An accumulation unit is used to compress and sum the multiple weighted operation results through an accumulation tree to obtain an accumulation result; The near-Massen number modulo reduction simple element is used to reduce the accumulated result moduloly without performing division operations based on the corresponding preset modulus, so as to obtain the modulo remainder.

[0073] In this embodiment, the block extraction unit is used to split the matrix elements in the integer matrix into multiple data blocks according to the data bit width of the integer data format. Let the matrix elements to be processed be... Its data bit width is The data width of the integer data format is Then matrix elements It can be split into multiple parts in order from least significant bit to most significant bit. Bit data block. When , At that time, the block extraction unit splits a 64-bit matrix element into eight 8-bit data blocks. to ,Right now:

[0074] For the preset modulus ,because:

[0075] Therefore, the bit weights of each data block can be converted to a preset modulus. Preset weighting coefficients:

[0076] Therefore, matrix element X with respect to the preset modulus The modulo remainder can be calculated based on each data block and its preset weight coefficient. The above block division and weighting operations can be performed on the magnitude of non-negative integers. When the matrix element to be processed is a negative integer, the near-Merson modulus unit can first calculate the modulo remainder of the absolute value of the matrix element with respect to the corresponding preset modulus, and then convert the calculated modulo remainder into a modulo remainder that is congruent to the negative integer with respect to the corresponding preset modulus, according to the sign of the matrix element. As another implementation, data blocks can also be extracted according to the two's complement representation of signed integers, and a correction term corresponding to the sign bit can be added during the weighting operation.

[0077] In this embodiment, the parallel weighted operation unit performs weighted operations on each data block according to its block order in the matrix elements, using corresponding preset weight coefficients. In the embodiments of the 64-bit integer and 8-bit integer data formats described above, the least significant bit data block... Corresponding preset weight coefficients If the value is 1, it can be directly sent to the accumulation unit; the remaining 7 data blocks to It can be executed by seven multipliers configured in parallel. Weighted operations.

[0078] In this embodiment, the accumulation unit can compress and sum the least significant data block and multiple weighted operation results using a Wallace accumulation tree to obtain the accumulation result. Using an accumulation tree can shorten the combinational logic path formed by serially adding the results of multiple weighted operations.

[0079] In this embodiment, the near-Massen number modulo reduced simple element is applied to the accumulation result according to the corresponding preset modulus. Modular reduction is performed to obtain the modulo remainder. ,Right now:

[0080] Specifically, the accumulated result can be divided into a high-order part and a low-order part, and the high-order part is folded into the low-order part. After one or more folding operations, the result is adjusted to the remainder range defined by the preset modulus through addition or subtraction operations. This process is mainly implemented through shift operations and addition / subtraction operations. No division operation is performed during the entire modulus reduction process, so there is no need to set up a general-purpose divider. When the preset modulus is 256, the modulo remainder can be obtained directly from the lower 8 bits of the accumulated result.

[0081] In this embodiment, the near-Mason numerator arithmetic unit adopts a three-stage pipeline architecture. The first stage of pipeline processing is completed by the block extraction unit, the second stage of pipeline processing is completed by the parallel weighted arithmetic unit, and the third stage of pipeline processing is completed by the accumulation unit and the near-Mason numerator arithmetic unit.

[0082] In this embodiment, multiple near-Mason digitization units can employ the same circuit topology and critical path design, and perform modulo operations at different preset moduli using different preset moduli and preset weighting coefficients. The identical circuit structure facilitates consistent processing timing across channels and allows for unified control of multiple channels performing modulo reduction in parallel.

[0083] In this embodiment, the parameters used by each channel (i.e., the near-Mason analog-to-digital unit) can be pre-calculated and stored in the corresponding configuration register. The parameters stored for each channel may include a preset modulus. Preset weighting coefficients to Remainder Reconstruction Parameters and channel enable parameters .in: It is the product of multiple preset moduli. , for about The reverse of the model.

[0084] Remainder Reconstruction Parameters It is used by the remainder reconstruction unit and does not participate in the process of calculating the modulo remainder by the near-Mason numerator unit.

[0085] Under normal circumstances, According to Determined, where b is the data bit width of the integer data format. In a specific embodiment where the data bit width of the integer data format is 8 bits, According to Obtained without separate storage. Preset weighting coefficients corresponding to the least significant data block. This can be set as a fixed constant. By pre-storing the preset weight coefficients corresponding to each preset modulus, the overhead of calculating the preset weight coefficients during operation can be reduced.

[0086] In this embodiment, each channel can be configured with a 191-bit configuration register to store the modulus parameter, seven 8-bit preset weight coefficients, remainder reconstruction parameters, and channel enable parameters. In one specific implementation, bits [190:183] are used to store the encoded value of the preset modulus, bits [182:127] are used to store the seven 8-bit preset weight coefficients, bits [126:1] are used to store the remainder reconstruction coefficients, and bit [0] is used to store the channel enable parameters. For a preset modulus of 256, it can be agreed that the encoded value 0 in the modulus parameter field represents the preset modulus of 256. When reading the modulus parameter, the encoded value is first decoded, and then the preset modulus and its corresponding value are determined. In other implementations, the data width of the modulus parameter field can be extended to directly store the preset modulus 256.

[0087] The configuration parameters for the 16 channels can be stored in 384 bytes of storage space. The storage medium can be an SRAM register set, eFUSE, or mask ROM. The SRAM register set is suitable for scenarios requiring updates to preset modulo values ​​or configuration parameters; eFUSE is suitable for scenarios requiring power-off retention of configuration parameters; and mask ROM is suitable for scenarios where the preset modulo set is fixed. The data type, bit width, register layout, and specific parameter values ​​for each of the 16 channels can be set according to the corresponding parameter table.

[0088] The near-Mersen number modular arithmetic unit array described above allows for the parallel computation of the modulo remainders of the same integer matrix elements with respect to multiple preset moduli. Each near-Mersen number modular arithmetic unit employs the same operational structure, but different channels are distinguished by different preset moduli and weighting coefficients, thus improving the parallelism of modular decomposition. Utilizing the numerical characteristics of near-Mersen numbers for modular reduction also simplifies the modular reduction circuit and reduces reliance on general-purpose dividers.

[0089] like Figure 3 As shown, in one embodiment of this application, the modulus matrix generation unit further includes: A fixed-point encoder is used to determine a scaling offset based on the exponent and mantissa of each matrix element in the first input matrix and the second input matrix, and to perform shifting and sign processing on the matrix elements in the first input matrix and the second input matrix based on the scaling offset to obtain the integer matrix; The matrix assembly and distribution buffer unit is used to arrange the modulo remainders calculated by each of the near-Mason modulo operation units according to the arrangement order of the matrix elements corresponding to each modulo remainder in the first input matrix or the second input matrix, and assemble them according to the matrix dimensions of the first input matrix and the second input matrix respectively, to obtain a pair of modulo matrices corresponding to each preset modulo.

[0090] like Figure 3 As shown, the modulus matrix generation unit may include a fixed-point encoder, a near-Mason numerator-matrix arithmetic unit array, and a matrix assembly and distribution buffer unit connected in sequence. The fixed-point encoder receives a first input matrix and a second input matrix in the target floating-point data format, and converts the two input matrices into integer matrices respectively; the near-Mason numerator-matrix arithmetic unit array calculates the modulo remainder of each matrix element in the integer matrix with respect to multiple preset moduli; the matrix assembly and distribution buffer unit arranges the matrix elements corresponding to each modulo remainder in the first or second input matrix according to their arrangement order, and assembles them according to the respective matrix dimensions of the first and second input matrices to obtain a pair of modulus matrices corresponding to each preset modulus.

[0091] A fixed-point encoder is used to scale and convert the first and second input matrices to fixed-point values. Specifically, the fixed-point encoder can parse the floating-point data of each matrix element, extract the sign, exponent, and mantissa of each matrix element, and determine the scaling offset based on the exponent and mantissa. The scaling offset is used to determine the shift direction and number of bits when converting matrix elements to integers, so that the converted integers can retain the required numerical information within a set integer bit width.

[0092] In one embodiment of this application, a first scaling offset can be determined for each row of the first input matrix, and a second scaling offset can be determined for each column of the second input matrix. The first scaling offset can be determined based on the exponent and mantissa of each matrix element in the corresponding row of the first input matrix, and the second scaling offset can be determined based on the exponent and mantissa of each matrix element in the corresponding column of the second input matrix.

[0093] The first scaling factor determined by each first scaling offset can form a first scaling factor diagonal matrix D, and the second scaling factor determined by each second scaling offset can form a second scaling factor diagonal matrix E. The first scaling factor diagonal matrix D is used to scale the first input matrix in the row direction, and the second scaling factor diagonal matrix E is used to scale the second input matrix in the column direction. If the two integer matrices after definite-point scaling are represented as A′=D×A and B′=B×E respectively, and their matrix multiplication result is represented as X=A′×B′, then the matrix multiplication result of the first input matrix and the second input matrix can be expressed as C=D -1 ×X×E -1 recover.

[0094] Each scaling factor can be an integer power of 2. Fixed-point encoders can use barrel shifters to shift the mantissa of matrix elements to the left or right based on the scaling offset, thus achieving the corresponding exponential scaling. Multiplying or dividing by an integer power of 2 can be converted into a shift operation in hardware, eliminating the need for general-purpose multipliers or dividers.

[0095] Sign processing is used to determine the sign of the converted integer based on the sign bit of the matrix element. Specifically, the fixed-point encoder first obtains the integer magnitude of the matrix element based on the exponent, mantissa, and scaling offset, and then converts the integer magnitude into the corresponding signed integer representation based on the sign bit. The scaling offset, or the scaling factor determined based on the scaling offset, can be recorded for the remainder reconstruction unit to perform inverse scaling after completing integer reconstruction.

[0096] In a specific embodiment of FP64 and INT8, the fixed-point encoder converts the FP64 matrix elements in the first input matrix A and the second input matrix B element-wise into 64-bit signed integers, obtaining integer matrices A′ and B′. These 64-bit integer matrices A′ and B′ serve as inputs to the near-Mason ADC array. The fixed-point encoder uses a barrel shifter to perform exponential scaling and generates 64-bit signed integers through a sign processing circuit, ensuring that the converted integers meet the dynamic range requirements of subsequent modulus decomposition.

[0097] The target floating-point data format and the integer bit width used after fixed-point encoding are not limited to FP64 and 64 bits. In other embodiments, the integer bit width used after fixed-point encoding can be determined based on the precision and numerical range of the target floating-point data format, the number of preset moduloes, and the reconstruction range required for subsequent operations. The scaling offset can also be determined jointly based on the matrix elements of the corresponding rows of the first input matrix, the matrix elements of the corresponding columns of the second input matrix, the inner product length of the matrix multiplication, and the product of multiple preset moduloes, so that the fixed-point integer matrix multiplication result falls within the reconstruction interval used by the remainder reconstruction unit. When using a zero-centered signed reconstruction interval, the absolute value of each integer matrix multiplication result can be made less than half of the product of multiple preset moduloes, so as to uniquely recover the corresponding signed integer matrix multiplication result based on the modulus remainder under each preset modulo.

[0098] The near-Mason numerator modular arithmetic unit array calculates the modulo remainders of each matrix element in integer matrices A′ and B′ with respect to multiple preset moduli. For each preset modulo, one near-Mason numerator modular arithmetic unit outputs the modulo remainder corresponding to the integer matrices A′ and B′. Since modular arithmetic processes matrix elements, the output of the near-Mason numerator modular arithmetic unit is a data stream of modulo remainders generated according to the operation sequence, and has not yet formed a modular matrix with a complete row and column structure.

[0099] The matrix assembly and distribution buffer unit receives the modulo remainder data streams output from each near-Mersen modulo operation unit. It arranges the matrix elements corresponding to each remainder according to their order in the corresponding input matrix and assembles them according to the matrix dimensions of the corresponding input matrix. Fixed-point encoding and modulo operations do not change the position of matrix elements in the input matrix, nor do they change the number of rows or columns of the input matrix. Therefore, each remainder can be assembled according to the matrix dimensions of the first and second input matrices while maintaining the original order of the corresponding matrix elements.

[0100] Specifically, when the first input matrix A has a matrix dimension of M×K and the second input matrix B has a matrix dimension of K×N, the matrix assembly and distribution buffer unit rearranges and assembles the modulo remainders corresponding to the first input matrix A under the same preset modulus according to the matrix dimension of M×K to obtain the modulo matrix Ai. It then rearranges and assembles the modulo remainders corresponding to the second input matrix B under the same preset modulus according to the matrix dimension of K×N to obtain the modulo matrix Bi. The modulo matrices Ai and Bi constitute a pair of modulo matrices corresponding to the preset modulus.

[0101] In specific embodiments of FP64 and INT8, the near-Mason digitization unit array includes 16 channels arranged in parallel. The matrix assembly and distribution buffer unit receives the 8-bit modulo-remainder data streams output from the 16 channels and assembles these modulo-remainder data streams into 16 pairs of complete 8-bit modulo matrices Ai and Bi, where i takes values ​​from 0 to 15.

[0102] The matrix assembly and distribution buffer unit can also distribute the assembled modulus pairs to the tensor computation cores. Specifically, based on the preset correspondence between moduli and tensor computation cores, each modulus pair can be sent to the corresponding tensor computation core input port via the on-chip interconnect network. When multiple tensor computation cores are configured, each modulus pair can be distributed in parallel to the corresponding multiple tensor computation cores via the on-chip interconnect network. Data transmission can adopt a unified burst transmission method to reduce the control overhead caused by directly transmitting scattered modulus data across units.

[0103] The matrix assembly and distribution buffer unit can employ a double-buffering mechanism, consisting of two buffers that work alternately. While one buffer stores and assembles the currently received modulo remainder data, the other buffer can distribute the already assembled modulo matrix to the tensor computation core. After a set of data is assembled or distributed, the two buffers exchange their working states. This double-buffering mechanism allows the loading of matrix blocks and the matrix multiplication operations of the tensor computation core to overlap, reducing the waiting time between modulo decomposition and matrix multiplication.

[0104] In a specific embodiment for high-performance computing scenarios in data centers, the matrix assembly and distribution buffer unit can adopt a double-buffered structure with a capacity of 128KB. The unified control unit can control the state switching of the two buffers according to the processing progress of matrix blocks, and coordinate the transmission of the modular matrix to multiple tensor computing cores via the on-chip interconnect network.

[0105] By using a matrix assembly and distribution buffer unit, the fragmented modular remainder data stream output from the near-Mason numerator array can be converted into a modular matrix with a complete matrix structure and transmitted according to the data input requirements of the tensor computation core. This helps reduce data fragmentation and intermediate data movement overhead between modular decomposition and matrix multiplication. The double buffering mechanism can also improve the pipelined processing continuity between modular decomposition, modular matrix transmission, and matrix multiplication.

[0106] In one embodiment of this application, the tensor computation core includes a modular reduction element and an output interface. The output interface has a result truncation mode corresponding to the integer data format. The modular reduction element is used to perform modular reduction on the matrix multiplication result for each preset modulus, to obtain a modular reduction result matrix corresponding to the preset modulus. The output interface is used to output the modular reduction result matrix in the integer data format under the result truncation mode.

[0107] like Figure 4As shown, in one embodiment of this application, the tensor computation core is a hardware matrix multiplication execution unit obtained by functionally extending a conventional tensor computation core. A conventional tensor computation core is used to perform matrix multiplication and accumulation operations on input matrices, and is typically optimized for specific integer data formats. For example, a conventional tensor computation core can receive matrix data in INT8 format and save or output the matrix multiplication and accumulation results in INT32 format.

[0108] The tensor computation core in this embodiment adds a modular reduction simplification element to the conventional tensor computation core and extends the output interface to provide a result truncation mode corresponding to integer data format. The modular reduction simplification element is used to perform modular reduction on the matrix multiplication result according to the corresponding preset modulus, and the output interface is used to output the modular reduction result matrix in the integer data format used by the modular matrix in the result truncation mode.

[0109] Specifically, the tensor computation core receives a pair of modular matrices corresponding to the same preset modulus, and a conventional matrix multiplication circuit performs matrix multiplication and accumulation operations on this pair of modular matrices. The accumulation result generated during matrix multiplication can use an accumulation data format with a data width greater than the data width of the modular matrix elements to reduce the possibility of data overflow during matrix multiplication accumulation. For example, when the input modular matrix is ​​in INT8 format, the matrix multiplication result can be in INT32 format.

[0110] When the result truncation mode is enabled, the matrix multiplication result is not directly used as the final output of the tensor computation core, but is instead fed into the modular reduction simplification element. The modular reduction simplification element performs modular reduction on each element of the matrix multiplication result based on the preset modulus corresponding to the currently processed pair of modular matrices, obtaining the modular reduction result matrix corresponding to that preset modulus. The modular reduction result matrix has the same matrix dimension as the matrix multiplication result, and each matrix element represents the modulo remainder of the corresponding matrix multiplication result with respect to that preset modulus.

[0111] It should be noted that the result truncation mode mentioned in this application does not directly discard the high-order bits of the matrix multiplication result. Instead, it first performs modular reduction on the matrix multiplication result according to a preset modulus, and then converts the modular reduction result into the integer data format used by the output interface. Directly truncating the low-order bits is equivalent to modular reduction only when the preset modulus is an integer power of 2. For other preset moduli, modular reduction must be performed first to ensure that the output result retains the modulo remainder information required for modulo reconstruction.

[0112] In result truncation mode, the output interface converts each matrix element in the modulo-reduction result matrix into the integer data format used by the modulo matrix and outputs the converted modulo-reduction result matrix. When the integer data format is unsigned integer, the modulo remainder can be represented by the corresponding unsigned integer; when the integer data format is signed integer, the modulo remainder can be converted into a congruent integer within the range of that signed integer data format.

[0113] In specific embodiments of FP64 and INT8, the conventional tensor computation core receives a pair of INT8 modular matrices and performs INT8 matrix multiplication and accumulation operations using INT32 or other accumulation data formats. After the result truncation mode is enabled, the modular reduction unit performs modular reduction on the INT32 matrix multiplication result based on the preset modulus corresponding to the pair of INT8 modular matrices. The output interface converts the modular reduction result to INT8 format and outputs an INT8 modular reduction result matrix. This INT8 modular reduction result matrix can be directly used as input to the remainder reconstruction unit.

[0114] The tensor computation core can also be configured with a modulus parameter receiving interface, which is connected to the configuration register group to receive the preset modulus corresponding to the current matrix multiplication operation. The preset modulus in the configuration register group can be written to and updated by the unified control unit. The modulus reduction simplifier performs modulus reduction based on the received preset modulus, and the output interface encodes the modulus reduction result into the integer data format and outputs it in result truncation mode.

[0115] When a tensor computation core processes multiple pairs of modular matrices corresponding to different preset moduli sequentially, the preset moduli corresponding to each pair of modular matrices can be configured before processing each pair. When multiple tensor computation cores are set, different preset moduli can also be configured for each tensor computation core, so that multiple tensor computation cores can process multiple pairs of modular matrices corresponding to different preset moduli in parallel.

[0116] In one embodiment of this application, the modular reduction simple element can adopt the same or similar operational structure as the aforementioned near-Mersen number modular reduction simple element. For a preset modulus in the near-Mersen number form, modular reduction can be completed through shift operations and addition / subtraction operations, reducing the reliance on a general-purpose divider.

[0117] The output interface can retain the conventional output mode. In conventional output mode, the tensor computation core outputs the matrix multiplication results in a standard accumulated data format. In result truncation mode, the tensor computation core outputs the modularly reduced result matrix in integer data format through the modularly reduced simple elements and the extended output interface. The two output modes can be switched using a mode selection signal, or corresponding output paths can be set separately.

[0118] In a specific embodiment for high-performance computing scenarios in data centers, the matrix computing device can be configured with 16 modified tensor computing cores, each supporting 16×16×16 INT8 matrix multiplication operations. The 16 pairs of INT8 modular matrices are transmitted to the corresponding tensor computing cores via an on-chip interconnect network, and each tensor computing core performs INT8 matrix multiplication, modular reduction, and INT8 format output, respectively.

[0119] This application, by setting a modular reduction simplifier and a result truncation mode in the tensor computation core, can directly obtain the modular reduction result required for remainder reconstruction after the tensor computation core completes matrix multiplication operations, avoiding the need to pass the large-bit-width accumulation result to other general-purpose computation units for modular reduction. Furthermore, outputting the modular reduction result in the integer data format used by the modular matrix helps reduce the amount of intermediate data transferred between the tensor computation core and the remainder reconstruction unit.

[0120] In one embodiment of this application, the matrix computing device may include multiple tensor computing cores, which constitute a tensor computing core array. When executing the current matrix computing task, different tensor computing cores correspond to different preset moduli. Each tensor computing core receives a pair of modulo matrices under its corresponding preset moduli and performs matrix multiplication and modulo reduction according to the preset moduli.

[0121] The correspondence between tensor computation cores and preset modules refers to the fact that when a tensor computation core performs a matrix multiplication operation, it uses the preset module to perform modular reduction on the result of the matrix multiplication operation. It does not require a tensor computation core to be permanently bound to a single preset module. The unified control unit can configure different preset modules to multiple tensor computation cores according to the current matrix computation task. When the matrix computation task changes, the preset module corresponding to each tensor computation core can be changed by updating the configuration parameters.

[0122] After assembling a pair of modular matrices corresponding to each preset modulus, the matrix assembly and distribution buffer unit sends each pair of modular matrices to the corresponding tensor computing core via the on-chip interconnect network, based on the correspondence between the preset modulus and the tensor computing core. The configuration register group also provides the preset modulus corresponding to each tensor computing core to the modular reduction simple element in that tensor computing core.

[0123] Multiple tensor computation cores process the same matrix block in parallel, handling pairs of modular matrices at different preset moduli. Each tensor computation core performs matrix multiplication on the received pair of modular matrices, and a modulo reduction element performs modulo reduction on the result of the matrix multiplication operation according to the corresponding preset modulo, obtaining the modulo-reduced result matrix corresponding to that preset modulo. In result truncation mode, each tensor computation core outputs the corresponding modulo-reduced result matrix in integer data format through the output interface.

[0124] The unified control unit can send start signals to multiple tensor computation cores via a broadcast distribution controller, enabling these cores to begin processing their respective modulo matrices within the same clock cycle. The unified control unit can also receive completion signals from each tensor computation core and align the modulo reduction results output by multiple cores using a convergence alignment buffer. The modulo reduction results received by the remainder reconstruction unit should belong to the same result matrix block and correspond to the same matrix element position within that block.

[0125] The number of tensor computing cores can be determined based on the number of preset moduli, chip hardware resources, and computational throughput requirements. In a typical implementation of the matrix computing device, the number P of tensor computing cores can be 1, 4, 16, or other numbers. P=1 corresponds to a single tensor computing core implementation; when P is greater than 1, multiple tensor computing cores can process modular matrices corresponding to different preset moduli in parallel.

[0126] When the number of tensor computation cores is less than the number of preset moduli, multiple preset moduli can be divided into multiple moduli batches. Within each moduli batch, multiple tensor computation cores process a pair of moduli matrices corresponding to different preset moduli in parallel. After completing the current moduli batch, the preset moduli corresponding to each tensor computation core are updated, and the next moduli batch is processed. The modulo reduction results corresponding to each modulo batch can be temporarily stored in a convergence alignment buffer. After the modulo reduction results for the same result matrix element under all preset moduli have been obtained, the set of modulo reduction results is then sent to the remainder reconstruction unit.

[0127] When the number of tensor computation cores equals the number of preset modules, each preset module can correspond to a tensor computation core, thereby performing matrix multiplication and modular reduction operations in parallel for all preset modules. In this embodiment, each preset module, the near-Mason numerator modular arithmetic unit corresponding to the preset module, and the tensor computation core corresponding to the preset module form a modular arithmetic processing path.

[0128] In a specific embodiment for high-performance computing scenarios in data centers, the matrix computing device includes 16 near-Mason numerator arithmetic units and 16 tensor computing cores, each corresponding to one of the 16 preset moduli. A matrix assembly and distribution buffer unit distributes 16 pairs of INT8 moduli matrices to the corresponding 16 tensor computing cores via an on-chip interconnect network. Each tensor computing core supports 16×16×16 INT8 matrix multiplication operations and performs modular reduction on the matrix multiplication results according to the corresponding preset moduli, outputting the reduced matrix in INT8 format. The 16 tensor computing cores process the moduli matrices under the 16 preset moduli in parallel, providing 16 channels of modular reduction results to the remainder reconstruction unit. The global moduli formed by the product of the 16 preset moduli is approximately 2 to the power of 125, providing the corresponding integer reconstruction range.

[0129] By setting up multiple tensor computation cores, each corresponding to a different preset modulus, matrix multiplication and modular reduction can be performed in parallel along the preset modulus dimension, reducing the waiting time caused by a single tensor computation core processing multiple pairs of modulo matrices sequentially. Each tensor computation core directly outputs the modular reduction result after completing the matrix multiplication operation, which also helps to reduce the data movement overhead caused by transferring large-bit-width accumulation results between different hardware units.

[0130] In one embodiment of this application, the remainder reconstruction unit includes: The segmented parallel multiplication unit is used to split the multiplication operation between each matrix element in the modulus reduction result matrix corresponding to the preset modulus and the remainder reconstruction coefficient corresponding to the preset modulus into multiple sub-multiplication operations for each preset modulus, execute the multiple sub-multiplication operations in parallel, and merge the results of each sub-multiplication operation to obtain the product result corresponding to each matrix element. The addition merging unit is used to sum the product results corresponding to the matrix elements at the same position in each of the modulo reduction result matrices through the addition tree to obtain the merged result matrix; A global modulo reduction simple element is used to perform modulo reduction on the merged result matrix based on a global modulus determined by the product of the multiple preset moduli, to obtain a reconstructed integer matrix; The floating-point format conversion and descaling unit is used to perform floating-point format conversion and descaling on the reconstructed integer matrix to obtain the matrix multiplication result, wherein the plurality of preset moduli are pairwise coprime.

[0131] In this embodiment, the remainder reconstruction unit can be implemented using dedicated pipelined hardware to recover the matrix multiplication result of the first input matrix and the second input matrix based on the modular reduction result matrices corresponding to multiple preset moduli. The remainder reconstruction unit may include a piecewise parallel multiplication unit, an addition merging unit, a global modular reduction simplex, and a floating-point format conversion and descaling unit connected in sequence.

[0132] In this embodiment, the remainder reconstruction unit reconstructs the remainders according to the positions of the matrix elements. For the same matrix element position in the matrix multiplication result, the remainder reconstruction unit obtains the modulo remainder at that position from each modulo reduction result matrix, obtaining a set of modulo remainders corresponding to multiple preset moduli, and recovers the corresponding matrix element based on this set of modulo remainders. The remainder reconstruction unit processes each matrix element position sequentially to obtain the complete matrix multiplication result.

[0133] In this embodiment, the multiple preset moduli are pairwise coprime, and the global moduli is the product of the multiple preset moduli. For the first... Preset modulus The intermediate parameters can be obtained by dividing the global modulus by the preset modulus. ,calculate about Model Reverse and will and The product of these factors serves as the remainder reconstruction coefficient corresponding to the preset modulus. Each remainder reconstruction coefficient can be pre-calculated and stored in the configuration register set, and read by the remainder reconstruction unit when performing remainder reconstruction.

[0134] In this embodiment, the piecewise parallel multiplication unit multiplies each element of the modulus reduction result matrix corresponding to each preset modulus with the remainder reconstruction coefficient corresponding to that preset modulus. Since the data bit width of the remainder reconstruction coefficient is usually larger than the data bit width of the modulus remainder, the piecewise parallel multiplication unit can split a large number multiplication into multiple sub-multiplication operations. Each sub-multiplication operation is executed in parallel by multiple multipliers with smaller bit widths. Then, according to the position order of each segment of data in the remainder reconstruction coefficient, the results of each sub-multiplication operation are shifted and merged to obtain the product result corresponding to each matrix element.

[0135] In specific embodiments of FP64 and INT8, the matrix elements in the modulo reduction result matrix adopt an 8-bit integer data format, and the data bit width of the remainder reconstruction coefficients can be up to 125 bits. The segmented parallel multiplication unit can divide the remainder reconstruction coefficients into a 64-bit low-order segment and a 61-bit high-order segment, and perform two sub-multiplication operations in parallel through an 8×64-bit multiplier and an 8×61-bit multiplier, respectively. For remainder reconstruction coefficients with a data bit width of less than 125 bits, zeros can be padded in the high-order bits. The sub-multiplication result corresponding to the high-order segment is shifted and then merged with the sub-multiplication result corresponding to the low-order segment to obtain the product result. The product result can be transmitted and stored using a 140-bit data path to accommodate the product data and the sign bit or guard bit required for the operation.

[0136] When 16 preset moduli are set, the segmented parallel multiplication unit can include 16 parallel multiplication circuits, which perform the above multiplication operation on the matrix elements in the 16 modulo reduction result matrices respectively. If the modulo reduction result is represented by a signed integer, the multiplication operation can be performed directly using the congruence relationship between the signed integer and the corresponding non-negative modulo remainder, or the signed integer can be converted into a non-negative modulo remainder within the range of the corresponding preset moduli first.

[0137] In this embodiment, the addition merging unit sums the product results corresponding to the matrix elements at the same position in each modulo-reduction result matrix according to the position of the matrix elements, using a multi-level addition tree, to obtain the merging result at that position. The merging results at each matrix element position form the merging result matrix. In a specific embodiment with 16 inputs, the addition merging unit can use a 16-way multi-level addition tree and store the merging result using a 141-bit or larger bit width that can cover the actual range of the merging result values.

[0138] In this embodiment, the global modulo reduction element performs modulo reduction on each matrix element in the merged result matrix based on the global modulus, reducing each merged result to a numerical range defined by the global modulus, thus obtaining a reconstructed integer matrix. In the embodiment using 16 preset moduli, the global modulus can be denoted as... .

[0139] In one specific embodiment of this application, to avoid directly using a large number divider to perform global modular reduction, the global modular reduction element can utilize the characteristic that the global modulus consists of multiple pairwise coprime preset moduloes, and employ a 4+4+4+4 block structure to complete the modular reduction. Specifically, the global modular reduction element can set up 16 modular reduction circuits with the same circuit structure as the aforementioned near-Merssen number modular reduction element, to calculate the modulo remainders of the merged result with respect to the 16 preset moduloes; then, the obtained 16 modulo remainders are divided into four groups, and within each group, the four modulo remainders are merged to obtain four intra-group merged results; subsequently, the four intra-group merged results are merged pairwise to obtain two inter-group merged results; finally, the two inter-group merged results are globally merged to obtain the reconstructed integer within the global modulus range. This process forms a three-level tree-like merging structure, realizing global modular reduction of the merged result.

[0140] The aforementioned tree-like merging structure can have 15 merging nodes. The modular inverse constant used in each merging node can be pre-determined and stored based on a preset modulus and merging order. In the specific embodiment using 16 preset moduli, the merging order can be set so that the 15 modular inverse constants are represented by smaller integers, thereby reducing the hardware implementation complexity of multiplication operations in the merging nodes. The modular reduction circuit used in the tree-like merging process can adopt the same circuit structure as the aforementioned near-Merssen number modular reduction element, completing modular reduction through shift operations and addition / subtraction operations. Here, the operational structure or circuit design of the near-Merssen number modular reduction circuit is reused; it is not required to return the merging result to the modular matrix generation unit for processing, nor is the global modulus represented. It is a near-Massen number.

[0141] In this embodiment, the matrix elements in the reconstructed integer matrix represent the integer reconstructed values ​​of the matrix multiplication result within the global modulus range. The number and values ​​of multiple preset moduli should ensure that the global modulus can cover the numerical range required for the integer matrix multiplication result. When the matrix multiplication result includes negative numbers, the sign of the reconstructed integers can be restored according to a preset signed reconstruction interval. For example, when using a zero-centered signed reconstruction interval, the reconstructed values ​​in the upper half of the global modulus can be converted into negative integer representations that are congruent to it.

[0142] In this embodiment of the application, the floating-point format conversion and descaling unit is specifically used to count leading zeros for each matrix element in the reconstructed integer matrix, determine the exponent of each matrix element in the target floating-point data format and construct the mantissa based on the leading zero count result, and descale the reconstructed integer matrix through shift operation to obtain the matrix multiplication operation result.

[0143] In this embodiment, the floating-point format conversion and descaling unit is used to convert the reconstructed integer matrix into a matrix multiplication result using the target floating-point data format. Specifically, leading zeros can be counted for each element in the reconstructed integer matrix, the position of the most significant bit can be determined based on the leading zero count result, and the exponent and mantissa in the target floating-point data format can be determined accordingly. For negative integers, leading zeros can be counted and the mantissa can be constructed for their absolute values, and then the sign bit of the target floating-point data can be set based on the sign recovery result.

[0144] The floating-point format conversion and descaling unit also descales the reconstructed integer matrix based on the scaling offset recorded by the fixed-point encoder. When the aforementioned fixed-point encoder uses a first scaling factor diagonal matrix D to scale the first input matrix in the row direction and a second scaling factor diagonal matrix E to scale the second input matrix in the column direction, the descaling bit depth of the matrix element can be determined based on the first scaling offset corresponding to the row of the reconstructed integer matrix element and the second scaling offset corresponding to the column. Descaling can be achieved through a shift operation in the opposite direction to the fixed-point processing. In a specific embodiment of FP64, descaling can be completed through a right shift operation, and the sign bit, exponent, and mantissa are generated according to the IEEE 754 floating-point data format to obtain the FP64 matrix element.

[0145] In this embodiment, the remainder reconstruction unit can employ a four-stage fixed-depth pipeline structure. The first stage of pipeline processing is completed by a piecewise parallel multiplication unit, the second stage by an addition merging unit, the third stage by a global modulo-reduction simple element, and the fourth stage by a floating-point format conversion and descaling unit. In this embodiment, the total processing depth of the four-stage pipeline can be 16 clock cycles. Once the pipeline is full, each clock cycle can receive a set of multiple modulo remainders corresponding to the same matrix element position and output a matrix element using the target floating-point data format.

[0146] Through the aforementioned dedicated hardware pipeline, the remainder reconstruction unit can automatically complete remainder reconstruction, sign restoration, descaling, and floating-point format conversion, and form a continuous data processing path with the upstream tensor calculation core and the downstream result buffer unit. This helps to reduce the control overhead and intermediate data movement overhead caused by the general-purpose computing unit cooperating with software to perform remainder reconstruction.

[0147] like Figure 2 As shown, in one embodiment of this application, the matrix calculation device of this application further includes: A unified control unit is used to provide the same clock enable signal to the plurality of near-Mason digitization units, so as to control the plurality of near-Mason digitization units to synchronously calculate the modulo remainder of each matrix element in the integer matrix with respect to its corresponding preset modulus.

[0148] like Figure 2 As shown, the unified control unit is connected to the modular matrix generation unit, the tensor calculation core, and the remainder reconstruction unit, respectively, and is used to schedule and synchronize the various hardware functional units in the matrix calculation device.

[0149] The unified control unit can be implemented based on a finite state machine and serves as the hardware scheduling center of the matrix computing device. Based on matrix computing instructions, the unified control unit controls input matrix loading, fixed-point encoding, modular decomposition, modular matrix transmission, matrix multiplication, modular reduction, remainder reconstruction, and result write-back, ensuring that each processing step is executed according to the defined data dependencies.

[0150] In this embodiment, the unified control unit provides the same clock enable signal to multiple near-Mason ADC units, enabling these units participating in the current computation task to begin processing input data within the same clock cycle. In one specific implementation, the multiple near-Mason ADC units employ the same circuit topology and critical path design, and have the same number of pipeline stages and processing delay. Therefore, under the control of the same clock enable signal, the multiple near-Mason ADC units can synchronously process the same matrix element and output the modulo remainder of that matrix element with respect to different preset moduli within the corresponding clock cycle.

[0151] It should be noted that the same clock enable signal is used to uniformly control multiple near-Mason ADCs to receive and process input data. Multiple near-Mason ADCs can share the same operating clock. The unified control unit broadcasts the clock enable signal to each near-Mason ADC participating in the current computing task, thereby reducing the inconsistency in processing timing caused by each channel being started separately.

[0152] In one embodiment of this application, the unified control unit may include a matrix block scheduler, a modular scheduler, a data layout engine, and a pipeline interlock synchronization module.

[0153] The matrix block scheduler parses the general matrix multiplication instructions sent from the upper layer and divides the matrix multiplication task into multiple matrix block tasks based on the matrix dimensions of the first and second input matrices. The matrix block scheduler can also determine the processing order of each matrix block and schedule the DMA controller to load the currently needed matrix block. In one specific embodiment, the input matrix can be divided into 128×128 matrix blocks, and loading and computation can be performed on a block-by-block basis.

[0154] The modular number scheduler manages the preset modules and their associated computational parameters. Based on the modular operation channels and tensor computation cores participating in the current computation task, the scheduler controls the writing and distribution of preset modules, preset weight coefficients, and remainder reconstruction coefficients. The scheduler can also control the verification and switching of configuration parameters for different batches, ensuring that all hardware functional units participating in the same computation task use the same set of configuration parameters.

[0155] The data placement engine coordinates the transmission of modular matrices output by the modular matrix generation unit to the tensor computation core via the on-chip interconnect network. Based on a preset correspondence between moduli and tensor computation cores, the data placement engine controls the matrix assembly and distribution buffer unit to send each pair of modular matrices to the corresponding tensor computation core. Modular matrices can be transmitted using a unified burst transmission method to reduce the control overhead caused by transmitting fragmented modular remainder data separately.

[0156] The pipelined interlock synchronization module coordinates the startup and completion states of the modular matrix generation unit, tensor computation cores, and remainder reconstruction unit. The pipelined interlock synchronization module can send startup signals to multiple tensor computation cores via a broadcast distribution controller, enabling these cores participating in the current computation task to begin processing their respective pairs of modular matrices within the same clock cycle. The broadcast distribution controller can also receive preparation signals from each tensor computation core and send a startup signal after all tensor computation cores have met their startup conditions.

[0157] The pipelined interlocking synchronization module can also align the modulo reduction results output by multiple tensor computation cores through a convergence alignment buffer. For the same matrix element position, the set of modulo reduction results is only sent to the remainder reconstruction unit after all the modulo reduction results required for remainder reconstruction have arrived. In an embodiment using 16 tensor computation cores, the pipelined interlocking synchronization module can also perform convergence detection on 16 completion signals to determine whether all tensor computation cores participating in the current matrix block task have completed computation, and control subsequent matrix block scheduling or configuration parameter switching accordingly.

[0158] With a unified control unit, matrix block scheduling, configuration parameter management, synchronous control of multiple modular arithmetic channels, and interlocking between pipeline processes at each level can be completed in hardware, reducing the scheduling overhead caused by software controlling each processing stage separately.

[0159] like Figure 2 As shown, in one embodiment of this application, the matrix calculation device further includes a configuration register group, a matrix buffer unit, and a result buffer unit. The configuration register group is used to store and provide configuration parameters required for modular decomposition, modular reduction, and remainder reconstruction; the matrix buffer unit is used to temporarily store the first input matrix and the second input matrix; and the result buffer unit is used to temporarily store the matrix multiplication operation results.

[0160] The configuration register set can be connected to the unified control unit. The unified control unit controls the writing, verification, switching, and distribution of configuration parameters through the modulus scheduler. The configuration parameters stored in the configuration register set may include each preset modulus, the preset weight coefficients corresponding to each preset modulus, the remainder reconstruction coefficients, and the channel enable parameters. The near-Mason modulus arithmetic unit obtains the corresponding preset modulus and preset weight coefficients from the configuration register set, the tensor computation core obtains the preset modulus corresponding to the current matrix multiplication operation, and the remainder reconstruction unit obtains the remainder reconstruction coefficients corresponding to each preset modulus.

[0161] In a specific embodiment employing 16 preset modulo values, the configuration register group may include configuration registers corresponding to the 16 modulo operation channels. Each configuration register may adopt the aforementioned 191-bit register layout, storing the modulo parameter corresponding to one modulo operation channel, 7 preset weight coefficients, remainder reconstruction coefficients, and channel enable parameters. A complete set of 16-channel configuration parameters can be stored using 384 bytes of storage space.

[0162] The configuration register set can adopt a dual-storage structure, including a current storage area and a background storage area. Each storage area can store a complete set of 16-channel configuration parameters. The current storage area is used to provide the configuration parameters used by the current computing task to each hardware functional unit, while the background storage area is used to write the configuration parameters used by the next computing task during the execution of the current computing task. After the analog-to-digital scheduler has completed writing the configuration parameters in the background storage area and passed the verification, it waits for the current computing frame to be processed before controlling the two storage areas to perform an atomic switch.

[0163] The computation frame here can be a single matrix block task executed using the same set of configuration parameters, or a series of consecutive matrix block tasks. By uniformly switching the configuration parameters at the boundary of the computation frame, it is possible to avoid using different batches of configuration parameters for different channels, and it is also unnecessary to clear the pipeline that is currently executing the computation frame.

[0164] The current storage area and the background storage area can be implemented using SRAM register sets. With a fixed preset modulus set, configuration parameters can also be stored in eFUSE or mask ROM and loaded into the SRAM register set when the matrix computing device starts. The unified control unit can set the enable parameters of each channel according to the computing task to select the near-Mason digitization unit to participate in the current computing task.

[0165] The matrix buffer unit is located on the input side of the modular matrix generation unit and is used to temporarily store the first and second input matrices, or matrix blocks obtained by dividing the first and second input matrices. The host computer can write the first and second input matrices into the chip's global memory via a high-speed interface. The unified control unit schedules the DMA controller according to the matrix block scheduling result, loading the currently processed matrix blocks into the matrix buffer unit. The modular matrix generation unit reads the matrix blocks from the matrix buffer unit and performs fixed-point encoding and modular decomposition on the read matrix elements.

[0166] In a specific embodiment of FP64, the matrix buffer unit can be an FP64 matrix buffer unit, used to store the matrix block Asub of the first input matrix and the matrix block Bsub of the second input matrix respectively, and to provide matrix elements according to the data input timing of the modular matrix generation unit.

[0167] The result buffer unit is located on the output side of the remainder reconstruction unit and is used to temporarily store the matrix multiplication results or result matrix blocks output by the remainder reconstruction unit. The unified control unit can schedule the DMA controller according to the storage state of the result buffer unit to write the calculated result matrix blocks into the chip's global memory. After all matrix blocks have been calculated, the complete matrix multiplication result can be returned to the host through a high-speed interface.

[0168] In a specific embodiment of FP64, the result buffer unit receives the FP64 matrix elements output by the remainder reconstruction unit and assembles them into FP64 result matrix blocks according to the position of each matrix element in the result matrix. The result buffer unit can also support burst write-back, allowing consecutive FP64 matrix elements to be written to the chip's global memory via the DMA controller.

[0169] By coordinating the configuration register group, matrix buffer unit, and result buffer unit with the unified control unit, configuration parameter distribution, input matrix loading, and calculation result writing-back can be accomplished. The unified control unit mainly transmits control signals, the configuration register group mainly provides configuration parameters, and the matrix buffer unit and result buffer unit are located on the data path of matrix calculation, respectively responsible for the temporary storage of input data and output results.

[0170] In one embodiment of this application, there are multiple tensor computation cores, each corresponding to a different preset modulus when performing matrix multiplication operations; the multiple tensor computation cores are used to perform matrix multiplication operations in parallel on a pair of modulo matrices corresponding to their respective preset modulo, and to perform modulo reduction on the matrix multiplication results according to their respective preset modulo.

[0171] In this embodiment, multiple tensor computation cores constitute a tensor computation core array. When performing the current matrix computation task, different tensor computation cores correspond to different preset moduli. Each tensor computation core receives a pair of modulo matrices under its corresponding preset moduli and performs matrix multiplication and modulo reduction according to the preset moduli.

[0172] The correspondence between tensor computation cores and preset modules refers to the fact that when a tensor computation core performs a matrix multiplication operation, it uses the preset module to perform modular reduction on the result of the matrix multiplication operation. It does not require a tensor computation core to be permanently bound to a single preset module. The unified control unit can configure different preset modules to multiple tensor computation cores according to the current matrix computation task. When the matrix computation task changes, the preset module corresponding to each tensor computation core can be changed by updating the configuration parameters.

[0173] After assembling a pair of modular matrices corresponding to each preset modulus, the matrix assembly and distribution buffer unit sends each pair of modular matrices to the corresponding tensor computing core via the on-chip interconnect network, based on the correspondence between the preset modulus and the tensor computing core. The configuration register group also provides the preset modulus corresponding to each tensor computing core to the modular reduction simple element in that tensor computing core.

[0174] Multiple tensor computation cores process the same matrix block in parallel, handling pairs of modular matrices at different preset moduli. Each tensor computation core performs matrix multiplication on the received pair of modular matrices, and a modulo reduction element performs modulo reduction on the result of the matrix multiplication operation according to the corresponding preset modulo, obtaining the modulo-reduced result matrix corresponding to that preset modulo. In result truncation mode, each tensor computation core outputs the corresponding modulo-reduced result matrix in integer data format through the output interface.

[0175] The unified control unit can send start signals to multiple tensor computation cores via a broadcast distribution controller, enabling these cores to begin processing their respective modulo matrices within the same clock cycle. The unified control unit can also receive completion signals from each tensor computation core and align the modulo reduction results output by multiple cores using a convergence alignment buffer. The modulo reduction results received by the remainder reconstruction unit should belong to the same result matrix block and correspond to the same matrix element position within that block.

[0176] The number of tensor computing cores can be determined based on the number of preset moduli, chip hardware resources, and computational throughput requirements. In a typical implementation of the matrix computing device, the number P of tensor computing cores can be 1, 4, 16, or other numbers. P=1 corresponds to a single tensor computing core implementation; when P is greater than 1, multiple tensor computing cores can process modular matrices corresponding to different preset moduli in parallel.

[0177] When the number of tensor computation cores is less than the number of preset moduli, multiple preset moduli can be divided into multiple moduli batches. Within each moduli batch, multiple tensor computation cores process a pair of moduli matrices corresponding to different preset moduli in parallel. After completing the current moduli batch, the preset moduli corresponding to each tensor computation core are updated, and the next moduli batch is processed. The modulo reduction results corresponding to each modulo batch can be temporarily stored in a convergence alignment buffer. After the modulo reduction results for the same result matrix element under all preset moduli have been obtained, the set of modulo reduction results is then sent to the remainder reconstruction unit.

[0178] When the number of tensor computation cores equals the number of preset modules, each preset module can correspond to a tensor computation core, thereby performing matrix multiplication and modular reduction operations in parallel for all preset modules. In this embodiment, each preset module, the near-Mason numerator modular arithmetic unit corresponding to the preset module, and the tensor computation core corresponding to the preset module form a modular arithmetic processing path.

[0179] In a specific embodiment for high-performance computing scenarios in data centers, the matrix computing device includes 16 near-Mason numerator-deterministic (NMD) arithmetic units and 16 tensor computing cores, each corresponding to one of the 16 preset moduli. A matrix assembly and distribution buffer unit distributes 16 pairs of INT8 moduli matrices to the corresponding 16 tensor computing cores via an on-chip interconnect network. Each tensor computing core supports 16×16×16 INT8 matrix multiplication operations and performs modular reduction on the matrix multiplication results according to the corresponding preset moduli, outputting the reduced matrix in INT8 format. The 16 tensor computing cores process the moduli matrices under the 16 preset moduli in parallel, providing 16 channels of modular reduction results to the remainder reconstruction unit. The product of the aforementioned 16 preset moduli can provide an integer reconstruction range of approximately 125 bits.

[0180] By setting up multiple tensor computation cores, each corresponding to a different preset modulus, matrix multiplication and modular reduction can be performed in parallel along the preset modulus dimension, reducing the waiting time caused by a single tensor computation core processing multiple pairs of modulo matrices sequentially. Each tensor computation core directly outputs the modular reduction result after completing the matrix multiplication operation, which also helps to reduce the data movement overhead caused by transferring large-bit-width accumulation results between different hardware units.

[0181] The solution of this application will now be described with reference to a specific embodiment.

[0182] This embodiment provides a specific implementation method for high-performance computing scenarios in data centers. The matrix computing device includes 16 parallel near-Mason numerator and 16 tensor computing cores. The global modulus formed by the product of the 16 preset moduli is approximately 2 to the power of 125. When the result of the fixed-point-reduced integer matrix multiplication falls within the reconstruction interval corresponding to this global modulus, the integer matrix multiplication result can be recovered based on the 16-way modulus reduction result, and then FP64 matrix multiplication can be completed using INT8 matrix operations.

[0183] The hardware components and connections in this embodiment are as follows: The Modular Matrix Generation Unit (MD-RU) comprises a fixed-point encoder, a near-Mason numerator-matrix (NM-MAU) array, and a matrix assembly and distribution buffer unit. The NM-MAU array includes 16 parallel NM-MAUs, each corresponding to one of the 16 preset moduli. The matrix assembly and distribution buffer unit employs a 128 KB double-buffered structure to assemble the 16 modulo remainder data streams into 16 pairs of INT8 modulo matrices and distribute each pair of INT8 modulo matrices to the corresponding tensor computation cores.

[0184] The configuration register set is implemented using static random access memory (SRAM), with each memory area having a configuration capacity of 384 bytes. It stores the preset modulus, preset weight coefficients, remainder reconstruction coefficients, and channel enable parameters for the 16 modulus operation channels. The modulus matrix generation unit, tensor calculation core, and remainder reconstruction unit each obtain their required configuration parameters from the configuration register set.

[0185] The tensor computation core array comprises 16 tensor computation cores (MTCs) with modular reduction functionality. Each of the 16 MTCs corresponds to a preset modulus and is connected to a matrix assembly and distribution buffer unit via an on-chip interconnect network. Each MTC supports 16×16×16 INT8 matrix multiplication operations and includes modular reduction elements and an output interface with INT8 result truncation mode.

[0186] The Remainder Reconstruction Unit (CRT-RPU) has 16 parallel inputs to receive the modular reduction results from the outputs of 16 tensor computation cores. The CRT-RPU employs a four-stage pipeline architecture, sequentially performing piecewise parallel multiplication, addition merging, global modular reduction, and floating-point format conversion and inverse scaling. In one specific implementation, the total processing depth of the four-stage pipeline is 16 clock cycles. Once the pipeline is full, it can output one FP64 matrix element per clock cycle.

[0187] The Unified Control Unit (UCU) is implemented based on a finite state machine and includes a matrix block scheduler, a modular scheduler, a data layout engine, and a pipeline interlock synchronization module. It is used to control matrix block loading, configuration parameter distribution, synchronous processing of multiple modular operation channels, synchronous startup of multiple tensor calculation cores, and convergence and alignment of modular reduction results.

[0188] The matrix computing device also includes a matrix buffer unit and a result buffer unit. The matrix buffer unit is used to temporarily store FP64 matrix blocks to be processed, and the result buffer unit is used to temporarily store FP64 result matrix blocks output by the remainder reconstruction unit. The matrix buffer unit and the result buffer unit respectively transmit data to the chip's global memory through the direct memory access controller.

[0189] The specific workflow of this embodiment is as follows: The host computer sends the first input matrix A and the second input matrix B, which are in FP64 format, to the chip's global memory via a high-speed interface.

[0190] The unified control unit parses the General Matrix Multiplication (GEMM) instruction, divides the first input matrix A and the second input matrix B into 128×128 matrix tiles, and schedules the direct memory access (DMA) controller to load the matrix tiles that need to be processed into the matrix buffer unit.

[0191] The modular matrix generation unit reads FP64 matrix blocks from the matrix buffer unit. The fixed-point encoder determines the scaling offset based on the exponent and mantissa of each FP64 matrix element, and converts each FP64 matrix element into a 64-bit signed integer through shifting and sign processing. The 16 near-Merson modulus arithmetic units calculate the modulo remainder of each 64-bit signed integer in parallel according to their respective preset modulo.

[0192] The matrix assembly and distribution buffer unit rearranges and assembles the 16 modulo remainder data streams according to the matrix dimensions of the first input matrix A and the second input matrix B, resulting in 16 pairs of INT8 modulo matrices corresponding to 16 preset moduli. When assembling the modulo matrices corresponding to the current matrix block, the double-buffered structure can simultaneously send the already assembled modulo matrices to the corresponding tensor computation cores via the on-chip interconnect network (NoC).

[0193] Sixteen tensor computation cores perform matrix multiplication operations in parallel on each pair of received INT8 modular matrices. Within each tensor computation core, a modular reduction element performs modular reduction on the matrix multiplication result based on a preset modulus corresponding to that tensor computation core. The output interface, in INT8 result truncation mode, outputs the corresponding modular reduction result matrix in INT8 format. The 16 modular reduction results are then converged and aligned before being sent to the remainder reconstruction unit.

[0194] The remainder reconstruction unit performs segmented parallel multiplication, addition merging, and global modular reduction sequentially for the 16-way modulo reduction results corresponding to the same result matrix element position to obtain the reconstructed integer; then it performs inverse scaling according to the scaling offset recorded during fixed-point encoding, and completes symbol recovery and FP64 format conversion to obtain FP64 result matrix blocks.

[0195] The remainder reconstruction unit writes the FP64 result matrix into blocks to the result buffer unit. The unified control unit schedules the direct memory access controller to write the result matrix back to the chip's global memory in blocks. After all matrix blocks have been calculated, the complete FP64 matrix multiplication result is returned to the host via a high-speed interface.

[0196] Through the aforementioned hardware structure and processing flow, this embodiment can perform modular decomposition, INT8 matrix multiplication, and modular reduction in parallel along a preset modular dimension, and recover the FP64 matrix multiplication results through a dedicated remainder reconstruction pipeline. Intermediate data is transferred between processing stages via on-chip interconnect networks and buffer units, which helps reduce the overhead of software scheduling and intermediate data movement.

[0197] The application areas and scenarios of the matrix computing device of this application are introduced below: The matrix computing device provided in this application is suitable for applications requiring high precision and efficiency in matrix multiplication, especially those using target floating-point data formats such as FP64 for matrix multiplication. Examples include scientific computing, climate modeling, materials computation, engineering simulation, large-scale numerical simulation, financial risk analysis, and artificial intelligence applications requiring high-precision numerical calculations. The matrix computing device can be implemented as a matrix computing acceleration chip or a high-performance computing acceleration card, or integrated into a high-performance computing server or computing cluster.

[0198] As can be seen from the above embodiments of this application, the matrix calculation device and matrix calculation method provided by this application achieve at least the following beneficial effects: 1. This application converts an input matrix using a target floating-point data format into multiple pairs of modulo matrices using a smaller data bit width integer data format. Matrix multiplication can be performed using a tensor computation core optimized for this integer data format, and the matrix multiplication result in the target floating-point data format can be recovered through remainder reconstruction, which is beneficial to improving the computational efficiency of target floating-point matrix multiplication.

[0199] 2. This application forms a hardware computing path through a modular matrix generation unit, a tensor calculation core, and a remainder reconstruction unit, which can reduce the task switching, waiting, and scheduling overhead caused by relying on software to execute each processing stage, and improve the utilization rate of hardware resources.

[0200] 3. In this application, each hardware functional unit transmits the modular matrix and modular reduction results through an on-chip interconnect network. The tensor computation core directly outputs the modular reduction results in integer data format, which helps reduce the amount of intermediate data transmission and data movement overhead. The double buffering mechanism also allows data assembly, transmission, and matrix multiplication operations to be performed concurrently.

[0201] 4. This application utilizes the numerical characteristics of near-Merssen numbers, enabling modular reduction through shift and addition / subtraction operations, thus reducing reliance on general-purpose dividers. Furthermore, by pre-storing preset weighting coefficients and remainder reconstruction coefficients, the overhead of calculating relevant parameters during operation can be reduced.

[0202] 5. This application uses multiple near-Mason numerator and tensor computation cores to process computation tasks with different preset moduli in parallel, and uses a unified control unit and remainder reconstruction pipeline for synchronous control and continuous processing, which is beneficial to improving the parallelism of matrix computation and the overall processing throughput.

[0203] Another aspect of this application also provides a matrix calculation method. Since the principle of solving problems by the matrix calculation method is the same as that of the matrix calculation device, the embodiments of the matrix calculation method can be referred to the embodiments of the matrix calculation device described above, and the repeated parts will not be described again.

[0204] like Figure 5 As shown, in one embodiment of this application, the matrix calculation method includes steps S101 to S103.

[0205] Step S101: Convert the first input matrix and the second input matrix, which adopt the target floating-point data format, into integer matrices respectively, and perform modular decomposition on the integer matrices based on multiple preset moduli to obtain a pair of moduli matrices corresponding to each preset moduli. The matrix elements of the moduli matrices adopt integer data format, and the data bit width of the integer data format is smaller than the data bit width of the target floating-point data format.

[0206] In this embodiment of the application, in this step, the modulus matrix generation unit converts the first input matrix and the second input matrix, which adopt the target floating-point data format, into integer matrices respectively, and performs modulus decomposition on the integer matrices based on multiple preset moduli to obtain a pair of modulus matrices corresponding to each preset moduli.

[0207] Step S102: For each preset modulus, perform matrix multiplication on a pair of modulo matrices corresponding to the preset modulus, and perform modulo reduction on the matrix multiplication result according to the preset modulus to obtain the modulo reduction result matrix corresponding to each preset modulus.

[0208] In this embodiment of the application, in this step, the tensor computation core performs matrix multiplication on a pair of modulo matrices corresponding to each preset modulus, and performs modulo reduction on the matrix multiplication result according to the preset modulus to obtain the modulo reduction result matrix corresponding to each preset modulus.

[0209] Step S103: Perform remainder reconstruction based on each of the modulo reduction result matrices to obtain the matrix multiplication result of the first input matrix and the second input matrix, wherein the matrix elements of the matrix multiplication result adopt the target floating-point data format.

[0210] In this embodiment of the application, in this step, the remainder reconstruction unit performs remainder reconstruction based on each of the modulo reduction result matrices to obtain the matrix multiplication result of the first input matrix and the second input matrix, wherein the matrix elements of the matrix multiplication result adopt the target floating-point data format.

[0211] In one embodiment of this application, the matrix calculation method further includes the following steps: The modular matrix generation unit calculates the modulo remainder of each matrix element in the integer matrix with respect to the preset modulus corresponding to each near-Mersen number modular arithmetic unit through multiple parallel near-Mersen number modular arithmetic units. Among them, the multiple preset moduli are pairwise coprime, and each preset modulus is a near-Mersen number.

[0212] In one embodiment of this application, the matrix calculation method further includes the following steps: The block extraction unit divides each matrix element in the integer matrix into multiple data blocks, using the data bit width of the integer data format as the unit. The parallel weighted operation unit uses a preset weight coefficient associated with the corresponding preset modulus to perform parallel weighted operation on the multiple data blocks to obtain multiple weighted operation results. Each preset weight coefficient is predetermined based on the data bit width of the integer data format, the corresponding preset modulus, and the block order of the corresponding data block. The accumulation unit compresses and sums the results of multiple weighted operations using an accumulation tree to obtain the accumulated result. The near-Massen number modular reduction simple element performs modular reduction on the accumulated result without performing division operations based on the corresponding preset modulus to obtain the modulo remainder.

[0213] In one embodiment of this application, the matrix calculation method further includes the following steps: The fixed-point encoder determines the scaling offset based on the exponent and mantissa of each matrix element in the first and second input matrices, and performs shifting and sign processing on the matrix elements in the first and second input matrices based on the scaling offset to obtain an integer matrix. The matrix assembly and distribution buffer unit arranges the modulo remainders calculated by each near-Mersen modulo operation unit according to the order of the matrix elements corresponding to each modulo remainder in the first input matrix or the second input matrix, and assembles them according to the matrix dimensions of the first input matrix and the second input matrix respectively, to obtain a pair of modulo matrices corresponding to each preset modulo.

[0214] In one embodiment of this application, the matrix calculation method further includes the following steps: The modular reduction element in the tensor computation core performs modular reduction on the matrix multiplication result for each preset modulus, thereby obtaining the modular reduction result matrix corresponding to the preset modulus. The output interface of the tensor computation core outputs the modular reduction result matrix in integer data format in result truncation mode.

[0215] In one embodiment of this application, the matrix calculation method further includes the following steps: For each preset modulus, the segmented parallel multiplication unit splits the multiplication operation between each matrix element in the modulus reduction result matrix corresponding to the preset modulus and the remainder reconstruction coefficient corresponding to the preset modulus into multiple sub-multiplication operations, executes multiple sub-multiplication operations in parallel, and merges the results of each sub-multiplication operation to obtain the product result corresponding to each matrix element. The addition merging unit sums the product results corresponding to the matrix elements at the same position in each of the modulo reduction result matrices through the addition tree to obtain the merged result matrix; The global modulo reduction simplification element reduces the merged matrix by a global modulus determined by the product of multiple preset moduli, and then obtains the reconstructed integer matrix. The floating-point format conversion and descaling unit performs floating-point format conversion and descaling on the reconstructed integer matrix to obtain the matrix multiplication result.

[0216] In one embodiment of this application, the matrix calculation method further includes the following steps: The unified control unit provides the same clock enable signal to multiple near-Mason digitization units to control the multiple near-Mason digitization units to synchronously calculate the modulo remainder of each matrix element in the integer matrix with respect to its corresponding preset modulus.

[0217] In one embodiment of this application, the matrix calculation method further includes the following steps: Multiple tensor computation cores correspond to different preset moduli when performing matrix multiplication operations. They perform matrix multiplication operations in parallel on a pair of moduli matrices corresponding to their respective preset moduli, and perform modulo reduction on the matrix multiplication results according to their respective preset moduli.

[0218] In another aspect, this application also provides a chip that includes the matrix computing device described in any of the above embodiments.

[0219] This chip can be a dedicated matrix computation acceleration chip or a high-precision computation acceleration chip, or it can be integrated as a dedicated computing unit into computing chips such as graphics processing units (GPUs). It can be applied to high-performance computing accelerator cards, high-performance computing servers, or computing clusters, and used for scientific computing, climate modeling, materials computation, engineering simulation, large-scale numerical simulation, financial risk analysis, and artificial intelligence fusion applications requiring high-precision matrix computation. By incorporating a modular matrix generation unit, a tensor computation core, and a remainder reconstruction unit within the chip, the integer matrix operation capabilities of the tensor computation core can be utilized to perform matrix multiplication in the target floating-point data format. This reduces the control overhead caused by software scheduling of different computing resources and the overhead of intermediate data movement, thus improving the computational efficiency of matrix multiplication in the target floating-point data format.

[0220] Another aspect of this application provides a board card including the aforementioned chip.

[0221] In another aspect of this application, an electronic device is also provided, including at least one of the above-described chip and the above-described circuit board.

[0222] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A matrix calculation device, characterized in that, include: The modular matrix generation unit is used to convert the first input matrix and the second input matrix, which adopt the target floating-point data format, into integer matrices respectively, and to perform modular decomposition on the integer matrices based on multiple preset moduli to obtain a pair of modular matrices corresponding to each preset moduli. The matrix elements of the modular matrices adopt integer data format, and the data bit width of the integer data format is smaller than the data bit width of the target floating-point data format. The tensor computation core is used to perform matrix multiplication on a pair of modulo matrices corresponding to each preset modulo, and perform modulo reduction on the matrix multiplication result according to the preset modulo to obtain the modulo reduction result matrix corresponding to each preset modulo. The remainder reconstruction unit is used to perform remainder reconstruction based on each of the modulo reduction result matrices to obtain the matrix multiplication result of the first input matrix and the second input matrix, wherein the matrix elements of the matrix multiplication result adopt the target floating-point data format.

2. The matrix computing device according to claim 1, characterized in that, The modular matrix generation unit is tightly coupled to the tensor computation core, and the modular matrix generation unit transmits the modular matrix to the tensor computation core through an on-chip interconnect network; The remainder reconstruction unit is tightly coupled to the tensor calculation core, and the tensor calculation core transmits the modulo reduction result matrix to the remainder reconstruction unit through an on-chip interconnect network.

3. The matrix computing device according to claim 1, characterized in that, The modular matrix generation unit includes a near-Mersen number modular arithmetic unit array, which includes multiple near-Mersen number modular arithmetic units that are arranged in parallel and correspond to each of the preset moduli, wherein the multiple preset moduli are pairwise coprime and each of the preset moduli is a near-Mersen number. Each of the near-Mason numerator modular arithmetic units is used to calculate the modulo remainder of each matrix element in the integer matrix with respect to its corresponding preset modulus.

4. The matrix computing device according to claim 3, characterized in that, The modulus matrix generation unit further includes: A fixed-point encoder is used to determine a scaling offset based on the exponent and mantissa of each matrix element in the first input matrix and the second input matrix, and to perform shifting and sign processing on the matrix elements in the first input matrix and the second input matrix based on the scaling offset to obtain the integer matrix; The matrix assembly and distribution buffer unit is used to arrange the modulo remainders calculated by each of the near-Mason modulo operation units according to the arrangement order of the matrix elements corresponding to each modulo remainder in the first input matrix or the second input matrix, and assemble them according to the matrix dimensions of the first input matrix and the second input matrix respectively, to obtain a pair of modulo matrices corresponding to each preset modulo.

5. The matrix computing device according to claim 3, characterized in that, The near-Mason digitization unit includes: The block extraction unit is used to split each matrix element in the integer matrix into multiple data blocks, using the data bit width of the integer data format as the unit; The parallel weighted operation unit is used to perform parallel weighted operation on the multiple data blocks using preset weight coefficients associated with the corresponding preset modulus, and to obtain multiple weighted operation results. The preset weight coefficients are predetermined based on the data bit width of the integer data format, the corresponding preset modulus, and the block order of the corresponding data blocks. An accumulation unit is used to compress and sum the multiple weighted operation results through an accumulation tree to obtain an accumulation result; The near-Massen number modulo reduction simple element is used to reduce the accumulated result moduloly without performing division operations based on the corresponding preset modulus, so as to obtain the modulo remainder.

6. The matrix computing device according to claim 1, characterized in that, The tensor computation core includes a modular reduction element and an output interface. The output interface has a result truncation mode corresponding to the integer data format. The modular reduction element is used to perform modular reduction on the matrix multiplication result for each preset modulus, so as to obtain the modular reduction result matrix corresponding to the preset modulus. The output interface is used to output the modulo-reduced result matrix in the integer data format under the result truncation mode.

7. The matrix computing device according to claim 1, characterized in that, The remainder reconstruction unit includes: The segmented parallel multiplication unit is used to split the multiplication operation between each matrix element in the modulus reduction result matrix corresponding to the preset modulus and the remainder reconstruction coefficient corresponding to the preset modulus into multiple sub-multiplication operations for each preset modulus, execute the multiple sub-multiplication operations in parallel, and merge the results of each sub-multiplication operation to obtain the product result corresponding to each matrix element. The addition merging unit is used to sum the product results corresponding to the matrix elements at the same position in each of the modulo reduction result matrices through the addition tree to obtain the merged result matrix; A global modulo reduction simple element is used to perform modulo reduction on the merged result matrix based on a global modulus determined by the product of the multiple preset moduli, to obtain a reconstructed integer matrix; The floating-point format conversion and descaling unit is used to perform floating-point format conversion and descaling on the reconstructed integer matrix to obtain the matrix multiplication result, wherein the plurality of preset moduli are pairwise coprime.

8. The matrix computing device according to claim 3, characterized in that, Also includes: A unified control unit is used to provide the same clock enable signal to the plurality of near-Mason digitization units, so as to control the plurality of near-Mason digitization units to synchronously calculate the modulo remainder of each matrix element in the integer matrix with respect to its corresponding preset modulus.

9. The matrix computing device according to claim 6, characterized in that, The tensor computation cores are multiple, and each of the multiple tensor computation cores corresponds to a different preset modulus when performing matrix multiplication operations; Multiple tensor computation cores are used to perform matrix multiplication operations in parallel on a pair of modulo matrices with their respective preset modulo values, and to perform modulo reduction on the matrix multiplication results according to their respective preset modulo values.

10. A matrix calculation method, characterized in that, include: The first and second input matrices, which adopt the target floating-point data format, are converted into integer matrices respectively. The integer matrices are then decomposed based on multiple preset moduli to obtain a pair of moduli matrices corresponding to each preset moduli. The matrix elements of the moduli matrices adopt integer data format, and the data bit width of the integer data format is smaller than the data bit width of the target floating-point data format. For each preset modulus, matrix multiplication is performed on the pair of modulo matrices corresponding to the preset modulus, and the result of the matrix multiplication is reduced by modulus according to the preset modulus to obtain the modulus reduction result matrix corresponding to each preset modulus; The remainders are reconstructed based on the modulo reduction result matrices to obtain the matrix multiplication result of the first input matrix and the second input matrix, wherein the matrix elements of the matrix multiplication result adopt the target floating-point data format.

11. A chip, characterized in that, The matrix computing device includes any one of claims 1 to 9.

12. A circuit board, characterized in that, Includes the chip described in claim 11.

13. An electronic device, characterized in that, It includes at least one of the chip of claim 11 and the board of claim 12.

Images