On-chip storage protocol device and method for high-precision matrix multiplication

Abstract

The invention provides an on-chip storage protocol device for high-precision matrix multiplication. The on-chip storage protocol device comprises a scheduling unit; the index and fragmentation preparation unit is used for reading single-precision floating point data from the on-chip first-level cache, performing index domain information extraction and low bit width conversion, generating a reference index and a low bit width fragmentation operand, and outputting the low bit width fragmentation operand and the reference index to the scheduling unit; the processing array is used for receiving the low-bit wide fragmentation operands and the reference indexes, executing matrix multiplication and generating multiple paths of intermediate multiplication results; the matrix reduction unit is used for receiving the multiple paths of intermediate multiplication results and the reference index, distributing the multiple paths of intermediate multiplication results to the corresponding memory banks according to the reference index and carrying out accumulation updating; and the normalization unit is used for receiving accumulated and updated results and carrying out floating point format reduction processing to generate final single-precision floating point data. According to the invention, the throughput and storage efficiency of the on-chip protocol are improved, and the hardware area and power consumption are reduced.

Description

Technical Field

[0001] This invention relates to the field of on-chip storage technology, and in particular to an on-chip storage reduction device and method for high-precision matrix multiplication. Background Technology

[0002] With the development of artificial intelligence and scientific computing workloads, high-precision matrix multiplication has become a core workload in training, inference, and high-performance computing of various high-performance computing systems. Single-precision floating-point numbers (FP32), as a standard-defined general-purpose high-precision data format, are widely used in deep learning model training because they can simultaneously meet the requirements of numerical range and precision. However, during on-chip storage and computation, the on-chip storage bandwidth of FP32 data is difficult to match the peak computing power of the computing unit, and the transfer of large amounts of data between off-chip storage and on-chip computing units has become a major obstacle to performance improvement.

[0003] To improve throughput and energy efficiency, existing systems often convert high-precision matrix multiplication into multiple low-bit-width partial results and reconstruct them on-chip, generating intermediate results that require further processing. Due to differences in exponent / bit weights, these intermediate results necessitate order alignment, shift alignment, and wide-bit accumulation during the reduction process, along with carry propagation and normalization requirements. Using a traditional element-wise floating-point accumulation path would result in numerous shifters and wide adders, a longer critical path, increased power consumption and area, and higher demands on on-chip memory ports and access parallelism, leading to problems such as bank conflicts and unstable throughput. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes an on-chip storage reduction device and method for high-precision matrix multiplication, which improves on-chip reduction throughput and storage efficiency while reducing hardware area and power consumption.

[0005] This invention provides an on-chip storage reduction device for high-precision matrix multiplication, comprising:

[0006] The scheduling unit is equipped with a first control port, a second control port, and a third control port;

[0007] The exponent and sharding preparation unit is connected to the scheduling unit through the first control port and to the on-chip level 1 cache and processing array through the data port. It is used to read single-precision floating-point data from the on-chip level 1 cache, extract exponent field information and perform low-bit width conversion, generate a base exponent and low-bit width sharding operands, and output the low-bit width sharding operands and the base exponent to the scheduling unit.

[0008] The processing array is connected to the scheduling unit through the second control port, and is used to receive the low-width slice operands and the base exponent and perform matrix multiplication to generate multiple intermediate multiplication results.

[0009] The matrix reduction unit includes an on-chip memory array, is connected to the scheduling unit through the third control port, and is connected to the processing array and the on-chip level 1 cache through the data port. It is used to receive the multi-path intermediate multiplication results and the base exponent, and allocate each intermediate multiplication result to the corresponding memory bank according to the base exponent and perform cumulative updates.

[0010] The normalization unit, connected to the matrix reduction unit, is used to receive the accumulated and updated results, perform floating-point format restoration processing, and generate the final single-precision floating-point data.

[0011] An on-chip level 1 cache is connected to the exponent and sharding preparation unit and the normalization unit via data ports, respectively, and is used to store the final single-precision floating-point data.

[0012] In one embodiment of the present invention, the index and sharding preparation unit includes:

[0013] The benchmark index generation module is used to extract the original index value from the index field of the single-precision floating-point data and select the maximum index value as the benchmark index within a preset matrix block granularity.

[0014] The subtraction module, connected to the benchmark index generation module, is used to calculate the difference between each original index value and the benchmark index, and generate the offset.

[0015] The shift alignment module is connected to the output of the subtraction module and is used to shift and align the mantissa field of the single-precision floating-point data according to the offset, and convert it into a fixed-point integer representation.

[0016] The low-bit-width segmentation generation module is connected to the output of the shift alignment module and is used to divide the fixed-point integer representation into low-bit-width segmentation operands according to a preset low-bit-width.

[0017] In one embodiment of the present invention, the index and sharding preparation unit further includes:

[0018] An anomaly detection module is used to output an anomaly flag bit of the single-precision floating-point data, generate flag information, and output it to the scheduling unit.

[0019] In one embodiment of the present invention, at least one accumulation entry is provided in each memory cell of the on-chip memory array, and the accumulation entry stores the entry index, carry vector and hold vector;

[0020] The matrix reduction unit further includes:

[0021] A shifter array is used to generate a shift amount based on the difference between the reference index and the entry index of the corresponding target accumulation entry read from the target memory, and to shift and align the intermediate multiplication results of each path.

[0022] A carry-retaining adder array is connected to the output of the shifter array and is used to merge the aligned multi-path intermediate multiplication results with the carry vector and the retain vector to obtain the updated carry vector and retain vector.

[0023] The memory selection module is connected to the output of the carry-hold adder array and is used to remap the updated carry vector and hold vector to the target memory.

[0024] The entry selection module is connected to the output of the memory selection module and to the on-chip memory array. It is used to locate the target accumulation entry in the target memory and rewrite the updated carry and retention vector back to the target accumulation entry.

[0025] In one embodiment of the present invention, the memory selection module is further configured to: select the target memory for mapping based on the reference exponent of the intermediate multiplication results of each path.

[0026] In one embodiment of the present invention, the normalization unit includes:

[0027] The carry-propagation addition module is used to receive the updated carry vector and the retained vector and perform carry-propagation addition on them to obtain the reduction result;

[0028] The floating-point normalization module, connected to the carry propagation addition module, is used to perform floating-point normalization processing on the reduction result based on the entry exponent and the base exponent. By adjusting the value of the exponent field and shifting the mantissa field, it conforms to the normalization format requirements of single-precision floating-point numbers and generates an intermediate floating-point result with an exponent field.

[0029] The rounding module, connected to the floating-point normalization module, is used to perform rounding processing on the mantissa field of the intermediate floating-point result, so that it is truncated or carried over to the specified mantissa length of the single-precision floating-point number.

[0030] An exception handling module, connected to the rounding module, is used to detect whether there are abnormal data such as non-numerical, infinity, overflow or underflow in the intermediate floating-point result, generate corresponding exception flag information, and remove abnormal data from the intermediate floating-point result to generate the final single-precision floating-point data.

[0031] The output control module, connected to the exception handling module, is used to output the generated final single-precision floating-point data and write it back to the on-chip level 1 cache.

[0032] Another aspect of the present invention provides an on-chip storage reduction method for high-precision matrix multiplication, comprising:

[0033] Extract exponent field information and perform low-bit width conversion on the input single-precision floating-point data to generate a base exponent and low-bit width slice operands;

[0034] Receive the low-width slice operand and the reference exponent, perform matrix multiplication, and generate multiple intermediate multiplication results;

[0035] Receive the multi-path intermediate multiplication results and the benchmark exponent, allocate each intermediate multiplication result to the corresponding storage bank according to the benchmark exponent, and perform cumulative updates;

[0036] The accumulated and updated results are restored to their floating-point format to generate the final single-precision floating-point data.

[0037] In one embodiment of the present invention, the step of extracting exponent field information and performing low-bit width conversion on the input single-precision floating-point data to generate a base exponent and low-bit width piecewise operands includes:

[0038] The original exponent value is extracted from the exponent field of the single-precision floating-point data, and the maximum exponent value is selected as the benchmark exponent within the preset matrix block granularity.

[0039] Calculate the difference between each original index value and the benchmark index to generate the offset;

[0040] The mantissa field of the single-precision floating-point data is shifted and aligned according to the offset.

[0041] Convert to fixed-point integer representation;

[0042] The fixed-point integer representation is divided into low-width segment operands according to a preset low-width segment.

[0043] In one embodiment of the present invention, the step of allocating the intermediate multiplication results of each path to the corresponding storage bank according to the benchmark index and accumulating and updating them includes:

[0044] The target storage for mapping is selected based on the benchmark exponent of each intermediate multiplication result;

[0045] Locate the target accumulation entry within the target storage body, the target accumulation entry storing items...

[0046] Point index, carry vector, and retention vector;

[0047] Read the carry vector, retention vector, and entry exponent of the target accumulation entry;

[0048] A shift amount is generated based on the difference between the benchmark index and the entry index, and the intermediate multiplication results of each path are shifted and aligned.

[0049] The merged and aligned multi-path intermediate multiplication results are combined with the carry vector and the retained vector to obtain the updated carry vector and retained vector;

[0050] The updated carry vector and retain vector are remapped to the target memory and written back to the target accumulation entry.

[0051] In one embodiment of the present invention, the step of receiving the accumulated and updated result, performing floating-point format restoration processing, and generating the final single-precision floating-point data includes:

[0052] Receive the updated carry vector and the retained vector, and perform carry propagation addition on them to obtain the reduction result;

[0053] The reduction result is normalized to floating point based on the entry index and the benchmark index. The exponent field value is adjusted and the mantissa field is shifted to meet the normalization format requirements of single-precision floating point numbers, thereby generating an intermediate floating point result with an exponent field.

[0054] The mantissa field of the intermediate floating-point result is rounded to truncate or carry over to the specified mantissa length for a single-precision floating-point number.

[0055] The intermediate floating-point result is checked for abnormal data such as non-numerical, infinity, overflow, or underflow, and corresponding abnormal flag information is generated. Abnormal data is removed from the intermediate floating-point result to generate the final single-precision floating-point data.

[0056] As can be seen from the above solutions, the advantages of the present invention are:

[0057] The on-chip memory reduction device provided by this invention reduces timing and area overhead during the reduction phase. Through structured organization of the reduction process, it reduces critical path pressure caused by order / width bit accumulation, preventing long carry chains from repeatedly entering the main path, thereby increasing achievable frequency and reducing hardware area and power consumption. It improves on-chip reduction throughput and storage efficiency by regularly allocating and grouping intermediate results, reducing storage port requirements and bank conflicts, and minimizing intermediate result read / write and transport overhead. This allows the reduction phase to stably follow the array output cycle, improving end-to-end throughput and energy efficiency. Attached Figure Description

[0058] Figure 1 This diagram illustrates the overall structure of an on-chip memory protocol device according to an embodiment of the present invention.

[0059] Figure 2 A schematic diagram of the specific structure of the exponent and slice preparation unit is shown;

[0060] Figure 3A schematic diagram of the matrix reduction unit is shown;

[0061] Figure 4 A schematic diagram of the specific structure of the normalization unit is shown;

[0062] Figure 5 A schematic diagram of the overall flow of an on-chip memory specification method provided in an embodiment of the present invention is shown;

[0063] Figure 6 It shows Figure 5 A detailed flowchart of step S2 is shown below;

[0064] Figure 7 It shows Figure 5 A detailed flowchart of step S3 is shown below;

[0065] Figure 8 It shows Figure 5 A schematic diagram of the specific process for step S4.

[0066] The attached figures are labeled as follows:

[0067] 10: Scheduling Unit

[0068] 20: Index and Fragmentation Preparation Unit

[0069] 201: Benchmark Index Generation Module

[0070] 202: Subtraction Module

[0071] 203: Shift Alignment Module

[0072] 204: Low-width segmentation generation module

[0073] 205: Anomaly Detection Module

[0074] 30: Processing Array

[0075] 40: Matrix Reduction Unit

[0076] 401: On-chip memory array

[0077] 402: Shifter Array

[0078] 403: Carry-Retaining Adder Array

[0079] 404: Memory Selection Module

[0080] 405: Item Selection Module

[0081] 50: Normalized Unit

[0082] 501: Carry Propagation Addition Module

[0083] 502: Floating-point normalization module

[0084] 503: Rounding module

[0085] 504: Exception handling module

[0086] 505: Output Control Module

[0087] 60: On-chip L1 cache. Detailed Implementation

[0088] It should be noted that, in this invention, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus.

[0089] In the absence of further restrictions, an element defined by the phrase "comprising a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0090] As previously mentioned, in existing technologies, the multiplication side can typically achieve certain benefits through splitting, rewriting, or low-precision reuse in the implementation of high-precision matrix multiplication. However, these methods inevitably produce multiple intermediate results with different exponents, and the reduction method directly determines end-to-end performance, energy efficiency, and area cost. Especially when array throughput continues to increase, if the reduction stage still relies on traditional element-wise order matching and wide-bit accumulation, it will not only become a critical path and power hotspot but also place excessive demands on on-chip storage ports and cause bank conflicts. To address this, this invention considers improving existing technologies by allocating and grouping intermediate results according to the exponent field in the back-end reduction of floating-point matrix multiplication, thereby reducing the critical path pressure and area / power consumption overhead caused by order matching / accumulation / normalization. Furthermore, it reduces bank conflicts and maintains array-level throughput without introducing large-scale multi-port SRAM.

[0091] Specifically, refer to Figure 1 As shown, Figure 1 A schematic diagram of the overall structure of an on-chip storage reduction device for high-precision matrix multiplication provided by an embodiment of the present invention is shown.

[0092] An on-chip storage reduction device for high-precision matrix multiplication includes:

[0093] The scheduling unit 10 is equipped with a first control port, a second control port, and a third control port.

[0094] The exponent and sharding preparation unit 20 is connected to the scheduling unit 10 through the first control port and to the on-chip level 1 cache 60 and the processing array 30 through the data port. It is used to read single-precision floating-point data from the on-chip level 1 cache 60, extract exponent field information and perform low-bit width conversion, generate a base exponent and low-bit width sharding operands, and output the low-bit width sharding operands and the base exponent to the scheduling unit 10.

[0095] The processing array 30 is connected to the scheduling unit 10 through the second control port. It is used to receive the low-width slice operands and the reference exponent and perform matrix multiplication to generate multiple intermediate multiplication results.

[0096] The matrix reduction unit 40 includes an on-chip memory array, is connected to the scheduling unit 10 through the third control port, and is connected to the processing array 30 and the on-chip level 1 cache 60 through the data port. It is used to receive the multi-path intermediate multiplication results and the reference exponent, and allocate each intermediate multiplication result to the corresponding memory bank according to the reference exponent and perform cumulative updates.

[0097] The normalization unit 50 is connected to the matrix reduction unit 40 through a data port. It is used to receive the accumulated and updated results, perform floating-point format restoration processing, and generate the final single-precision floating-point data.

[0098] The on-chip level 1 cache 60 is connected to the exponent and sharding preparation unit 20 and the normalization unit 50 via data ports, and is used to store the final single-precision floating-point data.

[0099] In this embodiment, a scheduling unit 10 (Scheduler) is used to uniformly orchestrate the data flow and coordinate the exponent and slicing preparation unit 20, the processing array 30, the matrix reduction unit 40, and the normalization unit 50. Specifically, the exponent and slicing preparation unit 20 (ESU) reads single-precision floating-point data from the on-chip L1 cache 60, extracts the exponent field information, and generates low-width slicing operands and a base exponent. Subsequently, these low-width slicing operands enter the processing array 30 (PE Arrays) to generate multiple intermediate multiplication results. Finally, the matrix reduction unit 40 (MRU) performs matrix-level exponent reduction and fast accumulation on the multiple intermediate multiplication results, and when output is required, sends the reduction result to the normalization unit 50 (Normalization Unit) to perform floating-point format restoration processing, outputting the final single-precision floating-point data. This on-chip storage reduction device elevates the "alignment and reduction" process from an element-wise floating-point path to a matrix-level organization, avoiding repeated wide-bit carry propagation and high-port storage access during the reduction phase. This reduces critical path pressure and on-chip storage costs, improves end-to-end throughput and energy efficiency, and enhances the performance of the on-chip storage system.

[0100] In one embodiment, reference Figure 2 As shown, Figure 2 A schematic diagram of the specific structure of the exponent and slice preparation unit 20 is shown. The exponent and slice preparation unit 20 preprocesses single-precision floating-point data, and its core function is to provide a traceable exponent reference and a controllable low-bit-width representation for the matrix reduction unit.

[0101] Specifically, the index and fragmentation preparation unit 20 includes:

[0102] The baseline exponent generation module 201 is used to extract the original exponent value from the exponent field of the single-precision floating-point data and select the largest exponent value within a preset matrix block granularity as the baseline exponent. The baseline exponent is exemplified as E_MAX in the diagram, used to describe the exponent reference of the data within this granularity. Specifically, the single-precision floating-point data (FP32) consists of 32 bits, comprising 1 sign bit, 8 exponent bits, and 23 mantissa bits. The exponent field stores the offset exponent value, using an offset representation (offset value of 127), and the mantissa field stores the fractional part of the significant digits. The 8-bit exponent field is directly read through bit operations to obtain the unbiased original exponent value (range -127 to 128). A predefined matrix block (Tile) granularity is used to process data in units of Tiles. For each Tile (such as a 16×16 or 32×32 submatrix), the exponent values ​​of all elements are traversed, and the largest exponent is selected as the baseline exponent.

[0103] The subtraction operation module 202 is connected to the benchmark index generation module 201 and is used to calculate the difference between each original index value and the benchmark index to generate an offset Δexp. The original index value is exemplified as x-exp in the figure.

[0104] The shift alignment module 203 is connected to the output of the subtraction operation module 202 and is used to shift and align the mantissa field of the single-precision floating-point data according to the offset, converting it into a fixed-point integer representation that is convenient for low-bit-width array calculation and back-end reduction.

[0105] The low-width segmentation generation module 204 is connected to the output of the shift alignment module and is used to divide the fixed-point integer representation into low-width segmentation operands according to a preset low-width segmentation.

[0106] The anomaly detection module 205 is used to detect whether there are abnormal data such as non-numerical, infinite, overflow or underflow in the single-precision floating-point data, output an anomaly flag, generate flag information and output it to the scheduling unit.

[0107] In this embodiment, the exponent and sharding preparation unit 20 transmits the base exponent, low-width sharding operands, and flag information (such as the non-numerical (NaN) and infinity (Inf) detection results) to the scheduling unit 10, which then distributes them to the processing array 30. Through the exponent and sharding preparation unit 20, the multiple intermediate multiplication results generated by the subsequent processing array 30 can have their bit weights quickly determined in the form of "base exponent + offset" during matrix reduction by the matrix reduction unit 40, thus laying the foundation for matrix-level exponent reduction and rapid accumulation.

[0108] The processing array 30, or PE array, consists of many PEs. Each PE typically includes a multiply-accumulate unit, a few registers, and a small amount of control logic. The interconnections between PEs vary greatly depending on the design. Each PE can perform calculations independently and output the results directly; adjacent PEs can transfer data; or even a router can be added for data transfer between PEs. In one embodiment, the processing array 30 receives low-width slice operands from the exponent and slice preparation unit and performs matrix multiplication, outputting multiple intermediate multiplication results. Since these intermediate multiplication results correspond to different bit weights (e.g., originating from different slices, different alignment offsets, or different multiplication paths), they carry or can be deduced from the corresponding exponent field information when entering the matrix reduction unit 40. In other words, each intermediate multiplication result can be abstracted as "numerical part + exponent description," where the exponent description can be obtained by combining a base exponent and an offset.

[0109] It should be emphasized that the reduction mechanism of the matrix reduction unit of the present invention is not bound to the generation method of the multi-path intermediate multiplication results of the processing array. As long as there are intermediate results with different exponents and bit weights that need to be aligned and accumulated on the chip, the matrix reduction unit 40 of the present invention can be used to reduce the intermediate multiplication results.

[0110] The exponent / weight differences in multiplication results from multiple intermediate paths trigger frequent alignment and shift operations. If processed element-wise, this introduces a large number of shifters and control logic on the reduction side, which rapidly expands with increasing parallelism. To address this, in one embodiment, the matrix reduction unit employs matrix-level exponential reduction, organizing alignment and mapping rules at a higher granularity, reducing repetitive alignment operations and improving the regularity of the reduction.

[0111] Traditional element-wise floating-point accumulation centered on wide-bit CPA (carry-propagating addition) requires wide adders, frequent alignment and normalization, resulting in long carry chains. This leads to long and large back-end critical paths, which worsen with increasing accumulation depth, often becoming a bottleneck in the reduction stage. This makes it difficult to increase frequency or requires deeper pipelines, incurring additional registers and control costs. In one embodiment, the matrix reduction unit uses a carry-preserving adder array for fast accumulation, organizing the reduction process into a more pipelined structure. This avoids repeatedly triggering long carry propagation on the main path, significantly reducing timing pressure.

[0112] Matrix reduction requires high-concurrency read and write operations on a large number of intermediate multiplication results. Traditional solutions often rely on multi-port SRAM, replicated memory, or complex crossbar switches to maintain parallel access, but these structures significantly increase area, power consumption, and design complexity. In one embodiment, the matrix reduction unit distributes multiple intermediate multiplication results to different memory banks and updates them locally based on exponent field information, achieving high throughput with fewer ports.

[0113] To simultaneously read and write large amounts of partial sums at high throughput, conventional solutions often require multi-port SRAM or replicated memory. However, this significantly increases area, and when multiple intermediate multiplication results access the same memory region simultaneously, bank conflicts arise, leading to insertion bubbles, throughput jitter, and increased tail latency. This is especially amplified in matrix-level pipelines, forcing an unfavorable trade-off between area and performance. In one embodiment, the matrix reduction unit performs more rational allocation and local merging as multiple intermediate multiplication results arrive, distributing and controlling hotspot accesses, thus significantly mitigating the conflict problem.

[0114] Based on this, the matrix reduction unit 40 includes an on-chip memory array 401, which is connected to the scheduling unit 10 through the third control port and to the processing array 30 and the on-chip level 1 cache 50 through the data port. It is used to receive the multi-path intermediate multiplication results and the reference exponent, and to allocate each intermediate multiplication result to the corresponding memory bank and perform cumulative updates based on the reference exponent.

[0115] Specifically, in one embodiment, further reference is made to... Figure 3 As shown, Figure 3 A schematic diagram of the matrix reduction unit is shown.

[0116] Matrix reduction unit 40 includes:

[0117] The on-chip memory array 401 has at least one accumulation entry in each memory cell, and the accumulation entry stores the entry index, carry vector and hold vector.

[0118] The shifter array 402 is used to generate a shift amount based on the difference between the reference index and the entry index of the corresponding target accumulation entry read from the target memory, and to shift and align the intermediate multiplication results of each path.

[0119] The carry-retaining adder array 403 is connected to the output of the shifter array 402 and is used to merge the aligned multi-path intermediate multiplication results with the carry vector and the retain vector to obtain the updated carry vector and retain vector.

[0120] The memory selection module 404 is connected to the output of the carry-holding adder array 403. It is used to select the target memory for mapping based on the reference exponent of each intermediate multiplication result, and to remap the updated carry vector and hold vector to the target memory.

[0121] The entry selection module 405 is connected to the output of the storage selection module 404 and is used to locate the target accumulation entry in the target storage and rewrite the updated carry and retention vector back to the target accumulation entry.

[0122] The on-chip memory array 401 is located between the memory selection module 404 and the entry selection module 405, and is connected to the output of the memory selection module 404 and the entry selection module 405.

[0123] In this embodiment, the matrix reduction unit adopts a "distribution by exponent field + accumulation of entries within the Bank" approach to achieve matrix-level reduction. First, the matrix reduction unit performs a first-level selection through the storage selection module 404 based on the exponent field information of the input multi-path intermediate multiplication results, mapping it to the target storage to achieve distributed storage and parallel processing according to the exponent range. Subsequently, the entry selection module 405 performs a second-level selection within the target storage to locate the accumulation entry (Entry / Chunk). This accumulation entry stores at least the entry exponent (used to describe the currently accumulated exponent reference) and the two reduction vectors: the carry vector and the save vector. During the specification process, the hardware can perform a "read-align-accumulate-write-back" update process in parallel for each memory bank: read the carry vector and save vector of the entry, as well as the entry index, from the target memory bank; generate a shift amount based on the difference between the base index and the entry index, and shift and align the intermediate multiplication results of each path; then merge the aligned and merged multiplication results of the multiple paths with the read carry vector and save vector through a carry-save adder array (CSA) to obtain the updated carry vector and save vector; finally, write the updated carry vector and save vector back to the same entry to complete one accumulation update. Because the carry-retaining adder array merging avoids carry propagation into the main path, the critical path of the matrix reduction unit mainly consists of shift alignment and adder cascade, which makes it easier to achieve high-frequency pipelined operation than traditional CPA accumulation. At the same time, the on-chip memory array reduces the dependence on multiple ports of a single memory bank and reduces conflicts and throughput jitter through parallel memory bank updates, thereby enabling the reduction stage to stably follow the array output beat.

[0124] When the final single-precision floating-point data needs to be generated (e.g., when a matrix block reduction is completed or the write-back phase is triggered), the matrix reduction unit outputs the carry vector and save vector saved in the selected entry to the normalization unit 50. The normalization unit 50 then performs further floating-point format restoration processing on the accumulated and updated result to generate the final single-precision floating-point data.

[0125] In one embodiment, reference Figure 4 As shown, Figure 4 A schematic diagram of the specific structure of the normalization unit is shown.

[0126] Normalization unit 50 includes:

[0127] Carry propagation addition module 501 (CPA) is used to receive the updated carry vector and the retained vector and perform carry propagation addition operation on them to obtain the reduction result.

[0128] The floating-point normalization module 502, connected to the carry propagation addition module 501, is used to perform floating-point normalization processing on the reduction result based on the entry exponent and the base exponent. By adjusting the value of the exponent field and shifting the mantissa field, it conforms to the normalization format requirements of single-precision floating-point numbers and generates an intermediate floating-point result with an exponent field.

[0129] The rounding module 503 is connected to the floating-point normalization module 502 and is used to perform rounding processing on the mantissa field of the intermediate floating-point result, so that it is truncated or carried over to the specified mantissa length of the single-precision floating-point number.

[0130] An exception handling module 504, connected to the rounding module 503, is used to detect whether there are abnormal data such as non-numerical, infinite, overflow or underflow in the intermediate floating-point result, generate corresponding exception flag information, and remove abnormal data from the intermediate floating-point result to generate the final single-precision floating-point data.

[0131] The output control module 505 is connected to the exception handling module 504 and is used to output the generated final single-precision floating-point data and write it back to the on-chip level 1 cache.

[0132] In this embodiment, the normalization unit delays CPA and floating-point normalization to the "final output moment" for unified execution. The main reduction path avoids repeated wide-bit carry propagation and normalization logic, thereby significantly reducing the critical path and area power consumption overhead in the reduction stage while maintaining the recoverability and precision semantics of floating-point results, and reducing the additional cost caused by repeated on-chip formatting of intermediate results.

[0133] Furthermore, in one embodiment, the scheduling unit is responsible for globally coordinating the operations of each unit. Each port corresponds to a different type of control signal for a different unit. For example, the first control port provides the exponentiation and sharding preparation unit 20 with control signals for data reading, preprocessing initiation, and exception response; the second control port provides the processing array 30 with control signals for multiplication initiation, data distribution, and pipeline cycle control; and the third control port provides the matrix reduction unit 40 with control signals for memory allocation, accumulation initiation, and result reading. The scheduling unit eliminates operational delay differences between units through a multi-level pipeline timing coordination mechanism, achieving bubble-free end-to-end pipeline operation. The entire matrix multiplication process is divided into four synchronous stages, each triggered uniformly by the scheduling unit: Preprocessing stage (T1-Tn): The exponent and fragmentation preparation unit 20 completes single-precision floating-point data reading, base exponent extraction, and low-bit-width fragmentation; Computation stage (Tn+1-T2n): The processing array performs low-bit-width matrix multiplication, generating multiple intermediate multiplication results; Reduction stage (T2n+1-T3n): The matrix reduction unit completes the partitioned storage and cumulative update of multiple intermediate multiplication results; Normalization stage (T3n+1-T4n): The normalization unit completes floating-point format restoration and result write-back. At the end of the preprocessing stage (Tn-2), the scheduling unit sends a pre-start signal to the PE array in advance, so that the processing array immediately enters the calculation state at the same time as the output of the exponent and sharding preparation unit 20, eliminating pipeline gaps; similarly, at the end of the calculation stage, a memory pre-allocation signal is sent to the matrix reduction unit 40 to ensure that the intermediate results can be directly written to the corresponding memory after they are generated.

[0134] In summary, the on-chip memory reduction device disclosed in this invention reduces timing and area overhead during the reduction phase. Through structured organization of the reduction process, it reduces critical path pressure caused by order / width bit accumulation, and avoids long carry chains repeatedly entering the main path, thereby increasing achievable frequency and reducing hardware area and power consumption. It improves on-chip reduction throughput and storage efficiency by regularly allocating and grouping intermediate results, reducing storage port requirements and bank conflicts, and minimizing intermediate result read / write and transport overhead. This allows the reduction phase to stably follow the array output cycle, improving end-to-end throughput and energy efficiency.

[0135] In one embodiment, an on-chip storage reduction method for high-precision matrix multiplication is further provided. This embodiment is a method embodiment corresponding to the above-described device embodiment, such as... Figure 5 As shown, Figure 5 This diagram illustrates the overall flow of an on-chip storage reduction method for high-precision matrix multiplication according to an embodiment of the present invention. This method implementation can be combined with the aforementioned device implementation. The relevant technical details mentioned in the device implementation remain valid in this method implementation, and will not be repeated here to avoid repetition.

[0136] An on-chip storage reduction method for high-precision matrix multiplication includes the following steps:

[0137] Step S1: Extract the exponent field information and perform low-bit width conversion on the input single-precision floating-point data to generate the base exponent and low-bit width slice operands.

[0138] Step S2: Receive the low-width segmented operands and the reference exponent, and perform matrix multiplication to generate multiple intermediate multiplication results.

[0139] Step S3: Receive the multi-path intermediate multiplication results and the reference exponent, allocate each intermediate multiplication result to the corresponding storage bank according to the reference exponent, and perform cumulative updates.

[0140] Step S4: Perform floating-point format restoration processing on the accumulated and updated results to generate the final single-precision floating-point data.

[0141] In one embodiment, further reference is made to Figure 6 As shown, in step S2, the step of extracting exponent field information and performing low-bit-width conversion on the input single-precision floating-point data to generate the base exponent and low-bit-width slice operands further includes:

[0142] Step S21: Extract the original exponent value from the exponent field of the single-precision floating-point data, and select the largest exponent value within the preset matrix block granularity as the base exponent.

[0143] Step S22: Calculate the difference between each original index value and the benchmark index to generate the offset.

[0144] Step S23: Shift the mantissa field of the single-precision floating-point data according to the offset.

[0145] Bit alignment, converted to fixed-point integer representation.

[0146] Step S24: Divide the fixed-point integer representation into low-width segment operands according to the preset low-width bit width.

[0147] In one embodiment, further reference is made to Figure 7 As shown, in step S3, the step of allocating the intermediate multiplication results of each path to the corresponding storage bank according to the benchmark exponent and accumulating and updating them further includes:

[0148] Step S31: Select the target memory for mapping based on the base exponent of each intermediate multiplication result. Step S32: Locate the target accumulation entry within the target memory.

[0149] It stores the entry index, carry vector, and retention vector.

[0150] Step S33: Read the carry vector, retention vector and entry index of the target accumulation entry.

[0151] Step S34: Generate a shift amount based on the difference between the base index and the entry index, and shift and align the intermediate multiplication results of each path.

[0152] Step S35: Merge the aligned multi-path intermediate multiplication results with the carry vector and the retained vector to obtain the updated carry vector and retained vector.

[0153] Step S36: Remap the updated carry vector and retain vector to the target memory and write them back to the target accumulation entry.

[0154] In one embodiment, further reference is made to Figure 8 As shown, in step S4, the step of receiving the accumulated and updated result, performing floating-point format restoration processing, and generating the final single-precision floating-point data further includes:

[0155] Step S41: Receive the updated carry vector and the retained vector, and perform carry propagation addition on them to obtain the reduction result.

[0156] Step S42: Perform floating-point normalization processing on the reduction result based on the entry exponent and the base exponent. By adjusting the value of the exponent field and shifting the mantissa field, the normalization format requirements of single-precision floating-point numbers are met, and an intermediate floating-point result with an exponent field is generated.

[0157] Step S43: Perform rounding on the mantissa field of the intermediate floating-point result to truncate or carry over to the specified mantissa length of a single-precision floating-point number.

[0158] Step S44: Detect whether there are abnormal data such as non-numerical, infinity, overflow or underflow in the intermediate floating-point result, generate corresponding abnormal flag information, remove abnormal data from the intermediate floating-point result to generate the final single-precision floating-point data.

[0159] In summary, the on-chip memory reduction method disclosed in this invention reduces the timing and area overhead of the reduction stage. Through structured organization of the reduction process, it reduces critical path pressure caused by order / width bit accumulation, and avoids long carry chains repeatedly entering the main path, thereby increasing the achievable frequency and reducing hardware area and power consumption. It improves on-chip reduction throughput and storage efficiency by regularly allocating and grouping intermediate results, reducing storage port requirements and bank conflicts, and minimizing intermediate result read / write and transport overhead. This allows the reduction stage to stably follow the array output cycle, improving end-to-end throughput and energy efficiency.

[0160] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of the present invention.

Claims

1. An on-chip storage reduction device for high-precision matrix multiplication, characterized in that, include: The scheduling unit is equipped with a first control port, a second control port, and a third control port; The exponent and sharding preparation unit is connected to the scheduling unit through the first control port and to the on-chip level 1 cache and processing array through the data port. It is used to read single-precision floating-point data from the on-chip level 1 cache, extract exponent field information and perform low-bit width conversion, generate a base exponent and low-bit width sharding operands, and output the low-bit width sharding operands and the base exponent to the scheduling unit. The processing array is connected to the scheduling unit through the second control port, and is used to receive the low-width slice operands and the base exponent and perform matrix multiplication to generate multiple intermediate multiplication results. The matrix reduction unit includes an on-chip memory array, is connected to the scheduling unit through the third control port, and is connected to the processing array and the on-chip level 1 cache through the data port. It is used to receive the multi-path intermediate multiplication results and the base exponent, and allocate each intermediate multiplication result to the corresponding memory bank according to the base exponent and perform cumulative updates. The normalization unit is connected to the matrix reduction unit through a data port. It is used to receive the accumulated and updated results, perform floating-point format restoration processing, and generate the final single-precision floating-point data. An on-chip level 1 cache is connected to the exponent and sharding preparation unit and the normalization unit via data ports, respectively, and is used to store the final single-precision floating-point data.

2. The on-chip memory protocol device according to claim 1, characterized in that, The index and fragmentation preparation unit includes: The benchmark index generation module is used to extract the original index value from the index field of the single-precision floating-point data and select the maximum index value as the benchmark index within a preset matrix block granularity. The subtraction module, connected to the benchmark index generation module, is used to calculate the difference between each original index value and the benchmark index, and generate the offset. The shift alignment module is connected to the output of the subtraction module and is used to shift and align the mantissa field of the single-precision floating-point data according to the offset, and convert it into a fixed-point integer representation. The low-bit-width segmentation generation module is connected to the output of the shift alignment module and is used to divide the fixed-point integer representation into low-bit-width segmentation operands according to a preset low-bit-width.

3. The on-chip memory protocol device according to claim 2, characterized in that, The index and fragmentation preparation unit also includes: An anomaly detection module is used to output an anomaly flag bit of the single-precision floating-point data, generate flag information, and output it to the scheduling unit.

4. The on-chip memory protocol device according to claim 1, characterized in that, Each memory cell of the on-chip memory array is provided with at least one accumulation entry, and the accumulation entry stores the entry index, carry vector and hold vector; The matrix reduction unit further includes: A shifter array is used to generate a shift amount based on the difference between the reference index and the entry index of the corresponding target accumulation entry read from the target memory, and to shift and align the intermediate multiplication results of each path. A carry-retaining adder array is connected to the output of the shifter array and is used to merge the aligned multi-path intermediate multiplication results with the carry vector and the retain vector to obtain the updated carry vector and retain vector. The memory selection module is connected to the output of the carry-hold adder array and is used to remap the updated carry vector and hold vector to the target memory. The entry selection module is connected to the output of the memory selection module and to the on-chip memory array. It is used to locate the target accumulation entry in the target memory and rewrite the updated carry and retention vector back to the target accumulation entry.

5. The on-chip memory protocol device according to claim 4, characterized in that, The storage selection module is also used to select the target storage for mapping based on the reference exponent of the intermediate multiplication results of each path.

6. The on-chip memory protocol device according to claim 4, characterized in that, The normalization unit includes: The carry-propagation addition module is used to receive the updated carry vector and the retained vector and perform carry-propagation addition on them to obtain the reduction result; The floating-point normalization module, connected to the carry propagation addition module, is used to perform floating-point normalization processing on the reduction result based on the entry exponent and the base exponent. By adjusting the value of the exponent field and shifting the mantissa field, it conforms to the normalization format requirements of single-precision floating-point numbers and generates an intermediate floating-point result with an exponent field. The rounding module, connected to the floating-point normalization module, is used to perform rounding processing on the mantissa field of the intermediate floating-point result, so that it is truncated or carried over to the specified mantissa length of the single-precision floating-point number. An exception handling module, connected to the rounding module, is used to detect whether there are abnormal data such as non-numerical, infinity, overflow or underflow in the intermediate floating-point result, generate corresponding exception flag information, and remove abnormal data from the intermediate floating-point result to generate the final single-precision floating-point data. The output control module, connected to the exception handling module, is used to output the generated final single-precision floating-point data and write it back to the on-chip level 1 cache.

7. A method for reducing on-chip storage for high-precision matrix multiplication, characterized in that, Include: Extract exponent field information and perform low-bit width conversion on the input single-precision floating-point data to generate a base exponent and low-bit width slice operands; Receive the low-width slice operand and the reference exponent, perform matrix multiplication, and generate multiple intermediate multiplication results; Receive the multi-path intermediate multiplication results and the benchmark exponent, allocate each intermediate multiplication result to the corresponding storage bank according to the benchmark exponent, and perform cumulative updates; The accumulated and updated results are restored to their floating-point format to generate the final single-precision floating-point data.

8. The method according to claim 7, characterized in that, The step of extracting exponent field information and performing low-bit width conversion on the input single-precision floating-point data to generate a base exponent and low-bit width piecewise operands includes: The original exponent value is extracted from the exponent field of the single-precision floating-point data, and the maximum exponent value is selected as the benchmark exponent within the preset matrix block granularity. Calculate the difference between each original index value and the benchmark index to generate the offset; The mantissa field of the single-precision floating-point data is shifted and aligned according to the offset. Convert to fixed-point integer representation; The fixed-point integer representation is divided into low-width segment operands according to a preset low-width segment.

9. The method according to claim 7, characterized in that, The step of allocating the intermediate multiplication results of each path to the corresponding storage bank and accumulating and updating them according to the benchmark index includes: The target storage for mapping is selected based on the benchmark exponent of each intermediate multiplication result; Locate the target accumulation entry within the target storage body, the target accumulation entry storing items... Point index, carry vector, and retention vector; Read the carry vector, retention vector, and entry exponent of the target accumulation entry; A shift amount is generated based on the difference between the benchmark index and the entry index, and the intermediate multiplication results of each path are shifted and aligned. The merged and aligned multi-path intermediate multiplication results are combined with the carry vector and the retained vector to obtain the updated carry vector and retained vector; The updated carry vector and retain vector are remapped to the target memory and written back to the target accumulation entry.

10. The method according to claim 7, characterized in that, The step of receiving the accumulated and updated result, performing floating-point format restoration processing, and generating the final single-precision floating-point data includes: Receive the updated carry vector and the retained vector, and perform carry propagation addition on them to obtain the reduction result; The reduction result is normalized to floating point based on the entry index and the benchmark index. The exponent field value is adjusted and the mantissa field is shifted to meet the normalization format requirements of single-precision floating point numbers, thereby generating an intermediate floating point result with an exponent field. The mantissa field of the intermediate floating-point result is rounded to truncate or carry over to the specified mantissa length for a single-precision floating-point number. The intermediate floating-point result is checked for abnormal data such as non-numerical, infinity, overflow, or underflow, and corresponding abnormal flag information is generated. Abnormal data is removed from the intermediate floating-point result to generate the final single-precision floating-point data.

Images