FP8 matrix multiplication and addition operation circuit based on RISC-V architecture

Abstract

The embodiment of the invention discloses an FP8 matrix multiplication and addition operation circuit based on an RISC-V framework, and relates to the technical field of processor microstructures and artificial intelligence calculation acceleration. The circuit comprises an input module used for receiving a first operand in an FP8 format, a second operand and a third operand in a preset format; the matrix multiplication and addition operation module is used for periodically carrying out row and column multiplication operation on a plurality of first floating point elements of a preset row number in the first operand and a second floating point element in the second operand in a matrix form; the accumulation operation module is used for performing accumulation calculation on the intermediate result and a third operand; the output module is used for outputting a target operand, the problem that in the prior art, FP8 matrix multiplication and addition lacks special hardware support is solved, the computing power density and the energy efficiency ratio in an AI scene are remarkably improved, high fusion with an RISC-V vector architecture can be achieved, and the method is suitable for a CPU, an AI accelerator and a heterogeneous computing platform.

Description

Technical Field

[0001] This application relates to the fields of processor microarchitecture and artificial intelligence computing acceleration technology, and in particular to an FP8 matrix multiply-accumulate circuit based on the RISC-V architecture. Background Technology

[0002] With the rapid development of deep learning and high-performance computing technologies, matrix multiplication and addition operations have become a core computational mode in AI accelerators and central processing unit vector units. In AI inference and some training scenarios, FP8 (Floating-Point 8) floating-point numbers (including E4M3 and E5M2) are used for computation, exhibiting a good balance between accuracy and energy efficiency. However, FP8 floating-point computation relies on GPUs / dedicated NPUs, making it difficult to integrate with RISC... The deep integration of the V-vector architecture has resulted in significant bottlenecks in FP8 floating-point computation in terms of computing power density, power efficiency, and storage bandwidth utilization. Summary of the Invention

[0003] To address the shortcomings of existing technologies, this application provides an FP8 matrix multiply-add operation circuit based on the RISC-V architecture, which can solve the problems of lack of dedicated hardware support, low energy efficiency, high bandwidth pressure, and scattered and inefficient special value processing in FP8 matrix multiply-add.

[0004] In a first aspect, this application provides an FP8 matrix multiply-add operation circuit based on a RISC-V architecture, comprising: The input module is configured to input operands, which include a first operand, a second operand, and a third operand. The first operand includes multiple first floating-point elements in FP8 format, the second operand includes multiple second floating-point elements in FP8 format, and the third operand includes multiple third floating-point elements in a preset format. The matrix multiplication and addition module is connected to the input module and is configured to periodically perform row and column multiplication operations on multiple first floating-point elements in a preset row number in the first operand and second floating-point elements in the second operand in the form of a matrix to obtain a fourth operand, which includes multiple fourth floating-point elements in a preset format. The accumulation operation module connects the matrix multiplication and addition operation module and the input module, and is configured to accumulate multiple fourth floating-point elements with third floating-point elements to obtain a fifth operand, which includes multiple fifth floating-point elements in a preset format. The output module is connected to the accumulation module and is configured to output the target operand based on multiple fifth floating-point elements.

[0005] In one embodiment, the FP8 matrix multiply-accumulate circuit based on the RISC-V architecture further includes a preprocessing module; The input end of the preprocessing module is connected to the input module, and the output end of the preprocessing module is connected to the matrix multiplication and addition operation module. The preprocessing module is configured to preprocess the first operand, the second operand, and the third operand respectively to obtain the preprocessed first operand, the second operand, and the third operand.

[0006] In one embodiment, the exponents of the floating-point elements in the third and fourth floating-point elements after preprocessing are represented using signed two's complement.

[0007] In one embodiment, when the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element reach the minimum negative value represented by the two's complement, independent protection processing is performed on the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element.

[0008] In one embodiment, the preprocessing module includes a splitting module, a special data detection module, and a hidden bit recovery module; Among them, the splitting module is connected to the input module, the hidden bit recovery module, the matrix multiplication and addition module and the accumulation operation module respectively; the special data detection module is connected to the input module, the hidden bit recovery module, the accumulation operation module and the accumulation operation module respectively; the hidden bit recovery module is connected to the accumulation operation module and the accumulation operation module respectively. The splitting module is configured to split the first floating-point element, the second floating-point element, and the third floating-point element into bits respectively, so as to obtain the sign bit, exponent, and mantissa in the first floating-point element, the second floating-point element, and the third floating-point element. The special data detection module is configured to perform normalization detection on the mantissas of the first floating-point element, the second floating-point element, and the third floating-point element respectively, and obtain the normalization detection results. The hidden bit recovery module is configured to recover the hidden bit of the mantissa in the first, second, and third floating-point elements based on the normalization detection results.

[0009] In one embodiment, the preprocessing module further includes a leading zero detection module, a mantissa normalization module, and an exponent adjustment module; Among them, the leading zero detection module is connected to the matrix multiplication and addition operation module, the accumulation operation module, the hidden bit recovery module and the mantissa normalization module respectively; the mantissa normalization module is connected to the matrix multiplication and addition operation module and the accumulation operation module respectively; and the exponent adjustment module is connected to the splitting module, the leading zero detection module and the matrix multiplication and addition operation module respectively. The leading zero detection module is configured to perform leading zero detection on the denormalized mantissas in the first floating-point element, the second floating-point element, and the third floating-point element respectively, and obtain the first leading zero detection result; The mantissa normalization module is configured to normalize the unnormalized mantissas in the first floating-point element and the second floating-point element based on the first leading zero detection result. Both the exponent adjustment module and the mantissa normalization module are configured to process the exponent and the denormalized mantissa in the third floating-point element based on the first leading zero detection result, so as to obtain the preprocessed third operand.

[0010] In one embodiment, the accumulation module includes an exponent comparison module, an alignment shift module, and an addition module; The exponent comparison module is connected to the preprocessing module, the alignment and shifting module, and the output module, respectively; the addition module is connected to the preprocessing module, the alignment and shifting module, and the output module, respectively. The exponent comparison module is configured to filter the largest exponent from the exponents of the floating-point elements in the preprocessed third operand and the exponents of multiple fourth floating-point elements to obtain the exponent of the fifth floating-point element. The alignment shift module is configured to right-shift and align the mantissas of multiple fourth floating-point elements based on the maximum exponent, resulting in multiple right-shifted and aligned mantissas. The addition module is configured to add the mantissa of the floating-point element in the preprocessed third operand to multiple right-shifted mantissas to obtain the mantissa of the fifth floating-point element.

[0011] In one embodiment, the output module is configured to perform leading zero detection on the mantissas of a plurality of fifth floating-point elements to obtain a second leading zero detection result, and to normalize the mantissas of the plurality of fifth floating-point elements based on the second leading zero detection result to generate the mantissa in the target operand; to adjust the exponents of the plurality of fifth floating-point elements based on the second leading zero detection result to generate the exponent in the target operand; and to output the target operand.

[0012] In one embodiment, the matrix multiply-add operation module includes a multiplexer and a multiplication operation module; The multiplexer is connected to the input module and the multiplication module, respectively, and the multiplication module is connected to the accumulation module. The multiplexer is configured to periodically output multiple first floating-point elements of a preset number of rows in the first operand; The multiplication module is configured to perform matrix multiplication on multiple first floating-point elements in a preset row number in the first operand and second floating-point elements in the second operand to obtain multiple fourth floating-point elements.

[0013] In one embodiment, the matrix multiplication and addition module further includes a processing module; The processing module is connected to the accumulation operation module; The multiplication module is configured to perform row and column multiplication on multiple first floating-point elements in a preset row number in the first operand and second floating-point elements in the second operand in the form of a matrix, to obtain multiple sixth floating-point elements; The processing module is configured to expand the mantissas of multiple sixth-point elements and align the exponents of multiple sixth-point elements to obtain a fourth operand.

[0014] Secondly, this application also provides a processor, including the FP8 matrix multiplication and addition circuit based on the RISC-V architecture provided in the first aspect.

[0015] Thirdly, this application also provides an electronic device, including the FP8 matrix multiply-add operation circuit based on the RISC-V architecture provided in the first aspect or the processor provided in the second aspect.

[0016] This application provides a RISC-V-based FP8 matrix multiplication and addition circuit, including an input module, a matrix multiplication and addition module, an accumulation module, and an output module. The input module is configured to input operands, which include a first operand, a second operand, and a third operand. The first operand includes multiple first floating-point elements in FP8 format, the second operand includes multiple second floating-point elements in FP8 format, and the third operand includes multiple third floating-point elements in a preset format. The matrix multiplication and addition module is configured to periodically multiply multiple first floating-point elements of a preset row number in the first operand with the second floating-point elements in the second operand to form a matrix multiplication and addition operation. The system performs matrix multiplication in a specific format to obtain a fourth operand, which includes multiple fourth floating-point elements in a preset format. The accumulation module is configured to accumulate the multiple fourth floating-point elements with the third floating-point element to obtain a fifth operand, which also includes multiple fifth floating-point elements in a preset format. The output module is configured to output the target operand based on the multiple fifth floating-point elements. This solves the problem of lack of dedicated hardware support for FP8 matrix multiplication and addition in the prior art, significantly improves the computing power density and energy efficiency in AI scenarios, and is highly integrated with the RISC-V vector architecture, making it suitable for CPUs, AI accelerators, and heterogeneous computing platforms. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A first schematic block diagram of the FP8 matrix multiply-add operation circuit based on the RISC-V architecture provided in the embodiments of this application; Figure 2 A second schematic block diagram of the FP8 matrix multiply-add operation circuit based on the RISC-V architecture provided in the embodiments of this application; Figure 3 A first architecture diagram of the FP8 matrix multiply-add operation circuit based on the RISC-V architecture provided in the embodiments of this application; Figure 4 This is a second architecture diagram of the FP8 matrix multiply-add operation circuit based on the RISC-V architecture provided in the embodiments of this application. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should also be further understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] Furthermore, in this application, unless otherwise explicitly specified or limited in the embodiments, the terms "installation," "connection," "joining," and "fixing" appearing in the embodiments should be interpreted broadly. For example, a connection can be a fixed connection, a detachable connection, or an integral part; it can also be a mechanical connection, an electrical connection, etc. Of course, it can also be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication between two components, or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific implementation.

[0024] In related technologies, deep learning models are continuously increasing their demands for energy efficiency and throughput in low-bit floating-point operations in scenarios such as artificial intelligence inference and training, and high-performance vector computing. Currently, FP16 or BF16 are mostly used for matrix multiplication and addition operations. Although they have high numerical accuracy, they have significant bottlenecks in terms of computing power density, power consumption control, and storage bandwidth. In particular, under the RISC-V architecture, the general floating-point pipeline is difficult to efficiently support large-scale parallel matrix operations in FP8 format. Furthermore, the handling of different FP8 encoding formats such as E4M3 / E5M2, denormalized numbers (Denorm), NaN, Inf, and exponential boundary anomalies is scattered across multiple logic units, resulting in high hardware complexity, high latency, and low energy efficiency.

[0025] Furthermore, during the accumulation process from FP8×FP8 to FP32, due to the large differences in the dynamic range of the exponent, the narrowness of the mantissa, and the tight coupling between rounding and normalization, precision loss or overflow errors are prone to occur, and there is a lack of a unified and predictable hardware co-processing mechanism.

[0026] To address this, this application provides an FP8 matrix multiplication and addition circuit based on the RISC-V architecture, which achieves efficient hardware implementation of FP8 matrix multiplication and addition operations, significantly improving computing power density and energy efficiency in AI scenarios. It natively supports FP8 dual formats, reducing data migration and bandwidth pressure. It is highly integrated with the RISC-V vector architecture, providing unified and predictable hardware processing for FP8 special values ​​and exponential boundaries, and is suitable for CPUs, AI accelerators, and heterogeneous computing platforms.

[0027] In the FP8 matrix multiply-add circuit based on the RISC-V architecture, three operands are received through the input module, including the first operand in FP8 format, the second operand in FP8 format, and the third operand in a preset format, thus realizing efficient processing of low-bit floating-point data.

[0028] This application utilizes a matrix multiplication and addition module to periodically perform matrix multiplication operations on multiple first floating-point elements in a preset number of rows of the first operand and second floating-point elements in the second operand, generating intermediate results. This addresses the lack of dedicated hardware support for FP8 matrix multiplication and addition, significantly improving computational efficiency. The accumulation module adds the intermediate results to the third operand, achieving precision conversion from FP8 to FP8 to FP32, effectively avoiding the accumulation of numerical errors due to insufficient precision. The output module outputs the target operand based on the accumulated results, ensuring the completeness and usability of the computational results.

[0029] Furthermore, this application also uses a preprocessing module to perform format adaptation and prenormalization on the input operands, solving the problem of handling special values ​​such as denormalized numbers, NaN, and Inf in FP8 data, and ensuring the numerical consistency of the calculations. At the same time, the exponents of the floating-point elements in the third and fourth floating-point elements of the preprocessed operands are represented using signed two's complement, simplifying the exponent comparison logic, supporting wide dynamic range mapping from FP8 to FP32, and ensuring correct alignment even in the smallest exponent scenario.

[0030] Meanwhile, independent protection is performed on the minimum negative exponent, eliminating the risk of numerical collapse under the worst boundary and ensuring the determinism and verifiability of the calculation results in the entire input domain.

[0031] The collaborative work of the splitting module, special data detection module, and hidden bit recovery module enables unified parsing of FP8 dual formats, explicitly exposing hidden bits and avoiding loss of mantissa precision. The cooperation of the leading zero detection module, mantissa normalization module, and exponent adjustment module eliminates computational redundancy caused by denormalized numbers, unifies the exponent reference system, and ensures accurate accumulation alignment.

[0032] The combination of the exponent comparison module, alignment shift module, and addition operation module ensures that the numerical accuracy of the accumulation process is not degraded, supports multi-channel parallel accumulation, and is compatible with the GRS bit generation required for RISC-V rounding mode. Simultaneously, the output module can also perform leading zero detection, normalization shift, and dynamic exponent adjustment to ensure that the output strictly conforms to the IEEE 754 FP32 format, supports five RISC-V rounding modes, and automatically sets the exception flag.

[0033] The multiplexer and multiplication module work together to achieve high-throughput matrix multiplication, supporting periodic issuance of RISC-V vector instructions and reducing the temporary storage area for multiplication results. The processing module performs mantissa expansion and exponent alignment on the floating-point elements output by the multiplication module, eliminating the data format gap between the multiplication and accumulation modules and avoiding truncation errors caused by bit width mismatch.

[0034] This application achieves efficient hardware implementation of FP8 matrix multiplication and addition operations through the organic combination of the above-mentioned technical features, significantly improving the computing power density and energy efficiency ratio in AI scenarios. It natively supports FP8 dual formats, reduces data migration and bandwidth pressure, and is highly integrated with the RISC-V vector architecture. It provides unified and predictable hardware processing for FP8 special values ​​and exponential boundaries, and is suitable for CPUs, AI accelerators and heterogeneous computing platforms.

[0035] Please see Figure 1 , Figure 1 This is a first schematic block diagram of an FP8 matrix multiply-add operation circuit based on the RISC-V architecture provided in an embodiment of this application. Figure 1As shown, this application provides an FP8 matrix multiply-add operation circuit based on the RISC-V architecture, including: The input module 110 is configured to input operands. The operands include a first operand, a second operand, and a third operand. The first operand includes multiple first floating-point elements in FP8 format, the second operand includes multiple second floating-point elements in FP8 format, and the third operand includes multiple third floating-point elements in a preset format. The matrix multiplication and addition module 120 is connected to the input module 110 and is configured to periodically perform row and column multiplication operations on multiple first floating-point elements in a preset row number of the first operand and second floating-point elements in the second operand in the form of a matrix to obtain a fourth operand, the fourth operand including multiple fourth floating-point elements in a preset format. The accumulation operation module 130 is connected to the matrix multiplication and addition operation module 120 and the input module 110, and is configured to accumulate multiple fourth floating-point elements with third floating-point elements to obtain a fifth operand, the fifth operand including multiple fifth floating-point elements in a preset format; Output module 140 is connected to accumulation module 130 and is configured to output target operands based on multiple fifth floating-point elements.

[0036] Specifically, this application provides a hardware native adaptation approach for the RISC-V Vector Extension (RVV) instruction set. Specifically, it can take FP32+=FP8×FP8 as the basic computation paradigm, modularly decouple and periodically pipeline the input data stream, matrix operation structure, accumulation alignment logic and output regularization process, and achieve high throughput, low power consumption and strong robust FP8 matrix multiplication and addition hardware acceleration while ensuring IEEE 754 compatibility and RISC-V rounding semantic consistency.

[0037] The input module 110 can be understood as an interface circuit for receiving three 512-bit wide data streams from the RISC-V vector register group (such as v0–v31), and its input port supports data transfer triggered by RISC-V vector loading instructions (such as vlw.v).

[0038] Both the first and second operands are in FP8 format, and the FP8 format includes either E4M3 format (1 sign bit + 4 exponent + 3 mantissa, exponent bias of 7) or E5M2 format (1 sign bit + 5 exponent + 2 mantissa, exponent bias of 15). The two formats can be dynamically switched using a control signal. The first and second operands can be understood as... Figure 4 The opa and opb in the text.

[0039] The third operand is in FP32 format, which corresponds to the RISC-V standard single-precision floating-point format (IEEE 754), including 1 sign bit, 8 exponent bits, and 23 mantissa bits.

[0040] Specifically, the input module 110 contains a buffer register array to temporarily store the block data of each operand in order to match the periodic processing rhythm of subsequent modules. The buffer depth can be set to 2–4 cycles according to the actual timing constraints, such as a double buffer structure or a four-cycle FIFO.

[0041] The matrix multiplication and addition module 120 can be understood as the core computing unit that generates intermediate results from FP8×FP8 to FP32. It is configured to use a matrix size of (8×8)×(8×8) to (8×8) as the basic computing granularity and completes the calculation of all 64 output elements through a 4-cycle time-sharing multiplexing structure.

[0042] Among them, such as Figure 3 As shown, each cycle processes 2 rows of results, and each row contains 8 FP8×FP8 multiplications. The matrix multiplication and addition module 120 may include a multiplexer and an FP8 multiplication module. The multiplexer periodically selects 16 FP8 elements from the first operand with a preset number of rows (e.g., 2 rows) and pairs them with the 8 FP8 elements in the corresponding column of the second operand.

[0043] The FP8 multiplication module performs floating-point multiplication on each pair of FP8 elements and outputs an intermediate result with an extended bit width. Its sign bit is obtained by XORing the two input sign bits, the exponent is obtained by subtracting the corresponding bias from the sum of the two input exponents and correcting it with leading zeros, and the mantissa is obtained by multiplying the two input mantissas (including the hidden bit recovery), and is extended to at least 16 bits.

[0044] The extended result format can be Sgn(1) + Exp(10) + Man(24), where Exp(10) is represented by signed two's complement to cover the wide exponent range [-11, 23] or [-17, 47] that may occur after multiplying E4M3 and E5M2.

[0045] The accumulation operation module 130 can be understood as a dedicated circuit for aligning the intermediate results of FP32 with the initial accumulated value (third operand), summing, and detecting anomalies; its input terminals receive the fourth operand (512 FP32 elements) from the matrix multiplication and addition operation module 120 and the third operand (64 FP32 elements) from the input module 110, respectively.

[0046] The accumulation operation module 130 may include an exponent comparison module, an alignment shift module, and an addition operation module; the exponent comparison module is configured to compare the exponent of a fourth floating-point element corresponding to each output position with the exponent of a third floating-point element, and select the largest exponent between the two as the exponent benchmark of the fifth floating-point element at that position.

[0047] The alignment shift module is configured to right-shift the mantissa of the operand corresponding to the smaller exponent based on the largest exponent. The number of right shifts is equal to the exponent difference, and guard, round, and sticky bits are generated in the shifted-out low bits, i.e., G bits / R bits / S bits.

[0048] The addition module is configured to perform signed addition on the nine aligned mantissas (including sign extension) and output the mantissa of the fifth floating-point element. The addition module supports a NaN propagation mechanism: if any input is DQNaN, it directly outputs DQNaN without performing subsequent alignment and addition; its hardware implementation uses a dual-path structure—the normal path performs aligned addition, while the abnormal path directly leads to NaN / Inf, reducing critical path latency.

[0049] The output module 140 can be understood as a final processing unit that performs normalization, rounding, overflow judgment and format encapsulation on the fifth floating-point element; the input terminal of the output module 140 can receive 64 FP32 elements of the fifth operand; at the same time, the input module 110 includes a leading zero detection module, a normalization shift module, a rounding module and an exception flag generation module.

[0050] The leading zero detection module is configured to perform a leading zero count (LZC) on the mantissa of each fifth floating-point element to determine the number of left-normalized bits; the normalization shift module is configured to perform a left shift on the mantissa based on the LZC result, so that the most significant bit is at the 23rd bit (i.e., in the form of 1.xxxx), and at the same time perform a corresponding subtraction adjustment on the exponent based on the number of left-shifted bits.

[0051] The rounding module is configured to generate a rounding increment based on the GRS bit and the current RISC-V rounding mode (RNE / RTZ / RDN / RUP / RMM), and add it to the normalized mantissa. If a carry occurs, a second right rounding is triggered (Exp+1, Man shifts right by 1 bit).

[0052] The exception flag generation module is configured to output five types of exception flags: NV, DZ, OF, UF, and NX, based on the final exponent and mantissa combination. The target operands output by the exception flag generation module are in standard FP32 format, which can be directly written back to the RISC-V vector register file and supports write-back using memory instructions such as vsw.v.

[0053] Specifically, this application constructs a fully pipelining hardware paradigm for FP8 matrix multiplication and addition for RISC-V architecture. The input module 110 serves as the data entry point, and the matrix multiplication and addition module 120 performs highly parallel FP8×FP8 multiplication. The accumulation module 130 then completes the exponent alignment and mantissa accumulation for FP32 precision. Finally, the output module 140 performs normalization, rounding, and anomaly detection. The modules can use a fixed-period handshake protocol to support continuous vector block computation and avoid cross-period state dependencies. Mechanisms such as FP8 format recognition, fast bypassing of special values, independent protection of the minimum exponent, and G-bit / R-bit / S-bit generation and rounding linkage are all embedded within the corresponding modules without introducing additional control overhead.

[0054] In this embodiment, when the RISC-V processor executes the vwmacc.vv ​​instruction, the input module 110 loads the first operand (vs1), the second operand (vs2), and the third operand (vs3) in parallel from the vector register. All three operands are 512 bits wide. The matrix multiply-add module 120 is scheduled by cycle. In the first cycle, it reads rows 0-1 of vs1 (16 FP8s) and all 8 columns of vs2 (64 FP8s). After multiplexing and multiplication array, it generates 128 FP8×FP8 products. The process yields 128 intermediate FP32 results (i.e., fourth floating-point elements). The accumulation module 130 simultaneously receives these 128 intermediate FP32 results and the 16 initial FP32 values ​​at the corresponding positions in vs3, performs exponent comparison, right shift alignment, and mantissa addition, generating 16 fifth floating-point elements. The output module 140 performs LZC detection, left normalization, rounding, and overflow judgment on these elements, generating 16 target FP32 results and writing them to the destination register. Subsequent cycles process the remaining lines sequentially, completing all 64 outputs in 4 cycles. Throughout the entire process, automatic FP8 format adaptation, Denorm front normalization, NaN pass-through, and minimum exponent protection are all handled autonomously by hardware without software intervention.

[0055] As an example, taking the convolutional kernel weight matrix (8×8) and activation feature matrix (8×8) of a certain layer in the ResNet-50 model as an example, both are quantized in E4M3 format and stored in a vector register; the third operand is the previous accumulation result of this layer, stored in FP32 format; after the circuit starts, the input module 110 loads vs1[0:1] (weights in rows 0–1), vs2 (all 8 columns of features), and vs3[0:1] (initial accumulation values ​​in rows 0–1) in cycle0; the matrix The multiply-add module 120 calculates 128 8-element dot products and outputs 128 FP32 intermediate values; the accumulation module 130 adds these 128 values ​​to the corresponding elements of vs3[0:1] respectively; the output module 140 completes normalization and rounding and outputs 16 FP32 results to vd[0:1]; cycle1–3 similarly completes the calculation of the remaining rows, and finally obtains a complete 8×8 FP32 output matrix in vd. The whole process only requires 4 vector instruction cycles and has no additional scalar intervention.

[0056] In this application, the input module 110 supports dynamic switching between E4M3 and E5M2 dual FP8 formats, thereby enabling compatibility with the different requirements of various AI models for numerical accuracy and dynamic range, thus improving hardware versatility; the matrix multiplication and addition module 120 adopts a 4-cycle time-sharing multiplexing structure, which can control the total latency of (8×8)×(8×8) matrix multiplication and addition within 4 cycles while maintaining a 512-bit vector bandwidth, significantly improving the computing power density per unit area.

[0057] Meanwhile, the accumulation module 130 has built-in exponent comparison and alignment shift hardware, and the output module 140 integrates LZC and rounding linkage logic, which avoids the overhead of multiple memory accesses or calls to scalar units for exponent adjustment in traditional solutions, and reduces overall power consumption.

[0058] In addition, each module sets up independent hardware paths for NaN, Inf, Denorm, and the minimum exponent, thus achieving deterministic exception handling and predictable timing, and meeting the strict consistency requirements of RISC-V vector extension for floating-point semantics. All modules are designed based on the RISC-V vector register interface and can be seamlessly integrated into the RISC-V CPU core or AI coprocessor, thereby providing underlying hardware support for the RVV instruction set.

[0059] In some embodiments, such as Figure 2As shown, the FP8 matrix multiplication and addition circuit based on the RISC-V architecture also includes a preprocessing module 150; wherein, the input terminal of the preprocessing module 150 is connected to the input module 110, and the output terminal of the preprocessing module 150 is connected to the matrix multiplication and addition module 120; the preprocessing module 150 is configured to preprocess the first operand, the second operand, and the third operand respectively to obtain the preprocessed first operand, the second operand, and the third operand.

[0060] In this embodiment, the preprocessing module 150 can be understood as a hardware logic unit that performs unified format adaptation, numerical normalization and anomaly prediction on the original input operands before the FP8 matrix multiplication and addition operation is started.

[0061] The preprocessing module 150 can convert raw data that cannot be directly used for high-precision accumulation, such as heterogeneous formats (FP8 format E4M3 / E5M2 and FP32), denormalized numbers (Denorm), hidden bit missing, and special values ​​(NaN / Inf / Zero), into intermediate representations that meet the stable operation requirements of subsequent matrix multiplication and accumulation modules.

[0062] Meanwhile, the preprocessing module 150 and the input module 110 form a serial data path, and its output strictly serves the pre-constraints proposed by the matrix multiplication and addition module 120 on operand format, exponent alignment and mantissa integrity.

[0063] The first operand is a multiplicand in FP8 format, such as 8-bit floating-point data encoded in E4M3 or E5M2. Before entering the preprocessing module 150, it has not undergone hidden bit recovery, exponent offset alignment, or normalization of denormalized numbers. The preprocessing module 150 can perform bit splitting, normalization detection, hidden bit recovery, and optional exponent adjustment on the first operand, and then output the preprocessed first operand. In the output data, the mantissa has been padded with hidden bits, the exponent has been mapped to a uniform extended bit width (not less than 10 bits), and the denormalized number has been left-normalized and the exponent has been corrected synchronously, thereby ensuring that it can perform lossless row and column multiplication operations with other operands in the matrix multiplication and addition operation module 120.

[0064] The second operand is a multiplier in FP8 format, with the same structure as the first operand, and can be in E4M3 or E5M2 format. The preprocessing module 150 can perform a preprocessing process on the second operand that is completely symmetrical to that on the first operand, including sign / exponent / mantissa separation, denorm detection and normalization, hidden bit insertion, and exponent field expansion. This ensures that the mantissa precision and exponent dynamic range of the second operand are consistent with those of the first operand. The two operands can achieve element-wise alignment and matching during the matrix multiplication stage, avoiding product truncation or overflow caused by asymmetry in mantissa bit width or difference in exponent bias.

[0065] The third operand is an initial value of accumulation in FP32 format, and its original exponent and mantissa bit width are significantly higher than those of the FP8 operand. The preprocessing module 150 performs exponent field remapping and mantissa zero extension on the third operand to keep its exponent representation compatible with the intermediate results generated by the first and second operands after preprocessing (such as Sgn(1) + Exp(10) + Man(24)).

[0066] Specifically, the preprocessing module 150 can map the 8-bit exponent of FP32 to a 10-bit signed two's complement representation through bias conversion, and extend the 23-bit mantissa by padding the high bits with zeros to at least 16 bits. This ensures that it can achieve bit-level alignment with the fourth operand (i.e., the intermediate result of FP32 output by matrix multiplication) in the subsequent accumulation operation module 130 in the exponent comparison, right shift alignment and mantissa addition reduction stages, preventing the loss of low-bit information or rounding deviation caused by bit width mismatch.

[0067] The preprocessing module 150 operates as follows: When the input module 110 sends the first operand, the second operand, and the third operand into the preprocessing module 150 in parallel, the module first obtains the sign bit, original exponent, and original mantissa of each operand through bit splitting logic. Subsequently, the special data detection module performs NaN / Inf / Zero / Denorm identification on the three data streams respectively. For data identified as Denorm, the leading zero detection module outputs the number of leading zeros, and the mantissa normalization module performs a left shift on the mantissa accordingly, while the exponent adjustment module synchronously corrects the exponent. For normalized data, the hidden bit recovery module inserts a hidden bit 1 before the most significant bit of the mantissa. Finally, all three operands are converted into an intermediate data stream with a unified exponent representation width (10-bit signed two's complement), an extended mantissa bit width (≥16 bits), and a standardized sign processing mechanism, and are synchronously output to the corresponding input port of the matrix multiplication and addition module 120 according to the periodic beat.

[0068] As an example, when the RISC-V vector instruction triggers the operation fp32+=fp8×fp8, vector register v0 provides 64 bytes of FP8 multiplicand (8×8 E4M3), v1 provides 64 bytes of FP8 multiplier (8×8 E5M2), and v2–v5 provide a total of 256 bytes of FP32 accumulation initial value (4×64 bytes). After receiving the data from v0 / v1 / v2, the preprocessing module 150 completes the parallel splitting and denorm detection of the three data streams in the first cycle; the second cycle starts the hidden bit recovery and mantissa normalization; the third cycle completes the exponent remapping and mantissa expansion; and the fourth cycle outputs the three preprocessed data streams to the matrix multiplication and addition module 120 in the form of a 512-bit wide bus. At this time, the mantissas of the first and second operands are both 16 bits and the exponents are both 10-bit signed two's complements, while the mantissa of the third operand is 24 bits and the exponent is 10-bit signed two's complements, all of which meet the consistency requirements of the matrix multiplication and addition and accumulation modules for the input data format.

[0069] In this application, by adding a preprocessing module 150 and using the preprocessing module 150 to unify the execution format and regularize the values ​​of the three operands, the precision of the mantissa and the compatibility of the exponent dynamic range are ensured between the first operand and the second operand in the FP8×FP8 multiplication process.

[0070] Meanwhile, the preprocessing module 150 performs exponent remapping and mantissa expansion on the third operand, which enables the initial value of FP32 accumulation to be aligned with the intermediate result of FP8 multiplication output without error in subsequent accumulation stages.

[0071] In addition, the preprocessing module 150 has built-in special data detection and independent protection logic, which can intercept Denorm, NaN and extremely small exponent scenarios in advance to avoid them causing chain anomalies in the main matrix multiplication and addition path, thereby improving the numerical robustness and operational predictability of the entire FP8 matrix multiplication and addition operation circuit.

[0072] In some embodiments, the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element are represented using signed two's complement.

[0073] In this embodiment, the exponent of the floating-point element in the third operand after preprocessing can be understood as the exponent value obtained after the preprocessing module 150 completes the format conversion, normalization and bias adjustment. The exponent value is represented in signed two's complement form with a bit width of not less than 10 bits, which can provide the subsequent exponent alignment module with a data format that can directly participate in arithmetic comparison and difference operation, avoiding the maximum value selection error caused by the distortion of negative exponent mapping under unsigned encoding. At the same time, the exponent and the exponent of the fourth floating-point element maintain the same format in the data path, and the two are jointly input to the exponent comparison module, forming a unified input basis for the exponent alignment operation.

[0074] In the preprocessed third operand, the exponent of the floating-point element is an integer value in two's complement format, with the highest bit being the sign bit and the remaining bits being the numeric bits. This does not change the mathematical meaning of the exponent of the original operand, but only its hardware encoding form. At the same time, it can also be synchronously converted to two's complement with the exponent of the fourth floating-point element in the circuit and sent to the exponent comparison module through the same bus structure. This allows the exponent comparison module to reuse the standard signed comparator unit without the need for additional sign judgment logic, thereby reducing the complexity of the control logic. In addition, this two's complement representation supports the direct calculation of negative exponent difference, which can provide a deterministic basis for the subsequent right shift alignment amount (i.e., exponent difference).

[0075] The exponent of the fourth floating-point element is the exponent part of the intermediate result generated by the matrix multiplication and addition module 120 after performing FP8×FP8 multiplication on the first operand and the second operand. Before output, the exponent is encoded in two's complement by the exponent adjustment module in the preprocessing module 150, and its bit width can be dynamically adapted according to the FP8 format type.

[0076] When the input is in E4M3 format, the two's complement exponent has a bit width of 10 bits; when the input is in E5M2 format, the two's complement exponent also has a bit width of 10 bits. Format compatibility can be achieved through high-bit sign extension. The exponents of multiple fourth floating-point elements and the exponents of floating-point elements in the preprocessed third operands jointly participate in the maximum exponent selection. Their numerical relationship directly determines the number of bits that the mantissas need to be shifted to the right. At the same time, since both use signed two's complement representation, the exponent difference can be directly output by the subtractor without the need for conditional branches or absolute value correction, which significantly improves the timing robustness of the alignment path.

[0077] Specifically, before the preprocessing module 150 sends the third operand to the accumulation module 130, and before the matrix multiplication and addition module 120 outputs the fourth operand, the preprocessing module 150 performs two's complement encoding on the exponent field: for the original exponent e4∈[0, 15] (with bias) in E4M3 format, it can be first restored to the unbiased exponent e4′=e4-7∈[-6, 8], and then encoded with 10-bit signed two's complement; for the original exponent e5∈[0, 31] (with bias) in E5M2 format, it is first restored to the unbiased exponent e5′=e5-15∈[-14, 16], and then encoded with 10-bit signed two's complement.

[0078] The encoding process can be completed collaboratively by the exponent adjustment module in the preprocessing module 150 and the exponent encoding module inside the matrix multiplication and addition module 120. The two modules are symmetrical in structure and aligned in timing, which can ensure that all exponent data entering the accumulation path has a unified two's complement semantics.

[0079] As an example, in an (8×8)×(8×8) matrix multiplication and addition operation, assuming that the first and second operands both use E5M2 format, the exponent of the fourth floating-point element output from a pair of FP8 element multiplications is -17 (corresponding to two's complement 10'b1110110111), while the exponent of the corresponding floating-point element in the preprocessed third operand is -12 (corresponding to two's complement 10'b1111010100). The exponent comparison module performs a signed comparison on these two sets of 10-bit two's complement data, identifies that -12>-17, and therefore selects -12 as the maximum exponent at that position. Subsequently, the alignment and shift module calculates the exponent difference Δ=5 based on this and performs a 5-bit right shift on the mantissa of the fourth floating-point element to complete the alignment. The entire process does not introduce any sign bit discrimination logic or conditional jumps, and is entirely implemented based on the standard two's complement arithmetic unit, meeting the requirements of RISC-V hardware pipelining for low latency and high determinism.

[0080] In this application, the exponents of the floating-point elements in the third and fourth floating-point elements after preprocessing are both represented using signed two's complement. The exponent comparison module can directly perform signed numerical comparisons, avoiding the problem of negative exponents being misjudged as extremely positive numbers under unsigned encoding. Two's complement representation naturally supports negative difference operations, and the alignment shift module can directly output the right shift number using a subtractor, eliminating the need for additional absolute value circuits and multiplexers. The exponents of the two FP8 formats, E4M3 and E5M2, can be mapped to a unified numerical range in the two's complement domain. The same set of exponent alignment hardware can be compatible with dual-format input, improving circuit reuse and architectural simplicity.

[0081] In some embodiments, when the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element reach the minimum negative value represented by the sign complement, independent protection processing is performed on the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element.

[0082] In this embodiment, the exponents of the floating-point elements in the third and fourth floating-point elements after preprocessing are represented using signed two's complement, with a bit width of not less than 10 bits. For example, it can be 10, 11, or 12 bits. The specific bit width can be set by balancing the precision requirements of FP8 operations in RISC-V vector extension with hardware resource constraints.

[0083] When the exponent of a floating-point element reaches the smallest negative value represented by the sign complement, it corresponds to the extreme form of all 1s followed by 0s in the complement encoding, such as -512 in 10-bit complement (i.e., binary 1000000000). Its value is usually subjected to conventional arithmetic operations such as negation, left shift (for normalization), addition (for exponent alignment offset), or maximum value filtering by the comparator. It is very easy to cause logical errors or undefined behavior due to overflow.

[0084] The minimum negative value is identified as a boundary exponent state requiring special handling in this application. Its functional meaning is to characterize the deepest underflow exponent level that the FP8 denormalized number may reach after preprocessing. It is named based on the fact that it constitutes a systematic risk source in the exponent alignment and normalization link.

[0085] Independent protection processing can be understood as setting up an extreme value detection and response module in the exponential data path that is independent of the main operation path. This module does not participate in regular arithmetic operations, but only performs pattern matching and condition replacement on the input exponential value.

[0086] When the exponents of the third and fourth floating-point elements in the currently processed operand reach the minimum negative value represented by the sign complement, an overflow occurs. This forms an action path with the exponent comparison module and the normalization module. That is, if the input contains the minimum negative value during the process of selecting the maximum exponent, the exponent comparison module must avoid misjudging it as the valid maximum value. If the input contains the minimum negative value during the left / right normalization adjustment, the normalization module must prevent the output of an illegal FP32 exponent due to exponent subtraction overflow. Through the above cooperation, even under the most unfavorable input combination (such as full Denorm input with multiple levels of accumulation), the controllability of the exponent field and the physical consistency of mantissa alignment can be maintained, thereby forming a stable and predictable floating-point operation state flow.

[0087] During the independent protection process of the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element, when it is detected that the exponent of any floating-point element in the preprocessed third operand or the exponent of any floating-point element in the fourth floating-point element is equal to the smallest negative value in the signed two's complement representation, a dedicated protection path is activated. This protection path does not rely on general arithmetic logic units, but is implemented through hard-wired logic.

[0088] Specifically, in the exponent comparison module, the smallest negative value can be set as an invalid candidate, forcibly excluding it from participating in the maximum value competition; in the alignment shift module, when the smallest negative value is used as the minuend in the exponent difference calculation, a fixed safety offset (e.g., 0 or 1) can be clamped to output instead of performing actual subtraction; in the normalization module, when the smallest negative value is used as the input exponent in the left normal exponent correction, the exponent update can be frozen and a preset compensation value can be enabled (e.g., the exponent can be forcibly set to -126 to match the lower limit of the FP32 minimum positive normalized number exponent).

[0089] In addition, when the exponent comparison module selects the largest exponent from the exponents of the floating-point elements in the preprocessed third operand and the exponents of multiple fourth floating-point elements, if any input is the smallest negative value, the input is automatically masked by the logic gate circuit, and only the valid exponents participate in the comparison. Subsequently, the alignment and shift module calculates the right shift number of the mantissa of each fourth floating-point element based on the selected largest exponent. At this time, if the exponent of a certain fourth floating-point element was originally the smallest negative value, its right shift number is forced to be set to the maximum allowed right shift amount (e.g., 23 bits), and its mantissa high bits are padded with the sign bit to maintain the numerical meaning. Thus, even under extreme underflow input conditions, the accumulation operation can still generate an intermediate result with physical interpretability, rather than triggering an abnormal interrupt or uncontrollable output.

[0090] In this application, when the minimum negative value occurs, protection logic independent of the general arithmetic path is enabled to avoid alignment failure caused by exponent inversion or subtraction overflow. At the same time, the protection logic can form a closed loop with the exponent comparison, alignment shift and normalization modules, thereby ensuring the numerical robustness of FP8 matrix multiplication and addition operations in the entire input domain. In addition, the protection action can be implemented based on hardware hardwired without software intervention, without introducing additional timing overhead, and maintaining the single-cycle issue characteristic of RISC-V vector instructions.

[0091] In some embodiments, the preprocessing module 150 includes a splitting module, a special data detection module, and a hidden bit recovery module. The splitting module is connected to the input module 110, the hidden bit recovery module, the matrix multiplication and addition module 120, and the accumulation module 130. The special data detection modules are all connected to the input module 110, the hidden bit recovery module, the accumulation module 130, and the accumulation module 130. The hidden bit recovery module is connected to the accumulation module 130 and the accumulation module 130. The splitting module is configured to split the first floating-point element, the second floating-point element, and the third floating-point element into bits to obtain the sign bit, exponent, and mantissa of the first, second, and third floating-point elements. The special data detection module is configured to perform normalization detection on the mantissa of the first, second, and third floating-point elements to obtain a normalization detection result. The hidden bit recovery module is configured to recover the hidden bits of the mantissa of the first, second, and third floating-point elements based on the normalization detection result.

[0092] In this embodiment, the splitting module can be understood as a logic circuit unit used to perform physical bit field separation on the input FP8 format floating-point elements, and is configured to parse each FP8 floating-point element into independent sign bit (S), exponent bit (E) and mantissa bit (M) according to a preset format.

[0093] Among them, the FP8 format includes the E4M3 format (1bitS+4bitE+3bitM) and the E5M2 format (1bitS+5bitE+2bitM).

[0094] The split module can be a combinational logic circuit, without introducing timing delays, and its output signal lines can correspond to three sets of parallel buses: S, E, and M.

[0095] Specifically, the input port of the splitting module interfaces with the 512-bit FP8 vector interface of the input module 110, supporting parallel splitting of 64 FP8 elements simultaneously. The sign bit, exponent bit, and mantissa bit output by the splitting module are transmitted to the special data detection module and the hidden bit recovery module, respectively. The splitting module can provide a structured input foundation for subsequent semantic recognition and numerical reconstruction. It and the input module 110 form the first stage of the preprocessing link, and it forms a data-driven relationship with the special data detection module. That is, only after the bit field splitting is completed can the subsequent modules accurately identify the meaning of each field, so that the FP8 dual-format input can be uniformly parsed in the same hardware path, avoiding the increase in control complexity caused by format discrimination in advance.

[0096] The special data detection module can be understood as a dedicated detection circuit used to perform semantic classification on the mantissa obtained by splitting. It is configured to determine the type of the current floating-point element based on the characteristics of the mantissa value and the corresponding exponent state, including five categories: normalized number (Norm), denormalized number (Denorm), zero value (Zero), infinity (Inf), and not-a-number (NaN).

[0097] The detection logic of the special data detection module is based on the FP8 encoding rules defined in [the documentation]. For the E4M3 format, when the exponent E=0 and the mantissa M=0, it is determined as Zero; when E=0 and M≠0, it is determined as Denorm; when E∈[1, 14], it is determined as Norm; when E=15, it is further distinguished as Inf (M=0) and NaN (M≠0) based on the M value. For the E5M2 format, when E=0 and M=0, it is determined as Zero; when E=0 and M≠0, it is determined as Denorm; when E∈[1, 30], it is determined as Norm; when E=31, it is distinguished as Inf (M=0) and NaN (M≠0) based on the M value.

[0098] The hidden bit recovery module can be understood as a numerical reconstruction unit that pads the hidden bits in the lower bits of the mantissa based on the normalization detection result. It is configured as follows: when the detection result is a normalized number (Norm), a hidden bit of 1 is added before the most significant bit of the mantissa to form a complete normalized mantissa; when the detection result is a denormalized number (Denorm), a hidden bit of 0 is added before the most significant bit of the mantissa to form a complete denormalized mantissa; when the detection result is Zero, Inf, or NaN, the original value of the mantissa is kept unchanged or set to zero.

[0099] The hidden bit recovery module can be composed of a multiplexer and splicing logic. Its inputs include the original mantissa output by the splitting module, the type code output by the special data detection module, and control signals. Its output is the extended mantissa (e.g., E4M3 format extended to 4-bit mantissa, E5M2 format extended to 3-bit mantissa), and the corresponding exponent value is updated synchronously (Denorm needs to correct the exponent according to the number of leading zeros).

[0100] The hidden bit recovery module can provide mantissa input with full precision expression capabilities for subsequent multiplication operations, exponent alignment and normalization processing; the hidden bit recovery module and the special data detection module constitute the third level of the preprocessing link, realize the conversion from semantic class to computable value, and ensure that the FP8 mantissa has a unified bit width and clear hidden bit semantics before entering the main operation path, avoiding multiplication precision loss or reduction logic error caused by the absence of hidden bits.

[0101] In this application, the splitting module, the special data detection module, and the hidden bit recovery module constitute a three-level pipelined preprocessing chain. The splitting module first completes the physical decoupling of S / E / M of FP8 elements. The special data detection module determines the semantic type based on the joint E and M. The hidden bit recovery module dynamically injects hidden bits according to the type code and outputs the normalized mantissa. This allows the same hardware circuit to process the two FP8 formats, E4M3 and E5M2, without ambiguity. It also provides structurally consistent, semantically clear, and numerically complete input data for the subsequent matrix multiplication and addition module 120 and accumulation module 130. This avoids changing the meaning of the original data values ​​and only completes the format parsing and numerical explicitation. All operations are completed in a single cycle and do not affect the overall pipeline depth.

[0102] In some embodiments, the preprocessing module 150 further includes a leading zero detection module, a mantissa normalization module, and an exponent adjustment module; wherein, the leading zero detection module is connected to the matrix multiplication and addition module 120, the accumulation module 130, the hidden bit recovery module, and the mantissa normalization module, respectively; the mantissa normalization module is connected to the matrix multiplication and addition module 120 and the accumulation module 130, respectively; and the exponent adjustment module is connected to the splitting module, the leading zero detection module, and the matrix multiplication and addition module 120, respectively; the leading zero detection module is configured to perform leading zero detection on the unnormalized mantissas in the first floating-point element, the second floating-point element, and the third floating-point element, respectively, to obtain a first leading zero detection result; the mantissa normalization module is configured to normalize the mantissas in the first floating-point element and the second floating-point element based on the first leading zero detection result; and both the exponent adjustment module and the mantissa normalization module are configured to process the exponent and the unnormalized mantissa in the third floating-point element based on the first leading zero detection result, respectively, to obtain the preprocessed third operand.

[0103] In this embodiment, the leading zero detection module can be understood as a logic unit used to perform a leading zero count (LZC) operation on the denormalized mantissa of the input floating-point element. Its output is a first leading zero detection result representing the number of consecutive zeros in the high bits of the denormalized mantissa. The first leading zero detection result is an unsigned integer, and the bit width is set according to the minimum supported denormalized mantissa length.

[0104] The leading zero detection module is responsible for the identification and quantization of denormalized numbers, and its detection results serve as the control basis for subsequent normalization shift and exponential compensation.

[0105] Meanwhile, the leading zero detection module is connected to the splitting module, hidden bit recovery module, mantissa normalization module, and matrix multiplication and addition module 120 via a multi-channel data bus and control signal line. The first leading zero detection result output by the module is synchronously distributed to the mantissa normalization module and the exponent adjustment module to achieve coordinated response of the three floating-point element processing paths. This allows the denormalized mantissa to complete the valid bit positioning before entering the multiplication operation, avoiding invalid low bits from participating in the operation, thereby reducing power consumption and improving computational efficiency.

[0106] The mantissa normalization module can be understood as performing a left shift alignment operation on the denormalized mantissa based on the first leading zero detection result, and simultaneously updating the combinational logic circuit of the implicit bits of the corresponding floating-point element and the mantissa precision representation.

[0107] The mantissa normalization module can be a pure combinational logic structure or it can include a first-level register for timing constraint optimization. Its inputs include the original mantissa from the splitting module, the normalization status flag from the special data detection module, and the first leading zero detection result from the leading zero detection module. Its output is the normalized mantissa and the corresponding normalization offset.

[0108] The mantissa normalization module can be connected to the leading zero detection module, the matrix multiplication and addition module 120, and the accumulation module 130 via a width-matched data path.

[0109] In this application, the mantissa normalization module is used to perform normalization processing on the unnormalized mantissas in the first floating-point element and the second floating-point element, so that they are uniformly converted into a normalized representation in the form of 1.xxxx, and the hidden bits are explicitly restored; it works in conjunction with the leading zero detection module to form the core execution unit for the Denorm to Norm conversion, so that FP8 multiplication operations can be performed under the premise of uniform normalization, ensuring the consistency of the product mantissa precision and the predictability of the exponent calculation.

[0110] The exponent adjustment module can be understood as an arithmetic logic unit that performs compensation correction on the exponent field of the third floating-point element based on the first leading zero detection result. The exponent adjustment module can be an adder or a lookup table structure, and its inputs include: the original exponent from the splitting module and the first leading zero detection result from the leading zero detection module; its output is the compensated exponent value.

[0111] The exponent adjustment module can be connected to the splitting module, the leading zero detection module, and the matrix multiplication and addition module 120 via a bus with matching exponent bit width.

[0112] In this application, the exponent adjustment module and the mantissa normalization module work together. For the unnormalized number in the third floating-point element (i.e., the accumulated input C), while normalizing its mantissa, the exponent is adjusted simultaneously to ensure that the exponent reference of the third floating-point element is consistent with the exponent reference of the first and second floating-point elements after normalization. The exponent compensation amount performed by the exponent adjustment module is equal to the number of left shifts indicated by the first leading zero detection result. This ensures that the subsequent exponent alignment module uses the exponent value under the same normalization level when comparing the exponents of multiple floating-point elements, eliminating the exponent alignment deviation caused by the third floating-point element not being synchronously normalized, and improving the accuracy and robustness of numerical alignment in the accumulation stage.

[0113] Specifically, when the input first, second, or third floating-point element is determined to be a denormalized number by the special data detection module, the leading zero detection module immediately performs a leading zero count on its mantissa and outputs the first leading zero detection result. The first leading zero detection result can drive the mantissa normalization module to perform a left shift LZC bit operation on the mantissa of the first and second floating-point elements, and fill the low bits with 0, while restoring the implicit bits to form a complete normalized mantissa. On the other hand, the first leading zero detection result can be synchronously sent to the exponent adjustment module to subtract LZC from the original exponent of the third floating-point element to offset the exponent reduction effect caused by the left shift of the mantissa, thereby maintaining numerical identity. This ensures that all input floating-point elements are in a uniform normalized state and their exponents are comparable before entering the matrix multiplication and addition operation module 120, thus laying the foundation for subsequent high-precision accumulation alignment.

[0114] As an example, before a certain FP8 matrix multiplication-addition operation is initiated, the input module 110 receives a set of operands in FP8 format. The third operand contains a denormalized number in E4M3 format with a mantissa of 0001000 and an exponent of 0000 (bias 7, actual exponent -7). The leading zero detection module detects that the number of leading zeros in the mantissa is 3 and outputs the first leading zero detection result as 3. The mantissa normalization module shifts the mantissas of the first and second floating-point elements left by 3 bits, pads them with 0s to form 10000000, and restores the implicit bit 1 to obtain the normalized mantissa 1.0000000. The exponent adjustment module subtracts 3 from the original exponent 0000 of the third floating-point element to obtain 1101 (two's complement representation -3), corresponding to an actual exponent of -10, which is then multiplied by the normalized mantissa 0.0010000 × 2. -7 =1.0000000×2 -10 Maintaining numerical equivalence; after this processing, the third floating-point element is directly comparable to the other two data in the exponent field, and the subsequent exponent alignment module can accurately select the largest exponent for right shift alignment, avoiding alignment errors or underflow misjudgments caused by uncorrected Denorm.

[0115] In this application, by setting a leading zero detection module, the valid starting position of the denormalized mantissa can be accurately identified; the mantissa normalization module performs left shift normalization on the mantissa of the first floating-point element and the second floating-point element based on the detection result, which can eliminate redundant low-bit operations when the denormalized number participates in multiplication and improve energy efficiency; the exponent adjustment module simultaneously performs compensation correction on the exponent of the third floating-point element, which can ensure the consistency of the three input data on the exponent reference, ensure the atomicity and timing determinism of the Denorm processing, avoid cross-module asynchronous errors, and thus support the high reliability and low power consumption implementation of FP8 to FP32 matrix multiplication and addition operations under the RISC-V architecture.

[0116] In some embodiments, the accumulation module 130 includes an exponent comparison module, an alignment shift module, and an addition module; wherein, the exponent comparison module is connected to the preprocessing module 150, the alignment shift module, and the output module 140, and the addition module is connected to the preprocessing module 150, the alignment shift module, and the output module 140, respectively; the exponent comparison module is configured to select the largest exponent from the exponents of the floating-point elements in the preprocessed third operand and the exponents of multiple fourth floating-point elements to obtain the exponent of the fifth floating-point element; the alignment shift module is configured to right-shift and align the mantissas of multiple fourth floating-point elements based on the largest exponent to obtain multiple right-shifted and aligned mantissas; the addition module is configured to perform an addition operation on the mantissas of the floating-point elements in the preprocessed third operand and the multiple right-shifted and aligned mantissas to obtain the mantissa of the fifth floating-point element.

[0117] In this embodiment, the exponent comparison module is a logic circuit unit used to perform parallel comparisons between the exponents of multiple input floating-point elements and output the maximum value. Its inputs include: the exponents of each floating-point element in the third operand output by the preprocessing module 150 (i.e., the exponents of the preprocessed FP32 format accumulated initial value), and the exponents of multiple fourth floating-point elements output by the matrix multiplication and addition module 120 (i.e., the intermediate FP32 precision exponents formed after the FP8×FP8 multiplication result is expanded).

[0118] The exponent comparison module uses a chain of bit-by-bit comparators or a tree-structured comparison module to filter the maximum value of multiple exponents. Its output is a set of exponent values ​​that correspond one-to-one with the fifth floating-point element. These exponent values ​​are synchronously distributed to the alignment shift module and the output module 140 and can be used as the reference for subsequent normalization and rounding.

[0119] The exponent comparison module can establish a unified exponent alignment benchmark, avoiding the loss of significant mantissa bits due to exponent dispersion. It and the alignment shift module form the starting link of the exponent-mantissa collaborative path. By outputting the maximum exponent, it drives the calculation of the right shift of all mantissa bits to be accumulated, thereby ensuring that fixed-point addition is completed on the same order of magnitude for multiple data.

[0120] The alignment shift module is a hardware shift unit used to perform an unsigned right shift and zero-padding / sign-padding of the mantissas of each path based on the maximum exponent. Its inputs include: the mantissas of each floating-point element in the preprocessed third operand, the mantissas of multiple fourth floating-point elements, and the maximum exponent output by the exponent comparison module.

[0121] The alignment and shift module first calculates the difference between the exponent corresponding to the mantissa of each path and the maximum exponent. This difference is the number of bits to be shifted right. For mantissa paths with a difference greater than 0, the corresponding number of bits is shifted right, and zeros are added to the high bits (for positive numbers) or a sign bit is added (for negative numbers, an arithmetic right shift is used). For paths with a difference equal to 0, the mantissa is output as is.

[0122] In addition, the alignment shift module is configured to synchronously generate a guard bit, a round bit, and a sticky bit during the right shift process. The sticky bit is the logical OR result of all the low-order bits that are shifted out.

[0123] The alignment and shift module can achieve lossless alignment of multi-precision floating-point data in the fixed-point domain. It works with the exponent comparison module to form an alignment decision-alignment execution closed loop, and together with the addition module, it forms the backbone of the alignment summation data flow. This allows the mantissas of the 8-way fourth floating-point elements and the mantissas of the 1-way third floating-point elements to participate in addition under the same exponent benchmark, ensuring that all valid information enters the subsequent reduction process.

[0124] The addition module is a tree-structured adder used to perform parallel addition of multiple fixed-point mantissas. Its inputs include: the mantissas of each floating-point element in the preprocessed third operand (already aligned to the maximum exponent) and the mantissas of multiple right-shifted aligned fourth floating-point elements.

[0125] The addition module can adopt a three-level addition tree structure. The first level combines the nine inputs pairwise to form five partial sums (including one carry-retaining input). The second level further reduces the five partial sums to three. The third level completes the final summation and outputs the single-channel mantissa result. The addition module supports full-precision arithmetic, and its mantissa bit width is not less than 24 bits to be compatible with FP32 mantissa precision and GRS bit extension requirements.

[0126] In this application, the addition module can complete the core numerical fusion of the accumulation stage. Its input comes directly from the output of the alignment shift module, and the output is sent to the output module 140 as the mantissa of the fifth floating-point element.

[0127] Specifically, the addition module, together with the exponent comparison module and the alignment shift module, constitutes the complete data path of the accumulation module 130, realizing a deterministic pipeline of exponent alignment, mantissa alignment, and multi-way summation, providing high-fidelity mantissa input for subsequent normalization.

[0128] Within each calculation cycle, the exponent comparison module receives one third operand exponent (8 elements in total, 1 exponent per element) from the preprocessing module 150 and eight fourth operand exponents (1 element per channel) from the matrix multiplication and addition module 120, for a total of nine exponent signals. The module internally selects the maximum value among the nine exponents through parallel comparison logic and broadcasts the maximum value to the alignment and shift module. Based on the maximum value, the alignment and shift module calculates the number of bits to be right-shifted for each mantissa (i.e., the maximum exponent minus the original exponent of that channel), performs an arithmetic right shift on the corresponding mantissa, and generates three GRS bits. The addition module receives nine aligned mantissas (one from the third operand and eight from the fourth operand), performs parallel reduction in the tree adder, and outputs a 24-bit mantissa result. This result, along with the maximum exponent, is sent to the output module 140 as the basis for the mantissa and exponent of the fifth floating-point element.

[0129] Taking an (8×8) matrix multiplication and addition as an example, the preprocessing module 150 outputs one FP32 format third operand (8 elements), and the matrix multiplication and addition module 120 outputs eight FP32 format fourth operands (each containing eight elements, for a total of 64 elements, organized into eight groups of eight by column). During the accumulation phase, for the j-th column (j=0~7), the exponent comparison module selects the maximum exponent E_max,j from the exponent of the j-th third operand element and the exponents of the j-th elements of the eight fourth operands. The alignment and shifting module calculates the right shift number Δe_i,j=E_max,j-e_i,j (i=0 for the third operand, i=1~8 for the fourth operand) and performs the right shift. The addition module inputs the nine right-shifted mantissas into a three-level addition tree and outputs the accumulated mantissa man_j of the j-th column. This process is performed in parallel on all eight columns, ultimately generating the mantissas and corresponding exponents of the eight fifth floating-point elements, forming the basis of the fifth operands.

[0130] In this application, the exponent comparison module uniformly selects the largest exponent, avoiding the truncation of significant bits due to excessive differences in exponents among multiple floating-point data; the alignment and shift module performs precise shifting based on the largest exponent and generates GRS bits, which can provide complete information support for subsequent rounding; the addition module adopts a tree structure to perform parallel reduction of the mantissas of 9 channels, completing all accumulation operations in a single cycle, meeting the high throughput timing constraints of the RISC-V vector unit, and ensuring the numerical integrity, precision controllability, and hardware execution efficiency of FP8 matrix multiplication and addition in the accumulation stage.

[0131] In some embodiments, the output module 140 is configured to perform leading zero detection on the mantissas of a plurality of fifth floating-point elements to obtain a second leading zero detection result, and to normalize the mantissas of the plurality of fifth floating-point elements based on the second leading zero detection result to generate the mantissa in the target operand; to adjust the exponents of the plurality of fifth floating-point elements based on the second leading zero detection result to generate the exponent in the target operand; and to output the target operand.

[0132] In this embodiment, the output module 140 can be used to perform final formatting on the fifth floating-point element formed after the accumulation operation. The output module 140 can convert the denormalized mantissa into the 1.xxxx normalized form required by the IEEE 754 FP32 standard and simultaneously correct the corresponding exponent, thereby ensuring that the output result meets the RISC-V floating-point specification requirements in terms of numerical representation, rounding behavior and exception flag generation.

[0133] The output module 140 works in conjunction with the exponent comparison module to receive the maximum exponent value of its output as a normalization reference. At the same time, the output module 140 can also work in conjunction with the addition module to receive the unnormalized mantissa and sign bit of its output, forming a complete input data stream. This enables integrated processing of mantissa left / right shift, exponent increment / decrement, GRS bit extraction and rounding decision, forming a closed-loop path from the accumulation result to the standard FP32 output.

[0134] Specifically, in the process of performing leading zero detection on the mantissa of multiple fifth floating-point elements to obtain the second leading zero detection result, this application can use a leading zero counter (LZC) circuit to perform parallel leading zero statistics on the mantissa of each fifth floating-point element output by the addition module to output the second leading zero detection result, which represents the number of consecutive zeros starting from the most significant bit.

[0135] The second leading zero detection result can be an integer between 0 and 23, where 0 indicates that the highest bit of the mantissa is 1 (normalized), and 23 indicates that the mantissa is all 0 (i.e., the result is zero).

[0136] Meanwhile, in the process of performing leading zero detection on the mantissas of multiple fifth floating-point elements to obtain the second leading zero detection result, this application can be executed based on the original code mantissa, or the mantissa in two's complement can be uniformly converted to the original code form after sign extension before execution; the second leading zero detection result can be directly used for subsequent normalized shift amount determination and GRS bit generation.

[0137] For example: when the second leading zero detection result is n, shifting left by n bits can align the most significant bit to the 23rd bit (the most implicit bit of the FP32 mantissa); when the second leading zero detection result is 0 and the two most significant bits of the mantissa are 10, it indicates that there is a carry overflow, and it is necessary to right-align by 1 bit and increment the exponent by 1.

[0138] In the process of normalizing the mantissas of multiple fifth floating-point elements based on the second leading zero detection result to generate the mantissas in the target operand, the programmable shifter can be controlled to perform left or right shift operations on the mantissas according to the second leading zero detection result.

[0139] When the second leading zero detection result n>0, a left shift of n bits is performed, with low-order bits padded with 0; when the second leading zero detection result n=0 and the high-order bit of the mantissa is 10, a right shift of 1 bit is performed, with high-order bits discarded and low-order bits padded with rounding bits; the shifted mantissa retains 23 significant decimal places and reserves 1 implicit bit space; this normalization process can be completed in conjunction with rounding operations, for example, during the left shift, low-order redundant bits are truncated simultaneously and three rounding auxiliary bits G / R / S are generated; or after the shift is completed, an independent rounding module generates the final Rnd bit based on the GRS bit and RISC-V rounding mode (RNE / RTZ / RDN / RUP / RMM) and performs addition; the normalized mantissa is a standard FP32 format 23-bit explicit mantissa, with its most significant bit always 1 (implicit) or all zeros (zero value output).

[0140] In the process of adjusting the exponents of multiple fifth floating-point elements based on the second leading zero detection result to generate the exponent in the target operand, the exponents of multiple fifth floating-point elements (denoted as Exp_max) output by the exponent comparison module can be arithmetically operated with the second leading zero detection result n.

[0141] When left-shifting by n bits, the exponent is updated to Exp_out = Exp_max - n; when right-shifting by 1 bit, the exponent is updated to Exp_out = Exp_max + 1. This exponent adjustment process is implemented using a signed adder, supporting a 10-bit signed exponent operation range [-162, 386]. The adjusted exponent is mapped to the FP32 exponent field [1, 254] after being limited: if Exp_out > 254, it is determined to be an overflow (OF), and Inf or a saturation value is output; if Exp_out < 1 and the mantissa is not zero, it is determined to be an underflow (UF), and a denormalized number or zero is output. This mapping process is completed synchronously with the exception flag generation logic.

[0142] Finally, during the output of the target operand, the normalized mantissa, adjusted exponent, and original sign bit can be packaged and output in FP32 format. The sign bit comes directly from the sign bit output by the addition module, the exponent is an 8-bit unsigned integer (bias 127), and the mantissa is a 23-bit explicit decimal. The output data is written to the vector register file in cycles via a 512-bit bus, with 8 FP32 elements output per cycle, for a total of 4 cycles to complete the output of 64 FP32 results. At the same time, the output process is controlled by the RISC-V vector instruction vfmopma.vv, supporting mask writing and broadcast modes.

[0143] The output module 140 operates as follows: it receives the unnormalized mantissa and sign bit from the addition module and the maximum exponent value from the exponent comparison module; it calls the LZC circuit to perform leading zero detection on each mantissa to obtain the second leading zero detection result; based on the result, it determines whether to shift left, shift right, or keep it unchanged, and drives the shifter to perform the corresponding operation; synchronously, it sends the second leading zero detection result to the exponent adjustment unit to calculate the output exponent; at the same time, during or after the shift, it generates three bits (G / R / S) based on the low-order truncation information of the mantissa and sends them to the rounding module to participate in the Rnd bit calculation; finally, it combines the sign bit, the adjusted exponent, the normalized mantissa, and the exception flag into a standard FP32 word, which is then driven to the external interface via the output buffer.

[0144] As an example, after performing matrix multiplication and addition from (8×8)×(8×8) to (8×8), the accumulation module 130 outputs 64 fifth floating-point elements, each containing a 32-bit mantissa field (including implicit bits), a 10-bit exponent field, and a 1-bit sign bit. The output module 140 performs LZC detection on all 64 mantissas in parallel, obtaining 64 sets of second leading zero detection results. For one set of mantissas with a detection result of 3, a left shift operation of 3 bits is performed, changing the original mantissa 0001_0101_… to 1010_1000_…, while decrementing the exponent by 3. For another set of mantissas with a detection result of 0 and a high-order bit of 10, a right shift operation of 1 bit is performed, changing the original mantissa 1011_… to 0101_…, while incrementing the exponent by 1. All shifted mantissas are processed by the rounding module, adding Rnd bits according to the RNE mode and truncating to 23 bits. Finally, 64 elements conforming to IEEE 754 are output. The target operand in FP32 format is written to four vector registers dst0–dst3, each register holding 16 FP32 elements.

[0145] In this application, the output module 140 performs normalization processing on the mantissa of the fifth floating-point element based on the second leading zero detection result, thereby restoring the denormalized mantissa to the standard form of 1.xxxx; the corresponding exponent is adjusted synchronously to ensure that the numerical precision is not lost due to shift; the normalization process is deeply coupled with GRS bit generation and rounding mode selection, and can support all five rounding behaviors defined by RISC-V; the exponent adjustment range covers [-162, 386] and overflow / underflow criteria are set, which can accurately generate abnormal flags such as OF / UF / NX, so that the final state normalized output of the FP8 matrix multiplication and addition link can be completed without introducing a new structure.

[0146] In some embodiments, the matrix multiplication and addition module 120 includes a multiplexer and a multiplication module; wherein the multiplexer is connected to the input module 110 and the multiplication module, and the multiplication module is connected to the accumulation module 130; the multiplexer is configured to periodically output a plurality of first floating-point elements in a preset number of rows of the first operand, and the multiplication module is configured to perform row and column multiplication operations on the plurality of first floating-point elements in the preset number of rows of the first operand and the second floating-point elements in the second operand in the form of a matrix to obtain a plurality of fourth floating-point elements.

[0147] In this embodiment, the multiplexer can be understood as a hardware selection circuit for periodically scheduling the input data stream. Its input terminal receives the complete first operand from the input module 110 (e.g., an 8×8 matrix consisting of 512-bit width and 64 FP8 elements), and its output terminal supplies the multiplication module with a preset number of FP8 elements in batches according to the instruction cycle.

[0148] The preset number of rows is 1 or 2. For example, in the scenario of (8×8)×(8×8) matrix calculation, 2 rows × 8 columns of 16 first floating-point elements are output in each cycle, corresponding to the i-th row and i+1-th row of the multiplicand matrix.

[0149] The multiplexer is implemented using a static decoding structure or a dynamic timing control structure. Its selection logic is driven by the cycle count signal of the RISC-V vector instruction vwmacc.vv.

[0150] In this application, the multiplexer can realize the spatiotemporal multiplexing and scheduling of input data. Together with the input module 110, it forms a periodic fragmented supply path for FP8 data. It works with the multiplication module to complete the directional pairing of row and column dimensions, thereby avoiding wiring congestion and power consumption increase caused by full matrix broadcasting, and supporting the 4-cycle pipelined matrix multiplication and addition execution process.

[0151] The multiplication module can be understood as a hardware multiplication unit dedicated to multiplying FP8 format floating-point numbers. It is configured to receive the first floating-point element of a preset row (e.g., 2 rows × 8 columns FP8) output by the multiplexer and the second operand (8 rows × 8 columns FP8) directly provided by the input module 110, and perform FP8 × FP8 multiplication operation on each row of the first floating-point element and the corresponding 8 second floating-point elements in the second operand according to the matrix multiplication rules.

[0152] The multiplication module supports two FP8 encoding formats, E4M3 and E5M2. Internally, it includes sign bit XOR logic, exponent adder (with bias correction), mantissa multiplier (4×4bit or 3×3bit unsigned array), hidden bit recovery circuit and normalization forward shift logic.

[0153] Specifically, the sixth floating-point element output by the multiplication module is in an extended intermediate format, with an exponent width of no less than 10 bits, represented by signed two's complement, and a mantissa width of no less than 16 bits. This format is compatible with subsequent exponent alignment and parallel reduction processing.

[0154] In this application, the multiplication module can form a scheduling-computation coupling unit with the multiplexer. Its input data source is controlled by the cycle of the multiplexer, and its output data granularity (i.e. the sixth floating-point element) directly determines the number of input paths of the parallel reduction module, realizing the fidelity conversion from FP8 input to high-precision intermediate results.

[0155] In the first cycle, the multiplexer outputs 16 first floating-point elements from rows 0 to 1 of the first operand, which are synchronously supplied to the multiplication module. The multiplication module pairs these 16 elements with 64 second floating-point elements from columns 0 to 7 of the second operand, performing 16 × 4 = 64 FP8 × FP8 multiplications to generate 64 fourth floating-point elements. These 64 fourth floating-point elements are then fed into the accumulation module 130 to participate in subsequent FP32 accumulation. This process is repeated over 4 cycles, covering all 8 rows, to complete the multiplication and addition of the entire (8 × 8) × (8 × 8) matrix.

[0156] As an example, using the RISC-V vector extension instruction vwmacc.vv ​​as the driving source, the vs1 register is configured to store the multiplicand matrix A (8×8 FP8), the vs2 register to store the multiplier matrix B (8×8 FP8), and the vd register is initially loaded with the accumulated value C (8×8 FP32). In cycle 0, the multiplexer selects the FP8 data in rows 0–1 from vs1, and the multiplication module performs 64 parallel multiplications with all 8 columns of FP8 data in vs2, generating 64 fourth floating-point elements. The 64 fourth floating-point elements are fed into the accumulation module 130 and merged with the corresponding FP32 accumulation values ​​in vd. Cycles 1-3 process rows 2-3, 4-5, and 6-7 in sequence, and finally complete the generation and updating of all 64 FP32 results within 4 cycles. Throughout the process, the scheduling timing of the multiplexer, the format adaptation capability of the multiplication module, and the tree topology of the floating-point parallel reduction module work together to ensure computational determinism, area efficiency, and consistency with RISC-V instruction semantics.

[0157] In this application, the multiplexer supplies the first floating-point element of a preset number of rows periodically, thereby reducing the input bus bandwidth pressure and improving data locality; the multiplication module supports E4M3 / E5M2 dual formats and outputs extended intermediate formats, thereby being compatible with different FP8 encodings and reserving space for subsequent precision improvement.

[0158] In some embodiments, the matrix multiplication and addition module 120 further includes a processing module; wherein the processing module is connected to the accumulation module 130; the multiplication module is configured to perform row and column multiplication operations on a plurality of first floating-point elements in a preset number of rows in the first operand and second floating-point elements in the second operand in the form of a matrix to obtain a plurality of sixth floating-point elements; the processing module is configured to extend the mantissas of the plurality of sixth floating-point elements and perform an alignment operation on the exponents of the plurality of sixth floating-point elements to obtain a fourth operand.

[0159] In this embodiment, the sixth floating-point element is an intermediate result output by the multiplication module after performing a matrix multiplication operation on multiple first floating-point elements in a preset number of rows in the first operand and second floating-point elements in the second operand.

[0160] The sixth floating-point element can serve as a transition point from FP8 multiplication precision to FP32 accumulation precision. It forms a format adaptation front-end with the processing module and achieves low-latency data flow through a direct signal path. As a result, the numerical integrity of the sixth floating-point element can be calibrated before entering the accumulation stage, thereby avoiding precision loss caused by mantissa truncation or exponent misalignment.

[0161] The processing module is configured to extend the mantissa of multiple sixth-point elements, thereby extending the mantissa of the sixth-point elements to 24 bits. The extension method can be to pad with zeros at the lower bits, specifically by appending multiple zero bits after the least significant bit of the mantissa, to reserve the precision margin required for subsequent rounding and normalization.

[0162] Meanwhile, the extended mantissa can participate in subsequent exponent alignment and right shift alignment, thus ensuring that the mantissa precision of the final fifth floating-point element meets the requirement of 23 valid mantissas in the FP32 format.

[0163] During the process of aligning the exponents in multiple sixth-point elements, the processing module can uniformly convert the extended exponents carried by different sixth-point elements into signed two's complement representations with the same bit width (e.g., 10 bits), and adapt their numerical range to the exponent comparison requirements of the FP32 accumulation stage.

[0164] The exponent alignment operation can be as follows: when the sixth floating-point element comes from E4M3 format input, its original 4-bit exponent is mapped to the 10-bit two's complement field after offset correction (-7) and LZC adjustment; when it comes from E5M2 format input, its original 5-bit exponent is mapped to the same 10-bit two's complement field after offset correction (-15) and LZC adjustment.

[0165] Specifically, the exponent alignment operation can be performed in conjunction with the mantissa expansion operation. Their connection is achieved through timing synchronization via shared control signals, which in turn provides a directly comparable exponent input to the exponent comparison module in the accumulation module 130. At the same time, the aligned exponent can participate in subsequent maximum exponent filtering and mantissa right shift alignment, which can ensure that multiple sixth floating-point elements and third floating-point elements are comparable and operable in the exponent dimension.

[0166] Specifically, during the process of mantissa expansion and exponent alignment of the sixth floating-point element, the processing module can output 8 groups of the sixth floating-point element in each calculation cycle. Each group contains 1 sign bit, 10 two's complement exponent, and 16 mantissa. After receiving the 8 groups of data, the processing module first performs low-bit zero-padding expansion on the mantissa of each group to generate 8 groups of 24-bit mantissa. At the same time, it performs sign bit expansion and field mapping verification on the 10-bit two's complement exponent of each group to ensure that its value is within the range of [-162, 386] and there is no overflow. Then, the expanded 24-bit mantissa and the aligned 10-bit exponent are packaged into a fourth operand of a unified format and sent to the accumulation operation module 130. This process does not introduce new functional units or control logic, but only relies on the existing bit width expander and exponent remapping circuit. Its hardware overhead is controllable and the timing path is stable.

[0167] As an example, taking (8×8) matrix multiplication and addition as an example, in the first calculation cycle, the processing module immediately performs mantissa expansion to 24 bits on the 8 sixth floating-point elements output by the multiplication module, and performs normalization alignment on their 10-bit two's complement exponents—unifying the exponent values ​​[-11, 23] generated by the E4M3 path and the exponent values ​​[-17, 47] generated by the E5M2 path to the effective subset of the 10-bit two's complement full range [-512, 511]. The 8 sets of data after expansion and alignment constitute the fourth operand, with a mantissa width of 24 bits, an exponent width of 10 bits, and a sign bit of 1 bit, which fully matches the interface requirements of the accumulation module 130 for the input data format, thereby ensuring that the subsequent accumulation calculation with the third floating-point element (FP32 format) is consistent in both numerical precision and dynamic range.

[0168] In this application, a processing module is added to the matrix multiplication and addition module 120 to perform mantissa expansion and exponent alignment operations on the seventh floating-point element output by the multiplication module, thereby eliminating the format gap between the multiplication result and the accumulated input. The mantissa expansion is 24 bits with zero padding at the low bits, which provides sufficient precision margin for subsequent addition operations and avoids the loss of low-bit information due to insufficient bit width.

[0169] In some embodiments, this application also provides a processor, including the FP8 matrix multiply-accumulate circuit based on the RISC-V architecture provided in this application.

[0170] In this application, the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0171] In some embodiments, this application also provides an electronic device, including the FP8 matrix multiply-accumulate circuit based on the RISC-V architecture provided in this application or the processor provided in this application.

[0172] In this embodiment, the electronic device may be equipped with a processor, which may be equipped with an FP8 matrix multiply-add operation circuit based on the RISC-V architecture and connected to the system bus to provide computing and control capabilities to support the operation of the electronic device.

[0173] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A matrix multiply-add operation circuit based on RISC-V architecture, characterized in that, include: The input module is configured to input operands, the operands including a first operand, a second operand and a third operand, the first operand including a plurality of first floating-point elements in FP8 format, the second operand including a plurality of second floating-point elements in FP8 format, and the third operand including a plurality of third floating-point elements in a preset format; A matrix multiplication and addition module is connected to the input module and is configured to periodically perform row and column multiplication operations on multiple first floating-point elements in a preset number of rows in the first operand and the second floating-point elements in the second operand in the form of a matrix to obtain a fourth operand, wherein the fourth operand includes multiple fourth floating-point elements in a preset format. An accumulation operation module is connected to the matrix multiplication and addition operation module and the input module, and is configured to accumulate multiple fourth floating-point elements with the third floating-point elements to obtain a fifth operand, wherein the fifth operand includes multiple fifth floating-point elements in a preset format; The output module is connected to the accumulation operation module and is configured to output the target operand based on a plurality of the fifth floating-point elements.

2. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 1, characterized in that, It also includes a pre-processing module; The input terminal of the preprocessing module is connected to the input module, and the output terminal of the preprocessing module is connected to the matrix multiplication and addition operation module. The preprocessing module is configured to preprocess the first operand, the second operand, and the third operand respectively to obtain the preprocessed first operand, the second operand, and the third operand.

3. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 2, characterized in that, The exponents of the floating-point elements in the third operand after preprocessing and the exponents of the fourth floating-point element are represented using signed two's complement.

4. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 3, characterized in that, When the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element reach the minimum negative value represented by the sign complement, independent protection processing is performed on the exponents of the floating-point elements in the preprocessed third operand and the fourth floating-point element.

5. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 2, characterized in that, The preprocessing module includes a splitting module, a special data detection module, and a hidden bit recovery module; The splitting module is connected to the input module, the hidden bit recovery module, the matrix multiplication and addition module, and the accumulation module, respectively. The special data detection module is connected to the input module, the hidden bit recovery module, the accumulation module, and the accumulation module, respectively. The hidden bit recovery module is connected to the accumulation module and the accumulation module, respectively. The splitting module is configured to split the first floating-point element, the second floating-point element, and the third floating-point element into bits respectively, so as to obtain the sign bit, exponent, and mantissa in the first floating-point element, the second floating-point element, and the third floating-point element. The special data detection module is configured to perform normalization detection on the mantissas of the first floating-point element, the second floating-point element, and the third floating-point element respectively, and obtain normalization detection results. The hidden bit recovery module is configured to recover the hidden bit of the mantissa in the first floating-point element, the second floating-point element, and the third floating-point element based on the normalization detection result.

6. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 5, characterized in that, The preprocessing module also includes a leading zero detection module, a mantissa normalization module, and an exponent adjustment module. The leading zero detection module is connected to the matrix multiplication and addition module, the accumulation module, the hidden bit recovery module, and the mantissa normalization module, respectively. The mantissa normalization module is connected to the matrix multiplication and addition module and the accumulation module, respectively. The exponent adjustment module is connected to the splitting module, the leading zero detection module, and the matrix multiplication and addition module, respectively. The leading zero detection module is configured to perform leading zero detection on the denormalized mantissas in the first floating-point element, the second floating-point element, and the third floating-point element respectively, to obtain the first leading zero detection result; The mantissa normalization module is configured to normalize the unnormalized mantissas in the first floating-point element and the second floating-point element based on the first leading zero detection result. Both the exponent adjustment module and the mantissa normalization module are configured to process the exponent and the denormalized mantissa in the third floating-point element based on the first leading zero detection result, so as to obtain the preprocessed third operand.

7. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 2, characterized in that, The accumulation operation module includes an exponent comparison module, an alignment shift module, and an addition operation module; The exponent comparison module is connected to the preprocessing module, the alignment shift module, and the output module, respectively, and the addition module is connected to the preprocessing module, the alignment shift module, and the output module, respectively. The exponent comparison module is configured to filter out the largest exponent from the exponents of the floating-point elements in the preprocessed third operand and the exponents of the multiple fourth floating-point elements to obtain the exponent of the fifth floating-point element. The alignment shift module is configured to right-shift and align the mantissas of multiple fourth floating-point elements based on the maximum exponent, thereby obtaining multiple right-shifted and aligned mantissas. The addition module is configured to perform an addition operation on the mantissa of the floating-point element in the preprocessed third operand and multiple right-shifted mantissas to obtain the mantissa of the fifth floating-point element.

8. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 7, characterized in that, The output module is configured to perform leading zero detection on the mantissas of the plurality of fifth floating-point elements to obtain a second leading zero detection result, and to normalize the mantissas of the plurality of fifth floating-point elements based on the second leading zero detection result to generate the mantissas in the target operand. Based on the second leading zero detection result, the exponents of multiple fifth floating-point elements are adjusted to generate the exponent in the target operand; Output the target operand.

9. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 1, characterized in that, The matrix multiplication and addition module includes a multiplexer and a multiplication module; The multiplexer is connected to the input module and the multiplication module, respectively, and the multiplication module is connected to the accumulation module. The multiplexer is configured to periodically output a plurality of the first floating-point elements in a preset number of rows of the first operand; The multiplication module is configured to perform row and column multiplication operations on multiple first floating-point elements in a preset number of rows in the first operand and second floating-point elements in the second operand in the form of a matrix, to obtain multiple fourth floating-point elements.

10. The FP8 matrix multiply-add operation circuit based on RISC-V architecture according to claim 9, characterized in that, The matrix multiplication and addition module also includes a processing module; The processing module is connected to the accumulation operation module; The multiplication module is configured to perform row and column multiplication on multiple first floating-point elements in a preset row number in the first operand and second floating-point elements in the second operand in the form of a matrix to obtain multiple sixth floating-point elements. The processing module is configured to expand the mantissas of the plurality of sixth floating-point elements and align the exponents of the plurality of sixth floating-point elements to obtain the fourth operand.

Images