A reconfigurable system on chip for AIoT multi-modal fusion computing and a reconfiguration method

Abstract

The application relates to the technical field of artificial intelligence hardware acceleration, and provides a reconfigurable system-on-chip for AIoT multi-modal fusion calculation and a reconfiguration method, wherein the system is internally configured with an RPU, an RNU and a reconfiguration execution block; the reconfiguration execution block is used for reconfiguring the first and second source data in accuracy according to instructions; the RPU is internally configured with a multi-precision multiplier; a mantissa calculation module in the multi-precision multiplier comprises a Booth encoder, a partial product generation unit and a Wallace compression tree; the encoder determines a working mode according to a mode selection signal and encodes the mantissa of the reconfigured first source data; the partial product generation unit generates a partial product according to an encoding result and the mantissa of the reconfigured second source data; and the Wallace compression tree compresses the partial product according to the accuracy. The computing path of the system can be adjusted according to the accuracy of data, and extreme wide-precision compatibility from INT4 to FP32 is realized.

Description

Technical Field

[0001] This application relates to the technical field of artificial intelligence hardware acceleration, and in particular to a reconfigurable system-on-chip and reconfiguration method for AIoT multimodal fusion computing. Background Technology

[0002] Multimodal sensing computing is a core application scenario in emerging AIoT (Artificial Intelligence and Internet of Things) systems, such as smart radar, assisted driving, and intelligent robots. Modern sensing systems typically require the execution of a complete processing pipeline, covering everything from front-end sensor signal processing and feature extraction to back-end AI inference and decision-making.

[0003] In existing technologies, general-purpose processors (such as CPUs or GPUs) are typically used to perform these heterogeneous computations for the aforementioned multimodal perception computing tasks. These general-purpose devices rely on fixed instruction sets and computing cores to schedule various algorithms sequentially or in parallel according to traditional computing models. Meanwhile, to overcome the energy efficiency limitations of general-purpose devices at the edge, existing technologies often employ domain-specific architecture (DSA) accelerators as specialized hardware solutions. For example, relevant literature uses the RedMulE architecture optimized for FP16 precision neural network training and inference at the edge, the DAP architecture for specific digital signal processing (DSP), and the Amber architecture for dense linear algebra. These domain-specific accelerators typically accelerate the computational data flow for specific precision or specific domain by configuring fixed data paths and a single type of multiply-accumulate array.

[0004] However, with the increasing demand for multimodal data fusion at the edge, the aforementioned existing technologies have the following technical shortcomings when processing complete perceptual computing pipelines: Existing domain-specific accelerators suffer from technical shortcomings, including extremely rigid data flows and an inability to adapt to dynamic tasks with multiple formats and modalities. Specifically, perceptual computing has drastically different underlying requirements for data formats at different stages: front-end DSPs often rely on high-precision floating-point operations and dedicated mathematical functions, while back-end AI inference is primarily dominated by dynamic, high-intensity fixed-point or low-precision floating-point parallel multiply-accumulate (MAC) units. Most existing edge accelerators employ fixed data flow designs, completely lacking unified underlying hardware support for mixed-precision multimodal operations, such as compatibility with multi-precision GEMMs and FFTs on the same hardware circuitry. This results in existing architectures being unable to flexibly change computational pathways when facing perceptual scenarios that require frequent switching between high-precision DSP tasks and low-precision AI inference tasks, leading to extremely poor system adaptability. Summary of the Invention

[0005] Based on this, it is necessary to provide a reconfigurable on-chip system and computing method for AIoT multimodal fusion computing to address the above-mentioned technical problems. This method can adjust the computing path of the system according to the accuracy of the data and achieve extremely wide accuracy compatibility from INT4 to FP32.

[0006] In the first aspect, this application provides a reconfigurable system-on-a-chip for AIoT multimodal fusion computing, including multiple computing cores, each of which is configured with an RPU, an RNU, and a reconfiguration execution block; The reconstructed configuration execution block is configured to receive instructions from the superior and reconstruct the first source data and the second source data in terms of precision according to the instructions from the superior. The RPU is equipped with a multi-precision multiplier, which includes: The first exponent and mantissa unpacking module is configured to unpack the reconstructed first source data and second source data to generate the exponent part and mantissa part of the source data; The exponent addition module is configured to add the exponent portions of the first source data and the second source data to generate the exponent calculation result; The mantissa calculation module includes a Booth encoder, a partial product generation unit, and a Wallace compression tree. The Booth encoder is configured to determine its operating mode based on the indication of a mode selection signal. In the current operating mode, the mantissa of the source data is input to the Booth encoder for Booth encoding. The partial product generation unit is configured to generate a partial product based on the Booth encoding result and the mantissa of the reconstructed second source data. The Wallace compression tree is configured to compress the partial product according to its precision to generate the mantissa calculation result. The first result generation module is configured to integrate the exponent calculation result and the mantissa calculation result to generate the final product output result.

[0007] In one embodiment, the Booth encoder's operating modes include FP32×1 mode, FP16×2 mode, FP8×4 mode, INT16×2 mode, INT8×4 mode, and INT4×8; In FP32×1 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into one FP32 mantissa segment. In FP16×2 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into two FP16 mantissa segments. In FP8×4 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into four FP8 mantissa segments. In INT16×2 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into two sets of 16-bit suboperands using a two-way integer multiplication parallel method. In INT8×4 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into four sets of 8-bit suboperands using a four-way integer multiplication parallel method. In INT4×8 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into eight sets of 4-bit suboperands using an eight-way integer multiplication parallel method.

[0008] In one embodiment, in FP32×1 mode, the partial product is entered into the Wallace compression tree via one FP32 mantissa multiplication, resulting in one compressed output. In FP16×2 and INT16×2 modes, the partial product is divided into two independent data streams and entered into the Wallace compression tree, resulting in two compressed outputs. In FP8×4 and INT8×4 modes, the partial product is divided into four independent data streams and entered into the Wallace compression tree, resulting in four compressed outputs. In INT4×8 mode, the partial product is divided into eight independent data streams and entered into the Wallace compression tree, resulting in eight compressed outputs.

[0009] In one embodiment, the exponent addition module includes: The carry-lookahead adder is configured to add the exponent portions of the first and second source data and subtract the corresponding precision offset to generate the exponent calculation result. The overflow / underflow detection module is configured to determine whether the initial value of the exponent part exceeds the representable range of the current floating-point format.

[0010] In one embodiment, the multiprecision multiplier further includes: An anomaly detection module is used to detect whether the first source data and the second source data belong to special value cases; if so, a corresponding anomaly status flag is generated and a corresponding calculation result is formed for the result generation module to select; the special values ​​include zero, infinity, and not a number.

[0011] In one embodiment, the RPU is further configured with a multi-precision adder, which includes: The second exponent and mantissa unpacking module is configured to unpack the first source data and the second source data to generate the exponent part and mantissa part of the source data. Exponential subtraction module: configured to compare the exponents of the first source data and the second source data, and calculate the difference between the two exponents; The mantissa alignment module is configured to perform mantissa preprocessing and exponent alignment operations on the first source data and the second source data; The mantissa addition module is configured to sum the mantissas of the first and second source data after alignment processing. The results generation module integrates the summation results to generate the final summation output.

[0012] In one embodiment, the multi-precision adder further includes: The second anomaly detection module is used to detect whether the first source data and the second source data belong to special value cases; if so, it generates a corresponding anomaly status flag and forms a corresponding calculation result for the result generation module to select; the special values ​​include zero, infinity, and not a number.

[0013] Secondly, this application also provides a reconfigurable on-chip system reconfiguration method for AIoT multimodal fusion computing, including: Acquire first source data, second source data, and mode selection signal; The first source data and the second source data are unpacked into an exponent part and a mantissa part; Add the exponential portions of the first source data and the second source data to generate the exponential calculation result; The operating mode of the Booth encoder is determined according to the indication of the mode selection signal, and the mantissa of the reconstructed first source data is input to the Booth encoder for Booth encoding. A partial product is generated based on the Booth encoding result and the mantissa of the reconstructed second source data; the partial product is compressed according to its precision to generate the mantissa calculation result. The results of the exponent calculation and the mantissa calculation are combined to generate the final product output.

[0014] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to perform the following steps: Acquire first source data, second source data, and mode selection signal; The first source data and the second source data are unpacked into an exponent part and a mantissa part; Add the exponential portions of the first source data and the second source data to generate the exponential calculation result; The operating mode of the Booth encoder is determined according to the indication of the mode selection signal, and the mantissa of the reconstructed first source data is input to the Booth encoder for Booth encoding. A partial product is generated based on the Booth encoding result and the mantissa of the reconstructed second source data; the partial product is compressed according to its precision to generate the mantissa calculation result. The results of the exponent calculation and the mantissa calculation are combined to generate the final product output.

[0015] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, performs the following steps: Acquire first source data, second source data, and mode selection signal; The first source data and the second source data are unpacked into an exponent part and a mantissa part; Add the exponential portions of the first source data and the second source data to generate the exponential calculation result; The operating mode of the Booth encoder is determined according to the indication of the mode selection signal, and the mantissa of the reconstructed first source data is input to the Booth encoder for Booth encoding. A partial product is generated based on the Booth encoding result and the mantissa of the reconstructed second source data; the partial product is compressed according to its precision to generate the mantissa calculation result. The results of the exponent calculation and the mantissa calculation are combined to generate the final product output.

[0016] This application employs the aforementioned reconfigurable on-chip system and computing method for AIoT multimodal fusion computing, which has the following beneficial effects: 1. The technical approach employed in this application, namely, achieving sign bit isolation through a "one-to-two" Booth multiplier structure and deeply reusing multi-level partial product compression trees and carry-lookahead adders within a single data path, enables extremely wide precision compatibility from INT4 to FP32 without increasing the input bus bandwidth burden. This full reuse of underlying operational logic allows the operator to complete multimodal data mixing calculations with minimal hardware area overhead within a uniform 3-clock-cycle delay, fundamentally eliminating the static power consumption and area waste caused by idle computational logic.

[0017] 2. By introducing a unified flow control instruction dispatch mechanism at the SoC top level, and supporting the dynamic reconfiguration of the computing array composed of adders and multipliers between three system modes—SIMD (Single Instruction Multiple Data), Pipeline, and SIMT (Single Instruction Multiple Threads)—the system can adaptively adjust the arithmetic strength at the hardware level based on specific algorithm characteristics. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the SoC system architecture in one embodiment; Figure 2 This is a schematic diagram of a multi-precision multiplier structure in one embodiment; Figure 3 This is a schematic diagram of the reconfigurable data path of the Booth encoder and the partial product generation module in one embodiment; Figure 4 This is a schematic diagram of Wallace tree compression and reconstruction in another embodiment; Figure 5 This is a schematic diagram of a multi-precision adder structure in one embodiment. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] This application provides a SoC system architecture, which consists of an AXI interconnect interface, a RISC-V E203 microcontroller, an ICB bus, a JTAG interface, and an accelerator subsystem. The accelerator subsystem is the core component for accelerating perceptual computing tasks, integrating DMA, a main control module, affine bus memory, a data / instruction scheduler, a data / reconfiguration network, and four reconfigurable computing cores. The RISC-V E203 is responsible for task configuration and execution control, the AXI / DMA / on-chip memory handles the main data transfer, and the four reconfigurable computing cores perform parallel computing and inter-core collaboration through the data / reconfiguration network. Each specific module is described below: The RISC-V E203 microcontroller connects to the external application programming interface (API) via the on-chip ICB bus and JTAG interface to receive external tasks and perform debugging operations. It also connects to the main control module in the accelerator subsystem to send task configuration information, operating modes, and scheduling commands to the accelerator. The RISC-V E203 microcontroller is the control core of the SoC system, primarily responsible for upper-layer task parsing, accelerator configuration, runtime status management, and task start / stop control. The RISC-V E203 microcontroller does not directly handle large-scale perceptual computing; instead, it acts as a software-programmable control node, uniformly configuring DMA transfer modes, on-chip memory organization, the operating modes of the four computing cores, and inter-core collaboration methods, thereby achieving the scheduling and management of the entire perceptual computing process.

[0021] The AXI interconnect interface, located between the SoC and external memory / peripherals, is one of the system's main data entry points. Directly connected to the DMA, the AXI interconnect interface forms the main data path for off-chip data to enter the on-chip accelerator subsystem. The AXI interconnect interface primarily handles high-speed data exchange between the SoC and off-chip memory or other system components. For example, the AXI interconnect interface, together with the DDR / DRAM control path, constitutes the external data access infrastructure. Sensing data, preprocessing results, and data to be computed can all be sent to the on-chip DMA via the AXI path, and then into the on-chip affine bus memory, providing data support for subsequent computation core execution.

[0022] The RISC-V E203 microcontroller connects to the JTAG interface via the ICB bus. The ICB bus is used to carry control and debugging information on the microcontroller side, enabling system configuration, debugging access, and status interaction.

[0023] The JTAG interface connects to the RISC-V E203 microcontroller via the ICB bus and is used for system debugging, testing, and program downloading.

[0024] The accelerator subsystem is the main body responsible for high-performance perceptual computing. It mainly consists of DMA, main control module, on-chip affine bus memory, unified data / instruction scheduler and four reconfigurable computing cores, supporting multiple execution modes such as SIMD, Pipeline and SIMT.

[0025] The DMA connects to the AXI interconnect interface on one end and the affine bus memory on the other, forming a main data transfer bridge between off-chip and on-chip memory. Its operating mode and transmission behavior are controlled by the main control module and processor. DMA is primarily used for efficient data transfer between external DDR / DRAM and on-chip affine bus memory. Furthermore, DMA and affine bus memory collaborate via a dedicated port, supporting sequential storage and transposed storage data layout methods. Therefore, DMA not only "moves data" but also plays a crucial role in organizing memory access for algorithm-oriented data layout.

[0026] The main control module is connected to the RISC-VE203 microcontroller, receiving control commands and task parameters from the microcontroller; it also provides control and configuration support to the DMA, data / instruction scheduler, and each reconfigurable computing core. The main control module includes a decoder, a register set, and a state machine. The decoder is responsible for parsing the task description, mode configuration, and control words from the RISC-VE203 microcontroller; the register set temporarily stores various configuration parameters, such as execution mode, address information, task length, and synchronization control parameters; and the state machine is responsible for switching workflows at different stages, such as the configuration stage, data transfer stage, computation execution stage, and result write-back stage.

[0027] The affine bus memory is connected to the DMA at the front end and to the data / instruction scheduler at the back end, and provides data support for the four reconfigurable computing cores through the data / reconfiguration network.

[0028] The data / instruction scheduler connects to the main control module on one side to receive task parsing results, configuration parameters, and control commands; it also connects to the affine bus memory for on-chip data organization. On the other side, it connects to the data / reconfiguration network, through which it distributes data and instructions to the four reconfigurable computing cores in a unified manner. The data / instruction scheduler needs to perform different scheduling tasks in different modes: in SIMD mode, it synchronously drives multiple computing cores; in Pipeline mode, it handles task transitions between different computational stages; and in SIMT mode, it is responsible for the differentiated distribution of execution content across different cores.

[0029] The data / reconfiguration network, located between the four reconfigurable computing cores and connected to the data / instruction scheduler, forms the backbone of the inter-core interconnection within the accelerator. Its core function is to support inter-core data transfer, configuration information propagation, and interconnection reconfiguration under various data flow modes.

[0030] Each reconfigurable computing core is connected to other on-chip computing cores and the data distribution module via a data / reconfiguration network; within the core, it consists of an RPU, an RNU, and a reconfiguration configuration execution block. The reconfigurable computing core is the basic execution entity in the accelerator that actually undertakes computational tasks. The four cores can execute the same type of task in parallel or handle different computational stages in a pipelined mode, thus forming a multi-core collaborative computing array for perceptual computing. The RPU is the main arithmetic execution unit in each computing core, primarily composed of multi-precision multipliers and multi-precision adders; the RNU serves two purposes: firstly, as a local buffer and data relay node within the core, improving the continuity of data supply; and secondly, as a data flow organization node, providing adaptation support for different algorithm data modes, which can be understood as an "internal data connection and scheduling unit." The reconfiguration configuration execution block receives configuration and control commands from the data / instruction scheduler and coordinates the operation of the RPU and RNU within the core.

[0031] In the first aspect, based on the above structure, this application provides a reconfigurable system-on-a-chip for AIoT multimodal fusion computing, including multiple computing cores, each of which is configured with an RPU, an RNU, and a reconfiguration execution block.

[0032] The reconfiguration execution block is configured to receive instructions from the data / instruction scheduler and reconstruct the first and second source data in terms of precision according to the instructions from the data / instruction scheduler. On the other hand, it is configured to adjust the computation path of the computing array composed of multi-precision multipliers and multi-precision adders according to the instructions from the data / instruction scheduler, so that the computing array supports multiple execution modes such as SIMD, Pipeline and SIMT. The RPU is equipped with a multi-precision multiplier, which includes: a first exponent and mantissa unpacking module, configured to unpack the reconstructed first and second source data to generate the exponent and mantissa parts of the source data; an exponent addition module, configured to add the exponent parts of the first and second source data to generate the exponent calculation result; a mantissa calculation module, including a Booth encoder, a partial product generation unit, and a Wallace compression tree. The Booth encoder is configured to determine its operating mode based on the mode selection signal, and in the current operating mode, input the mantissa of the source data to the Booth encoder for Booth encoding; the partial product generation unit is configured to generate a partial product based on the Booth encoding result and the mantissa of the reconstructed second source data; the Wallace compression tree is configured to compress the partial product according to its precision to generate the mantissa calculation result; and a first result generation module, configured to integrate the exponent calculation result and the mantissa calculation result to generate the final product output result.

[0033] In one embodiment, the exponent addition module includes: a carry-lookahead adder configured to add the exponent portions of the first source data and the second source data and subtract an offset corresponding to the precision to generate an exponent calculation result; and an overflow / underflow detection module configured to determine whether the initial value of the exponent portion exceeds the representable range of the current floating-point format.

[0034] In one embodiment, the multi-precision multiplier further includes: an anomaly detection module, used to detect whether the first source data and the second source data belong to special numerical cases; if so, a corresponding anomaly status flag is generated, and a corresponding calculation result is formed for the result generation module to select; the special numerical cases include zero, infinity, and NOT.

[0035] This application's multi-precision multiplier addresses the multiplication requirements of various floating-point and fixed-point data formats. The inputs include a mode selection signal, a data validity signal, first source data, and second source data. The outputs include a calculation completion signal, a product output result, and an output result without rounding. The entire multiplier mainly consists of a first exponent and mantissa unpacking module, a first anomaly detection module, an exponent addition module, a mantissa calculation module, a first normalization module, and a first result generation module. Specifically, the exponent and mantissa unpacking module parses the input operands according to the current precision mode; the first anomaly detection module handles special input cases; the exponent addition module and the mantissa calculation module perform the core operations of the exponent and mantissa paths, respectively; the first normalization module normalizes the multiplication result; and the first result generation module integrates the outputs of all modules to encapsulate and output the final result.

[0036] Specifically, the exponent and mantissa unpacking module is located at the data entry point of the entire multi-precision multiplier. Its input receives the mode selection signal, the data validity signal, the first source data, and the second source data. Its output is connected to the first anomaly detection module, the exponent addition module, and the mantissa calculation module, respectively. The unpacked exponent portion is sent to the exponent addition module, and the unpacked mantissa portion is sent to the mantissa calculation module. Simultaneously, the operand segment information is provided to the anomaly detection module for special case judgment.

[0037] The anomaly detection module receives floating-point data field information from the input operand path and connects to the first result generation module. It outputs an anomaly flag signal and, in special cases, directly provides the abnormal product result, which is bypassed under controlled conditions in fixed-point mode. This module detects whether the input operand belongs to a special numerical case in floating-point multiplication, such as zero, infinity, or Not-a-N (NaN). When a special input is detected, the module generates a corresponding anomaly status flag and can directly form the corresponding abnormal product result for the result generation module to use preferentially. This prevents abnormal cases from continuing into subsequent regular exponent and mantissa operation paths, reduces invalid calculations, and ensures the correctness of the multiplication result for special numerical values.

[0038] The exponent addition module receives the exponent parts of the two operands from the exponent and mantissa unpacking module and sends the calculated exponent result to the normalization module. Simultaneously, the overflow / underflow flags output by this module and the possible overflow / underflow products are sent to the result generation module. This module performs the exponent path operations in floating-point multiplication. It adds the exponent parts of the two input operands and subtracts the corresponding precision bias to obtain the initial exponent of the multiplication result. Since the exponent result of floating-point multiplication may exceed the range allowed by the current format, this module also performs preliminary overflow and underflow detection on the exponent result, providing a basis for subsequent normalization and result selection. This module is the core processing unit of the exponent path in the entire floating-point multiplier. The carry-lookahead adder performs the addition, and the overflow / underflow detection module determines whether the initial exponent value exceeds the representable range of the current floating-point format.

[0039] The mantissa calculation module receives the mantissa portions of two operands from the exponent and mantissa unpacking module, and outputs the mantissa multiplication compression result to the normalization module. This module further includes a Booth encoder, a partial product generation unit, and a Wallace compression tree.

[0040] First, the multiplier is Booth-encoded to reduce the number of partial products. Then, partial products at each level are generated based on the encoding results. Finally, a compression tree structure is used to efficiently compress the partial products, forming an intermediate mantissa result suitable for subsequent normalization. The Booth encoder is located at the front end of the mantissa calculation module. Its input is the mantissa of the second operand or its corresponding encoded segment, and its output is the Booth-encoded result, which is sent to the partial product generation unit. This module performs radix-4 Booth encoding on the mantissa multiplier, converting the original multiplication operation into a smaller number of addition, subtraction, and shift combinations, thereby reducing the number of partial products and lowering the hardware complexity and latency of subsequent partial product compression. The partial product generation unit receives the encoding control information output from the Booth encoder and the mantissa data of another operand, and outputs multiple sets of partial products to the Wallace compression tree. This module generates partial products at each level based on the Booth-encoded results. During the generation process, issues such as positive and negative partial products, shift alignment, and sign extension need to be considered to form a standard set of partial products suitable for further processing by the compression tree. The Wallace compression tree receives the multiple partial products output from the partial product generation unit and sends the compressed mantissa result to the normalization module. This module is used for fast parallel compression of multi-way partial products, progressively compressing the original multi-row partial product into two intermediate results, such as sum and carry, or an equivalent two-way compressed form. Compared to the serial accumulation method, the Wallace compression tree can significantly shorten the delay of the mantissa multiplication path.

[0041] The normalization module receives the exponent result from the exponent addition module and the mantissa compression result from the mantissa calculation module, and sends the normalized exponent and mantissa to the result generation module. This module is used to normalize intermediate multiplication results. Since the result after mantissa multiplication may have carry-in at the most significant bit, changes in the leading bit position, or expansion of the effective bit width, it is necessary to adjust the mantissa representation through normalization operations and simultaneously correct the exponent value so that the result meets the normalization requirements of the target floating-point format.

[0042] The result generation module is located at the output of the entire multi-precision multiplier. Inputs include: anomaly flag signals and abnormal product results from the first anomaly detection module; overflow / underflow flag signals and overflow / underflow product results from the exponent addition module; and normalized exponent and normalized mantissa from the first normalization module. The output generates a calculation completion signal, a product output result, and an unrounded output result. This module synthesizes the results from each operation path and generates the final output according to priority. Specifically, when the anomaly detection module indicates a special value input, the abnormal product result is selected first; when the exponent path detects overflow or underflow, the corresponding overflow / underflow result is selected based on the overflow / underflow flags; under normal conditions with no anomalies and no out-of-bounds errors, the exponent and mantissa output by the normalization module are repackaged according to the current precision format to form the final product output result. Simultaneously, to facilitate subsequent expansion or debugging, this module also outputs unrounded intermediate results and a calculation completion signal to indicate that the current operation has ended.

[0043] For the mantissa calculation module, hardware reuse is mainly reflected in the reconfigurable data path design of the Booth encoder, partial product generation unit, and Wallace tree compression network. To support multiple precision and parallel modes such as INT4×8, INT8×4, INT16×2, FP8×4, FP16×2, and FP32×1, this design does not build independent mantissa multiplication paths for each data format. Instead, it uses unified basic hardware resources and reconfigures the input data path, partial product organization method, and compression path through mode control, thereby achieving hardware reuse in multiple precision modes.

[0044] like Figure 3 As shown, the Booth encoder and partial product generation module are implemented using a unified reconfigurable structure. Considering that the parallelism is highest in integer mode, where INT16×2, INT8×4, and INT4×8 modes, after parallel expansion, all require up to 16 channels of Booth encoding / partial product generation resources, this application uses 16 radix-4 Booth encoders and 16 partial product generation units as unified basic hardware resources.

[0045] In this structure, the first source data input first enters the encoding reconstruction configuration execution block. Under the control of the mode selection signal, the first source data is segmented, concatenated, or padded with zeros according to the current precision mode, and the reconstructed encoded input is sent to 16 Booth encoders. Correspondingly, the second source data input enters the second source data reconstruction configuration execution block. Under the control of the mode selection signal, the multiplicand data path is reconstructed and sent to each partial product generation unit. Each Booth encoder generates a Booth encoded signal according to the corresponding encoded input, which controls the subsequent partial product generation unit to perform operations such as shifting, inverting, and incrementing on the second source data, thereby obtaining the corresponding partial product result for each mode.

[0046] The reconfigurable data path operates as follows for different modes: In FP32×1 mode, the encoding reconstruction configuration execution block sends the first source data to the encoder in a single-channel FP32 mantissa multiplication manner, and the partial product generation module outputs all the partial products required for the single-channel floating-point mantissa multiplication. In FP16×2 mode, the input data is divided into two groups of half-precision mantissa segments, which are fed in parallel into the same encoder array and partial product generation unit, thereby simultaneously generating the partial products required for two channels of FP16 multiplication. In FP8×4 mode, the input is further divided into four groups of shorter mantissa segments, which reuse the same set of hardware resources in parallel. In INT16×2, INT8×4, and INT4×8 modes, the input data is segmented and mapped in parallel using 2-channel, 4-channel, and 8-channel integer multiplication methods, respectively, so that the 16 Booth encoders and 16 partial product generation units are fully reused. For paths not used by the current mode, bypassing, shutting down, or zero-padding can be performed through mode control.

[0047] More specifically, the Booth encoder operates in FP32×1, FP16×2, FP8×4, INT16×2, INT8×4, and INT4×8 modes. In FP32×1 mode, the reconstruction configuration execution block reconstructs the mantissa of the first source data and the mantissa of the second source data into one FP32 mantissa segment. In FP16×2 mode, the reconstruction configuration execution block reconstructs the mantissa of the first source data and the mantissa of the second source data into two FP16 mantissa segments. In FP8×4 mode, the reconstruction configuration execution block reconstructs the mantissa of the first source data and the mantissa of the second source data into two FP16 mantissa segments. The mantissa is reconstructed into four FP8 mantissa segments; in INT16×2 mode, the reconstruction configuration execution block reconstructs the mantissa of the first source data and the mantissa of the second source data into two groups of 16-bit suboperands respectively in a two-way integer multiplication parallel manner; in INT8×4 mode, the reconstruction configuration execution block reconstructs the mantissa of the first source data and the mantissa of the second source data into four groups of 8-bit suboperands respectively in a four-way integer multiplication parallel manner; in INT4×8 mode, the reconstruction configuration execution block reconstructs the mantissa of the first source data and the mantissa of the second source data into eight groups of 4-bit suboperands respectively in an eight-way integer multiplication parallel manner.

[0048] Figure 3 The structure shown is essentially based on 16-way unified Booth encoding and partial product generation resources. It dynamically reconstructs the input data path through encoding reconstruction configuration execution block and second source data reconstruction configuration execution block, thereby being compatible with the mantissa or integer partial product generation requirements in various floating-point / fixed-point precision modes.

[0049] like Figure 4 As shown, the Wallace compression tree uses pp1 to pp16 as a unified input interface to compress the partial product results from the previous partial product generation unit. Since the number of partial products, bit width, and grouping methods differ under different data precision modes, this application does not design independent compression trees for each mode. Instead, it constructs a reconfigurable compression network consisting of multiple 4-2 compressors, 3-2 compressors, and reconfiguration paths, and changes the connection relationships and compression levels of each partial product entering the compressor under the control of the mode selection signal.

[0050] In one embodiment, in FP32×1 mode, the partial product is entered into the Wallace compression tree via one FP32 mantissa multiplication, resulting in one compressed output. In FP16×2 and INT16×2 modes, the partial product is divided into two independent data streams and entered into the Wallace compression tree, resulting in two compressed outputs. In FP8×4 and INT8×4 modes, the partial product is divided into four independent data streams and entered into the Wallace compression tree, resulting in four compressed outputs. In INT4×8 mode, the partial product is divided into eight independent data streams and entered into the Wallace compression tree, resulting in eight compressed outputs.

[0051] In FP32×1 mode, the Wallace compression tree uses the complete compression path, and all partial products enter the multi-stage compression network in the manner of single-path FP32 mantissa multiplication, ultimately obtaining the compression result in single-path FP32 mode. In FP16×2 and INT16×2 modes, the partial products are divided into two independent data streams, which are sent to the corresponding compressor cascade paths through reconstruction paths, thus obtaining two compression results simultaneously. In FP8×4 and INT8×4 modes, the partial products are further divided into four groups, which are mapped to shallower compression paths through reconstruction paths to obtain four parallel results. In INT4×8 mode, since the number of partial products corresponding to each multiplication is minimal, the compression requirement is significantly reduced, so it is no longer necessary to go through the complete compression tree, but can directly output 8 results through mode selection.

[0052] More specifically, refer to Figure 4 The multi-stage compression network of this application consists of three compressor stages: the first stage has three 4-2 compressors, the second stage has one 4-2 compressor and one 3-2 compressor, and the third stage has one 4-2 compressor. The compression path differs depending on the precision of the data.

[0053] from Figure 4 As can be seen, the compression network contains several 4-2 and 3-2 compressors with different bit widths, such as 28-bit, 12-bit, and 44-bit compressors for front-end compression, and 27-bit, 35-bit, and 18-bit compressors for back-end result integration. Each compressor does not serve a fixed mode; instead, under the control of the mode selection signal, it dynamically changes the input source and output destination through the reconstruction path shown in the dashed box, allowing the same compression hardware to be multiplexed across multiple modes such as FP32×1, FP16×2, FP8×4, INT16×2, and INT8×4.

[0054] therefore, Figure 4 As shown, the core idea of ​​the Wallace compression tree in this application is: based on a unified partial product input set and unified compressor resources, the partial product group method, compression level and compressor connection relationship are dynamically reconstructed through mode control, thereby realizing hardware reuse of the mantissa compression path under different precision modes.

[0055] This application provides complete embodiments for calculating data at different precision levels: When the mode selection signal indicates that the mantissa calculation module is working in INT4×8 mode, the encoding reconstruction module divides the first source data into 8 groups of 4-bit suboperands. Each group of 4-bit suboperands forms 2 radix-4 Booth encoding windows, which are then mapped to 16 Booth encoders. The second source data reconstruction module synchronously divides the second source data into 8 groups of 4-bit suboperands and sends them to the 16 partial product generation units corresponding to the 16 Booth encoders.

[0056] In INT4×8 mode, each 4-bit multiplication produces only a small number of partial products. Therefore, the 16 partial products output by the partial product generation unit are organized into 8 pairs, and the 8 sets of multiplication results are directly output through the reconstruction path without going through the subsequent Wallace tree compression network. In other words, in INT4×8 mode, the mode selection signal shuts down or bypasses the compressor path, and the mantissa calculation module uses a direct output path to complete the 8-way parallel INT4 multiplication operation.

[0057] When the mode selection signal indicates that the mantissa calculation module is working in INT8×4 mode, the encoding reconstruction module divides the first source data into 4 groups of 8-bit suboperands. Each group of 8-bit suboperands forms 4 Booth encoding windows, and the 4 groups form a total of 16 Booth encoding inputs, which are then sent to 16 Booth encoders respectively. The second source data reconstruction module divides the second source data into 4 groups of 8-bit suboperands, which are then sent to the corresponding 16 partial product generation units respectively.

[0058] In INT8×4 mode, the 16 partial products are grouped into 4 INT8 multiplication paths, with each group containing 4 partial products. The mode selection signal controls the reconstruction path to send each group of partial products into the first-level 4-2 compression path for compression. The compressed result directly forms 4 groups of INT8×4 output results, without entering a deeper level of compression network. In other words, in INT8×4 mode, the mantissa calculation module reuses the first-level compression resources to achieve parallel output of four 8-bit multiplications.

[0059] When the mode selection signal indicates that the mantissa calculation module is working in INT16×2 mode, the encoding reconstruction module divides the first source data into two groups of 16-bit suboperands. Each group of 16-bit suboperands forms 8 Booth encoding windows, and the two groups form a total of 16 Booth encoding inputs, which are then sent to 16 Booth encoders respectively. The second source data reconstruction module divides the second source data into two groups of 16-bit suboperands, which are then sent to the corresponding partial product generation units respectively.

[0060] In INT16×2 mode, the 16 partial products are grouped into two INT16 multiplication paths, with each group containing 8 partial products. The mode selection signal first controls the reconstruction path to send each group of partial products into the first-stage 4-2 compression path. After the first-stage compression, the products are sent to the 3-2 compression path through the next-stage reconstruction path for further compression, ultimately forming two groups of INT16×2 output results. In other words, in INT16×2 mode, the mantissa calculation module reuses the first-stage compression path used in INT8×4 mode and adds an additional compression stage afterward, thereby achieving parallel output of two 16-bit multiplications.

[0061] When the mode selection signal indicates that the mantissa calculation module is working in FP8×4 mode, the encoding reconstruction module extracts the four FP8 mantissa segments from the first source data, and performs zero padding, alignment and rearrangement after restoring the hidden bits, so that the four FP8 mantissas are mapped to 16 Booth encoders; the second source data reconstruction module performs the same reconstruction on the four FP8 mantissa segments in the second source data and sends them to 16 partial product generation units.

[0062] In FP8×4 mode, the partial product is organized according to four groups of floating-point mantissa multiplication paths. The mode selection signal controls the right-hand reconstruction path to send the partial product into the first-level 4-2 compression path for compression. The first-level compression result directly forms four groups of FP8×4 output results, without entering deeper floating-point compression levels. In other words, in FP8×4 mode, the mantissa calculation module expands the floating-point mantissa field with zeros and performs group mapping, reuses unified Booth encoding and partial product generation resources, and achieves four-way FP8 mantissa multiplication output through a first-level floating-point compression path.

[0063] In one implementation, when the mode selection signal indicates that the mantissa calculation module is operating in FP16×2 mode, the encoding reconstruction module extracts the two FP16 mantissa segments from the first source data, and after restoring the hidden bits, performs zero padding, alignment, and rearrangement before sending them to 16 Booth encoders. For unused Booth encoding paths, the encoding reconstruction module can prevent the corresponding Booth encoder from participating in effective calculations by zero padding or invalid masking. The second source data reconstruction module processes the two FP16 mantissa segments from the second source data in the same way and sends them to 16 partial product generation units.

[0064] In FP16×2 mode, the partial product is grouped according to two FP16 mantissa multiplication paths. The mode selection signal first controls the right-side reconstruction path to allow the partial product to enter the first-level floating-point 4-2 compression path. The first-level compression result is then sent to the second-level 4-2 compression path through the intermediate reconfig path for further compression, ultimately forming two sets of FP16×2 output results. In other words, in FP16×2 mode, the mantissa calculation module adds an extra level of compression on top of the first-level floating-point compression used in FP8×4 mode to adapt to the compression requirements of wider mantissa segments.

[0065] When the mode selection signal indicates that the mantissa calculation module is operating in FP32×1 mode, the encoding reconstruction module extracts a single FP32 mantissa segment from the first source data, and after restoring the hidden bits, sends it to 16 Booth encoders. The valid Booth encoded input corresponds to the encoding window required for FP32 mantissa multiplication, and the remaining unused paths are closed by padding with zeros or invalid masking. The second source data reconstruction module performs the same processing on the single FP32 mantissa segment in the second source data and sends it to the partial product generation unit.

[0066] In FP32×1 mode, all valid partial products are treated as a single group of FP32 mantissa multiplication partial products and uniformly fed into the Wallace tree compression network. The mode selection signal controls the reconstruction path, causing it to sequentially pass through the first-level compression path, the second-level compression path, and the third-level compression path. This completes the compression of a single-channel FP32 mantissa multiplication, ultimately forming a single group of FP32×1 output results. In other words, in FP32×1 mode, the mantissa calculation module utilizes a complete multi-level compression path to meet the partial product compression requirements of single-channel high-precision floating-point mantissa multiplication.

[0067] In one embodiment, the RPU is further configured with a multi-precision adder, which includes: a second exponent and mantissa unpacking module, configured to unpack the first source data and the second source data to generate the exponent and mantissa portions of the source data; an exponent subtraction module, configured to compare the exponents of the first source data and the second source data and calculate the exponent difference between them; a mantissa alignment module, configured to perform mantissa preprocessing and exponent alignment operations on the first source data and the second source data; a mantissa addition module, configured to sum the mantissas of the aligned first source data and the second source data; and a result generation module, which integrates the summation results to generate the final summation output result.

[0068] In one embodiment, the multi-precision adder further includes: a second anomaly detection module, used to detect whether the first source data and the second source data belong to special numerical cases; if so, a corresponding anomaly status flag is generated, and a corresponding calculation result is formed for the result generation module to select; the special numerical cases include zero, infinity, and NOT.

[0069] The exponent subtraction module receives the exponent information of the two operands from the exponent and mantissa unpacking module, and its output is connected to the mantissa alignment module and the mantissa addition module. Figure 5 The key signals output by this module include information related to the exponent size relationship or exponent difference, such as `exp_large`. This module is used to compare the exponent sizes of two input operands and calculate the exponent difference between them. For floating-point addition, before adding the mantissas, the mantissa of the operand with the smaller exponent must be shifted to the right to bring it to the same exponent level as the operand with the larger exponent. Therefore, the size relationship and difference information output by the exponent subtraction module are prerequisites for the mantissa alignment module to function. The carry-lookahead adder is used to quickly perform exponent difference calculations.

[0070] The mantissa alignment module receives the mantissa segment from the exponent and mantissa unpacking module, and the exponent size relationship and exponent difference information from the exponent subtraction module. Its output is connected to the mantissa addition module, outputting aligned mantissa addition inputs (e.g., `mantissa_add_src`). This module performs mantissa preprocessing and exponent alignment operations in floating-point addition. Since the exponents of two floating-point inputs are usually different, before mantissa addition, the mantissa of the operand with the smaller exponent needs to be right-shifted according to the exponent difference to ensure both operands have the same effective exponent. Simultaneously, when the input operands have different signs and a subtraction operation is actually performed, the mantissa alignment module can also perform preprocessing such as complement calculation so that subsequent adders can uniformly perform sign-dependent addition and subtraction operations.

[0071] The mantissa addition module receives the aligned mantissa input from the mantissa alignment module and the exponent size relationship or larger exponent information from the exponent subtraction module. Internally, the module includes a mantissa addition unit and a normalization unit. Its output is connected to the result generation module, outputting the final exponent and mantissa results used for encapsulation. In this module, the mantissa addition unit first sums the two aligned mantissas; if the addition result contains carry, leading zeros, or other factors affecting normalization, the normalization unit shifts and adjusts the mantissa, simultaneously correcting the corresponding exponent value, thus ensuring that the output result meets the normalization requirements of the target floating-point format.

[0072] The result generation module is located at the output of the entire multi-precision adder. Its inputs include: exception and sum_exception signals from the exception detection module; the normalized exponent and normalized mantissa from the mantissa addition module; and format information related to the current mode. Its outputs include a calculation completion signal, the summation result, and an unrounded output result. This module is used to comprehensively select and finally encapsulate the results from each data path of the adder. When the exception detection module indicates a special case in the input, the result generation module prioritizes generating the corresponding output based on the exception information; otherwise, it re-encapsulates the normalized exponent and mantissa output from the mantissa addition module according to the current precision format to generate the final summation output result.

[0073] The multiplexing logic for multi-precision adders is as follows: For floating-point modes, the multiplexing method of the exponent and mantissa unpacking module and the anomaly detection module is consistent with the corresponding modules in the multi-precision multiplier. That is, the bit width and anomaly judgment rules are resolved by controlling the field through the mode selection signal, and the same set of basic logic is reused in different precision modes. Multiplexing of the exponent subtraction module: In the exponent subtraction module, BF16 mode requires two 8-bit CLAs; FP32 and FP16 modes both reuse the exponent difference path in the BF16 mode. In other words, this module does not build independent exponent comparators and subtractors for the three floating-point formats separately, but rather uses the exponent processing resources required by BF16 as a basis, and achieves unified reuse of the exponent path by controlling the exponent bit width requirements of FP32 and FP16 through mode control. Multiplexing of the mantissa alignment module: The resource requirements of the mantissa alignment module vary with the precision mode: FP32 mode requires one set of 48-bit shifters; FP16 mode requires two sets of 22-bit shifters; BF16 mode requires two sets of 19-bit shifters. After considering the resource reuse requirements, the system uses one 48-bit shifter and one 22-bit shifter as unified basic resources. The shift width, input concatenation method, and complement path are controlled by mode selection signals, thereby achieving mantissa alignment and preprocessing reuse in FP32, FP16, and BF16 modes. Reuse in the mantissa addition and normalization stages: In the mantissa addition and subsequent normalization stages, the three floating-point modes are uniformly reused: one 48-bit CLA for mantissa summation; one 47-bit shifter and one 21-bit shifter for normalization processing. Through mode control, data paths can be switched between single-channel high precision and multi-channel low precision, enabling the same mantissa summation and normalization hardware to support floating-point addition operations in different precision modes.

[0074] For integer modes, INT32, INT16, INT8, and INT4 all uniformly reuse four 32-bit CLA adders. Since integer addition does not involve the exponent comparison, mantissa alignment, and normalization processes specific to floating-point operations, in integer mode, the dedicated floating-point logic can be bypassed or disabled, and the segmented parallel addition of integer data of different bit widths can be directly performed through multiple 32-bit CLAs. The mode selection signal controls the segmentation method of the input data in the four 32-bit CLAs and the result concatenation method, thereby achieving unified hardware multiplexing across multiple integer modes including INT32, INT16, INT8, and INT4.

[0075] Secondly, this application also provides a computing method for AIoT multimodal fusion computing, including: Acquire first source data, second source data, and mode selection signal; The first source data and the second source data are unpacked into an exponent part and a mantissa part; Add the exponential portions of the first source data and the second source data to generate the exponential calculation result; The operating mode of the Booth encoder is determined according to the indication of the mode selection signal, and the mantissa of the reconstructed first source data is input to the Booth encoder for Booth encoding. A partial product is generated based on the Booth encoding result and the mantissa of the reconstructed second source data; the partial product is compressed according to its precision to generate the mantissa calculation result. The results of the exponent calculation and the mantissa calculation are combined to generate the final product output.

[0076] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0077] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0078] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0079] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0080] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0081] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A reconfigurable system on chip for AIoT multi-modal fusion computing, comprising a plurality of computing cores, characterized in that, The computing core is configured with an RPU, RNU, and a reconfiguration execution block; The reconstructed configuration execution block is configured to receive instructions from the superior and reconstruct the first source data and the second source data in terms of precision according to the instructions from the superior. The RPU is equipped with a multi-precision multiplier, which includes: The first exponent and mantissa unpacking module is configured to unpack the reconstructed first source data and second source data to generate the exponent part and mantissa part of the source data; The exponent addition module is configured to add the exponent portions of the first source data and the second source data to generate the exponent calculation result; The mantissa calculation module includes a Booth encoder, a partial product generation unit, and a Wallace compression tree. The Booth encoder is configured to determine its operating mode based on the indication of a mode selection signal. In the current operating mode, the mantissa of the source data is input to the Booth encoder for Booth encoding. The partial product generation unit is configured to generate a partial product based on the Booth encoding result and the mantissa of the reconstructed second source data. The Wallace compression tree is configured to compress the partial product according to its precision to generate the mantissa calculation result. The first result generation module is configured to integrate the exponent calculation result and the mantissa calculation result to generate the final product output result.

2. The system of claim 1, wherein, The Booth encoder has the following operating modes: FP32×1 mode, FP16×2 mode, FP8×4 mode, INT16×2 mode, INT8×4 mode, and INT4×8. In FP32×1 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into one FP32 mantissa segment; in FP16×2 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into two FP16 mantissa segments; in FP8×4 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into four FP8 mantissa segments; in INT16×2 mode, the reconstruction configuration execution block reconstructs the mantissas of the first and second source data into two sets of 16-bit suboperands in parallel with two-way integer multiplication. In INT8×4 mode, the refactoring configuration execution block refactors the mantissa of the first source data and the mantissa of the second source data into four groups of 8-bit suboperands in a parallel four-way integer multiplication method. In INT4×8 mode, the refactoring configuration execution block refactors the mantissa of the first source data and the mantissa of the second source data into eight groups of 4-bit suboperands in parallel eight-way integer multiplication.

3. The system according to claim 2, characterized in that, In FP32×1 mode, the partial product is entered into the Wallace compression tree in one FP32 mantissa multiplication method, and one compressed result is output. In FP16×2 and INT16×2 modes, the partial product is divided into two independent data streams that enter the Wallace compression tree, and two compressed results are output. In FP8×4 and INT8×4 modes, the partial product is divided into four independent data streams and fed into the Wallace compression tree to obtain four-way compression results. In INT4×8 mode, the partial product is divided into eight independent data streams that enter the Wallace compression tree, and the eight-way compression result is output.

4. The system of claim 1, wherein, The exponent addition module includes: The carry-lookahead adder is configured to add the exponent portions of the first and second source data and subtract the corresponding precision offset to generate the exponent calculation result. The overflow / underflow detection module is configured to determine whether the initial value of the exponent part exceeds the representable range of the current floating-point format.

5. The system of any one of claims 1 to 4, wherein, Multi-precision multipliers also include: An anomaly detection module is used to detect whether the first source data and the second source data belong to special value cases; if so, a corresponding anomaly status flag is generated and a corresponding calculation result is formed for the result generation module to select; the special values ​​include zero, infinity, and not a number.

6. The system of any one of claims 1 to 4, wherein, The RPU also includes a multi-precision adder, which includes: The second exponent and mantissa unpacking module is configured to unpack the first source data and the second source data to generate the exponent part and mantissa part of the source data. Exponential subtraction module: configured to compare the exponents of the first source data and the second source data, and calculate the difference between the two exponents; The mantissa alignment module is configured to perform mantissa preprocessing and exponent alignment operations on the first source data and the second source data; The mantissa addition module is configured to sum the mantissas of the first and second source data after alignment processing. The results generation module integrates the summation results to generate the final summation output.

7. The system of claim 6, wherein, Multi-precision adders also include: The second anomaly detection module is used to detect whether the first source data and the second source data belong to special value cases; if so, it generates a corresponding anomaly status flag and forms a corresponding calculation result for the result generation module to select; the special values ​​include zero, infinity, and not a number.

8. A reconfigurable on-chip system reconfiguration method for AIoT multimodal fusion computing, characterized in that, include: Acquire first source data, second source data, and mode selection signal; The first source data and the second source data are unpacked into an exponent part and a mantissa part; Add the exponential portions of the first source data and the second source data to generate the exponential calculation result; The operating mode of the Booth encoder is determined according to the indication of the mode selection signal, and the mantissa of the reconstructed first source data is input to the Booth encoder for Booth encoding. A partial product is generated based on the Booth encoding result and the mantissa of the reconstructed second source data; The partial product is compressed based on its precision to generate the mantissa calculation result; The results of the exponent calculation and the mantissa calculation are combined to generate the final product output.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method of claim 8.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method of claim 8.

Images