Low-bit-width high-energy-efficiency floating point storage and calculation integrated circuit based on partial pre-alignment architecture

Abstract

The invention belongs to the technical field of storage and calculation integration, and particularly relates to a low-bit-width and high-energy-efficiency floating point storage and calculation integrated circuit based on a partial pre-alignment framework. The circuit comprises a memory array, a pre-calculation unit, an adder tree, a configurable arithmetic unit and a normalization unit, and supports mixed precision operation of FP8MACFP4 and FP8MACFP8. The method is characterized in that a partial pre-alignment strategy dominated by an activation value is adopted, the maximum index of the activation value is dynamically counted, the mantissa of the maximum index is aligned, and multiple partial pre-alignment intermediate results are pre-calculated and latched for reuse; in combination with a customized lookup table and a multiplexer, a pre-calculation result is directly selected to replace real-time multiplication and displacement; and through the reconfigurable hardware, the FP8MACFP8 high-precision operation is realized by utilizing the FP8MACFP4 unit combination. According to the method, complete online floating point multiplication and addition operation is realized, and excellent energy efficiency ratio and operation speed are obtained while high precision is kept.

Description

Technical Field

[0001] This invention belongs to the field of in-memory computing technology, specifically relating to a floating-point in-memory computing circuit. Background Technology

[0002] Currently, there is the ReCIM processor (as described in the 2022 IEEE International Solid-State Circuits Conference (ISSCC) paper, "A 28nm 29.2TFLOPS / W BF16 and 36.5TOPS / W INT8 Reconfigurable Digital CIM Processor with Unified FP / INT Pipeline and BitwiseIn-Memory Booth Multiplication for Cloud Deep Learning Acceleration." This technology was proposed by a research team from Tsinghua University and the University of California, Santa Barbara). It aims to address the challenges of supporting high-precision floating-point calculations, improving energy efficiency, and optimizing resource utilization when applying computing-in-memory (CIM) technology to cloud-based deep learning. See [link to relevant documentation]. Figure 1 Its core solution is as follows: (1) In-Memory Alignment-Free FP MACPpipeline: One of the key innovations of this technology is the adoption of an "in-memory unaligned" floating-point multiply-accumulate design. Specifically, the exponent of the input data is uniformly aligned to its local maximum value (EImax) in the input alignment unit (IAU), while the exponent of the weights is aligned offline to its column maximum value (EWmax) after model training and stored in the CIM macro in a specific "Mantissa+" format (a two's complement form containing the sign bit, hidden bit, and mantissa bit). After alignment, the input and weights only need to perform an integer mantissa multiply-accumulate operation (INT Mantissa MAC), thereby avoiding complex real-time exponent alignment operations inside the in-memory array and simplifying the CIM structure.

[0003] (2) Bitwise In-Memory Booth Multiplication (BM2): To accelerate the core's integer multiplication and addition operations, this technology designs a BM2 architecture. The input is bitwise bootth encoded in the BM2 controller to generate control signals such as NEG, TWO, and ZERO. In the digitized CIM macro, the partial product is recoded using customized logic circuits (such as a 4T XOR gate based on a transmission gate and additional control transistors), enabling it to directly generate {0, ±W, ±2W}, rather than the traditional multi-bit partial product.

[0004] (3) Unified reconfigurable FP / INT data flow: Through the pre-alignment technology and reconfigurable accumulator design described above, ReCIM can efficiently support BF16 / FP32 floating-point operations and INT8 / INT16 integer operations on the same hardware. Its accumulator can be reconfigured at both the subarray level and the macro level according to the operation type and precision. For example, multiple adders can be combined to support the higher precision accumulation required by FP32, thereby improving the utilization of hardware resources and avoiding the area and power consumption overhead caused by the separate FP and INT engines in traditional designs.

[0005] The aforementioned prior art has the following drawbacks: (1) Limitations and Complexity Transfer Issues in the Computational Architecture: Although this technology avoids the hardware complexity caused by alignment operations through a pre-alignment strategy, its core computation still relies on peripheral digital logic units to complete the key steps of floating-point computation. Specifically, pre-alignment operations such as weight exponent comparison and mantissa shifting need to be completed in dedicated digital units, while the in-memory computing (CIM) macro is only responsible for the multiplication and accumulation operations of fixed-point mantissas. This architecture fails to achieve truly complete floating-point in-memory computing, resulting in frequent data transfer between peripheral logic and in-memory arrays, limiting further improvements in energy efficiency. The essence of this solution is the transfer of computational complexity rather than a fundamental solution. It removes the most complex exponent alignment operation in floating-point computation from the CIM core and entrusts it to peripheral dedicated digital circuits. This design strategy does not eliminate computational complexity, but only redistributes its physical location on the chip. More importantly, this complexity transfer leads to a high degree of coupling between the computing unit and external circuits. The efficient operation of the CIM core strongly depends on the correctness and real-time performance of the peripheral preprocessing circuits, forming a tight pipeline dependency between the two.

[0006] (2) Accuracy Guarantee Depends on Model Characteristics: This technique claims that "the numerical locality of deep learning models means that the exponent shift is usually less than 4 bits, and the shift operation will not cause accuracy loss." However, this conclusion heavily depends on the value locality of the specific model. For different deep learning models or application scenarios (such as scientific computing and high-precision inference), the exponent shift range may significantly exceed 4 bits, resulting in the loss of significant bits during mantissa shift, thus introducing non-negligible computational errors. This accuracy dependency limits the versatility and reliability of this technique, and cannot guarantee that the required computational accuracy can be maintained in all application scenarios.

[0007] The root cause of these shortcomings is: (1) The design adopts a compromise architecture, attempting to simplify CIM implementation by separating preprocessing and core computation, but in reality, it fails to integrate the complete floating-point computation process into the in-memory computing unit; (2) Its accuracy guarantee relies on statistical assumptions about the model data, rather than strict mathematical equivalence relations. This technique avoids accuracy loss through the empirical observation that "the exponential offset is usually less than 4 bits", but this conclusion varies from model to model and cannot provide deterministic guarantees for all scenarios, lacking a universal accuracy control scheme. Summary of the Invention

[0008] The purpose of this invention is to provide a low-bit-width, high-efficiency floating-point in-memory computing circuit based on a partially pre-aligned architecture, so as to solve the technical problems of low energy efficiency, poor flexibility, and loss of accuracy in floating-point computing in in-memory computing architecture.

[0009] The low-bit-width, high-efficiency floating-point in-memory computing circuit based on a partially pre-aligned architecture provided by this invention is described in the following architecture: Figure 2 As shown, it supports FP8_MAC_FP4 and FP8_MAC_FP8 formats, specifically including: memory array, pre-computation unit (PCU), adder tree (ADT), configurable arithmetic unit (CAC), and normalization unit (Norm). Among them: The memory array is used to store weight data (Weight, FP8 / FP4 formats). The pre-computation unit (PCU), connected to the memory array, is used to parse and preprocess the exponent and mantissa of the input activation value, and generate an initial partial product (Prod_xx) based on the weight data to facilitate subsequent addition reduction; the preprocessing includes shift addition operation, two's complement operation, and coarse-grained pre-alignment operation based on the maximum exponent of the current activation value group obtained by statistics, and pre-computes and generates multiple intermediate results which are latched into registers; The addition tree (ADT) is connected to the pre-computation unit and is used to perform parallel addition reduction on the partial product output by the pre-computation unit. The configurable arithmetic unit (CAC) is connected to the addition tree and is used to select and perform different arithmetic operations according to the operation mode. The normalization unit (Norm), connected to the configurable arithmetic unit, is used to normalize the operation results to conform to the floating-point standard BF16 representation.

[0010] The overall architecture can perform efficient multiply-add and post-processing operations in mixed precision FP8_MAC_FP4 and FP8_MAC_FP8. The circuit adopts a segmented alignment architecture, pre-calculates to reduce power consumption, and does not use approximate calculations to ensure accuracy. Specifically, the pre-calculation unit first performs coarse-grained pre-alignment of activation values, and the result is reused in multiple subsequent calculation cycles. In subsequent calculations, the customized barrel shifter performs supplementary fine-grained exponential compensation.

[0011] The pre-computation unit (PCU) specifically includes: (1) Shift-add array, two's complement unit, and alignment unit; wherein: Shifter Adder (SA) is used for pre-computed mantissa multiplication operations. Shifter Adder is used to handle the case where the mantissa of the weight is 2'b11. Direct copying is used to handle the mantissa of the weight is 2'b10 or 2'b01. Pre-computed operations are not required for the mantissa of the weight is 2'b00. The Two's Complement (TC) unit is used to implement sign extension and two's complement conversion, and to handle the positive and negative signs of the weight sign bit. Exponent Alignment (EA): Used for coarse-grained alignment of the floating-point exponent of the activation value.

[0012] Specifically, the architecture includes MAX_EXP_TREE and EXP_DIFF modules, which respectively implement exponent comparison and difference calculation in the Activation vector to drive the alignment logic of EA.

[0013] This entire module executes in parallel the difference calculation by the MAX_EXP_TREE and EXP_DIFF modules and the mantissa multiplication operation by the shift-and-add module. The two results obtained from the mantissa multiplication are first taken in two's complement, and then a certain shift space is reserved. Then, the right shift is performed according to the difference of the exponent terms for coarse-grained alignment, resulting in four results P01, N01, P11, and N11, which correspond to the four results after Act (original value of Activation), ~Act (original value of Activation taken in two's complement), Act<<1 + Act (original value of Activation shifted and added), and ~(Act<<1 + Act) (original value of Activation shifted and added and then taken in two's complement), respectively.

[0014] (2) Pre-computation Unit (PCU) result storage and selection After the above processing is completed, the results of P01, N01, P11, and N11 are sent to four sets of registers. These pre-calculated results, which are dynamically latched in the registers, can be reused for the subsequent 128 calculation cycles, and their calculation power consumption can be spread over these 128 cycles, achieving the goal of high energy efficiency.

[0015] (3) Final output of the pre-computation unit (PCU) Specifically, it is implemented using a custom cell with a 5-to-1 non-NMOS transmission gate structure (5Mux1B). This custom cell has no static power consumption and is smaller and faster than the standard cell library.

[0016] During computation, the target budget result is first selected from the register set (four pre-computed results) and the all-zero result (corresponding to the case with a mantissa of 2'b00) by a multiplexer (implemented by 5Mux1B). Then, the BarrelShift (also implemented by 5Mux1B) shift unit completes fine-grained shifting to resolve the slight shift deviation caused by the weights. Finally, the order of the multiplication operation is aligned, and the result can be directly entered into the addition tree for merge addition.

[0017] This is a method using a custom lookup table (LUT) and a multiplexer (Mux). The Mux directly selects the corresponding pre-computed result from the LUT, avoiding real-time multiplication operations and reducing the power consumption of the computation logic.

[0018] See above Figure 2 The right side of the architecture.

[0019] The Configurable Arithmetic Unit (CAC) selects different arithmetic operation modes based on control signals, exhibiting strong adaptability. Specifically, the CAC supports high-bit-width operations built by reusing low-bit-width computing hardware: FP8_MAC_FP8 can be configured to be implemented by combining two FP8_MAC_FP4 computing units with auxiliary compensation logic (an additional shifter and an additional adder). This unit achieves efficient hardware reuse of computing resources in different precision modes, providing a new computing paradigm for the floating-point operators of ActivationStationary.

[0020] See above Figure 2 The left side of the architecture.

[0021] The segment alignment technology of the low-bit-width, high-efficiency floating-point in-memory computing circuit of the present invention has the following advantages: (1) In terms of timing, the lookup table operation of the activation value is performed first, including shift and addition operation, two's complement operation and coarse-grained pre-alignment operation based on the maximum exponent of the current activation value group obtained by statistics. This result can be reused in all subsequent calculation cycles to form an equivalent pipeline structure. Subsequent calculations only need to perform supplementary fine-grained exponent compensation and lookup table query, which significantly improves throughput. (2) In terms of power consumption, the power consumption of the highly complex pre-alignment operation is spread over the entire multiplexing cycle, while the supplementary alignment is implemented using a customized NMOS transmission gate, which effectively reduces dynamic power consumption. (3) In terms of area, by decomposing the traditional large bit-width shifter into multi-level small-scale alignment units (pre-alignment module, alignment barrel shifter, supplementary alignment shifter), the rational allocation of hardware resources is achieved. Finally, coordinated optimization is achieved in terms of timing, power consumption and area. Attached Figure Description

[0022] Figure 1 This is the existing framework diagram of the Reconfigurable Digital CIM Processor.

[0023] Figure 2 This is a low-bit-width, high-efficiency floating-point CIM framework diagram based on a partially pre-aligned architecture. Detailed Implementation

[0024] The mixed-precision floating-point in-memory computing architecture described in this embodiment is as follows: Figure 2 As shown, it mainly consists of a memory array, a pre-computation unit (PCU), an adder tree (ADT), a configurable arithmetic unit (CAC), a normalization unit (Norm), and refined functional units such as shift, alignment, and complement operations. The overall architecture can perform efficient multiplication, addition, and post-processing operations in FP8_MAC_FP4 and FP8_MAC_FP8 mixed precision.

[0025] (1) Storage and pre-computation On the left side of the architecture, there is an S8T1 SRAM Array (256b×128) for storing weights (in FP8 / FP4 formats). Next is the PCU (Pre-Compute Unit), which parses and preprocesses the input data for the exponent and mantissa, and generates an initial partial product (Prod_xx) for subsequent addition reduction.

[0026] (2) Addition of last digits and normalization To the right of the PCU, an ADT (Adder Tree) is arranged. The Adder Tree implements the parallel addition and merging operation of multi-way partial products. In this embodiment, a single ADT can complete the mantissa addition part of FP8_MAC_FP4.

[0027] The output of the addition tree is further fed into the CAC (Configurable Arithmetic Cell) and the Norm (Normalization Cell). The CAC can select different arithmetic operation modes according to the control signal, and has strong adaptability; the Norm normalizes the result to ensure that the output conforms to the floating-point standard representation.

[0028] The detailed structure of the pre-computation unit expansion further includes: (1) Shift, alignment and two's complement operations SA (Shifter Adder): Implements shift-add operations. TC (Two's Complement): Implements sign extension and two's complement conversion; EA (Exponent Alignment): Used to align floating-point exponents, ensuring that operands of different precisions or numerical ranges can correctly participate in addition and normalization.

[0029] In addition, the architecture also includes the MAX_EXP_TREE and EXP_DIFF modules, which respectively implement the Activation exponent comparison and difference calculation to drive the alignment logic of EA.

[0030] (2) Result storage and selection After the above processing is completed, the results are sent to four key registers: P01, N01, P11, and N11, which correspond to the four results Act, ~Act, Act<<1 + Act, and ~(Act<<1 + Act), respectively.

[0031] By storing candidate results in parallel, the system can flexibly select the required computational output in subsequent stages.

[0032] (3) PCU final output In the output stage, the target budget result is first selected from the register by a multiplexer (implemented by 5Mux1B), and then fine-grained shifting is completed by shift units such as BarrelShift (also implemented by 5Mux1B) to finally form an output result that conforms to the target format (BF16).

[0033] This invention also included simulation tests. First, one operation was defined as representing either a multiplication or an addition, and the results were measured in dense mode. Based on this standard, M. Li et al.'s "SLAM-CIM: A Visual SLAM BackendProcessor With Dynamic-Range-Driven-Skipping Linear-Solving FP-CIM Macros," implementing an FP16 precision SLAM processor on a 28nm process, achieved an energy efficiency of 13.325 TFLOPS / W under 0.5V and 12.5MHz operating conditions. This work, using a hybrid precision architecture combining FP8 multiplication-accumulation and FP4 precision on a 12nm process, achieved an energy efficiency of 12.96 TFLOPS / W under 0.8V and 384MHz operating conditions; under the same conditions, the FP8 multiplication-accumulation architecture achieved an energy efficiency of 4.73 TFLOPS / W.

[0034] In summary, this invention achieves mixed-precision multiply-accumulate operations of FP8_MAC_FP4 and FP8_MAC_FP8 by organically combining a memory array, a pre-computation unit, an addition tree, a configurable arithmetic unit, and a normalization unit, supplemented by auxiliary modules such as alignment, shifting, and two's complement. This architecture ensures both computational accuracy and flexibility and energy efficiency, making it suitable for core computing scenarios in deep learning accelerators.

Claims

1. A low-bit-width, high-efficiency floating-point in-memory computing circuit based on a partially pre-aligned architecture, characterized in that, Supports FP8_MAC_FP4 and FP8_MAC_FP8 formats, specifically including: memory array, pre-computation unit (PCU), adder tree (ADT), configurable arithmetic unit (CAC), and normalization unit (Norm); performs efficient multiply-add and post-processing operations in FP8_MAC_FP4 mixed precision; among which: The memory array is used to store weight data; The pre-computation unit (PCU), connected to the memory array, is used to parse and preprocess the exponent and mantissa of the input activation value, and generate an initial partial product based on the weight data to facilitate subsequent addition reduction. The preprocessing includes shift addition operation, two's complement operation, and coarse-grained pre-alignment operation based on the maximum exponent of the current activation value group obtained by statistics, and pre-computes and generates multiple intermediate results which are latched into registers. The addition tree (ADT) is connected to the pre-computation unit and is used to perform parallel addition reduction on the partial product output by the pre-computation unit. The configurable arithmetic unit (CAC) is connected to the addition tree and is used to select and perform different arithmetic operations according to the operation mode. The normalization unit (Norm) is connected to the configurable arithmetic unit and is used to normalize the operation results to conform to the floating-point standard BF16 representation. The overall architecture performs efficient multiplication, addition, and post-processing operations in mixed precision FP8_MAC_FP4 and FP8_MAC_FP8. The circuit adopts a segmented alignment architecture, pre-calculates to reduce power consumption, and does not use approximate calculations to ensure accuracy. Specifically, the pre-calculation unit first performs a lookup table operation on the mantissa of the activation value, including shift-add operation, two's complement operation, and coarse-grained pre-alignment operation. The result is reused in multiple subsequent calculation cycles. In subsequent calculations, the customized barrel shifter performs supplementary fine-grained exponential compensation.

2. The floating-point in-memory computing circuit according to claim 1, characterized in that, The pre-computation unit (PCU) specifically includes: (1) Shift-add array, two's complement unit, and alignment unit; wherein: The shift-add array (SA) is used for pre-computed mantissa multiplication operations. Shift-add is used when the mantissa of the weight is 2'b11, direct connection assignment is used when the mantissa of the weight is 2'b10 or 2'b01, and no pre-computed operation is required when the mantissa of the weight is 2'b00. The two's complement unit (TC) is used to implement sign extension and two's complement conversion, and to handle the positive and negative signs of the weight sign bit; Alignment cells (EAs) are used to perform coarse-grained alignment of the floating-point exponents of the activation values; Specifically, the architecture includes MAX_EXP_TREE and EXP_DIFF modules, which respectively implement exponent comparison and difference calculation in the Activation vector to drive the alignment logic of EA; This part executes the difference calculation by the MAX_EXP_TREE and EXP_DIFF modules and the mantissa multiplication operation by the shift-add array module in parallel. The two results obtained from the mantissa multiplication are first taken as two's complement, and then a certain shift space is reserved. Then, the right shift is performed according to the difference of the exponent terms for coarse-grained alignment, resulting in four results: P01, N01, P11, and N11, which correspond to the original value of Activation, the original value of Activation taken as two's complement, the original value of Activation shifted and added, and the original value of Activation shifted and added and then taken as two's complement after calculation. (2) Pre-computation Unit (PCU) result storage and selection After the above processing is completed, the results of P01, N01, P11, and N11 are sent to four sets of registers; these pre-calculated results, which are dynamically latched in the registers, are reused for the subsequent 128 calculation cycles, and their calculation power consumption is spread out over these 128 cycles to achieve the goal of high energy efficiency. (3) Final output of the pre-computation unit (PCU) It is specifically implemented by a custom 5-to-1 non-NMOS transmission gate structure (5Mux1B). Specifically, the target budget result is first selected from the register group of four pre-calculated results and the all-zero result corresponding to the case of mantissa 2'b00 by a multiplexer implemented by 5Mux1B. Then, fine-grained shifting is completed by the BarrelShift shifting unit, which is also implemented by 5Mux1B, to solve the slight shift deviation caused by the weight. Finally, the order alignment of the multiplication operation is completed, and the result is directly entered into the addition tree for merge addition.

3. The floating-point in-memory computing circuit according to claim 2, characterized in that, The configurable arithmetic unit (CAC) supports high-bit-width operations built by reusing low-bit-width computing hardware: FP8_MAC_FP8 is configured to be implemented by two FP8_MAC_FP4 computing units plus an auxiliary compensation logic combination consisting of additional shifters and additional adders; this unit realizes efficient hardware reuse of computing resources in different precision modes, providing a new computing paradigm for the floating-point computing operators of Activation Stationary.

Images