A mixed-precision dequantization system and method for in-memory computing

Abstract

The application discloses a mixed-precision dequantization system and method for in-memory computing, and belongs to the field of integrated circuits and artificial intelligence hardware. The system comprises a digital domain pre-alignment module, an analog in-memory computing core and a general dequantization DQ-SIMD unit. The pre-alignment module completes floating-point activation processing and generates an activation value compensation term. The analog in-memory computing core completes multiplication and accumulation and eliminates physical offset errors based on a 1T1R single-terminal array. The dequantization unit realizes fixed-point to floating-point conversion and asymmetric quantization inverse transformation through a three-stage pipeline. The application proposes a single-terminal storage scheme of non-negative shift mapping in view of the core contradiction between mixed-precision computing requirements and the non-negativity of device conductance when deploying a large language model inference on a 1T1R structure RRAM array. The single-terminal storage scheme is matched with a double-error elimination mechanism of software and hardware cooperation to realize single-terminal storage of signed weights by superimposing a fixed offset in the analog domain and to eliminate physical errors introduced by the offset in real time through a pipeline circuit in the digital domain.

Description

Technical Field

[0001] This invention relates to the fields of integrated circuit design, in-memory computing (CIM) and artificial intelligence hardware acceleration technology, specifically to a mixed-precision inverse quantization system and method for in-memory computing, which is particularly suitable for in-memory computing acceleration chips based on a 1T1R structure for Large Language Model (LLM) inference. Background Technology

[0002] With the exponential growth of parameters in large language models, low-power, high-density deployment on edge devices has become a key bottleneck for the industrialization of artificial intelligence. In-memory computing technology, by performing matrix-vector multiplication (MVM) in situ within the storage array, fundamentally eliminates the limitations of traditional von Neumann architectures. The "memory wall" problem of the von Neumann architecture is a core technological path to overcome the energy efficiency limits of large language model inference. RRAM arrays based on the 1T1R (one transistor, one resistive random access memory) structure have become an ideal carrier for in-memory computing acceleration chips for large language model inference due to their advantages of high storage density and low operating power consumption.

[0003] Existing technologies for deploying large language models onto RRAM-based in-memory computing chips with a 1T1R structure suffer from two major technical drawbacks: First, the conflict between physical device characteristics and algorithm accuracy requirements. The intrinsic non-negativity of the conductance of RRAM devices prevents direct physical mapping of the signed integer weights prevalent in neural networks. Current mainstream solutions employ a 2T2R differential pair structure, storing positive and negative components in a pair of devices and then subtracting the currents to obtain the signed output. This approach directly halves the storage density while doubling the array area and power consumption, significantly diminishing the technical advantages of RRAM in-memory computing. Second, mixed precision... The circuit overhead and efficiency issues of dequantization: Large language model inference generally adopts an asymmetric mixed-precision quantization mode that combines floating-point activation values ​​(FP8 / FP16 / BF16) with fixed-point weights (INT4 / INT8). The fixed-point multiplication and accumulation results completed in the analog domain need to be restored to floating-point numbers with high dynamic range. However, the existing RRAM in-memory computing architecture lacks efficient on-chip dequantization circuits and often relies on off-chip general-purpose processors to complete the dequantization process. This introduces a lot of data transfer delay and additional system power consumption. At the same time, off-chip processing is difficult to match the parallel computing characteristics of analog in-memory computing, resulting in a significant decrease in overall inference throughput.

[0004] Furthermore, existing on-chip dequantization solutions often employ simple linear scaling methods, which cannot support the complete inverse transformation of asymmetric quantization, resulting in significant accuracy loss and failing to meet the high-precision requirements of large language model inference. Simultaneously, the timing compatibility between the dequantization circuit and the in-memory computing core is poor, hindering the formation of a deep pipeline and making it difficult to support the high-throughput inference requirements of continuous token streams in large language models. Therefore, the industry urgently needs a hybrid precision dequantization algorithm and circuit implementation scheme that supports 1T1R single-ended arrays, high storage density, high energy efficiency, on-chip closed-loop operation, and high precision. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a mixed-precision dequantization system and method for in-memory computation, coupled with a dual error elimination mechanism involving both hardware and software: in the analog domain, signed weighted single-ended storage is achieved by superimposing a fixed offset; in the digital domain, the physical error introduced by this offset is eliminated in real time through a pipelined circuit. This design achieves high-energy-efficiency and high-precision floating-point and fixed-point mixed computation while doubling the storage density.

[0006] To achieve the above objectives, the technical solution of the present invention is implemented as follows: A mixed-precision dequantization system for in-memory computing includes a digital domain pre-alignment module, an analog in-memory computing core, and a general-purpose dequantization DQ-SIMD unit connected in sequence by signals. The digital field pre-alignment module is configured to receive floating-point format activation vectors and externally provided shared maximum exponents, dynamically align the mantissas of each vector and convert them into binary two's complement, and simultaneously generate activation value compensation terms Psum through parallel accumulation via a bypass addition tree. The analog in-memory computing core adopts a 1T1R single-ended RRAM array structure. It maps signed logical weights with fixed offsets to non-negative physical weights, performs analog domain multiplication and accumulation operations, and in the digital shift and accumulation stage, it first converts Psum from the complete fixed-point aligned domain to the bit-slice reconstructed domain, then multiplies it by the fixed offset Offset and subtracts it from the accumulation result all at once to eliminate offset error. When the weight is INT4, Offset=8, the operation is equivalent to performing an arithmetic right shift of Psum by 17 bits and then multiplying it by 8, outputting the digital part and Isum. The general-purpose dequantization DQ-SIMD unit is connected to the analog in-memory computing core. It adopts a three-stage pipeline architecture to convert the Isum into a standard floating-point format and complete the inverse quantization transformation to output a high-precision floating-point result vector.

[0007] Furthermore, the fixed offset is Offset=2. (N-1) Where N is the bit width of the signed logical weights, such that the mapped physical weights take values ​​in the range [0, 2]. N-1[, maintain non-negativity.]

[0008] Furthermore, the digital domain pre-alignment module includes a barrel shifter array, a two's complement conversion logic, a bit slicing unit, and a bypass adder tree; the input of the bypass adder tree is coupled to the output of the two's complement conversion logic, and the Psum is obtained by parallel calculation based on the signed two's complement data, and the Psum does not enter the 1T1R RRAM array of the analog memory computing core, but is directly transmitted to the back-end digital circuit for error correction.

[0009] Furthermore, the analog in-memory computing core does not contain a 2T2R differential device pair, but is composed of a digital-to-analog converter (DAC), a 1T1RRRAM array, an analog-to-digital converter (ADC), and a digital shift and accumulation circuit. The digital shift and accumulation circuit is used to reassemble multi-cycle bit slice data into complete values, and performs a one-time subtraction correction during the accumulation completion stage to eliminate the systematic physical offset error introduced by the fixed offset.

[0010] Furthermore, the three-stage pipeline of the general-purpose dequantization DQ-SIMD unit is arranged in the order of data processing as scalar preprocessing stage, vector transformation stage, and SIMD arithmetic logic stage. The three-stage pipeline adopts a time-overlapping parallel processing method.

[0011] Furthermore, the scalar preprocessing stage is configured to obtain the actual base index by subtracting a preset Block offset 25 from the shared maximum index; wherein the Block offset 25 consists of an FP16 mantissa width of 10 and an FP16 exponent offset of 15, and converts the externally input 64-bit double-precision scaling factor Scale into a 16-bit half-precision floating-point format, while handling data underflow and saturation issues. The converted scaling factor is used for global broadcasting of subsequent vector calculations.

[0012] Furthermore, the vector conversion stage is a multi-channel parallel structure, with each channel integrating a leading zero detector and a barrel shifter to extract the absolute value of Isum, detect leading zeros, and normalize it in real time. The exponent is dynamically corrected according to the shift amount and packaged to generate a standard IEEE FP16 floating-point vector.

[0013] Furthermore, the SIMD arithmetic logic level includes 64 parallel FP16 subtractors and 64 parallel FP16 multipliers, which sequentially perform a linear transformation by subtracting the sum of activation values ​​offset Asum and multiplying by the quantization scaling factor Scale. The calculation formula is Result=(O float A sum )×Scale, where Result is the final output standard FP16 floating-point result, O float A is the intermediate floating-point vector after converting an integer to a floating-point number. sumThe sum of activation values ​​is offset, and Scale is the quantization scaling factor; both the subtractor and multiplier are implemented using synthesizable digital logic, with built-in logic for automatic alignment, mantissa operation, and result normalization.

[0014] To achieve the above objectives, the present invention also provides a mixed-precision dequantization method for in-memory computation, applied to the aforementioned mixed-precision dequantization system, comprising the following steps: S1: During the compilation phase, signed logical weights are superimposed with a fixed offset Offset=2. (N-1) , mapped to non-negative physical weights and written into the 1T1R single-ended RRAM array of the analog memory computing core; S2: During the hardware operation phase, the digital field pre-alignment module performs dynamic mantissa alignment and binary two's complement conversion on the floating-point activation vector, generates a fixed-point two's complement sequence and activation value compensation term Psum, and caches the total offset of activation values ​​Asum. S3: The analog in-memory computing core receives the fixed-point two's complement sequence and performs analog domain multiplication and accumulation operations. After ADC quantization, the digital shift and accumulation circuit reconstructs the multi-cycle bit slice data. In the accumulation completion stage, Psum is first arithmetically shifted right by 17 bits to align with the reconstructed domain of the bit slice. Then, it is multiplied by a fixed offset Offset and deducted from the accumulation result all at once. When the weight is INT4, Offset=8. The operation is equivalent to arithmetically shifting Psum right by 17 bits and then multiplying by 8. The output is the digital part and Isum that eliminate the physical offset error. S4: The general-purpose dequantization DQ-SIMD unit receives the Isum and sequentially completes scalar preprocessing, integer-to-float conversion, and floating-point linear transformation through a three-stage pipeline, outputting a high-precision floating-point result vector.

[0015] Furthermore, the floating-point linear transformation in step S4 is executed in parallel across multiple channels. First, the sum of global broadcast activation values ​​offset Asum is subtracted from the normalized floating-point data. Then, the subtraction result is multiplied element-wise by the preprocessed quantization scaling factor Scale to complete the on-chip inverse transformation process of "device-side offset compensation + algorithm-side linear dequantization". The device side is used to eliminate the system terms introduced by the fixed offset, and the algorithm side performs linear dequantization with zero-point terms. The entire transformation process is completed in a closed loop on-chip without the participation of an off-chip processor.

[0016] Beneficial effects: 1. Addressing the physical limitation of non-negative conductance in RRAM devices, this invention abandons the traditional inefficient 2T2R differential pair mapping architecture and innovatively adopts single-ended non-negative mapping and a real-time compensation mechanism for digital domain Psum. This design improves the effective area utilization of the analog array by approximately 2 times under the same storage capacity. Simultaneously, by eliminating the differential pair structure, the number of associated sensing amplifiers, reference current sources, and peripheral drive circuits is reduced proportionally, significantly lowering the overall hardware cost and design complexity of the chip, laying the foundation for the mass production of large-scale in-memory computing chips.

[0017] 2. This invention constructs a multi-stage deep pipeline architecture (7 stages of preprocessing + 1 stage of array + 5 stages of post-processing + 3 stages of dequantization), decoupling the complex mixed-precision dequantization process into three time-overlapping parallel stages: "digital preprocessing domain" → "analog computation domain" → "digital post-processing domain," forming a highly efficient pipelined ping-pong operation. This design effectively masks the combinational logic delays caused by floating-point operations, leading zero detection, and multi-stage shifting, enabling the processing speed of the digital circuit to match or even exceed the computation cycle of the analog array. Therefore, when processing continuous data streams (such as LLM token sequences), it achieves practical performance close to the peak theoretical throughput.

[0018] 3. This circuit natively supports the linear inverse quantization formula Result=(O) at the hardware level. float A sum The complete inverse transformation of )×Scale avoids the accuracy loss caused by traditional symmetric quantization or simple linear scaling, ensuring that the end-to-end computational accuracy of LLM inference is consistent with that of software simulation.

[0019] 4. By integrating a complete mixed-precision inverse quantization circuit on-chip, this invention achieves end-to-end on-chip closed-loop processing from "analog partial sum" to "standard FP16 floating-point output," completely avoiding the overhead of transferring the massive intermediate sum (Isum, typically 32-bit × 64 channels = 2KB / time) to an off-chip CPU / GPU for post-processing. This design significantly reduces the amount of data transferred between chips, overcoming the bandwidth limitations caused by the "von Neumann bottleneck" and "memory wall." The overall system energy efficiency ratio (TOPS / W) is estimated to improve by 3-5 times, making it particularly suitable for battery-powered edge AI devices and low-power smart terminals. Attached Figure Description

[0020] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an undue limitation of the invention. In the drawings: Figure 1This is a schematic diagram of the structure of the hybrid precision dequantization system for in-memory computing as described in an embodiment of the present invention; Figure 2 This is a functional schematic diagram of a hybrid precision dequantization system for in-memory computing according to an embodiment of the present invention; in the figure, (a) is a digital domain pre-alignment module, (b) is an analog in-memory computing core, and (c) is a general dequantization DQ-SIMD unit; Figure 3 This is a schematic diagram of the three-stage pipeline architecture of the general dequantization DQ-SIMD unit of the mixed-precision dequantization system for in-memory computing as described in an embodiment of the present invention. Figure 4 This is a schematic diagram of the four-stage pipeline hardware architecture of the digital domain pre-alignment module according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the digital shift accumulation circuit of the analog in-memory computing core of the hybrid precision dequantization system for in-memory computing described in an embodiment of the present invention. Figure 6 This is a schematic diagram illustrating the integer-to-floating point principle of the vector conversion stage of the general dequantization DQ-SIMD unit in the mixed-precision dequantization system for in-memory computing described in this embodiment of the invention. Figure 7 This is a complete timing diagram of the hybrid precision dequantization system for in-memory computing described in this embodiment of the invention; Figure 8 This is a flowchart of a mixed-precision dequantization method for in-memory computation as described in an embodiment of the present invention. Detailed Implementation

[0021] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other.

[0022] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0023] Example 1 See Figures 1-7 A mixed-precision dequantization system for in-memory computing includes a digital domain pre-alignment module, an analog in-memory computing core, and a general-purpose dequantization DQ-SIMD unit connected in sequence. The digital field pre-alignment module (Pre_aligner) is configured to receive floating-point format activation vectors and externally provided shared maximum exponents, dynamically align the mantissas of each vector and convert them into binary two's complement, and simultaneously generate activation value compensation terms Psum through parallel accumulation via a bypass addition tree; The digital domain pre-alignment module in this embodiment adopts a parallel pipeline architecture and serves as a bridge connecting the floating-point arithmetic domain and the fixed-point physical domain, responsible for processing floating-point format input activation values. It receives the input activation vector and its shared maximum exponent (max_e), and dynamically aligns the mantissas of all floating-point activation values ​​through internal shift logic, thereby converting the floating-point data into a fixed-point representation based on the shared exponent. The aligned mantissas are further converted into two's complement to meet the input requirements of the CIM array's digital interface (DAC). After performing the two's complement conversion, the circuit simultaneously sends the generated signed two's complement data to the serialization logic and the splitter path. The adder tree on the splitter path synchronously calculates the signed sum of all channels, generating an activation value compensation term (Psum) for subsequent elimination of systematic errors introduced by the weight offset.

[0024] The analog in-memory computing core adopts a 1T1R single-ended RRAM array structure. Signed logic weights are superimposed with fixed offsets and mapped to non-negative physical weights. Analog domain multiplication and accumulation operations are performed. In the digital shift and accumulation stage, Psum is first converted from the full fixed-point aligned domain to the bit slice reconstruction domain (the pre-alignment Stage2 concatenates 11 bits of the FP16 mantissa with 16 bits of zeros to form a 28-bit aligned value, and the bit slice is aligned[26:17], so the slice LSB weight is bit17; Psum=Σfull_aligned, after shifting right by 17 bits, the dimension is consistent with the bit slice reconstruction domain, and then multiplied by Offset to obtain the error compensation amount). After multiplying by the fixed offset Offset, it is deducted from the accumulation result all at once to eliminate the offset error. When the weight is INT4, Offset=8. The operation is equivalent to performing an arithmetic right shift of Psum by 17 bits and then multiplying by 8 (i.e., arithmetic right shift of 14 bits), and outputting the digital part and Isum. In this embodiment, the core storage unit for analog in-memory computing adopts a 1T1R structure. To address the conflict between signed integer weights (Logic Weight, Wsym) and the positive conductance characteristics of RRAM, this embodiment employs a non-negative mapping mechanism of "Logic Weight + Fixed Offset" to map the signed weights to non-negative physical conductance. The array receives the fixed-point complement sequence output by the digital domain pre-alignment module and performs a parallel voltage-conductance multiplication and accumulation operation in the analog domain. At this time, the analog current output by the array contains systematic physical errors introduced by the fixed offset. After being quantized by the analog-to-digital converter (ADC), the analog output enters the digital shift-accumulator circuit, which reconstructs the multi-cycle bit slice data into a complete value. During the accumulation phase, Psum, output by the pre-alignment module, is introduced to perform a one-time subtraction correction on the reconstructed full-precision result, thereby removing the systematic physical errors introduced by the fixed offset (Offset) at the analog end in real time in the digital domain.

[0025] The general-purpose dequantization DQ-SIMD unit is connected to the analog in-memory computing core. It adopts a three-stage pipeline architecture to convert the Isum into a standard floating-point format and complete the inverse quantization transformation to output a high-precision floating-point result vector.

[0026] The general-purpose dequantization DQ-SIMD unit in this embodiment receives the physically debiased digital part sum (Isum), uses a three-stage pipeline architecture to recover the fixed-point integer to high-precision floating-point number, and completes the inverse transformation of asymmetric quantization; The scalar preprocessing stage calculates the actual base exponent by subtracting the preset Block offset of 25 from the shared maximum exponent (max_e) of the external input, providing a base for subsequent floating-point format packaging. At the same time, it converts the 64-bit double-precision scaling factor Scale of the external input into a 16-bit half-precision floating-point format to handle data underflow and saturation issues. The converted scaling factor is stored in a register for global broadcasting in subsequent vector calculations.

[0027] Vector conversion stage: It is a 64-channel parallel structure, with each channel integrating a leading zero detector and a barrel shifter. It receives the 32-bit fixed-point integer Isum output from the analog in-memory computing core, first extracts the sign bit of Isum and takes its absolute value, and then detects the number of leading zeros in the absolute value through the leading zero detector. Based on this, it controls the barrel shifter to perform real-time normalization of the mantissa, aligning the mantissa to the standard IEEE FP16 implicit 1.xxxx layout. At the same time, it dynamically corrects the exponent according to the shift amount, and combines the actual reference exponent output by the scalar preprocessing stage to package the sign, exponent and mantissa into a standard IEEE FP16 floating-point vector.

[0028] SIMD Arithmetic Logic Level: Contains a 64-way parallel FP16 subtractor and a 64-way parallel FP16 multiplier. Both subtractors and multipliers are implemented using synthesizable digital logic, with built-in automatic alignment, mantissa operation, and result normalization logic. The unit first subtracts the sum of global broadcast activation values ​​offset Asum from the normalized FP16 floating-point vector using the FP16 subtractor. The subtraction result is then fed into the FP16 multiplier, where it is multiplied element-wise with the half-precision floating-point scaling factor Scale output from the scalar preprocessing stage, performing a linear transformation Result=(O float A sum )×Scale, and finally output a high-precision floating-point result vector in standard FP16 format.

[0029] This embodiment establishes a core architecture that replaces the 2T2R differential structure with a single-ended 1T1R, fundamentally resolving the conflict between the non-negative conductance and signed weights of RRAM; it achieves on-chip closed-loop processing of "floating-point preprocessing → analog calculation debiasing → fixed-point to floating-point dequantization", avoiding off-chip data transfer; and it clarifies the core design of the three-stage pipeline dequantization, laying the foundation for subsequent module refinement and ensuring the high throughput and high precision characteristics of the circuit.

[0030] In a specific instance, the fixed offset is Offset=2. (N-1) Where N is the bit width of the signed logical weights, such that the mapped physical weights take values ​​in the range [0, 2]. N-1 [, maintain non-negativity.]

[0031] This embodiment provides a quantifiable and implementable signed weighted non-negative mapping scheme, which, compared with fuzzy mapping without fixed parameters, ensures the uniqueness and accuracy of the weight mapping; it is compatible with different bit width weights such as INT4 / INT8, improving circuit compatibility, and eliminating the need to redesign the mapping logic for different bit widths; it provides an accurate error benchmark for subsequent digital Psum compensation, ensuring that physical offset errors can be completely eliminated.

[0032] In a specific example, the digital domain pre-alignment module includes a barrel shifter array, a two's complement conversion logic, a bit slicing unit, and a bypass adder tree; the input of the bypass adder tree is coupled to the output of the two's complement conversion logic, and the Psum is obtained by parallel calculation based on the signed two's complement data, and the Psum does not enter the 1T1RRRAM array of the analog memory computing core, but is directly transmitted to the back-end digital circuit for error correction.

[0033] This embodiment refines the hardware implementation of the pre-alignment module to ensure efficient and error-free floating-point to fixed-point format conversion, adapting to the DAC input requirements of analog in-memory computation; the bypass-style adder tree generates Psum in parallel without occupying the timing of the main data path, avoiding pipeline delays caused by compensation term calculation; Psum bypasses the analog array to prevent analog domain noise from interfering with the compensation term, ensuring efficient and high-precision floating-point to fixed-point format conversion.

[0034] In a specific example, the analog in-memory computing core does not contain a 2T2R differential device pair, but consists of a digital-to-analog converter (DAC), a 1T1R RRAM array, an analog-to-digital converter (ADC), and a digital shift-accumulator circuit. The digital shift-accumulator circuit is used to reassemble multi-cycle bit slice data into complete values, and performs a one-time subtraction correction during the accumulation completion stage to eliminate the systematic physical offset error introduced by the fixed offset.

[0035] This embodiment completely abandons the 2T2R differential structure, increasing storage density by 2 times while reducing the peripheral circuitry associated with the differential structure, thus lowering chip area and power consumption. The digital shift-accumulation circuit deducts Psum in one go, simplifying circuit logic, reducing timing overhead, and improving computational efficiency compared to multi-cycle step-by-step compensation. It achieves hardware and software collaborative error elimination by combining analog domain computation with digital domain bias correction, utilizing the parallelism of analog in-memory computation while ensuring the accuracy of error elimination.

[0036] In a specific example, the three-stage pipeline of the general dequantization DQ-SIMD unit is arranged in the order of data processing as scalar preprocessing stage, vector transformation stage, and SIMD arithmetic logic stage. The three-stage pipeline adopts a time-overlapping parallel processing method.

[0037] This embodiment establishes a high-efficiency pipeline architecture for dequantization, decoupling the complex dequantization process into three parallel stages, masking the combinational logic delays of floating-point operations and shift detection; the time-overlapping parallel processing enables the digital circuit processing speed to match the analog array computing cycle, ensuring the high throughput performance of the system in continuously processing token streams; the three-stage pipeline is a modular design, which facilitates circuit simulation, placement and routing, and improves hardware feasibility.

[0038] In a specific instance, the scalar preprocessing stage is configured to obtain the actual base exponent by subtracting a preset Block offset 25 from the shared maximum exponent. The Block offset 25 consists of an FP16 mantissa width of 10 and an FP16 exponent offset of 15, which is used to map the integer field reconstruction result to the FP16 exponent base and convert the externally input 64-bit double-precision scaling factor Scale into a 16-bit half-precision floating-point format. At the same time, it handles data underflow and saturation issues. The converted scaling factor is used for global broadcasting of subsequent vector calculations.

[0039] The accurate calculation of the baseline exponent in this embodiment provides a unified standard for subsequent floating-point format packaging, ensuring the correctness of the integer to floating-point format conversion; the format conversion and exception handling of Scale are adapted to subsequent vector parallel computing, avoiding double-precision data from consuming a large amount of hardware resources and improving vector computing efficiency; the global broadcast design reduces redundant data transmission, reduces the pressure on the on-chip bus bandwidth, and further improves energy efficiency.

[0040] In a specific example, the vector conversion stage is a multi-channel parallel structure, with each channel integrating a leading zero detector and a barrel shifter to extract the absolute value of Isum, detect leading zeros, and normalize it in real time. The exponent is dynamically corrected according to the shift amount and packaged to generate a standard IEEE FP16 floating-point vector.

[0041] The multi-channel parallel structure of this embodiment is adapted to the synchronous conversion of 64 Isums, ensuring the parallelism and high throughput of dequantization; the real-time normalization of the leading zero detector + barrel shifter ensures that the mantissa is accurately aligned with the IEEE FP16 1.xxxx standard layout, avoiding the precision loss caused by format deviation; the dynamic exponent correction based on the shift amount ensures the numerical accuracy of the floating-point conversion, realizing error-free conversion from fixed point to floating point.

[0042] In a specific example, the SIMD arithmetic logic level includes 64 parallel FP16 subtractors and 64 parallel FP16 multipliers, which sequentially perform a linear transformation by subtracting the sum of activation values ​​offset Asum and multiplying by the quantization scaling factor Scale. The calculation formula is Result=(O float A sum )×Scale, where Result is the final output standard FP16 floating-point result, O float A is the intermediate floating-point vector after converting an integer to a floating-point number. sum The sum of activation values ​​is offset, and Scale is the quantization scaling factor; both the subtractor and multiplier are implemented using synthesizable digital logic, with built-in logic for automatic alignment, mantissa operation, and result normalization.

[0043] This embodiment features 64 parallel computing units that match the output scale of the vector transformation stage, enabling fully parallel processing of dequantization and significantly improving processing speed. At the hardware level, it natively supports device offset compensation and linear dequantization combined with inverse transformation. Compared to traditional simple linear scaling, this significantly reduces the accuracy loss introduced by quantization and ensures high consistency between end-to-end results and software simulation, guaranteeing consistent end-to-end accuracy. It can synthesize digital logic and incorporate built-in normalization logic to ensure that the computation results conform to the IEEE FP16 standard, improving circuit compatibility and facilitating large-scale mass production.

[0044] Example 2 To achieve the above objectives, see Figure 8 The present invention also provides a mixed-precision dequantization method for in-memory computation, applied to the aforementioned mixed-precision dequantization system, comprising the following steps: S1: During the compilation phase, signed logical weights are superimposed with a fixed offset Offset=2. (N-1) , mapped to non-negative physical weights and written into the 1T1R single-ended RRAM array of the analog memory computing core; S2: During the hardware operation phase, the digital field pre-alignment module performs dynamic mantissa alignment and binary two's complement conversion on the floating-point activation vector, generates a fixed-point two's complement sequence and activation value compensation term Psum, and caches the total offset of activation values ​​Asum. S3: The analog in-memory computing core receives the fixed-point two's complement sequence and performs analog domain multiplication and accumulation operations. After ADC quantization, the digital shift and accumulation circuit reconstructs the multi-cycle bit slice data. In the accumulation completion stage, Psum is first arithmetically shifted right by 17 bits to align with the reconstructed domain of the bit slice. Then, it is multiplied by a fixed offset Offset and deducted from the accumulation result all at once. When the weight is INT4, Offset=8. The operation is equivalent to arithmetically shifting Psum right by 17 bits and then multiplying by 8. The output is the digital part and Isum that eliminate the physical offset error. S4: The general-purpose dequantization DQ-SIMD unit receives the Isum and sequentially completes scalar preprocessing, integer-to-float conversion, and floating-point linear transformation through a three-stage pipeline, outputting a high-precision floating-point result vector.

[0045] This embodiment implements a software and hardware co-design of "weight mapping in the compilation stage + pipelined processing in the hardware stage", decoupling offline processing from online computing and improving hardware operating efficiency; the method steps correspond one-to-one with the system modules, ensuring the feasibility of the circuit architecture, and making the solution of this embodiment form a complete closed loop from hardware design to actual application; the entire process is completed on-chip, significantly reducing off-chip data transfer overhead, improving the system energy efficiency ratio, and adapting to low-power edge devices.

[0046] In a specific example, the floating-point linear transformation in step S4 is executed in parallel across multiple channels. First, the sum of global broadcast activation values ​​offset Asum is subtracted from the normalized floating-point data. Then, the subtraction result is multiplied element-wise by the preprocessed quantization scaling factor Scale to complete the on-chip inverse transformation process of "device-side offset compensation + algorithm-side linear inverse quantization". The device side is used to eliminate the system terms introduced by the fixed offset, and the algorithm side performs linear inverse quantization with zero-point terms. The entire transformation process is completed in a closed loop on-chip without the participation of an off-chip processor.

[0047] This embodiment's multi-channel parallel execution ensures that the transformation speed matches that of the preceding modules, avoiding single-channel serial processing from becoming a bottleneck for system throughput; it strictly follows the asymmetric quantization inverse transformation rule of "subtracting the offset first and then multiplying by the scaling" to ensure lossless accuracy at the algorithm level and complete consistency with the software-side quantization logic; the on-chip closed-loop processing completely breaks through the limitations of von Neumann. Overcoming the Neumann bottleneck reduces inter-chip data transmission, further lowering system power consumption and latency, and improving the usability of edge devices.

[0048] Example 3 The mixed-precision dequantization system and method of this embodiment are applicable to in-memory computing acceleration chips for large language model inference based on a 1T1R structure. Taking the mixed-precision configuration of INT4 weights and FP16 activation values ​​as a specific embodiment, the technical solution of the present invention will be described in detail. The present invention is also applicable to other mixed-precision configurations such as INT8 weights and FP8 / BF16 activation values.

[0049] 1. System Overall Architecture and Interfaces The hybrid precision dequantization system of this invention is driven by a global clock and forms a five-stage pipeline data path. Externally, it is configured with 128 parallel FP16 floating-point vector input interfaces, a quantization parameter configuration interface (receiving the shared maximum exponent max_e, 64-bit double-precision scaling factor Scale, and activation value sum offset Asum), and 64 parallel FP16 floating-point result output interfaces (the 128 input activations form the input vector x for matrix-vector multiplication; the RRAM array has 128 columns, where each adjacent pair of columns is spatially merged by digital post-processing (Col_LSB + Col_MSB × 4) to encode an INT4 weight; the 128 columns actually carry 64 weight rows, corresponding to 64 output neurons; the 128 inputs → 64 outputs represent a natural dimensionality reduction of the MVM computation semantics, rather than data truncation or precision loss). The five-stage pipeline, in the order of data processing, is as follows: pre-alignment stage, in-memory computation and physical debiasing stage, scalar preprocessing stage, integer to floating-point and normalization stage, and SIMD arithmetic logic stage. At the circuit implementation level, each module is further refined into a total of 14 register pipeline stages; taking the INT4 weight configuration as an example, the end-to-end latency for processing a complete frame of data is 14 clock cycles; these three correspond to three different abstraction levels: the functional architecture layer, the RTL implementation layer, and the scene timing layer. The overall functional principle architecture of the system is as follows: Figure 2 As shown in the figure, a is the digital domain pre-aligner, b is the analog in-memory computing core, and c is the general dequantization DQ-SIMD unit.

[0050] 2. Specific implementation of the numeric field pre-alignment module (pre-alignment level) In this embodiment, the digital domain pre-alignment module receives 128 parallel FP16 floating-point activation vectors and a 5-bit shared maximum exponent max_e[4:0]. The 128 parallel subtractors within the module generate shift control signals for the mantissas of each channel based on max_e. These signals drive a barrel shifter array composed of cascaded multi-stage multiplexers (MUX), completing the dynamic logical right shift alignment of all mantissas through a 7-stage pipeline. The aligned mantissas are converted to binary two's complement by two's complement conversion logic, and then divided into a main data path and a bypass path. The two's complement data of the main data path is sent to the bit-slicing unit, decomposed into a 2-bit low-width bit-slice sequence, and output to the DAC of the analog in-memory computing core. The two's complement data of the bypass path is input to a bypass-style adder tree, which uses a 5-level binary tree structure. This parallel bypass design avoids the timing bottleneck caused by serial computation. Figure 4 As shown, pipeline registers are inserted between each stage to ensure timing, 128 channels of data are accumulated in parallel, and a 32-bit wide activation value compensation term Psum is generated. This Psum is directly transmitted to the digital shift and accumulation circuit of the analog memory computing core for subsequent physical offset error elimination.

[0051] 3. Specific implementation of the simulated in-memory computing core (in-memory computing and physical de-partitioning) In this embodiment, the logical weight is INT4 bits wide, and the fixed offset is Offset=2. (4-1) =8, physical weight W_phy=W_sym+8, so that the physical weight value range is [0,15], which satisfies the non-negative storage requirement of the 1T1R single-ended RRAM array; the DAC of the analog in-memory computing core converts the 2-bit bit slice sequence into an analog voltage signal. The 1T1R RRAM array receives the analog voltage signal and performs parallel "voltage-conductance" multiplication and accumulation operations in the analog domain, outputting an analog current signal containing systematic physical offset error; the analog current signal is quantized into a 10-bit wide digital signal by the ADC and then sent to the digital shift and accumulation circuit.

[0052] like Figure 6As shown, this circuit is responsible for restoring the numerical value and simultaneously eliminating systematic physical errors. First, the portions stored in different physical columns are merged using hard-wired shifting (Col_L + Col_H << 2). Then, the data enters a closed-loop accumulation unit consisting of a barrel shifter, a main adder, a 32-bit accumulator register, and a feedback multiplexer. The barrel shifter dynamically shifts the input left based on the slice index idx (shift amount = idx × 2) to restore its bit weights. The feedback multiplexer is controlled by the "first slice flag is_first," selecting zero to initialize the accumulator on the first cycle and selecting the feedback value in subsequent cycles, thus achieving automatic initialization and cyclic accumulation. On the last cycle of accumulation (T10 cycle), the activation value compensation term Psum is arithmetically right-shifted by 17 bits to align with the fixed-point bit weights, multiplied by 8, and fed into the subtractor at the output. This subtracted value is then subtracted from the complete accumulation result to offset the physical error introduced by the fixed offset, ultimately outputting a 32-bit wide digital portion and Isum, eliminating the physical offset error.

[0053] 4. Specific implementation of the general dequantization DQ-SIMD unit (scalar preprocessing stage + vector transformation stage + SIMD arithmetic logic stage) The principle of integer to floating-point conversion at the vector level is as follows: Figure 5 As shown, the specific implementation is as follows: Scalar preprocessing stage: The actual base exponent Base_Exp is calculated by subtracting the preset Block offset of 25 from the shared maximum exponent max_e, which is used for subsequent FP16 format packaging; at the same time, the external input 64-bit double-precision scaling factor Scale is compressed into 16-bit half-precision floating-point format through the format conversion unit to handle data underflow and saturation issues. The converted Scale_fp16 is stored in a register for global broadcasting in subsequent vector calculations.

[0054] Vector Transformation Stage: A 64-channel parallel structure, each channel integrates a leading zero detector (LZD) and a barrel shifter. After receiving the digital portion and Isum, it first extracts the sign bit and takes its absolute value. The LZD detects the number of leading zeros in the absolute value, and accordingly controls the barrel shifter to perform real-time normalization of the mantissa, aligning the mantissa to the implicit 1.xxxx layout of IEEE FP16. The shifted amount is synchronously sent to the exponent calculation unit, added to Base_Exp and corrected, finally packaging the sign, exponent, and mantissa parts to generate a standard IEEE FP16 floating-point vector O. float .

[0055] SIMD Arithmetic Logic Level: Equipped with 64 FP16 subtractors and 64 FP16 multipliers, it first uses the FP16 subtractors to start from O... floatSubtract the sum of global broadcast activation values ​​offset Asum from the result, and then feed the result into the FP16 multiplier. Multiply it element-wise with Scale_fp16 and perform a linear transformation Result=(O float A sum The subtractor and multiplier both have built-in automatic alignment, mantissa operation and result normalization logic to ensure that the operation results conform to the IEEE FP16 standard. Finally, they output a 64-channel parallel FP16 high-precision floating-point result vector and generate a valid pulse signal simd_valid, which marks the end of the complete processing of a frame of data.

[0056] 5. Complete system working sequence The complete timing diagram of the mixed-precision dequantization system of this invention is shown in Figure 7: T0 cycle: When both input_valid and input_ready signals are high, the input vector and max_e are sampled. T1~T7 cycles: The pre-alignment stage performs pipelined processing, and in T7 cycle, a valid activation value compensation term Psum is generated. T6~T10 cycles: The analog in-memory computing core performs operations and accumulation, and in T10 cycle, Psum is subtracted once, outputting the digital part and Isum after eliminating physical offset errors. T11~T13 cycles: The DQ-SIMD unit performs scalar preprocessing, integer-to-floating-point normalization, and SIMD arithmetic logic operations. T14 cycle: The simd_valid signal goes high, and the 64-channel FP16 high-precision floating-point result vector is output validly.

[0057] Furthermore, the timing diagram shows that the minimum input interval between two consecutive data entries is 11 clock cycles (T0 to T10 is the busy state, and T11 resumes input_ready). This deep pipeline design enables the system to achieve throughput close to the theoretical peak when processing continuous token streams of large language models.

[0058] 6. Performance Verification: To quantitatively illustrate the technical advancement of this invention, the solution of this invention is compared with existing mainstream architectures, as detailed in Table 1: Table 1: Performance Comparison Table of the Invention Solution and Existing Technical Architecture

[0059] In summary, this invention achieves an extremely high energy efficiency of 38.2 TOPS / W and a storage density of 28.5 Mb / mm² using a 28nm process. Compared to existing technologies, this solution increases storage density by approximately two times and area efficiency by more than 1.5 times while maintaining quantization accuracy, breaking through the area and energy consumption bottlenecks of traditional architectures when processing large-scale mixed-precision models.

[0060] The modules, units, circuits and methods not described in detail in this invention are all conventional technical means known to those skilled in the art, and can be conventionally adjusted and optimized according to actual chip design requirements without affecting the technical effect of this invention.

[0061] Industrial applicability This invention presents a mixed-precision inverse quantization system and method for in-memory computing, employing standard synthesizable digital logic and a 1T1R RRAM array design. It is fully compatible with existing integrated circuit designs and in-memory computing chip manufacturing processes, and can complete system circuit simulation, placement and routing, and tape-out using existing EDA design tools, facilitating large-scale mass production. Furthermore, the system and method of this invention can adapt to the inference needs of large language models of varying scales, significantly improving storage density, computational energy efficiency, and throughput performance while reducing hardware costs. It can be widely applied to edge AI devices, low-power smart terminals, large language model inference acceleration chips, and other products, demonstrating extremely high industrial applicability.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A mixed-precision inverse quantization system for in-memory computing, characterized in that, It includes a digital domain pre-alignment module, an analog in-memory computing core, and a general-purpose dequantization DQ-SIMD unit connected in sequence; The digital field pre-alignment module is configured to receive floating-point format activation vectors and externally provided shared maximum exponents, dynamically align the mantissas of each vector and convert them into binary two's complement, and simultaneously generate activation value compensation terms Psum through parallel accumulation via a bypass addition tree. The analog in-memory computing core adopts a 1T1R single-ended RRAM array structure. It maps signed logical weights with a fixed offset to non-negative physical weights, performs analog domain multiplication and accumulation operations, and in the digital shift and accumulation stage, first converts the activation value compensation term Psum from the complete fixed-point aligned domain to the slice reconstruction domain, then multiplies it by the fixed offset and deducts it from the accumulation result all at once to eliminate the offset error. When the weight is INT4, Offset=8, the operation is equivalent to performing an arithmetic right shift of the activation value compensation term Psum by 17 bits and then multiplying it by 8, outputting the digital part and Isum. The general-purpose dequantization DQ-SIMD unit is connected to the analog in-memory computing core. It adopts a three-stage pipeline architecture to convert the digital part and Isum into standard floating-point format, and completes the inverse quantization transformation to output a high-precision floating-point result vector.

2. The hybrid precision inverse quantization system according to claim 1, characterized in that, The fixed offset Offset=2 (N-1) Where N is the bit width of the signed logical weights, such that the mapped physical weights take values ​​in the range [0, 2]. N-1 [, maintain non-negativity.] 3. The hybrid precision inverse quantization system according to claim 1, characterized in that, The digital domain pre-alignment module includes a barrel shifter array, a two's complement conversion logic, a bit slicing unit, and a bypass adder tree. The input of the bypass adder tree is coupled to the output of the two's complement conversion logic. The activation value compensation term Psum is obtained by parallel calculation based on the signed two's complement data. The activation value compensation term Psum does not enter the 1T1R RRAM array of the analog memory calculation core, but is directly transmitted to the back-end digital circuit for error correction.

4. The hybrid precision inverse quantization system according to claim 1, characterized in that, The analog in-memory computing core does not contain a 2T2R differential device pair and consists of a digital-to-analog converter (DAC), a 1T1R RRAM array, an analog-to-digital converter (ADC), and a digital shift and accumulation circuit. The digital shift and accumulation circuit is used to reassemble multi-cycle bit slice data into complete values ​​and perform a one-time subtraction correction during the accumulation completion stage to eliminate the systematic physical offset error introduced by the fixed offset.

5. The hybrid precision inverse quantization system according to claim 1, characterized in that, The three-stage pipeline of the general-purpose dequantization DQ-SIMD unit consists of a scalar preprocessing stage, a vector transformation stage, and a SIMD arithmetic logic stage, arranged in the order of data processing. The three-stage pipeline adopts a time-overlapping parallel processing method.

6. The hybrid precision inverse quantization system according to claim 5, characterized in that, The scalar preprocessing stage is configured to obtain the actual base exponent by subtracting a preset Block offset of 25 from the shared maximum exponent. The Block offset of 25 consists of an FP16 mantissa width of 10 and an FP16 exponent offset of 15, which is used to map the integer field reconstruction result to the FP16 exponent base and convert the externally input 64-bit double-precision scaling factor Scale into a 16-bit half-precision floating-point format. At the same time, it handles data underflow and saturation issues. The converted scaling factor is used for global broadcasting of subsequent vector calculations.

7. The hybrid precision inverse quantization system according to claim 5, characterized in that, The vector conversion stage is a multi-channel parallel structure. Each channel integrates a leading zero detector and a barrel shifter to extract the absolute value, detect leading zeros, and normalize the digital part and Isum in real time. The exponent is dynamically corrected according to the shift amount and packaged to generate a standard IEEE FP16 floating-point vector.

8. The hybrid precision inverse quantization system according to claim 5, characterized in that, The SIMD arithmetic logic level includes 64 parallel FP16 subtractors and 64 parallel FP16 multipliers, which sequentially perform a linear transformation by subtracting the sum of activation values ​​offset Asum and multiplying by the quantization scaling factor Scale. The calculation formula is Result=(O float A sum )×Scale, where Result is the final output standard FP16 floating-point result, O float A is the intermediate floating-point vector after converting an integer to a floating-point number. sum The sum of activation values ​​is offset, and Scale is the quantization scaling factor; both the subtractor and multiplier are implemented using synthesizable digital logic, with built-in logic for automatic alignment, mantissa operation, and result normalization.

9. A mixed-precision dequantization method for in-memory computation, applied to the mixed-precision dequantization system according to any one of claims 1-8, characterized in that, Includes the following steps: S1: During the compilation phase, signed logical weights are superimposed with a fixed offset Offset=2. (N-1) , mapped to non-negative physical weights and written into the 1T1R single-ended RRAM array of the analog memory computing core; S2: During the hardware operation phase, the digital field pre-alignment module performs dynamic mantissa alignment and binary two's complement conversion on the floating-point activation vector, generates a fixed-point two's complement sequence and activation value compensation term Psum, and caches the total offset of activation values ​​Asum. S3: The analog in-memory computing core receives the fixed-point two's complement sequence and performs analog domain multiplication and accumulation operations. After ADC quantization, the digital shift and accumulation circuit reconstructs the multi-cycle bit slice data. During the accumulation completion stage, the activation value compensation term Psum is arithmetically shifted 17 bits to the right to align with the reconstructed domain of the bit slice. Then, it is multiplied by a fixed offset and deducted from the accumulation result all at once. When the weight is INT4, the fixed offset is 8. The operation is equivalent to arithmetically shifting the activation value compensation term Psum 17 bits to the right and then multiplying by 8. The output is the digital part and Isum that eliminate the physical offset error. S4: The general-purpose dequantization DQ-SIMD unit receives the digital part and Isum, and completes scalar preprocessing, integer-to-float conversion and floating-point linear transformation in sequence through a three-stage pipeline, and outputs a high-precision floating-point result vector.

10. The hybrid precision inverse quantization method according to claim 9, characterized in that, The floating-point linear transformation in step S4 is executed in parallel across multiple channels. First, the sum of global broadcast activation values ​​offset Asum is subtracted from the normalized floating-point data. Then, the subtraction result is multiplied element-wise by the preprocessed quantization scaling factor Scale to complete the on-chip inverse transformation process of "device-side offset compensation + algorithm-side linear inverse quantization". The device side is used to eliminate the system terms introduced by the fixed offset, and the algorithm side performs linear inverse quantization with zero-point terms. The entire transformation process is completed in a closed loop on-chip without the participation of an off-chip processor.

Images