A mixed-precision adaptive quantization circuitry

Abstract

The application discloses a hybrid precision adaptive quantization circuit system, and belongs to the technical field of digital integrated circuits and special computing chips. The system comprises a multi-path heterogeneous ALU array, a curvature detection circuit and a fixed-point format switching circuit; the multi-path heterogeneous ALU array constructs a closed-loop tensor calculation pipeline, and the path bit width of each stage ALU is configured according to the operation error propagation sensitivity. The application controls the floating-point operation precision of the first-end ALU in real time through the curvature scalar hardware hardwiring, and relies on the adaptive switching of the data dynamic range to switch the fixed-point format of the last-end ALU. The whole precision switching is completed by pure hardware logic, and the dynamic precision adaptation can be realized in a single cycle. The application effectively suppresses the error accumulation of the pipeline, reduces the hardware overhead and operation power consumption, and can be widely applied to high-density tensor calculation scenes such as numerical physics simulation, fluid mechanics solution and deep learning tensor operation, and is suitable for the high energy efficiency calculation requirements of various special tensor computing chips, end-side domain controllers and space computing chips.

Description

Technical Field

[0001] This invention belongs to the field of digital integrated circuits and dedicated computing chips, specifically relating to a hybrid precision adaptive quantization circuit system for general-purpose tensor computation. This invention is applicable to various high-density tensor computation scenarios, such as numerical physics simulation, fluid dynamics calculation, and deep learning tensor computation. It can be integrated into various dedicated tensor computation chips, edge domain controllers, and spatial computing chips, providing hardware-level adaptive precision control capabilities for general-purpose high-speed tensor numerical computation. Background Technology

[0002] Dedicated tensor computing chips integrate various heterogeneous computational units, each with significantly different requirements for numerical precision and dynamic range. Specifically, matrix inversion operations are highly sensitive to mantissa precision, difference and subtraction operations have stringent requirements for exponent dynamic range, accumulation and integration operations prioritize dynamic range and have lower requirements for mantissa precision, while threshold comparison and state determination operations have extremely low precision sensitivity. The numerical format requirements of these various operations cannot be simultaneously met by a unified precision configuration, making it difficult to achieve an optimal balance between computational precision, numerical stability, hardware power consumption, and chip area.

[0003] Existing tensor computation precision management solutions are mainly divided into three categories. Each type of solution not only has flawed computation scheduling strategies, but also suffers from irreconcilable hardware bottlenecks in its supporting hardware architecture:

[0004] One approach is a globally unified precision hardware solution, where the entire chip's computational path is fixed to a single, fixed floating-point format, and all hardware multiplication arrays and arithmetic units adopt a unified high bit width design. Low-precision arithmetic units have a large amount of bit width hardware redundancy, resulting in wasted power consumption and chip area, while high-precision sensitive arithmetic units will accumulate iteration errors due to insufficient bit width, leading to poor numerical stability.

[0005] The second is a static mixed-precision hardware solution, in which the bit width and numerical format of the operation path of various operators are fixed in the hardware during the tape-out stage. There is no dynamic switching of the operation path at runtime, and it is impossible to adaptively adjust the operation precision according to the dynamic range of the input data and the matrix condition number and other real-time characteristics at runtime.

[0006] Thirdly, there is the fixed-point static configuration hardware scheme. The ratio of integer and decimal bit widths in the fixed-point arithmetic unit is fixed and cannot be adjusted in the hardware circuit. When the dynamic range of input data fluctuates drastically, the hardware cannot switch the fixed-point format in real time, which can easily lead to numerical overflow and a significant decrease in arithmetic accuracy.

[0007] Taking molecular dynamics simulations as an example, the numerical magnitude difference between intermolecular van der Waals forces and electrostatic forces can reach [missing information]. The condition number of the tensor matrix fluctuates drastically with the molecular conformation. If high-precision fp32 is used throughout the process, more than 75% of the bit width resources of the computational unit in the structurally stable region are wasted. If low-precision fp16 is used throughout the process, the tiny error generated by matrix inversion in the high curvature region is amplified by thousands of iterations, directly causing the simulation trajectory to completely fail.

[0008] Meanwhile, high-speed tensor computation scenarios on the edge impose stringent real-time and power consumption constraints on precision scheduling. Terminal devices are power-constrained, necessitating the reduction of hardware computing power through low-bit-width quantization; however, the statistical features of input data such as images, point clouds, and motion states fluctuate dramatically, making statically fixed quantization hardware paths prone to computational failures, while full-precision floating-point hardware paths suffer from excessive power consumption. Furthermore, closed-loop tasks such as robot control and spatial computing have extremely high latency requirements. Traditional solutions relying on processor software scheduling for precision switching are limited by bus and instruction scheduling latency, failing to meet the hardware-level real-time computing demands.

[0009] In summary, existing precision configuration schemes are all static fixation strategies at compile time or before tape-out. They lack dedicated hardware adaptive detection and switching circuits and hardware-level runtime adaptive quantization adjustment mechanisms. They cannot simultaneously adapt to complex computing scenarios involving heterogeneous tensor operation units and dynamic input data, which is the core technical bottleneck of current general-purpose tensor computing chips that balance high precision, low power consumption, and low latency. Summary of the Invention

[0010] Inventive concept

[0011] The core concept of this invention is that the multi-channel ALU is a reconfigurable heterogeneous computing unit that can merge and split computing tasks according to computing power requirements, forming a closed-loop tensor computing pipeline covering metric tensor solving, connection symbol calculation, covariant gradient difference, numerical integration, and trace purity determination. Based on the error propagation sensitivity level of each computing node in the pipeline, the ALU at each level is configured with differentiated heterogeneous bit width and precision paths. The high-precision sensitive computing configuration at the front end of the pipeline supports dynamic switching between high and low precision data paths, the mid-range computing configuration is a medium fixed bit width path, and the low-sensitivity determination computing configuration is a low bit width, low power consumption fixed path.

[0012] This invention independently sets up two hardware detection logics: a curvature detection circuit and a data range detection unit. These are combined with a precision mode register and a multiplexer to achieve zero-latency hardware precision switching. The curvature detection circuit is the main control detection path, which collects and analyzes tensor curvature scalar parameters in real time, driving the front-end high-precision computing unit to dynamically switch precision operating modes. The data range detection unit is the auxiliary control detection path, which calculates the maximum absolute value of all elements of the current tensor in real time as the dynamic range characteristic parameter, controlling the end computing unit to complete the adaptive format switching between fixed-point integers and decimal places.

[0013] By using hardware-level hierarchical precision configuration and a dual-path independent detection and switching mechanism, all heterogeneous computing units are adapted to real-time computing conditions, ensuring the numerical stability of the entire pipeline while allowing each module to operate at its optimal power consumption and area operating point.

[0014] Purpose of the invention

[0015] This invention aims to overcome the technical shortcomings of existing tensor computing chips, such as fixed precision configuration and inability to adaptively adjust during runtime based on the dynamic characteristics of input data. It addresses the problems of traditional solutions, such as difficulty in simultaneously adapting to the differentiated precision requirements of heterogeneous computing units, insufficient numerical stability, power and area redundancy, and excessive software precision switching latency. The invention provides a hybrid precision adaptive quantization circuit system. Based on error sensitivity, this invention implements heterogeneous bit width configurations for each stage of the pipeline's ALUs, optimizing chip power consumption and area overhead from the hardware source. It achieves zero-latency adaptive precision control through a dual hardware path composed of a curvature detection circuit and a data range detection unit, respectively improving the precision instability of highly sensitive tensor operations and the overflow distortion defects of low-sensitivity fixed-point operations, enabling the entire tensor computing pipeline to continuously maintain optimal precision and power consumption balance under dynamic operating conditions.

[0016] Overall technical solution

[0017] This invention provides a hybrid precision adaptive quantization circuit system, the core of which is summarized as follows:

[0018] A hybrid precision adaptive quantization circuit system, characterized in that it comprises:

[0019] A multi-channel reconfigurable ALU array is provided, wherein the multiple ALUs are cascaded to form a closed-loop tensor computation pipeline, and each ALU is configured with an independent precision mode register; the ALU array is configured with differentiated bit-width paths according to the pipeline error sensitivity, the front-end ALU supports dynamic precision switching, the middle ALU is configured with a fixed mid-bit-width path, and the end ALU is configured with a low-bit-width fixed-point path.

[0020] The curvature detection circuit interacts with the front-end ALU signal to acquire the curvature scalar output by the front-end ALU and outputs control signals to write to the precision mode register of the front-end ALU, so as to realize the real-time dynamic adjustment of the front-end calculation precision.

[0021] The data range detection unit interacts with the pipeline data path to sample the dynamic range feature values ​​of the tensor in real time and outputs a fixed-point format control signal to write to the precision mode register of the end ALU to adaptively switch the bit width allocation of the end fixed-point value.

[0022] The curvature detection circuit and data range detection unit directly drive the precision mode register of the corresponding ALU through hardware hard-wired connections, without the need for software intervention, and the precision switching delay is less than a single clock cycle.

[0023] Furthermore, the multi-channel ALU array can split or merge computing resources to fully realize closed-loop iterative operations such as metric tensor solving, connection symbol calculation, covariant gradient difference, numerical integration, and trace purity convergence determination; the curvature detection circuit has a built-in curvature comparator, precision mode encoder, and hardwired writing logic; the data range detection unit has a built-in maximum value comparison tree, fixed-point format encoder, and hardwired writing logic.

[0024] This invention addresses the shortcomings of traditional tensor chips, such as statically fixed precision, high software switching latency, redundant power consumption and area, and poor numerical stability in dynamic scenarios, by combining a hierarchical heterogeneous ALU precision path with two independent hardware real-time detection and control paths. Based on the characteristics of real-time tensor data, the hardware adaptively adjusts the precision of each level of operation, ensuring the stability of pipeline numerical calculations while enabling each operation unit to work at the optimal power consumption and area balance point.

[0025] Mathematical definition of curvature scalar R

[0026] The curvature scalar R described in this invention is the Riemann scalar curvature (Ricci scalar), derived from the metric tensor. The calculation formula is obtained by combining its first and second partial derivatives:

[0027] ,

[0028] in To measure the inverse of a tensor matrix, For Christofel's symbol;

[0029] Simplified hardware implementation: For tensor iteration calculation scenarios, a trace simplification approximation scheme is adopted to reduce the amount of computation. The ratio of the trace of the metric tensor to the inverse trace is taken as the approximate scalar curvature R, which greatly reduces division and differentiation operations and is compatible with ALU fixed-point pipeline.

[0030] Specific circuit structure of each hardware unit

[0031] (I) Numerical Format Definition Logic

[0032] The system predefines two types of numerical formats: floating-point and fixed-point.

[0033] Floating-point formats include: 64-bit floating-point (fp64, 1-bit sign bit, 11-bit exponent, 52-bit mantissa), 32-bit floating-point (fp32, 1-bit sign bit, 8-bit exponent, 23-bit mantissa), 16-bit floating-point (fp16, 1-bit sign bit, 5-bit exponent, 10-bit mantissa), bfloat16 (16-bit, 1-bit sign bit, 8-bit exponent, 7-bit mantissa), and 8-bit floating-point fp8_e4m3 and fp8_e5m2.

[0034] Fixed-point formats include: Q4.4, Q3.4, Q3.8, Q3.12 (16-bit, range [−4,3]), Q3.16, Q5.10 (16-bit, range [−16,15]), and Q8.8.

[0035] The first-end ALU supports dynamic switching combinations of high-precision floating-point and low-precision floating-point, including: 64-bit / 32-bit, 32-bit / 16-bit, 32-bit / fp8, and 16-bit / fp8. Each format is uniquely determined by the total bit width, the exponent / mantissa bit width for floating-point format, or the integer and fractional bit width for fixed-point format.

[0036] The first-end ALU has a built-in metric tensor solving submodule, which sequentially performs: metric tensor element-wise multiplication and accumulation, Christofer symbolic parallel computation, tensor trace solving, and inverse matrix approximation. All operations reuse the native multiply-accumulate array of ALU1, without the need for additional independent operation units. After the operation is completed, the scalar curvature R is output to the curvature detection circuit in a single cycle.

[0037] (ii) Curvature detection circuit

[0038] The curvature detection circuit includes: a curvature comparator, a precision mode encoder, and hardwired write logic; the two inputs of the curvature comparator receive the curvature scalar R and the output of the preset curvature threshold register, respectively; the precision mode encoder converts the comparison result into a precision mode selection signal, which is sent to the precision mode register of the ALU at the beginning via the hardwired write logic.

[0039] As a preferred hardware topology, the curvature detection circuit consists of a first numerical comparator, an edge-triggered precision mode latch register, and a cascaded multiplier bit-width configuration multiplexer. The numerical comparator acquires the curvature scalar R and compares it with a preset threshold, then outputs a control latch register; the latch register output directly drives the multiplexer, switching between high-precision and low-precision floating-point multiplier paths.

[0040] In this preferred embodiment, the curvature threshold is set to 0.5. When R > 0.5, the high-precision floating-point path is selected; when R ≤ 0.5, the low-precision floating-point path is enabled, achieving hardware-free switching. This threshold is only an exemplary parameter. Those skilled in the art can flexibly select other thresholds according to the actual computing task and chip process conditions. Such parameter adjustments are all within the protection scope of the claims of this invention.

[0041] (III) Data Range Detection Unit

[0042] The data range detection unit includes: a maximum value comparison tree, a fixed-point format encoder, and hardwired write logic; the maximum value comparison tree takes all tensor elements of the current period as parallel input and outputs the maximum absolute value of all elements. The dynamic range comparator receives signals from its two inputs respectively. The preset threshold register output is connected to the comparator tree output at the input of the fixed-point format encoder. The hard-wired write logic connects the output of the fixed-point format encoder to the end ALU precision mode register write port.

[0043] in , which represents all tensor elements involved in the computation within the current iteration cycle, and is used to characterize the overall dynamic range of the current input data.

[0044] In this preferred embodiment, the dynamic range threshold value is 1.8. Switch to wide integer format Q5.10; Switch to the high decimal precision format Q3.12. This threshold is only an exemplary configuration parameter for this embodiment. Those skilled in the art can flexibly select other thresholds according to the actual computing task and chip process conditions. Such parameter adjustments are all within the protection scope of the claims of this invention.

[0045] (iv) Hardware implementation of switching delay

[0046] The curvature detection circuit and the comparison path within the data range detection unit are purely combinational logic. The comparison result is latched into the precision mode register after a combinational logic delay of less than 0.5 clock cycles. On the next rising clock edge, the register output signal controls the corresponding path multiplexer to complete the data path switching, with an overall switching delay of less than 1 clock cycle. Taking a typical operating frequency of 1GHz as an example, the switching delay is less than 1ns. This delay is determined only by the gate propagation delay and the register setup time, requiring no software intervention, no bus scheduling, and no instruction delay.

[0047] (v) Accuracy – Area Pareto Scan Engine (Preferred Solution)

[0048] The system is equipped with a precision-enhancing area Pareto scan engine. This module is a front-end offline simulation tool, not on-chip hardware; it is used to iterate through operator calculation errors and hardware area under various numerical formats and output the results. MAE and area estimation results. The area fitting model is based on TSMC's 28nm multiplier. .

[0049] (vi) Streamline-level hierarchical quantization analyzer (preferred option)

[0050] The pipeline-level hierarchical quantization analyzer is a design-stage auxiliary tool and is not integrated into the tape-out hardware. It supports independent configuration of input, computation, and output numerical formats for each stage of the pipeline, and calculates single-stage errors and pipeline cumulative errors. When the cumulative error exceeds the limit, it outputs precision configuration optimization suggestions for parameter tuning before tape-out.

[0051] (vii) Basic function generator (preferred option)

[0052] The circuit system can integrate three sets of basic function generators: the sigmoid function generator uses a 32-bit floating-point path, and the gaussian and exp function generators use a 16-bit floating-point path.

[0053] Threshold parameter design basis

[0054] Curvature threshold 0.5: corresponds to the condition number boundary value of the tensor matrix;

[0055] When the approximate scalar curvature R ≤ 0.5, the matrix condition number is low and the numerical error propagation is weak, so low bit width fixed-point can be switched; when R > 0.5, the matrix is ​​ill-conditioned and the error is amplified, so the high-precision floating-point path is forced to be enabled; 0.5 is the optimal boundary threshold obtained by offline simulation of multiple sets of typical geometric tensor samples, which takes into account both accuracy and power consumption.

[0056] Dynamic range threshold 1.8: Upper limit threshold for normalized tensor amplitude;

[0057] Normalize tensor elements ,when The tensor has a large dynamic range and is prone to fixed-point overflow, so switch to the high-bit-width fractional-digit Q format; Employs short, small-digit, low-power fixed-point architecture; 1.8 Based on overflow simulation calibration of mainstream 32 / 16-bit tensor datasets.

[0058] Supplement the text description of the mapping relationship table:

[0059] Curvature scalar R, dynamic range Each of the four precision modes is mapped to a specific level. The threshold is used as the decision boundary for switching multiplexers and is stored in an on-chip constant register, which can be reconfigured by software.

[0060] System overall workflow and adaptive quantitative operation mechanism

[0061] Multi-channel ALUs support merging and splitting computational tasks, and can be flexibly configured with pipelines of different granularities:

[0062] Option 1 (N=3 Simplified Pipeline):

[0063] First-end ALU: Unifies the handling of metric tensor solving and matrix inversion;

[0064] Mid-range ALU: Combines three types of intermediate operations: connection symbol calculation, covariant gradient difference, and numerical integration;

[0065] Terminal ALU: Performs trace purity determination and threshold comparison operations.

[0066] Option 2 (N=5, preferred production line):

[0067] The first ALU completes the metric tensor solution and matrix inversion operations, and the output is sent to the first intermediate ALU to complete the connection symbol difference operation. The operation results of the first intermediate ALU are passed to the second intermediate ALU and the third intermediate ALU in sequence to complete the integral accumulation and covariant gradient difference operation, respectively. Finally, the terminal ALU completes the terminal trace purity determination and threshold output.

[0068] Regardless of whether N=3, N=5, or N=7 is used for partitioning, each ALU strictly follows the configuration rule that the higher the error sensitivity, the higher the bit width. Each ALU completes steady-state operation based on the preset heterogeneous bit width configuration, achieving a balance between pipeline basic accuracy and power consumption.

[0069] Second, the front-end floating-point precision adaptive switching process based on curvature scalar

[0070] During pipeline operation, the metric tensor calculation unit outputs a curvature scalar R to the curvature detection circuit in real time. The curvature comparator continuously compares the real-time R value with a preset threshold in hardware.

[0071] When the tensor curvature scalar R is detected to be greater than the threshold, it is determined that the current matrix is ​​ill-conditioned and has high computational sensitivity. The hardware immediately writes a high-precision floating-point mode instruction to the precision mode register of the ALU at the beginning, and the path is switched to high-bit-width floating-point operation to suppress the generation of pipeline front-end error.

[0072] When R falls back to or below the threshold, the tensor is determined to be stable, and it automatically switches to a low-precision floating-point path to reduce computational power consumption and hardware area overhead.

[0073] The entire switching process is pure combinational logic with register latching, completed within a single cycle, without interrupting pipeline operations.

[0074] Third, the adaptive switching process for end-point fixed-point format based on the dynamic range of data.

[0075] During the pipeline iterative computation synchronization process, the data range detection unit continuously calculates the dynamic range estimate of the current tensor dataset. The data is then fed into the fixed-point format switching circuit. The hardware comparison logic determines the data amplitude range in real time.

[0076] when If the value exceeds the preset threshold, it indicates that the tensor data has a large dynamic range and a high range of integer amplitudes. The terminal ALU will be automatically switched to the wide integer Q5.10 format to avoid fixed-point overflow distortion.

[0077] when If the data is less than or equal to the threshold and has a limited dynamic range, switch to the high decimal precision Q3.12 format to improve the accuracy of the final determination.

[0078] Fourth, the collaborative operation mechanism of the entire system.

[0079] Front-end floating-point dynamic precision control suppresses pipeline source errors, while end-point fixed-point format adaptive switching prevents back-end quantization overflow and precision loss. Mid-level ALUs operate in a stable state with a fixed optimal bit width. Two independent hard-wired detection circuits operate in parallel without interference, enabling the entire closed-loop pipeline to dynamically match the optimal numerical format based on the real-time tensor state in each iteration cycle. This achieves a hardware adaptive quantization mechanism characterized by "high ill-conditioned high precision, stable low power consumption, large dynamic overflow prevention, and small dynamic precision improvement."

[0080] Beneficial effects of the present invention

[0081] Compared with existing static precision configuration and globally unified precision architecture, this invention has the following significant technical advantages:

[0082] (i) Based on error sensitivity, heterogeneous hierarchical precision configuration eliminates power consumption and area redundancy from the hardware source.

[0083] This invention abandons the traditional model of uniform precision or static precision allocation based on operator type. The ALU supports merging and splitting computational tasks, and can freely combine pipelines of any size (N≥3). It achieves heterogeneous bit width configuration based on the error propagation sensitivity level of each computational unit in the closed-loop pipeline. High-sensitivity operations at the beginning are configured with dynamically switchable high-bit-width floating-point paths, mid-range operations with fixed-bit-width floating-point paths, and low-sensitivity decision operations at the end with low-bit-width paths. This hierarchical architecture completely solves the defects of low-sensitivity unit bit width redundancy and severe error accumulation in high-sensitivity units, which exist in globally homogeneous precision schemes, and is adaptable to arbitrary pipeline granularity.

[0084] (ii) Dual-path hard-wired adaptive switching enables precise control during zero-latency operation.

[0085] This invention independently sets up two sets of pure hardware detection logic: a curvature detection circuit and a data range detection unit. The entire process of precision comparison, encoding, and register writing is implemented through hard-wired connections, requiring no software scheduling and eliminating bus latency. The overall switching latency is less than a single clock cycle, allowing for real-time updates of the precision mode without interrupting the pipeline. This real-time performance is far superior to software dynamic scheduling schemes.

[0086] (iii) Dynamic adaptation of end-point format, taking into account both overflow prevention and high-precision quantization.

[0087] This invention captures the overall amplitude range of the tensor in real time through a data range detection unit and adaptively switches the end-point format; it uses a wide integer format to prevent overflow for large dynamic data and a high decimal place to improve the judgment accuracy for stable data, thus solving the inherent defects of traditional static fixed-point data that are prone to overflow and have degraded quantization accuracy.

[0088] (iv) The hardware architecture is standardized and highly feasible, supporting the mass production and implementation of chips.

[0089] All core units are standard digital circuits, which can directly generate RTL netlists for backend design. Multiple ALUs support reconfiguration of computation tasks, enabling the complete execution of a full pipeline for metric tensor-based physics simulations, or allowing for the trimming of computation steps to adapt to general matrix and tensor computation tasks, demonstrating strong hardware reusability. The core switching logic has been verified through behavioral-level simulation, exhibiting stable timing without glitches, and presents no obstacles to mass production.

[0090] (v) It is adapted to various low-power real-time computing scenarios, and the system energy efficiency is greatly improved.

[0091] This invention enables the high-precision path only when the tensor is ill-conditioned or the data amplitude changes abruptly, and normally maintains low bit width and low power consumption operation; the nanosecond-level switching latency meets the requirements of various closed-loop hard real-time tasks, and can be packaged as a standard digital IP and integrated into a variety of computing hardware, making it highly versatile. Attached Figure Description

[0092] Figure 1 is a schematic diagram of the overall architecture of the hybrid precision adaptive quantization circuit system of the present invention, showing that a closed-loop iterative computation pipeline is formed by a five-level cascaded reconfigurable ALU array (ALU1 to ALU5);

[0093] Solid arrows represent the forward tensor operation data flow, while dashed lines represent the loop back propagation link of unconverged iteration data. Unconverged operation results flow back from ALU5 to ALU1 for another iteration, and converged operation results are output from ALU5.

[0094] The system is equipped with two independent hardware detection and control circuits: curvature detection circuit and data range detection unit;

[0095] The ALU1 has a built-in metric tensor operation module that outputs a curvature scalar R in real time. The curvature detection circuit acquires R and generates a control signal, which is hardwired into the ALU1's independent precision mode register to adjust the ALU1's floating-point operation bit width.

[0096] Dynamic range estimates are obtained by sampling tensor amplitude values ​​in real time throughout the pipeline. The data range detection unit is based on Generate a fixed-point format selection signal to control the switching of fixed-point operation format of ALU5;

[0097] Each ALU is equipped with an independent precision mode register, which can be configured with floating-point and fixed-point numerical formats respectively, enabling pipelined segmented differentiated mixed precision operations.

[0098] Figure 2 is a schematic diagram of the bit width allocation of various numerical formats used in this invention, showing 8-bit to 64-bit floating-point and fixed-point formats in sequence, with the allocation of sign bit, exponent bit, mantissa bit, and integer fractional bits marked, covering fp64, fp32, fp16, bfloat16, fp8 series and Q-class fixed-point formats;

[0099] The markings in the diagram are explained as follows: S represents a 1-bit sign bit, E (n) represents an n-bit exponent segment, and M (n) represents an n-bit mantissa segment; fixed-point format is divided into integer bit segment and fractional bit segment;

[0100] The bit structure correspondences for each floating-point format are as follows: fp64 is S+E (11)+M (52), fp32 is S+E (8)+M (23), fp16 is S+E (5)+M (10), bfloat16 is S+E (8)+M (7), FP8 (E5M2) is S+E (5)+M (2), and FP8 (E4M3) is S+E (4)+M (3).

[0101] Figure 3 is a schematic diagram of the adaptive precision switching working timing of the two sets of hardware detection circuits, the curvature detection circuit and the data range detection unit, of the present invention.

[0102] The signals in the diagram are defined as follows:

[0103] CLK is the system global clock;

[0104] R, For curvature scalar, tensor dynamic range amplitude The input signal transitions at the threshold crossing point; the comparator output is a pure combinational logic path output signal, which completes the toggle within a combinational delay of less than 0.5 clock cycles after the threshold is crossed.

[0105] The latch register is the ALU precision mode configuration register, which latches the precision control code output by the comparator at the rising edge of CLK at time T1;

[0106] The data path is the ALU operation data flow path. After register latching, the precision path switching is completed synchronously, marked "Switching complete";

[0107] The bottom of the diagram indicates that the total delay is less than one clock cycle, which visually demonstrates that the entire hardware adaptive precision switching process does not exceed a single clock cycle and there is no software scheduling delay. Detailed Implementation

[0108] Terminology Definition

[0109] Multi-channel ALU array:

[0110] A reconfigurable computing array containing N ALUs, where N≥3. A single ALU can handle one or more types of tensor operations. Those skilled in the art can merge or split computing tasks according to computing power requirements to form pipelines of different granularities, such as N=3, N=5, and N=7. Regardless of the splitting method, the configuration principle of higher error propagation sensitivity and higher data path width is followed.

[0111] Closed-loop computation pipeline:

[0112] The complete computation chain formed by cascading multiple ALUs can fully cover the entire process from metric tensor solving, connection symbol calculation, covariant gradient difference, numerical integration to trace purity determination; intermediate computation steps can also be trimmed, retaining only some ALUs to perform simplified computation tasks such as general matrix multiplication and tensor inference.

[0113] Error propagation sensitivity level:

[0114] The degree to which errors generated by the arithmetic unit are amplified and propagated to subsequent circuits is characterized; the first stage matrix inversion error is propagated to all subsequent stages, exhibiting the highest sensitivity; the last stage threshold comparison has no subsequent operations, resulting in the lowest sensitivity.

[0115] Curvature scalar R:

[0116] The output scalar after solving the metric tensor represents the curvature and condition number of the matrix; the larger the R value, the higher the ill-conditionedness of the matrix and the higher the requirement for computational precision.

[0117] Data dynamic range estimate :

[0118] The maximum absolute value of all tensor elements in the current iteration cycle is obtained by parallel calculation of the maximum value comparison tree within the data range detection unit, and is used to characterize the overall amplitude range of the data stream.

[0119] Hard connection:

[0120] The physical metal wires are directly connected to the circuit, without bus arbitration, microcode, or software scheduling logic; only the precision mode register is a writable storage element.

[0121] Precision mode register:

[0122] An m-bit edge-triggered D flip-flop array outputs an encoded signal that directly controls a bit-width multiplexer of the multiplication array.

[0123] Example 1: Area-Power Consumption Comparison of Heterogeneous Precision Allocation for Multi-Channel ALUs

[0124] This embodiment demonstrates the hardware overhead optimization effect under two pipeline architectures with N=3 and N=5.

[0125] Simplified pipeline configuration with AN=3

[0126] First-end ALU: Metric tensor solving + matrix inversion, highest sensitivity, supports switching between high and low precision;

[0127] Mid-range ALU: Connectivity symbol, integral, gradient difference merging operations, intermediate sensitivity. floating point;

[0128] Terminal ALU: trace purity determination, threshold comparison, minimum sensitivity, 8-bit floating-point and fixed-point.

[0129] Option BN=5: Optimal pipeline configuration

[0130] First-end ALU: Metric tensor solving and matrix inversion, with the highest sensitivity, supporting dynamic switching between high and low precision;

[0131] First-level intermediate ALU: Handshake symbolic computation (including multi-step subtraction and differential), second-highest sensitivity, fixed 32-bit floating point;

[0132] Second intermediate ALU: Step-size accumulation and integration operations, intermediate sensitivity, fixed bfloat16;

[0133] Third intermediate ALU: covariant gradient difference operation, intermediate sensitivity, fixed fp16;

[0134] Terminal ALU: Trace purity determination and threshold comparison, minimum sensitivity, fixed fp8 or fixed-point format.

[0135] The hardware area of ​​an integrated circuit is primarily determined by the multiplier array, and is approximately proportional to the square of the bit width. For the same throughput, a globally uniform high-bit-width architecture incurs enormous area overhead. This invention configures a high bit width only at the highly sensitive front end, gradually reducing the bit width at the middle and end stages, significantly reducing the hardware area. Changing the floating-point switching combination at the beginning only requires adjusting the multiplier bus; the entire detection and switching hardware topology remains unchanged.

[0136] Taking a 5-stage heterogeneous ALU pipeline (Example Scheme B) as an example, this invention utilizes the TSMC 28nm multiplier area estimation model based on the built-in precision-area Pareto scanning engine. Based on a comparison of theoretical areas using the bit width configurations of ALUs at each level, the total equivalent multiplication array area of ​​the heterogeneous hierarchical configuration of this invention is approximately 3149. All numbers of the same level adopt the same FP32 high-bit-width baseline scheme, with a total area of ​​6144. The hardware area was reduced by approximately 48.7%.

[0137] Based on the statistical characteristics of tensor iteration scenarios, under typical operating conditions, approximately 70% of the iteration cycles are scalar curvature. The first-end ALU switches to a low-precision path. Based on the well-known law that CMOS dynamic power consumption is proportional to the square of the operation bit width, the overall equivalent power consumption in the adaptive precision switching mode is reduced by approximately 59.2% compared to the global FP32 benchmark scheme.

[0138] The above area and power consumption values ​​are directly derived from the bit width parameters and built-in area model recorded in the manual. They are only theoretical estimation comparison results, used to intuitively demonstrate the hardware overhead optimization advantages of the hierarchical heterogeneous adaptive precision architecture.

[0139] Example 2: Adaptive switching of fixed-point format to prevent numerical overflow

[0140] This embodiment demonstrates the adaptive control mechanism of the data range detection unit.

[0141] The terminal ALU uses Q3.12 (3-bit integer, 12-bit decimal) by default; the data range detection unit calculates the maximum absolute value of all tensor elements in real time and in parallel. The hardware was switched to the wide integer Q5.10 format to avoid overflow; Switching back to high decimal precision Q3.12. The entire switching process is completed using pure hardware combinational logic, requiring no software intervention.

[0142] Technical effect: It can adaptively match the amplitude of the data stream, taking into account both the two major requirements of preventing overflow of large dynamic data and high precision of small dynamic data, and overcoming the inherent defects of static fixed-point format.

[0143] Example 3: Automatic switching of high-precision mode driven by curvature

[0144] The first-end ALU operates on a low-precision path under normal conditions; the curvature detection circuit continuously monitors the curvature scalar R; if R > 0.5, the matrix is ​​deemed ill-conditioned, and the hardware switches to high-bit-width floating-point in a single cycle to suppress initial errors; when R falls back, it automatically switches back to low precision to reduce power consumption, and the pipeline is uninterrupted throughout.

[0145] Technical effect: The accuracy is dynamically adjusted according to the ill-conditioning of the matrix, taking into account both low power consumption under normal conditions and numerical stability under ill-conditioning conditions.

[0146] Example 4: System-level Logic Verification Scheme

[0147] Based on the complete circuit topology disclosed in this invention, those skilled in the art can use conventional digital IC front-end simulation processes to complete the verification of system functions, timing and hardware overhead. Industry standard RTL simulation tools such as Synopsys VCS and Cadence Xcelium can be used, along with the Design Compiler logic synthesis tool. Verification can be carried out based on the TSMC 28nm HPC+ process library. The simulation clock can be configured to 1GHz (1ns cycle). Waveform debugging and code coverage collection are common methods in digital IC design, and a complete verification platform can be built.

[0148] The test stimulus design approach is as follows:

[0149] To fully cover all operating scenarios of the two adaptive precision switching paths—curvature detection circuit and data range detection unit—those skilled in the art can configure four types of tensor iteration test stimuli to complete full-coverage functional verification without creative effort:

[0150] (a) Low curvature steady-state excitation: set , This is used to verify the steady-state operation logic where the first-end ALU continuously operates in a low-precision floating-point path and the last-end ALU is fixed in the Q3.12 fixed-point format.

[0151] (ii) High curvature transient excitation: maintain If the value is low, let R step cross the 0.5 judgment threshold to verify the dynamic control behavior of the curvature detection circuit driving the first-end ALU to switch to the high-precision floating-point path;

[0152] (III) Large dynamic range transient excitation: Keep R at a low value, let If the step value exceeds the 1.8 threshold, the control unit ALU of the verification data range detection unit switches to the Q5.10 wide integer fixed-point format;

[0153] (iv) Composite transient excitation: making R and Simultaneously crossing their respective judgment thresholds, the parallel operation of the two independent detection pathways was verified, with no signal competition or logic conflict.

[0154] Each group of excitations can run for tens of thousands of clock cycles. After the pipeline operation state stabilizes, the switching behavior of the precision mode registers of each ALU is observed to fully verify the effectiveness of the adaptive precision control logic.

[0155] Functional verification expected results

[0156] Based on the pure combinational logic comparison path architecture of this invention, the theoretical total delay of precision switching is less than 1 clock cycle: after the output of the detection circuit is encoded by combinational logic, it is latched into the corresponding ALU precision mode register on the next rising edge of the clock. The multiplexer synchronously completes the data path switching, and there are no issues such as pipeline bubbles, operation interruption, numerical overflow or glitches. There are no race conditions in the dual-path synchronous switching scenario, and the adaptive control logic is stable and reliable.

[0157] Hardware area theoretical comparison method

[0158] The hardware area can be estimated based on the TSMC 28nm multiplier estimation model disclosed in this specification. Theoretical calculations were performed, using the bit width configuration of each ALU level as input, to compare the equivalent multiplication array area of ​​3-level and 5-level heterogeneous ALU pipelines with a globally unified FP32 benchmark scheme, visually demonstrating the area reduction advantage of the hierarchical heterogeneous architecture. These calculations were completed solely based on the parameters disclosed in the specification and the built-in estimation model, providing a theoretical analysis method that can be directly reproduced by those skilled in the art.

[0159] General methods for timing verification

[0160] After logic synthesis, the setup and hold times of all critical paths in the precision control path and the high- and low-precision arithmetic data paths can be checked using static timing analysis (STA) tools. Based on the circuit topology of this invention, the paths from each detection circuit to the ALU register and each stage of the ALU multiplier can meet the timing convergence requirements and there is no metastability risk; the low-precision arithmetic path has a smaller bit width and a better timing margin than the high-precision mode, making it suitable for long-term low-curvature steady-state iteration scenarios.

[0161] Verification Summary

[0162] Using the aforementioned conventional digital IC verification methods, those skilled in the art can fully reproduce all functions of the adaptive precision switching of this invention, verifying core design indicators such as switching delay, timing convergence, and hardware area optimization; combined with the tensor iteration condition, most cycles are in The low-precision mode also allows for theoretical estimation of system dynamic power consumption based on the well-known law that CMOS dynamic power consumption is positively correlated with the square of the bit width, fully demonstrating the comprehensive optimization effect of this invention in terms of accuracy, area, and power consumption.

[0163] Example 5: Pipeline-level Quantization Error Analysis (Offline Auxiliary Tool)

[0164] The pipeline-level hierarchical quantization analyzer is a front-end offline tool, not integrated into the hardware, used to pre-evaluate the cumulative error under different bit width configurations. It employs a three-stage simplified pipeline test, where the original configuration has a high cumulative MAE. The tool automatically outputs a fixed-point format optimization scheme, which significantly improves error metrics after adjustment, mitigating the risk of quantization failure before tape-out.

Claims

1. A hybrid precision adaptive quantization circuit system, characterized in that, include: The system includes a multi-channel ALU array, a curvature detection circuit, and a data range detection unit. The multi-channel ALU array contains N ALUs, where N≥3. Each ALU is a reconfigurable arithmetic unit with independent data path widths and independent precision mode registers. The multi-channel ALUs can merge and split computation tasks to form a closed-loop computation pipeline. The pipeline can fully perform metric tensor solving, connection symbol calculation, numerical integration, covariant gradient difference operations, and trace purity determination. The data path width of each ALU is configured according to its error propagation sensitivity level in the closed-loop pipeline. The higher the error propagation sensitivity level, the higher the configured data path width. The multi-channel ALU array includes: The first-end ALU is configured with a data path that can dynamically switch between high-precision floating-point format and low-precision floating-point format. It is used to perform metric tensor solving and matrix inversion operations at the beginning of the closed-loop pipeline and has the highest level of error propagation sensitivity. At least one mid-range ALU is configured with a medium-width fixed-point floating-point data path for performing at least one of subtraction and difference, accumulation and integration, and gradient difference operations, with an error propagation sensitivity level of medium or sub-high. The terminal ALU is configured with a low-bit-width floating-point or fixed-point format fixed data path and is used to perform pipeline end threshold comparison and status determination operations. It has the lowest level of error propagation sensitivity. The curvature detection circuit's input is connected to the output of the metric tensor calculation unit, and its output is hardwired to the precision mode register write port of the first-end ALU. The curvature detection circuit is used to monitor the curvature scalar R in real time. When R is greater than a preset curvature threshold, a high-precision floating-point mode selection signal is written to the precision mode register of the first-end ALU. When R is less than or equal to the preset curvature threshold, a low-precision floating-point mode selection signal is written to the first-end ALU. The hardwire is a physical metal wire direct connection, without passing through the bus arbiter, microcode controller, software interrupt and scheduling logic; the signal path does not contain any other writable storage elements except for the precision mode register. The data range detection unit's input is connected to the pipeline data stream, and its output is hardwired to the precision mode register write port of the final ALU; the data range detection unit is used to monitor the estimated dynamic range of data in real time. ;when When the value is greater than or equal to the preset threshold, the first fixed-point format selection signal is written to the precision mode register of the terminal ALU; When the value is less than the preset threshold, a second fixed-point format selection signal is written to the terminal ALU.

2. The hybrid precision adaptive quantization circuit system as described in claim 1, characterized in that: The intermediate ALU is configured with a medium-width fixed floating-point data path using a 16-bit to 32-bit floating-point format; the terminal ALU is configured with a low-width floating-point format of 8-bit floating-point.

3. The hybrid precision adaptive quantization circuit system as described in claim 1, characterized in that, The high-precision floating-point format and low-precision floating-point format combination used by the first-end ALU are selected from any of the following groups: Group 1: High-precision floating-point format is 64-bit floating-point, and low-precision floating-point format is 32-bit floating-point; The second group: high-precision floating-point format is 32-bit floating-point, and low-precision floating-point format is 16-bit floating-point format fp16 or bfloat16; The third group: high-precision floating-point format is 32-bit floating-point, and low-precision floating-point format is 8-bit floating-point format fp8_e4m3 or fp8_e5m2; Group 4: High-precision floating-point format is 16-bit floating-point format fp16 or bfloat16, and low-precision floating-point format is 8-bit floating-point format fp8_e4m3 or fp8_e5m2.

4. The hybrid precision adaptive quantization circuit system as described in claim 1, characterized in that, The curvature detection circuit includes: a curvature comparator, a precision mode encoder, and hardwired write logic. The curvature comparator receives a curvature scalar R and a preset curvature threshold register output from its two inputs, respectively; the precision mode encoder input is connected to the curvature comparator output; and the hardwired write logic connects the precision mode encoder output to the precision mode register write port of the ALU at the beginning.

5. The hybrid precision adaptive quantization circuit system as described in claim 1, characterized in that, The data range detection unit includes: a maximum value comparison tree, a fixed-point format encoder, and hardwired write logic; The maximum value comparison tree takes all tensor elements of the current period as parallel input and outputs an estimated dynamic range value of the data. The fixed-point encoder receives two inputs respectively.

1. Preset threshold register output; hard-wired write logic connects the fixed-point format encoder output to the end ALU precision mode register write port.

6. The hybrid precision adaptive quantization circuit system as described in claim 5, characterized in that, The estimated dynamic range of the data satisfies: In the formula This refers to all tensor elements that participate in the computation within the current iteration cycle.

7. The hybrid precision adaptive quantization circuit system as described in claim 1, characterized in that, The curvature detection circuit and the comparison logic inside the data range detection unit adopt pure combinational logic; the comparison result is latched into the precision mode register of the corresponding ALU after being delayed by combinational logic; on the next rising edge of the clock, the output signal of the precision mode register controls the corresponding path multiplexer to complete the data path switching, and the overall switching delay is less than 1 clock cycle.

8. The hybrid precision adaptive quantization circuit system as described in claim 4, characterized in that, The curvature detection circuit is composed of a first numerical comparator, an edge-triggered precision mode latch register, and a first multiplier array bit-width configuration multiplexer cascaded together. The inputs of the first numerical comparator are connected to the curvature scalar R bus and the preset curvature threshold fixed value register, respectively; the output of the first numerical comparator is connected to the clock enable terminal of the precision mode latch register; the output of the precision mode latch register is hardwired to the control terminal of the multiplexer; the two inputs of the multiplexer are connected to the high-precision floating-point multiplier array and the low-precision floating-point multiplier array, respectively; the output of the multiplexer is the core operation data input port of the first-end ALU.

9. A hybrid precision adaptive quantization method based on the hybrid precision adaptive quantization circuit system according to any one of claims 1 to 8, characterized in that, include: The curvature detection circuit acquires the curvature scalar R in real time. When R crosses the preset curvature threshold, it sends a precision mode selection signal to the precision mode register of the first-end ALU through a hard connection to control the first-end ALU to switch between high-precision floating-point path and low-precision floating-point path. Real-time data dynamic range estimation value of fixed-point format switching circuit ,exist When the preset threshold is exceeded, a fixed-point format selection signal is sent to the precision mode register of the terminal ALU via hardwired connection to control the terminal ALU to switch between two fixed-point formats; All comparison, encoding, and register write operations are executed independently in hardware, without relying on software interrupts or operating system scheduling. The overall precision mode switching delay is less than one clock cycle.

10. A tensor computing chip, characterized in that, Includes the hybrid precision adaptive quantization circuit system as described in any one of claims 1 to 8.

Images