Accumulator-based adaptive quantization system and accumulator

Abstract

The application relates to the technical field of adaptive quantization processing of accumulators, in particular to an accumulator-based adaptive quantization system and an accumulator. The system comprises a statistical module, a safety factor calculation module and a quantization execution module connected in sequence. The statistical module is used for extracting statistical features of an activation value matrix to be quantized and a corresponding weight matrix respectively, and calculating a first statistical ratio of the activation value matrix and a second statistical ratio of the weight matrix based on the statistical features. The safety factor calculation module is used for calculating a safety factor based on the first statistical ratio and the second statistical ratio, and calculating a dynamic quantization range of the activation value matrix in combination with a hardware constraint parameter of the accumulator. The quantization execution module is used for determining a scaling factor according to the dynamic quantization range, and performing a quantization mapping operation on the activation value matrix through the scaling factor to obtain a quantized activation value matrix. The application can guarantee the accuracy of model reasoning under the premise of meeting the anti-overflow requirement of the accumulator hardware.

Description

Technical Field

[0001] This invention relates to the field of adaptive quantization processing technology of accumulators, and more particularly to an adaptive quantization system and accumulator based on accumulators. Background Technology

[0002] As the number of parameters in deep learning models continues to surge, quantization technology has become a core means of reducing model memory usage and improving hardware inference speed. Its core principle is to map floating-point numbers to low-bit integers for computation. In hardware such as AI accelerator chips, to pursue computational density and energy efficiency, the accumulator for matrix multiplication is often designed with a low bit width, and its numerical representation range is strictly limited. If the matrix multiplication sum exceeds this range, overflow will occur, directly leading to incorrect calculation results and severely affecting the accuracy of model inference.

[0003] Currently, the mainstream quantization schemes mainly include two categories: maximum scaling and fixed maximum quantization. Maximum scaling calculates a scaling factor based on the absolute maximum value of the data and maps the data to the corresponding integer range; fixed maximum quantization, on the other hand, pre-sets a fixed quantization upper limit and maps the data to that fixed range. Both methods are widely used in various quantization scenarios for neural network inference.

[0004] Existing quantization schemes have obvious technical flaws. Maximum value scaling only focuses on the mapping boundary of a single data element. In the case of low-bit-width accumulators, the accumulation of multiple elements is prone to accumulation and overflow. Fixed maximum value quantization is a static scheme and lacks flexibility. Setting the quantization upper limit too high can easily lead to overflow, while setting it too low will waste the range of integer representation and cause a decrease in quantization accuracy. Neither of these methods can meet the requirements of preventing numerical overflow and maintaining quantization accuracy under the hardware constraints of low-bit-width accumulators. Summary of the Invention

[0005] This invention provides an adaptive quantization system and accumulator based on an accumulator to solve the problem that existing quantization schemes cannot simultaneously prevent numerical overflow and maintain quantization accuracy under the hardware constraints of low-bit-width accumulators.

[0006] In a first aspect, embodiments of the present invention provide an adaptive quantization system based on an accumulator, comprising: The statistical module, the safety factor calculation module, and the quantitative execution module are connected in sequence. The statistical module is used to extract statistical features from the activation value matrix to be quantized and the corresponding weight matrix, and to calculate the first statistical ratio of the activation value matrix and the second statistical ratio of the weight matrix based on the statistical features. The security factor calculation module is used to calculate the security factor based on the first statistical ratio and the second statistical ratio, and to calculate the dynamic quantization range of the activation value matrix in combination with the hardware constraint parameters of the accumulator. The quantization execution module is used to determine a scaling factor based on the dynamic quantization range, and to perform a quantization mapping operation on the activation value matrix through the scaling factor to obtain a quantized activation value matrix.

[0007] In one possible implementation, the statistical module includes: a statistical feature extraction unit and a statistical ratio calculation unit; The statistical feature extraction unit is used to extract the standard deviation, maximum absolute value, and equivalent statistics of the activation value matrix to be quantized and the corresponding weight matrix, respectively. The statistical ratio calculation unit is connected to the output of the statistical feature extraction unit and is used to calculate a first statistical ratio based on the standard deviation, maximum absolute value and equivalent statistics of the activation value matrix, and to calculate a second statistical ratio based on the standard deviation, maximum absolute value and equivalent statistics of the weight matrix.

[0008] In one possible implementation, the security factor calculation module is used to calculate based on... ,or Calculate the safety factor; in, Indicates safety factor, This represents the empirical proportionality coefficient. Indicates the first statistical ratio. This indicates the second statistical ratio.

[0009] In one possible implementation, the security factor calculation module is used to calculate based on... Calculate the dynamic quantization range of the activation value matrix; in, Indicates the dynamic quantization range. This represents the maximum safe value threshold that the accumulator in the hardware constraint parameters supports without overflowing. This represents the maximum quantized value of the weights in the hardware constraint parameters. This indicates the cumulative length of the segments in the hardware constraint parameters. This indicates rounding down to the nearest integer.

[0010] In one possible implementation, the security factor calculation module is further configured to, based on Limit the range of the calculated dynamic quantization; in, This indicates the final dynamic quantization range. This represents the truncation function.

[0011] In one possible implementation, the quantization execution module is used to, according to Determine the scaling factor; in, Indicates the scaling factor. Represents the activation value matrix. This represents the absolute maximum value of the activation value matrix.

[0012] In one possible implementation, the quantization execution module is used to, according to Perform a quantization mapping operation on the activation value matrix to obtain a quantized activation value matrix; in, This represents the quantized activation value matrix. This represents the floor function. This represents the lower limit of the final quantized positive number range of the activation value. This represents the upper limit of the final quantized positive number range of the activation value.

[0013] In one possible implementation, the statistical features include: the mean distribution of the activation value matrix and the corresponding weight matrix; It also includes a centralized correction module; The centering correction module is connected to the statistics module and is used to introduce a zero-point offset mechanism to center the activation value matrix and / or the weight matrix if the distribution mean of the activation value matrix and / or the distribution mean of the weight matrix does not meet the requirements, so that the distribution mean of the corrected activation value matrix and / or weight matrix meets the requirements, and then send the corrected activation value matrix and / or weight matrix to the statistics module for subsequent operations.

[0014] Secondly, embodiments of the present invention provide an artificial intelligence (AI) acceleration chip that integrates an accumulator-based adaptive quantization system as described in the first aspect or any possible implementation of the first aspect.

[0015] Thirdly, embodiments of the present invention provide an accumulator, including the AI ​​acceleration chip in the implementation of the second aspect described above.

[0016] This invention provides an adaptive quantization system and accumulator based on an accumulator. Through a statistical module, it extracts statistical features from the activation value matrix and the corresponding weight matrix to be quantized, and calculates a first statistical ratio of the activation value matrix and a second statistical ratio of the weight matrix based on the statistical features. This incorporates the distribution characteristics of the weight matrix into the consideration dimensions of quantization parameter calculation, breaking through the limitation of only performing quantization analysis on a single activation value matrix, and providing a comprehensive data distribution basis for the accurate calculation of subsequent quantization parameters. The security factor calculation module is used to calculate the security factor based on the first statistical ratio and the second statistical ratio, so that the determination of the security factor is directly related to the distribution characteristics of the two types of data, and the calculation of the quantization parameters can be adapted to different distribution combinations of activation values ​​and weights. Then, the dynamic quantization range of the activation value matrix is ​​calculated in combination with the hardware constraint parameters of the accumulator, so as to realize the combination of data distribution characteristics and hardware calculation requirements, and make the quantization range both fit the actual distribution characteristics of the data and meet the hardware operation constraints of the accumulator. The quantization execution module is used to determine a scaling factor based on the dynamic quantization range, and perform a quantization mapping operation on the activation value matrix through the scaling factor to obtain a quantized activation value matrix. The quantization scaling ratio of the activation value matrix is ​​determined by the dynamic quantization range that adapts to hardware constraints and data distribution, rather than a fixed ratio or a ratio derived from a single maximum value. This allows the quantized activation value matrix to adapt to the computational requirements of a low-bit-width accumulator, effectively avoiding numerical overflow when quantized data participates in matrix multiplication and accumulation operations, reducing accuracy loss during the quantization process, ensuring the numerical validity of the quantized activation value matrix, and thus improving the accuracy of model inference. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram of the structure of the adaptive quantization system based on accumulator provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an adaptive quantization system based on an accumulator provided in another embodiment of the present invention. Detailed Implementation

[0019] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0020] See Figure 1 This illustration shows a schematic diagram of an adaptive quantization system based on an accumulator according to an embodiment of the present invention. The adaptive quantization system based on an accumulator is used to adaptively quantize the activation values ​​involved in matrix multiplication operations during neural network inference under hardware constraints such as low-bit-width accumulators, in order to improve quantization accuracy and computational stability as much as possible while avoiding numerical overflow. The adaptive quantization system based on an accumulator includes: a statistics module 1, a safety factor calculation module 2, and a quantization execution module 3 connected in sequence. The statistics module 1 is used to extract statistical features from the activation value matrix and the corresponding weight matrix to be quantized, and to calculate the first statistical ratio of the activation value matrix and the second statistical ratio of the weight matrix based on the statistical features. The safety factor calculation module 2 is used to calculate the safety factor based on the first statistical ratio and the second statistical ratio, and to calculate the dynamic quantization range of the activation value matrix in combination with the hardware constraint parameters of the accumulator. The quantization execution module 3 is used to determine the scaling factor based on the dynamic quantization range, and to perform a quantization mapping operation on the activation value matrix through the scaling factor to obtain the quantized activation value matrix.

[0021] Statistical Module 1, as the basic processing module of the adaptive quantization system, is primarily used to extract full-dimensional statistical features from the activation value matrix to be quantized and the weight matrix that matches it for matrix multiplication. Based on the extracted statistical features, it calculates the first statistical ratio representing the distribution characteristics of the activation value matrix and the second statistical ratio representing the distribution characteristics of the weight matrix. This provides accurate data distribution basis for subsequent calculation of the safety factor and determination of the dynamic quantization range, and is a prerequisite for realizing data distribution-aware adaptive quantization.

[0022] In the matrix multiplication scenario of neural network inference, the object processed by statistics module 1 is the activation value matrix to be quantized. and the corresponding weight matrix These two are the two core operands of matrix multiplication. The statistics module 1 performs independent and standardized statistical feature extraction processes on the two matrices respectively. The extracted statistical features include at least the absolute maximum value and standard deviation of the data. Alternatively, equivalent feature parameters can be selected. The absolute maximum value is used to characterize the boundary of the numerical distribution range of the data, and the standard deviation is used to characterize the dispersion of the data. The combination of the two can accurately depict the overall distribution pattern of the data.

[0023] In one embodiment, such as Figure 2 As shown, the statistical module 1 includes: a statistical feature extraction unit 11 and a statistical ratio calculation unit 12; The statistical feature extraction unit 11 is used to extract the standard deviation, maximum absolute value and equivalent statistics of the activation value matrix to be quantized and the corresponding weight matrix, respectively. The statistical ratio calculation unit 12 is connected to the output of the statistical feature extraction unit 11 and is used to calculate the first statistical ratio based on the standard deviation, maximum absolute value and equivalent statistics of the activation value matrix, and to calculate the second statistical ratio based on the standard deviation, maximum absolute value and equivalent statistics of the weight matrix.

[0024] Optionally, the equivalent statistic for the standard deviation can be the mean absolute deviation. The equivalent statistic for the maximum absolute value can be the 99.9th percentile.

[0025] Optionally, the statistical ratio calculation unit 12 is based on Calculate the first statistic, where, Indicates the first statistical ratio. Represents the activation value matrix. This represents the absolute value of the activation value matrix. This represents the maximum absolute value of the activation value matrix. This represents the standard deviation of the activation value matrix.

[0026] The formula for calculating the first statistic here can include... Replace with the mean absolute deviation of the activation value matrix. It can be replaced with the 99.9th percentile.

[0027] Statistical ratio calculation unit 12 according to Calculate the second statistical ratio, where, Indicates the second statistical ratio. Represents the weight matrix. Represents the absolute value of the weight matrix. This represents the maximum absolute value of the weight matrix. This represents the standard deviation of the weight matrix.

[0028] The formula for calculating the second statistic here can include... Replace with the mean absolute deviation of the weight matrix. It can be replaced with the 99.9th percentile.

[0029] The first and second statistical ratios mentioned above are key indicators for measuring the sharpness of data distribution. Different types of typical data distributions correspond to fixed statistical ratio reference values. For example, for normally distributed data, the statistical ratio is about 3.5; for uniformly distributed data, the statistical ratio is about 1.732. By calculating these statistical ratios, statistical module 1 can directly quantify the actual distribution of the activation value matrix and weight matrix, allowing subsequent modules to determine the data distribution type based on the quantified numerical indicators, and thus achieve targeted adjustment of quantitative parameters.

[0030] When performing statistical feature extraction and calculating statistical ratios (first and second statistical ratios), the statistical module 1 processes the data according to the statistical assumptions of this embodiment, namely, that the mean of the activation value matrix and the weight matrix is ​​close to zero. This assumption is highly consistent with the actual engineering scenarios of deep neural network inference: on the one hand, activation values ​​are usually normalized by Layer Normalization, RMS Normalization, etc., before entering matrix multiplication operations, and their mean is stable in the range of zero or close to zero; on the other hand, the weight parameters are initialized with symmetric distribution and regularized during model training, and their statistical mean naturally tends to zero. If there is a significant mean shift in the activation value matrix or weight matrix in actual applications, it is necessary to correct the activation value matrix and / or weight matrix so that the mean of the corrected data satisfies the statistical assumption of being close to zero, ensuring the accuracy of subsequent statistical feature extraction and ratio calculation.

[0031] In one embodiment, the statistical features include the distribution mean of the activation value matrix and the weight matrix; like Figure 2 As shown, the adaptive quantization system based on the accumulator also includes: a centralized correction module 4; The centering correction module 4 is connected to the statistics module 1. If the distribution mean of the activation value matrix and / or the distribution mean of the weight matrix do not meet the requirements, a zero-point offset mechanism is introduced to center the activation value matrix and / or the weight matrix, so that the distribution mean of the corrected activation value matrix and / or the weight matrix meets the requirements. The corrected activation value matrix and / or weight matrix are then sent to the statistics module 1 for subsequent operations.

[0032] Here, the mean distribution of the activation value matrix and / or the mean distribution of the weight matrix do not meet the requirements; that is, the mean distribution of the activation value matrix and / or the mean distribution of the weight matrix are significantly not close to zero. For example, the mean distribution of the activation value matrix and / or the mean distribution of the weight matrix are not within the preset range. The preset range can be […]. , ],in, This represents the standard deviation of the activation value matrix or weight matrix. After centering correction, the mean of the corrected activation value matrix data and / or weight matrix distribution meets the requirements, that is, the mean of the corrected activation value matrix data and / or weight matrix distribution is close to zero, i.e., it falls within the preset range. , ].

[0033] Through the aforementioned zero-point offset processing, even if the mean of the original data distribution deviates significantly from zero, the subsequent statistical modeling assumptions based on variance propagation can still be satisfied, enabling the adaptive quantization method proposed in this embodiment to continuously and stably characterize the numerical accumulation characteristics during the accumulation process. In different embodiments, the zero-point offset mechanism can be applied to activation values, weights, or any combination of both. Zero-point offset can be implemented using either symmetric or asymmetric quantization methods; this embodiment does not limit its implementation.

[0034] The output of Statistical Module 1 is the absolute maximum value, standard deviation, and first statistical ratio of the activation value matrix. And the absolute maximum value, standard deviation, and second statistic of the weight matrix. The output result will be transmitted in real time to the security factor calculation module 2, which is logically connected to it, as the core calculation basis for the security factor and dynamic quantization range. This embodiment does not limit the specific implementation form of the statistics module 1. In practical applications, the statistics module 1 can be implemented by software, hardware circuit, or a combination of software and hardware. As long as it can complete the above-mentioned functions of extracting statistical features and calculating statistical ratios of the activation value matrix and weight matrix, it falls within the protection scope of this invention.

[0035] The setup of the statistics module 1 enables the adaptive quantization method of this invention to accurately perceive the actual statistical characteristics of the data involved in the current calculation before quantization is performed. This breaks the limitation of traditional quantization methods that only focus on a single data dimension and ignore data distribution characteristics. At the same time, by quantizing the distribution characteristics of activation values ​​and weights into statistical ratios, the quantitative correlation between data distribution characteristics and subsequent hardware constraint parameters is realized. This lays the data foundation for balancing the contradiction between "preventing numerical overflow" and "maintaining quantization accuracy" under the condition of limited hardware accumulator bit width.

[0036] The processing flow of the security factor calculation module 2 is divided into two core steps: security factor calculation and dynamic quantization range calculation. It sequentially completes the quantitative mapping from data distribution characteristics to quantization range. The whole process organically integrates the joint distribution characteristics of activation values ​​and weights and the physical constraints of hardware accumulators to ensure that the calculated quantization range not only meets the numerical boundary requirements of the hardware, but also maximizes the use of the integer representation range.

[0037] The safety factor calculation module is based on statistical derivation results and utilizes the first statistical ratio. Second statistical ratio The core calculation model for the safety factor is constructed using the product of matrix multiplication. This model is designed to ensure that the piecewise accumulation of matrix multiplication does not exceed the hardware accumulator's upper limit within a range of 3 times the standard deviation, while also reserving a reasonable safety margin for engineering implementation to avoid numerical overflow issues caused by data distribution deviating from the ideal state. Optionally, the 3 times standard deviation range set here can be adjusted to 2 times or 4 times the standard deviation range based on the hardware's fault tolerance capability, by adjusting the corresponding empirical proportional coefficient.

[0038] In one embodiment, the safety factor calculation module 2 is used to calculate based on Calculate the safety factor; in, Indicates safety factor, Indicates the first statistical ratio. Indicates the second statistical ratio. The empirical proportionality coefficient is a fault tolerance coefficient determined based on theoretical derivation and extensive experimental fitting. Its value can be set to 1 / 6, which maintains the rigor of theoretical derivation while taking into account the uncertainty and conservative requirements in engineering implementation. This empirical proportionality coefficient can be adjusted automatically under different hardware overflow tolerance requirements.

[0039] To prevent due to , A value that is too small will cause instability in the safety factor value, which in turn will lead to abnormal calculation of the quantization range. In this embodiment, a lower limit value of 0.2 is set for the safety factor, that is, according to Calculate the safety factor.

[0040] This safety factor has a clear physical meaning and can directly characterize the degree of constraint imposed by the combination of data distributions on the quantization range: the closer the distributions of the activation value matrix and the weight matrix are to a normal distribution, the better. , The larger the value of , the greater the security factor, indicating that the dispersion of the data distribution is more in line with statistical laws, and the quantization range can be appropriately relaxed to improve quantization accuracy; when the distribution of the activation value matrix and the weight matrix is ​​closer to a uniform distribution, , The smaller the value of , the lower the safety factor, down to as low as 0.2, which means that the variance of the data distribution is greater and the risk of cumulative overflow is higher. It is necessary to actively shrink the quantization range to avoid overflow problems.

[0041] After obtaining the safety factor, the safety factor calculation module 2 introduces the hardware constraint parameters of the accumulator and calculates the maximum permissible quantized integer value of the activation value matrix using a quantitative formula. (i.e., dynamic quantization range), and for Implement engineering scope constraints to ensure that it meets the actual application requirements of hardware quantification.

[0042] In one embodiment, the safety factor calculation module 2 is used to calculate based on Calculate the dynamic quantization range of the activation value matrix; in, Indicates the range of dynamic quantization.

[0043] This represents the maximum safe value threshold that the accumulator in the hardware constraint parameters can support without overflowing. Its value is determined by the data bit width of the accumulator in the hardware; for example, the accumulator can be an 8-bit, 12-bit, 16-bit, or higher bit width integer accumulator. The values ​​can be 127, 2047, 32767, etc.

[0044] Optional, , This indicates the effective bit width of the accumulator; the safe range of the accumulator is then... Usually Introduced as an explicit parameter, this application can be naturally adapted to accumulator hardware under different bit-width constraints.

[0045] This represents the maximum quantized value of the weights in the hardware constraint parameters, and its value is related to the bit width of the weight quantization. Related, the general calculation formula is as follows For example, during 4-bit weight quantization When quantizing for 7-bit and 8-bit weights The value is 127, which can be flexibly adjusted according to the actual bit width of the weight quantization.

[0046] This represents the piecewise accumulation length in the hardware constraint parameters, indicating the number of elements participating in a single local accumulation during matrix multiplication or tensor multiplication. Its value is related to hardware parallelism, computation array size, or pipeline design, and its range is [range missing]. Preferably, the segment length is 32, 64, 128, 256, or an integer multiple thereof. If the hardware supports variable segment length, it can be... It serves as a dynamic input parameter to adapt to different hardware configurations.

[0047] This indicates rounding down. Rounding the calculated result down in the formula is to ensure... It is an integer, meeting the hardware requirements for low-bit integer quantization, where The introduction of is to match the statistical characteristic that the standard deviation of the piecewise sum increases with the square root of the summation length, thus avoiding the spillover risk caused by the accumulation effect from the formula level.

[0048] To ensure The value of conforms to the engineering implementation requirements of hardware quantization, avoiding severe loss of quantization accuracy due to an excessively small value or an excessively large value exceeding the integer range of hardware quantization. This embodiment will... Within a reasonable engineering range [4,127], the final dynamic quantization range is determined using a truncation function. In one embodiment, the safety factor calculation module is further used to determine the safety factor calculation range based on... Limit the range of the calculated dynamic quantization; in, This indicates the final dynamic quantization range. This represents the truncation function.

[0049] The final output of safety factor calculation module 2 is the calculated safety factor. The final dynamic quantization range is then transmitted in real time to the quantization execution module 3, which is logically connected to it. This serves as the core basis for calculating the quantization scaling factor of the activation value matrix, linear quantization, and truncation, ensuring that subsequent quantization operations fully adapt to the current data distribution characteristics and hardware accumulator constraints.

[0050] This embodiment does not limit the specific implementation of the safety factor calculation module 2. In practical applications, this module can be implemented together with the statistics module 1 through software, or it can be implemented in a software and hardware co-operation manner according to hardware development requirements. As long as the core functions of "calculating the safety factor based on the first statistical ratio and the second statistical ratio" and "calculating the final dynamic quantization range in combination with hardware constraint parameters" can be completed, they all fall within the protection scope of this invention.

[0051] The inclusion of the safety factor calculation module 2 enables this embodiment to successfully establish a direct mapping link from "joint data distribution characteristics" to "hardware security quantization boundary," breaking the limitations of traditional quantization methods that separate data processing from hardware constraints. Through dynamic adjustment of the safety factor and explicit introduction of hardware parameters, the quantization range can be adaptively adjusted according to changes in data distribution. When the data distribution is "dangerous" (prone to overflow), the quantization range automatically shrinks, and when the data distribution is "safe" (not prone to overflow), the quantization range automatically expands. This lays the core parameter foundation for subsequent overflow-free, high-precision model inference on low-bit-width accumulator hardware.

[0052] Optionally, to reduce computational overhead, one can... , Discretization, pre-calculating the corresponding , Stored in a lookup table, the real-time floating-point calculation is replaced by table lookup during inference calculations.

[0053] The quantization execution module 3, as the terminal execution unit of the accumulator-based adaptive quantization system, is logically connected to the safety factor calculation module 2. It receives the final dynamic quantization range of the activation value matrix output by the safety factor calculation module. Its core function is to determine the scaling factor of the activation value quantization based on the final dynamic quantization range, and to perform standardized quantization mapping and numerical constraint operations on the original activation value matrix through the scaling factor. Finally, it outputs a quantized activation value matrix that meets the hardware calculation requirements of the low-bit-width accumulator. At the same time, it synchronously outputs the key parameters in the quantization process, providing accurate and compliant operands for subsequent matrix multiplication and accumulation operations. It is the core link in realizing the implementation of activation value quantization.

[0054] The core processing flow of the quantization execution module 3 is divided into two key steps: scaling factor calculation and quantization mapping and truncation. The entire process strictly follows the basic principle of linear quantization. At the same time, it is specifically adapted to the final dynamic quantization range output by the safety factor calculation module to ensure that the quantized activation value will not exceed the numerical range of the hardware accumulator and cause overflow when participating in matrix multiplication and accumulation operations, while maximizing the use of the integer representation range to ensure quantization accuracy.

[0055] The scaling factor is the core proportional parameter for mapping the original floating-point activation value to low-bit integers. Its function is to linearly map the numerical range of the original activation value matrix to the integer interval defined by the final dynamic quantization range. The quantization execution module 3 calculates the scaling factor based on the final dynamic quantization range output by the security factor calculation module 2 and the absolute maximum value of the original activation value matrix.

[0056] In one embodiment, the quantization execution module is used to, according to Determine the scaling factor; in, Indicates the scaling factor. Represents the activation value matrix. This represents the absolute maximum value of the activation value matrix.

[0057] The design logic of this calculation formula is to match the numerical boundaries of the activation value matrix with the integer boundaries of the final dynamic quantization range in a proportional manner, so as to ensure that the numerical distribution characteristics of the activation values ​​are preserved to the greatest extent after quantization, while avoiding loss of quantization accuracy or numerical out-of-bounds due to improper scaling.

[0058] After obtaining the scaling factor, the quantization execution module 3 performs linear quantization and numerical truncation operations on the original activation value matrix to complete the mapping from floating-point numbers to low-bit integers. In one embodiment, the quantization execution module is used to, according to Perform a quantization mapping operation on the activation value matrix to obtain the quantized activation value matrix; in, This represents the quantized activation value matrix. This represents a truncation function, which restricts the input value to a specified range of upper and lower limits, with the lower limit being 0. The upper limit is The setting of this interval is consistent with the symmetric quantization representation rule of low-bit integers, ensuring that the quantization result is adapted to the integer storage and calculation logic of the hardware. This represents the floor function. This represents the lower limit of the final quantized positive number range of the activation value. This represents the upper limit of the final quantized range of positive numbers for the activation values. In the formula above for calculating the quantized activation value matrix, To divide each element in the original activation value matrix by a scaling factor and perform rounding on the calculation result to obtain a preliminary quantized integer result, this implements a basic mapping from floating-point numbers to integers, allowing the activation values ​​to meet the hardware's integer calculation requirements.

[0059] To ensure that the quantized integer elements strictly fall within the legal range defined by the final dynamic quantization range, the quantization execution module truncates the initial quantized integer results using a truncation function. Values ​​exceeding the range are forcibly corrected to the range boundary values, ultimately resulting in a compliant quantized activation value matrix. The constraint formula is as follows: .

[0060] The quantized activation value matrix obtained after the above steps When performing matrix multiplication and piecewise accumulation with the quantized weight matrix, it can be guaranteed that the accumulated sum will not exceed the maximum value of the hardware accumulator in most cases. This avoids the risk of numerical overflow at the terminal execution level. At the same time, since the final dynamic quantization range is dynamically determined by combining data distribution characteristics and hardware constraints, the quantization result makes full use of the integer representation range, effectively avoiding unnecessary loss of quantization accuracy.

[0061] The final output of the quantization execution module is not only the quantized activation value matrix. Instead, it includes a combination of parameters such as the quantized activation value matrix, scaling factor, and dynamic quantization range, i.e. ( , , The synchronous output of this parameter combination has significant engineering implications: on the one hand, the quantized activation matrix serves as a direct operand for the hardware to perform matrix multiplication, participating in subsequent accumulation calculations; on the other hand, the scaling factor and dynamic quantization range, as key parameters in the quantization process, can be used in subsequent dequantization operations to restore integer results to floating-point results, ensuring the integrity and accuracy of the model inference results.

[0062] This embodiment does not limit the specific implementation of the quantization execution module 3. In practical applications, considering that quantization operations need to be performed frequently and efficiently to match the inference speed requirements of AI acceleration chips, this module can preferably be implemented in hardware, such as using a dedicated quantization operation circuit to complete scaling, rounding, and clamping operations. In scenarios with strict requirements on hardware resource usage or low inference speed requirements, a software and hardware co-implementation approach can also be used, or it can be implemented in software together with the statistics module 1 and the security factor calculation module 2. Regardless of the implementation method, as long as the core functions of "calculating the scaling factor according to the dynamic quantization range" and "performing quantization mapping and truncation on the activation value matrix" can be completed and the corresponding parameter combinations can be output, they all fall within the protection scope of this application.

[0063] The quantization execution module 3 completes the closed loop from quantization parameter calculation to quantization operation implementation. Its quantization mapping operation, based on dynamic quantization range execution, breaks the limitation of traditional fixed quantization schemes that rely on a single scaling ratio to adapt to all data distributions. This deeply binds the quantization operation to data distribution characteristics and hardware computational constraints. The quantized activation value matrix output by this module satisfies the computational requirements of the low-bit-width accumulator hardware while preserving the numerical information of the original activation values ​​to the maximum extent. Ultimately, it achieves the invention goal of "maintaining quantization accuracy" while "preventing numerical overflow," effectively improving the inference accuracy and stability of deep learning models on low-bit-width AI acceleration chips.

[0064] This invention provides an adaptive quantization system based on an accumulator, comprising: a statistics module, a safety factor calculation module, and a quantization execution module connected in sequence; the statistics module is used to extract statistical features from the activation value matrix to be quantized and the corresponding weight matrix, and to calculate a first statistical ratio of the activation value matrix and a second statistical ratio of the weight matrix based on the statistical features; the safety factor calculation module is used to calculate a safety factor based on the first and second statistical ratios, and to calculate the dynamic quantization range of the activation value matrix in conjunction with the hardware constraint parameters of the accumulator; the quantization execution module is used to determine a scaling factor according to the dynamic quantization range, and to perform a quantization mapping operation on the activation value matrix through the scaling factor to obtain the quantized activation value matrix.

[0065] In this embodiment of the invention, the statistical module simultaneously extracts statistical features from the activation value matrix and the weight matrix and calculates the corresponding statistical ratios. The distribution features of the weight matrix are incorporated into the consideration dimensions of the quantification parameter calculation, breaking through the limitation of only performing quantification analysis on a single activation value matrix. This enables dual perception of the distribution features of the two types of core data involved in matrix multiplication, providing a comprehensive data distribution basis for the accurate calculation of subsequent quantification parameters. The safety factor calculation module calculates the safety factor based on the statistical ratio of the activation value matrix and the weight matrix, so that the determination of the safety factor is directly related to the distribution characteristics of the two types of data. This allows the calculation of quantization parameters to adapt to different distribution combinations of activation values ​​and weights. At the same time, it combines the accumulator hardware constraint parameters to calculate the dynamic quantization range, realizing the combination of data distribution characteristics and hardware calculation requirements. This ensures that the quantization range not only fits the actual distribution characteristics of the data, but also meets the hardware operation constraints of the accumulator. The quantization execution module determines the scaling factor and performs quantization mapping based on the dynamic quantization range. This ensures that the quantization scaling ratio of the activation value matrix is ​​determined by the dynamic quantization range that adapts to hardware constraints and data distribution, rather than a fixed ratio or a ratio derived from a single maximum value. This allows the quantized activation value matrix to adapt to the computational requirements of the low-bit-width accumulator, effectively avoiding numerical overflow when the quantized data participates in matrix multiplication and accumulation operations. The accumulator-based adaptive quantization system consists of a statistical module, a safety factor calculation module, and a quantization execution module connected in sequence, forming a standardized quantization processing link. It forms a closed loop from data statistical feature extraction and quantization parameter calculation to quantization operation execution. The functions of each module are connected and the division of labor is clear, realizing the automated completion of activation value matrix quantization processing without manual intervention to adjust quantization parameters, improving the efficiency of quantization processing, and ensuring the consistency and stability of the quantization process. The dynamic quantization range is dynamically calculated based on data distribution characteristics and hardware constraints, enabling the quantization operation to adjust the quantization scale according to the actual distribution of the activation value matrix and weight matrix. Under the premise of meeting the overflow prevention requirements of the accumulator hardware, it maximizes the use of the integer representation range, reduces the accuracy loss in the quantization process, ensures the numerical validity of the activation value matrix after quantization, and thus improves the accuracy of model inference.

[0066] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0067] This invention provides an AI acceleration chip that integrates an accumulator-based adaptive quantization system according to any of the above embodiments.

[0068] This invention provides an accumulator, including the AI ​​acceleration chip described above.

[0069] In the above embodiments, the descriptions of each embodiment have their own emphasis. Parts not detailed or described in a particular embodiment can be referred to in the relevant descriptions of other embodiments. Unless otherwise specified or in conflict with logic, the terminology and / or descriptions between different embodiments are consistent and can be referenced interchangeably. Technical features in different embodiments can be combined to form new embodiments based on their inherent logical relationships.

[0070] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included within the protection scope of the present invention.

Claims

1. An adaptive quantization system based on an accumulator, characterized in that, include: The statistical module, the safety factor calculation module, and the quantitative execution module are connected in sequence. The statistical module is used to extract statistical features from the activation value matrix to be quantized and the corresponding weight matrix, and to calculate the first statistical ratio of the activation value matrix and the second statistical ratio of the weight matrix based on the statistical features. The security factor calculation module is used to calculate the security factor based on the first statistical ratio and the second statistical ratio, and to calculate the dynamic quantization range of the activation value matrix in combination with the hardware constraint parameters of the accumulator. The quantization execution module is used to determine a scaling factor based on the dynamic quantization range, and to perform a quantization mapping operation on the activation value matrix using the scaling factor to obtain a quantized activation value matrix.

2. The adaptive quantization system based on accumulator according to claim 1, characterized in that, The statistical module includes: a statistical feature extraction unit and a statistical ratio calculation unit; The statistical feature extraction unit is used to extract the standard deviation, maximum absolute value, and equivalent statistics of the activation value matrix to be quantized and the corresponding weight matrix, respectively. The statistical ratio calculation unit is connected to the output of the statistical feature extraction unit and is used to calculate a first statistical ratio based on the standard deviation, maximum absolute value and equivalent statistics of the activation value matrix, and to calculate a second statistical ratio based on the standard deviation, maximum absolute value and equivalent statistics of the weight matrix.

3. The adaptive quantization system based on accumulator according to claim 1, characterized in that, The safety factor calculation module is used to calculate based on ,or Calculate the safety factor; in, Indicates safety factor, This represents the empirical proportionality coefficient. Indicates the first statistical ratio. This indicates the second statistical ratio.

4. The adaptive quantization system based on accumulator according to claim 3, characterized in that, The safety factor calculation module is used to calculate based on Calculate the dynamic quantization range of the activation value matrix; in, Indicates the dynamic quantization range. This represents the maximum safe value threshold that the accumulator in the hardware constraint parameters can support without overflowing. This represents the maximum quantized value of the weights in the hardware constraint parameters. This indicates the segmented accumulation length in the hardware constraint parameters. This indicates rounding down to the nearest integer.

5. The adaptive quantization system based on an accumulator according to claim 4, characterized in that, The safety factor calculation module is also used to calculate based on Limit the range of the calculated dynamic quantization; in, This indicates the final dynamic quantization range. This represents the truncation function.

6. The adaptive quantization system based on an accumulator according to claim 5, characterized in that, The quantization execution module is used to, according to Determine the scaling factor; in, Indicates the scaling factor. Represents the activation value matrix. This represents the absolute maximum value of the activation value matrix.

7. The adaptive quantization system based on an accumulator according to claim 6, characterized in that, The quantization execution module is used to, according to Perform a quantization mapping operation on the activation value matrix to obtain a quantized activation value matrix; in, This represents the quantized activation value matrix. This represents the floor function. This represents the lower limit of the final quantized positive number range of the activation value. This represents the upper limit of the final quantized positive number range of the activation value.

8. The adaptive quantization system based on an accumulator according to any one of claims 1-7, characterized in that, The statistical features also include: the mean distribution of the activation value matrix and the corresponding weight matrix; The accumulator-based adaptive quantization system also includes a centralized correction module; The centering correction module is connected to the statistics module and is used to introduce a zero-point offset mechanism to center the activation value matrix and / or the weight matrix if the distribution mean of the activation value matrix and / or the distribution mean of the weight matrix does not meet the requirements, so that the distribution mean of the corrected activation value matrix and / or weight matrix meets the requirements, and then send the corrected activation value matrix and / or weight matrix to the statistics module for subsequent operations.

9. An artificial intelligence (AI) acceleration chip, characterized in that, An adaptive quantization system based on an accumulator, as described in any one of claims 1-8.

10. An accumulator, characterized in that, Including the AI ​​acceleration chip described in claim 9 above.

Images