A model self-adaptive fixed-point quantization method for overall performance optimization

Abstract

This invention discloses a model adaptive fixed-point quantization method for overall performance optimization. The fixed-point quantization method includes: a symmetric fixed-point representation based on bit width; constructing an initial fixed-point bit width based on feature statistics and alignment calculation; and joint optimization of fixed-point quantization-aware training and bit-width adaptation for overall network performance. Through the quantization-aware training process, the model can gradually adapt to the accuracy perturbations caused by fixed-point computation during the training phase, making the model's forward computation behavior during training approximate the fixed-point execution form of the final embedded inference stage, effectively reducing the performance degradation caused by training-deployment inconsistency.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a model adaptive fixed-point quantization method for overall performance optimization. Background Technology

[0002] With the rapid development of deep learning technology, deep neural network models, represented by deep convolutional neural networks (CNNs), have demonstrated excellent performance in tasks such as object detection, object recognition, and scene understanding, and have been widely applied in fields such as complex environment perception, precise object localization, and type recognition. In practical applications, especially in scenarios such as autonomous driving and drone operations in embedded intelligent terminals, models often need to be deployed on hardware platforms with limited computing power, storage, and power consumption, such as dedicated acceleration chips, embedded processors, or edge computing devices.

[0003] Due to the aforementioned hardware limitations, directly deploying high-precision floating-point models would result in significant storage overhead and computational latency, making it difficult to meet real-time performance and energy efficiency requirements. Therefore, fixed-point quantization of deep neural network models—converting model weights and intermediate features from high-precision floating-point representations to low-precision fixed-point representations—has become a key technical approach in the engineering deployment of models. Fixed-point quantization can significantly reduce model storage requirements and improve inference efficiency by utilizing fixed-point arithmetic units, thereby meeting the comprehensive performance and resource constraints of practical systems.

[0004] However, due to the complex structure and deep layers of deep neural network models, the feature distributions of different layers and channels vary significantly. Simple fixed-point quantization strategies often introduce large numerical errors, leading to a significant decrease in model accuracy. In complex application scenarios, especially in tasks requiring high accuracy in target localization and recognition, the accuracy loss caused by quantization becomes a major factor restricting the practical application of the model. Summary of the Invention

[0005] In view of the above problems, the present invention is proposed to provide a model adaptive fixed-point quantization method for overall performance optimization that overcomes or at least partially solves the above problems.

[0006] According to one aspect of the present invention, an adaptive fixed-point quantization method for models aimed at overall performance optimization is provided, the fixed-point quantization method comprising: Symmetric fixed-point representation based on bit depth; Construct an initial fixed-point number of bits based on feature statistics and alignment calculation; Joint optimization of fixed-point quantization-aware training and bit-width adaptive optimization for overall network performance.

[0007] Optionally, the bit-based symmetric fixed-point representation specifically includes: Symmetric fixed-point quantization is used to uniformly represent the feature values, weights, and biases in the network. In the quantization method, all values ​​are quantized with zero as the center; In terms of fixed-point representation, a fixed-point coefficient representation method with 2 raised to the power of N as the quantization scale factor is adopted, and floating-point values ​​are represented as the product of integers and scale factors. The quantization scale is limited to powers of 2, and floating-point multiplication and division operations are converted into shift operations. The fixed-point bit parameter is introduced to characterize the accuracy and dynamic range of fixed-point numbers.

[0008] Optionally, the fixed-point bit depth is used to characterize the number of bits occupied by the fractional part in the fixed-point number. Different layers or channels can be configured with different fixed-point bit depths to adapt to numerical distribution characteristics and accuracy requirements. By reasonably defining and constraining the fixed-point bit depth, we can ensure the accuracy of numerical representation while avoiding the risk of overflow, and provide a unified quantitative expression basis for subsequent bit-width joint optimization based on overall performance.

[0009] Optionally, the fixed-point representation specifically includes: Let the floating-point number be Its corresponding fixed-point representation is Fixed-point numbers use signed fixed-point representation, and their total bit width is denoted as . This includes the sign bit, integer bits, and decimal bits, defined as follows: in, Indicates the number of bits for the sign, when When both positive and negative values ​​are included When only non-negative or non-positive values ​​are included. ; Indicates the number of integer digits, used to limit the representable range of fixed-point numbers; Indicates the number of decimal places, used to limit the precision of fixed-point numbers; fixed point number and floating-point numbers The two satisfy a correspondence relationship: The equivalent representation is: The round operation uses either rounding to the nearest integer or rounding to the nearest whole number. Under the fixed-point partitioning, the range of floating-point numbers represented by fixed-point numbers is: ; The minimum resolvable precision is: .

[0010] Optionally, the construction of the initial fixed-point bit depth based on feature statistics and alignment calculation specifically includes: In the early stages of model training, the already trained floating-point model and training data are used to perform statistical analysis on the features and parameter distribution of each layer and channel in the network, and to construct an initial fixed-point bit configuration that satisfies the alignment calculation constraints.

[0011] Optionally, the construction of the initial fixed-point bit configuration that satisfies the alignment calculation constraints specifically includes: Characteristic distribution statistics and initial number of bits determination: The trained floating-point model is used to perform forward inference on the training data, and the floating-point distribution range of the output features of each layer and channel in the network is statistically analyzed to obtain the corresponding minimum and maximum values. Based on the preset feature retention ratio, and on the premise of covering most feature distributions, the corresponding fixed-point decimal places are adaptively determined for each channel feature, so that the fixed-point representation range can effectively cover the main feature distributions. Formal representation: For any convolutional layer The layer's output is used to perform forward inference on the training data using the trained floating-point model, resulting in the set of floating-point features corresponding to each channel. in Indicates the first The sample at the th The output feature values ​​of the layer To count the number of samples; Based on the above floating-point characteristics and statistical distribution range, we obtain: Introduce a preset feature retention ratio Select a feature set with a coverage ratio of 1. Key characteristic intervals: ; The main feature intervals satisfy at least Each feature value lies within its corresponding feature interval; Let the first The feature fixed-point representation of the layer adopts a symmetric fixed-point format, and its fixed-point number is formally represented as: in, These are floating-point eigenvalues; A fixed-point number in integer form; The number of fixed-point decimal places corresponding to the channel; Based on the aforementioned main feature intervals, they are determined as follows: in, and These are the number of fixed-point features, respectively. The following represents the minimum and maximum values; This indicates the floor function; Once the calculation is complete, the number of digits in the integer part is also determined accordingly. Calculation of alignment bits for weights and biases: For the weights after merging convolutional layers and batch normalized layers; Based on the determined number of feature bits, the fixed-point number of weights and biases is calculated layer by layer according to the alignment calculation principle. The fixed-point number of weights is determined according to their floating-point distribution range and precision requirements. The fixed-point number of biases is determined by the corresponding number of weight decimals and the number of feature decimals in the previous layer.

[0012] Optionally, the formal representation of the fixed-point number specifically includes: For the first convolutional layer and batch normalized layer after merging Layer Channel convolution, with weight parameters set as follows: The bias parameter is The output features of the previous layer are ; First, the merged weight parameters... By performing floating-point statistics and preserving the main weight distribution, we obtain its minimum floating-point value. and maximum value Determine the number of decimal places for the weights. : Given a total bit width of weight Under the condition of integer number of digits in the weights satisfy: in, The number of bits representing the sign of the weights; bias parameter The number of fixed-point digits is determined according to the alignment calculation principle; Let the upper layer input features The number of decimal places is Then the first Number of decimal places in layer bias parameters Defined as: ; This ensures that the bias parameters are aligned to the same decimal place scale as the convolution summation result, eliminating the need for additional scaling or rescaling operations during fixed-point computation. The number of integer bits in the offset is determined based on the total offset bit width and the number of decimal bits. The total bit width of the bias is greater than the weight, ensuring that the bias does not overflow within the fixed-point representation range.

[0013] Optionally, the joint optimization of fixed-point quantization-aware training and bit-width adaptive training for overall network performance specifically includes: Quantization-based perception training process based on fixed-point simulation: After completing the initial fixed-point bit construction based on feature statistics and alignment calculation, a quantization-aware training mechanism is introduced during the training phase. While keeping the overall network structure and inference process unchanged, a fixed-point simulation operator is explicitly introduced during the forward propagation process to simulate the fixed-point behavior of the floating-point calculation process, so that the model parameters gradually adapt to the numerical error brought about by fixed-point quantization. Bit-width joint optimization mechanism oriented towards overall network performance feedback: A feedback mechanism oriented towards overall network performance is introduced under the quantization-aware training framework to jointly optimize the fixed-point bit configuration of each layer in the model; During the backpropagation phase, the gradient of the bit offset parameter is approximated by combining the pass-through estimation method, and optimized based on the overall task loss during end-to-end training.

[0014] Optionally, the quantitative perception training process based on fixed-point simulation specifically includes: Parameter structure consistency processing: The parameters of the convolutional layer and its subsequent batch normalization layer are merged to obtain equivalent convolution weights and bias parameters consistent with the actual inference stage; Fixed-point simulation and quantization constraint introduction: For the merged weight parameters, bias parameters and intermediate features of the network, a simulated quantization operator is introduced during the forward propagation process to perform fixed-point conversion operations such as rounding, truncation and saturation constraint on floating-point values, and then the fixed-point numbers are restored to simulate the operation behavior under finite bit width fixed-point representation. Approximate gradient backpropagation mechanism: In the backpropagation stage, for the non-differentiable rounding and truncation operations in the quantization operator, the gradient is approximated by a pass-through estimation method; For quantization operators Its backward gradient is defined as: Within the representable range, the quantization operator approximates an identity mapping during backpropagation.

[0015] This invention provides a model adaptive fixed-point quantization method for overall performance optimization. The fixed-point quantization method includes: a symmetric fixed-point representation based on bit width; constructing an initial fixed-point bit width based on feature statistics and alignment calculation; and joint optimization of fixed-point quantization-aware training and bit-width adaptation for overall network performance. Through the quantization-aware training process, the model can gradually adapt to the precision perturbations caused by fixed-point computation during the training phase, making the model's forward computation behavior during training approximate the fixed-point execution form of the final embedded inference stage, effectively reducing the performance degradation caused by training-deployment inconsistency.

[0016] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 The flowchart illustrates a model adaptive fixed-point quantization method for overall performance optimization provided in this embodiment of the invention. Detailed Implementation

[0019] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0020] The terms "comprising" and "having," and any variations thereof, in the specification, embodiments, claims, and drawings of this invention are intended to cover non-exclusive inclusion, such as including a series of steps or units.

[0021] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0022] This invention proposes an adaptive fixed-point quantization method for overall performance optimization. Under the premise of satisfying the constraints of fixed-point arithmetic and alignment computation on general-purpose hardware platforms, this method takes the overall model performance index as an explicit optimization objective, introduces an overall performance feedback mechanism, and combines feature statistics, quantization-aware training, and bit-width joint optimization. It jointly and adaptively adjusts the number of fixed-point bits in each layer and channel of the neural network, thereby obtaining a fixed-point quantization model with optimal overall performance under given bit-width resource constraints.

[0023] This invention avoids the problem of overall model performance degradation caused by local bit-width joint adjustment in existing technologies by constructing a closed-loop optimization mechanism of "feature statistics - simulated quantization training - overall performance evaluation - bit width joint adjustment".

[0024] like Figure 1 As shown, an adaptive fixed-point quantization method for model optimization aimed at overall performance includes: Symmetric fixed-point representation based on bit depth; Construct an initial fixed-point number of bits based on feature statistics and alignment calculation; Joint optimization of fixed-point quantization-aware training and bit-width adaptive optimization for overall network performance.

[0025] The technical solution specifically includes: Symmetric fixed-point representation based on bit depth To adapt to resource-constrained embedded inference platforms and reduce the complexity of fixed-point computation, this invention employs symmetric fixed-point quantization to uniformly represent feature values, weights, and biases in the network. Under this quantization method, all values ​​are quantized with zero as the center, without introducing independent zero-point parameters, thereby simplifying hardware implementation logic and reducing additional computational overhead.

[0026] In terms of fixed-point representation, this invention employs a fixed-point coefficient representation with a quantization scale factor of 2 raised to the power of N, representing floating-point values ​​as the product of an integer and the scale factor. By limiting the quantization scale to a power of 2, floating-point multiplication and division operations can be transformed into shift operations, thereby significantly reducing computational complexity and power consumption on embedded platforms.

[0027] Building upon this, the present invention characterizes the precision and dynamic range of fixed-point numbers by introducing a fixed-point bit depth parameter. The fixed-point bit depth is used to represent the number of bits occupied by the fractional part of the fixed-point number. Different layers or channels can be configured with different fixed-point bit depths to adapt to their numerical distribution characteristics and precision requirements. Through a reasonable definition and constraint of the fixed-point bit depth, overflow risks can be avoided while ensuring numerical representation precision, and a unified quantization basis can be provided for subsequent bit-width joint optimization based on overall performance.

[0028] The formal representation is shown below.

[0029] Let the floating-point number be Its corresponding fixed-point representation is Fixed-point numbers use signed fixed-point representation, and their total bit width is denoted as . This includes the sign bit, integer bits, and decimal bits, defined as follows: in, Indicates the number of bits for the sign, when When both positive and negative values ​​are included When only non-negative or non-positive values ​​are included. ; Indicates the number of integer digits, used to limit the representable range of fixed-point numbers; Indicates the number of decimal places, used to limit the precision of fixed-point numbers.

[0030] Under the above definition, fixed-point numbers and floating-point numbers The following correspondence exists between them: Or equivalently represented as: The above round operation uses rounding or rounding to the nearest integer to ensure that the quantization error is minimized; Under this fixed-point bit division, the range of floating-point numbers that the fixed-point number can represent is: The minimum resolvable precision is: .

[0031] A method for constructing the initial fixed-point bit depth based on feature statistics and alignment calculation In the early stages of model training, using the already trained floating-point model and training data, statistical analysis is performed on the feature and parameter distribution of each layer and channel in the network to construct an initial fixed-point bit configuration that satisfies the alignment calculation constraints. This includes the following steps: Characteristic distribution statistics and initial number determination The trained floating-point model is used to perform forward inference on the training data to statistically analyze the floating-point distribution range of the output features of each layer and channel in the network, and obtain the corresponding minimum and maximum values. Based on the preset feature retention ratio, while covering most feature distributions, the corresponding fixed-point decimal places for each channel feature are adaptively determined to ensure that the fixed-point representation range can effectively cover the main feature distribution and reduce the impact of outliers on the bit width configuration.

[0032] The formal representation is shown below.

[0033] For any convolutional layer (or operator) the th The layer's output is used to perform forward inference on the training data using the trained floating-point model, resulting in the floating-point feature set corresponding to that channel. in Indicates the first The sample at the th The output feature values ​​of the layer To count the number of samples.

[0034] Based on the above floating-point characteristics, and by statistically analyzing its distribution range, we obtain: Considering the potential presence of outliers in the feature distribution, a preset feature retention ratio is introduced to reduce the impact of outliers on the fixed-point configuration. In the above floating-point feature set, a coverage ratio of is selected. The main characteristic intervals: The main feature intervals satisfy at least The eigenvalues ​​are located within this interval.

[0035] Let the first The feature fixed-point representation of the layer adopts a symmetric fixed-point format, and its fixed-point number is formally represented as: in These are floating-point eigenvalues; A fixed-point number in integer form; This represents the number of fixed-point decimal places corresponding to this channel.

[0036] Based on the aforementioned main feature intervals, the above is determined in the following manner: in and These are the number of fixed-point features, respectively. The following represents the minimum and maximum values; This indicates a floor operation, used to ensure that the fixed-point representation range can cover the main feature distribution and avoid numerical overflow during fixed-point calculation; Once the calculation is complete, the integer digits are determined accordingly based on the previous relationships.

[0037] Calculation of alignment bits for weights and biases For the weights after merging convolutional layers and batch normalized layers (CONV+BN) Based on the determined number of feature bits, the fixed-point bits of the weights and biases are calculated layer by layer according to the alignment calculation principle. The number of fixed-point bits for the weights is determined based on their floating-point distribution range and precision requirements; the number of fixed-point decimal places for the biases is jointly determined by the corresponding number of decimal places for the weights and the number of decimal places for the features in the previous layer, to ensure consistency in numerical representation and hardware feasibility during convolution operations and accumulation.

[0038] The formal expression is as follows.

[0039] For the first convolutional layer and batch normalization layer (CONV+BN) after merging convolutional layers, Layer Channel convolution, let its weight parameters be... The bias parameter is The output features of the previous layer are .

[0040] First, the merged weight parameters... By performing floating-point statistics and preserving the main weight distribution, we obtain its minimum floating-point value. and maximum value This determines the number of decimal places for the weights. : Given a total bit width of weight Under the condition of integer number of digits in the weights satisfy: in, The number of bits used to represent the sign of a weight is typically 1 for a weight.

[0041] To ensure consistency in fixed-point calculations during convolution and accumulation, the bias parameter in this invention... The fixed-point number of bits is determined according to the alignment calculation principle.

[0042] Let the upper layer input features The number of decimal places is Then the first Number of decimal places in layer bias parameters Defined as: By using the above method, the bias parameter and the convolution summation result are aligned to the same decimal place scale, so that no additional scaling or rescaling operations are needed during fixed-point computation.

[0043] The integer number of bits for the bias is determined based on the total bit width of the bias and the number of decimal places. Generally, the total bit width of the bias is greater than that of the weight to ensure that the bias does not overflow within the fixed-point representation range.

[0044] A joint optimization method for fixed-point quantization-aware training and bit-width adaptive training, aimed at improving overall network performance, includes: After completing the initial fixed-point bit depth construction based on feature statistics and alignment calculation, this invention further introduces a feedback mechanism for overall network performance during the quantization-aware training process to jointly optimize the fixed-point bit depth of weights, features and biases of each layer of the model, so as to avoid the local optimum problem caused by adjusting the bit width based only on local statistical information.

[0045] Quantization-based perception training process based on fixed-point simulation After completing the initial fixed-point bit construction based on feature statistics and alignment calculation, this invention further introduces a Quantization Aware Training (QAT) mechanism in the training phase. While keeping the overall network structure and inference process unchanged, fixed-point simulation operators are explicitly introduced during the forward propagation process to simulate the fixed-point behavior of the floating-point calculation process, so that the model parameters gradually adapt to the numerical errors brought about by fixed-point quantization.

[0046] The present invention mainly includes the following steps during the training process: Parameter structure consistency processing The parameters of the convolutional layer and its subsequent batch normalization layer (CONV+BN) are merged to obtain equivalent convolutional weights and bias parameters consistent with the actual inference stage, ensuring consistency in parameter form and computation path between the training and deployment stages.

[0047] The formal representation is as follows.

[0048] Let the training samples be The original floating-point model parameters are The network is mainly composed of It consists of n convolutional layers. For the nth The convolutional layers and their subsequent batch normalized layers have convolutional weights of 10 ... , bias is The BN parameter is the scaling factor. Offset mean ,variance The stable term is .

[0049] The equivalent convolution parameters after merging are expressed as: in , Equivalent weights and bias parameters used consistently during the training and inference phases.

[0050] Fixed-point simulation and introduction of quantization constraints For the merged weight parameters, bias parameters, and intermediate features of the network, a simulated quantization operator is introduced during the forward propagation process to perform fixed-point conversion operations such as rounding, truncation, and saturation constraints on floating-point values. Then, the fixed-point values ​​are restored to simulate the operation behavior under a finite-width fixed-point representation.

[0051] Let the first The floating-point characteristics of the layer input are The corresponding fixed-point bit configuration is as follows: ,in Indicates the number of sign bits (value is 1). Indicates the number of bits for the sign bit. Indicates the number of decimal places.

[0052] With any floating-point variable Taking (weights, biases, or features) as an example, its simulation fixed-point transformation process is defined as follows: Among them, quantization operator It consists of the following steps: Scaling and rounding: Cutoff and saturation constraints: Restore to floating-point representation: The above These are fixed-point analog values ​​represented in the floating-point field, used for forward computation.

[0053] No. The formal representation of the forward computation of the fixed-point simulation is as follows: Among them This represents a convolution operation, where all weights, biases, and features involved in the computation are constrained using simulated quantization operators; input... As the output feature of the previous layer, the output As input features for the next layer.

[0054] Approximate gradient backpropagation mechanism During the backpropagation phase, for non-differentiable operations such as rounding and truncation in the quantization operator, the gradient is approximated by methods such as Straight-Through Estimator (STE), so that the error information can be effectively backpropagated under fixed-point constraints, thereby continuously optimizing the model parameters.

[0055] For quantization operators Its backward gradient is defined as: That is, within the representable range, the quantization operator is approximately an identity mapping in backpropagation, so that the gradient of the loss function can be effectively transmitted to the floating-point parameters.

[0056] Through the above-mentioned quantitative perception training process, the model can gradually adapt to the accuracy perturbation caused by fixed-point computation during the training phase, so that the forward computation behavior of the model during the training process is close to the fixed-point execution form of the final embedded inference stage, effectively reducing the performance degradation caused by training-deployment inconsistency.

[0057] In this stage, the fixed-point bit configuration corresponding to the weights, features, and biases of each layer is not simply fixed, but participates in the subsequent optimization process as an adjustable parameter, laying the foundation for overall performance-driven joint bit width optimization. The specific value of the fixed-point bit will be uniformly optimized in the subsequent joint bit width optimization mechanism that introduces feedback on overall network performance.

[0058] Bit-width joint optimization mechanism for overall network performance feedback To address the problem that existing fixed-point quantization methods generally rely on local feature statistics or adjust bit width independently layer by layer, and are prone to getting trapped in local optima, this invention further introduces a feedback mechanism oriented towards overall network performance within the quantization-aware training framework, and jointly optimizes the fixed-point bit configuration of each layer in the model.

[0059] 1) Introduction and definition of bit offset parameter Specifically, this invention does not directly perform discrete search or manual adjustment of the fixed-point bit depth of each layer. Instead, based on the initial fixed-point bit depth determined by feature statistics and alignment calculations, it introduces a learnable bit offset parameter for each layer in the network to describe the direction and magnitude of the bit width adjustment of that layer relative to the initial configuration. The actual fixed-point bit depth of each layer is determined by its initial fixed-point bit depth and the corresponding bit offset, and is dynamically updated during training as the overall network performance changes.

[0060] The formal representation is as follows.

[0061] Suppose the network contains The quantizable layer, the first The number of decimal places obtained by the layer in the initial fixed-point bit construction phase is: The number of integer digits is Introduce the corresponding learnable bit offset parameter This describes the adjustment amount of the fixed-point bit depth of this layer relative to the initial configuration. Then the... The actual number of decimal places for a layer during the training phase is defined as follows: in, Parameters are learned continuously; This represents the floor function, used to ensure that the number of bits in the forward propagation stage is configured as discrete integers; Always satisfy the preset bit constraint range: The above constraint interval is based on The vertical fluctuation is determined by a certain factor. Correspondingly, the fixed-point number of weights, features, and biases can all be controlled using an offset mechanism, provided that alignment is satisfied.

[0062] 2) Forward fixed-point simulation calculation driven by bit offset During the forward propagation phase, the aforementioned bit offset parameters are mapped to discrete fixed-point bit adjustment amounts through rounding and other methods, ensuring that the model calculations always conform to the actual execution constraints of the fixed-point hardware. During the forward propagation phase, the first Any floating-point variable in the layer The formal representation of fixed-point simulation calculations (including merged weights, biases, or output features) is as follows: Therefore, the first The forward propagation process of a layer can be represented as: All quantization operators explicitly depend on the bit offset parameter, thus allowing the impact of different bit width configurations on the network output to be directly reflected in the overall forward computation results.

[0063] Let the overall task loss function of the network be... Its form can be classification loss, detection loss, or a combination of both. Therefore, in the quantitative perception training process, the optimization objective is uniformly represented as: in It represents a floating-point parameter model; This represents the set of bit offset parameters for each layer. The network output is influenced by both the model parameters and the bit width configuration. A bit width regularization term or complexity constraint term can be introduced into the loss function.

[0064] During the backpropagation phase, the gradient of the bit offset parameter is approximated by combining the pass-through estimation method, so that it can be optimized based on the overall task loss during end-to-end training.

[0065] By incorporating the bit offset parameter and model parameters into a unified optimization framework, this invention achieves the following joint optimization effects: The adjustment of the number of fixed points in each layer is directly driven by the overall task loss, rather than relying on local statistical information of a single layer or a single channel; The bit width configuration between different layers is adjusted collaboratively by sharing overall performance feedback, avoiding the local optimum problem caused by independent layer-by-layer configuration; Without introducing discrete search, genetic algorithms, or hardware-dependent structures, it effectively reduces the bit-width configuration space and improves training efficiency and engineering feasibility.

[0066] Through the aforementioned bit width joint optimization mechanism based on overall network performance feedback, this invention can achieve an overall optimal balance between the accuracy and computational efficiency of the fixed-point quantization model while satisfying the given bit width and hardware constraints.

[0067] Beneficial effects: A method for constructing the initial fixed-point bit depth based on feature statistics and alignment calculation By using training data to statistically analyze the floating-point distribution range of output features of each layer and channel of the network, the number of decimal places in the fixed-point representation is adaptively determined while covering the main feature distribution. The initial fixed-point bit configuration that meets the hardware implementation requirements is constructed by combining the alignment calculation principle.

[0068] Fixed-point simulation quantization perception training mechanism consistent with the inference phase During the training phase, parameters of the convolutional layer and the batch normalization layer are merged, and a fixed-point simulation operator is introduced during the forward propagation process to perform rounding, truncation and saturation constraints on weights, biases and intermediate features. At the same time, a pass-through estimation method is used to approximate the operator gradient during the backpropagation process, so that the training process approximates the actual fixed-point inference behavior.

[0069] Fixed-point bit width representation based on learnable bit offset parameters Based on the initial fixed bit width configuration, learnable bit offset parameters are introduced for each layer of the network to describe the adjustment direction and magnitude of the fixed bit width of each layer relative to the initial configuration, thereby avoiding direct discrete configuration of the absolute bit width or manual search.

[0070] Bit-width joint optimization mechanism for overall network performance feedback By incorporating bit offset parameters and model parameters into a unified quantization-aware training optimization framework, the adjustment of fixed-point bit width in each layer is driven by the overall network performance indicators, achieving collaborative optimization of bit width configuration between different layers and avoiding the local optimum problem caused by adjusting based solely on local statistical information.

[0071] The above specific embodiments further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A model adaptive fixed-point quantization method for overall performance optimization, characterized in that, The fixed-point quantization method includes: Symmetric fixed-point representation based on bit depth; Construct an initial fixed-point number of bits based on feature statistics and alignment calculation; Joint optimization of fixed-point quantization-aware training and bit-width adaptive training for overall network performance.

2. The adaptive fixed-point quantization method for model optimization oriented towards overall performance, as described in claim 1, is characterized in that... The bit-based symmetric fixed-point representation specifically includes: Symmetric fixed-point quantization is used to uniformly represent the feature values, weights, and biases in the network. In the quantization method, all values ​​are quantized with zero as the center; In terms of fixed-point representation, a fixed-point coefficient representation method with 2 raised to the power of N as the quantization scale factor is adopted, and floating-point values ​​are represented as the product of integers and scale factors. The quantization scale is limited to powers of 2, and floating-point multiplication and division operations are converted into shift operations. The fixed-point bit parameter is introduced to characterize the accuracy and dynamic range of fixed-point numbers.

3. The adaptive fixed-point quantization method for model optimization oriented towards overall performance, as described in claim 2, is characterized in that... The fixed-point bit depth is used to characterize the number of bits occupied by the fractional part in the fixed-point number. Different layers or channels can be configured with different fixed-point bit depths to adapt to numerical distribution characteristics and accuracy requirements. By reasonably defining and constraining the fixed-point bit depth, we can ensure the accuracy of numerical representation while avoiding the risk of overflow, and provide a unified quantitative expression basis for subsequent bit-width joint optimization based on overall performance.

4. The adaptive fixed-point quantization method for model optimization oriented towards overall performance, as described in claim 2, is characterized in that... The fixed-point representation specifically includes: Let the floating-point number be Its corresponding fixed-point representation is Fixed-point numbers use signed fixed-point representation, and their total bit width is denoted as . This includes the sign bit, integer bits, and decimal bits, which are defined as follows: in, Indicates the number of bits for the sign, when When both positive and negative values ​​are included When only non-negative or non-positive values ​​are included. ; Indicates the number of integer digits, used to limit the representable range of fixed-point numbers; Indicates the number of decimal places, used to limit the precision of fixed-point numbers; fixed point number and floating-point numbers The two satisfy a correspondence relationship: The equivalent representation is: The round operation uses either rounding to the nearest integer or rounding to the nearest whole number. Under the fixed-point partitioning, the range of floating-point numbers represented by fixed-point numbers is: ; The minimum resolvable precision is: .

5. The adaptive fixed-point quantization method for model optimization oriented towards overall performance as described in claim 1, characterized in that, The construction of the initial fixed-point bit depth based on feature statistics and alignment calculation specifically includes: In the early stages of model training, the already trained floating-point model and training data are used to perform statistical analysis on the features and parameter distribution of each layer and channel in the network, and to construct an initial fixed-point bit configuration that satisfies the alignment calculation constraints.

6. The adaptive fixed-point quantization method for model optimization oriented towards overall performance, as described in claim 1, is characterized in that... The specific steps for constructing the initial fixed-point bit configuration that satisfies the alignment calculation constraints include: Characteristic distribution statistics and initial number determination By using the trained floating-point model to perform forward inference on the training data, the floating-point distribution range of the output features of each layer and channel in the network is statistically analyzed, and the corresponding minimum and maximum values ​​are obtained. Based on the preset feature retention ratio, and on the premise of covering most feature distributions, the corresponding fixed-point decimal places are adaptively determined for each channel feature, so that the fixed-point representation range can effectively cover the main feature distributions. Formal representation: For any convolutional layer The layer's output is used to perform forward inference on the training data using the trained floating-point model, resulting in the set of floating-point features corresponding to each channel. in Indicates the first The sample at the th The output feature values ​​of the layer To count the number of samples; Based on the above floating-point characteristics and statistical distribution range, we obtain: Introduce a preset feature retention ratio Select a feature set with a coverage ratio of 1. Key characteristic intervals: ; The main feature intervals satisfy at least Each feature value lies within its corresponding feature interval; Let the first The feature fixed-point representation of the layer adopts a symmetric fixed-point format, and its fixed-point number is formally represented as: in, These are floating-point eigenvalues; A fixed-point number in integer form; The number of fixed-point decimal places corresponding to the channel; Based on the aforementioned main feature intervals, they are determined as follows: in, and These are the number of fixed-point features, respectively. The following represents the minimum and maximum values; This indicates the floor function; Once the calculation is complete, the number of digits in the integer part is also determined accordingly. Calculation of alignment bits for weights and biases: For the weight and bias parameters after merging convolutional and batch normalized layers, based on the determined number of feature bits, the fixed-point number of weights and biases is calculated layer by layer according to the alignment calculation principle. The fixed-point number of weights is determined based on their floating-point distribution range and precision requirements. The fixed-point number of biases is determined by the corresponding number of weight decimals and the number of feature decimals in the previous layer.

7. The adaptive fixed-point quantization method for model optimization oriented towards overall performance as described in claim 1, characterized in that, The formal representation of fixed-point numbers specifically includes: For the first convolutional layer and batch normalized layer after merging Layer Channel convolution, with weight parameters set as follows: The bias parameter is The output features of the previous layer are ; First, the merged weight parameters... By performing floating-point statistics and preserving the main weight distribution, we obtain its minimum floating-point value. and maximum value Determine the number of decimal places for the weights. : Given a total bit width of weight Under the condition of integer number of digits in the weights satisfy: in, The number of bits representing the sign of the weights; bias parameter The number of fixed-point digits is determined according to the alignment calculation principle; Let the upper layer input features The number of decimal places is Then the first Number of decimal places in layer bias parameters Defined as: ; This ensures that the bias parameters are aligned to the same decimal place scale as the convolution summation result, eliminating the need for additional scaling or rescaling operations during fixed-point computation. The number of integer bits in the offset is determined based on the total offset bit width and the number of decimal bits. The total bit width of the bias is greater than the weight, ensuring that the bias does not overflow within the fixed-point representation range.

8. The adaptive fixed-point quantization method for model optimization oriented towards overall performance as described in claim 1, characterized in that, The joint optimization of fixed-point quantization-aware training and bit-width adaptive training for overall network performance specifically includes: Quantization-based perception training process based on fixed-point simulation: After completing the initial fixed-point bit construction based on feature statistics and alignment calculation, a quantization-aware training mechanism is introduced during the training phase. While keeping the overall network structure and inference process unchanged, a fixed-point simulation operator is explicitly introduced during the forward propagation process to simulate the fixed-point behavior of the floating-point calculation process, so that the model parameters gradually adapt to the numerical error brought about by fixed-point quantization. Bit-width joint optimization mechanism oriented towards overall network performance feedback: A feedback mechanism oriented towards overall network performance is introduced under the quantization-aware training framework to jointly optimize the fixed-point bit configuration of each layer in the model; During the backpropagation phase, the gradient of the bit offset parameter is approximated by combining the pass-through estimation method, and optimized based on the overall task loss during end-to-end training.

9. The adaptive fixed-point quantization method for model optimization oriented towards overall performance, as described in claim 8, is characterized in that... The quantitative perception training process based on fixed-point simulation specifically includes: Parameter structure consistency processing: The parameters of the convolutional layer and its subsequent batch normalization layer are merged to obtain equivalent convolution weights and bias parameters consistent with the actual inference stage; Fixed-point simulation and quantization constraint introduction: For the merged weight parameters, bias parameters and intermediate features of the network, a simulated quantization operator is introduced during the forward propagation process to perform fixed-point conversion operations such as rounding, truncation and saturation constraint on floating-point values, and then the fixed-point numbers are restored to simulate the operation behavior under finite bit width fixed-point representation. Approximate gradient backpropagation mechanism: In the backpropagation stage, for the non-differentiable rounding and truncation operations in the quantization operator, the gradient is approximated by a pass-through estimation method; For quantization operators Its backward gradient is defined as: Within the representable range, the quantization operator approximates an identity mapping during backpropagation.

Images