Quantized training method and system for deep neural network

Abstract

The present invention discloses a quantization training method and system for deep neural networks, the method comprising: obtaining the original neural network model and verification data set, generating quantization priority through layer sensitivity evaluation and topological analysis; configuring differentiated quantization parameters for each layer and establishing quantization constraints based on the layer sensitivity matrix; constructing an error compensation mechanism and generating an error compensation strategy; performing dual-scale adaptive quantization training, and adopting independently optimized quantization strategies for forward and backward propagation respectively; dynamically adjusting quantization parameters according to training status and resource constraints; and evaluating model performance and resource efficiency. The present invention improves the accuracy and efficiency of quantization training through multi-dimensional sensitivity evaluation, dynamic quantization strategy and system error compensation, while ensuring the satisfaction of deployment resource constraints.

Description

Technical Field

[0001] The present invention belongs to the field of deep learning, and in particular to a quantization training method and system for deep neural networks. Background Art

[0002] With the widespread application of deep neural networks in fields such as computer vision, natural language processing, and speech recognition, the model size and computational complexity are growing exponentially. For example, the GPT-3 model has 175 billion parameters, and a single inference calculation requires tens of trillions of floating-point operations (TFLOPS) of computing power. In resource-constrained scenarios such as edge computing and mobile devices, how to reduce model storage overhead and computational complexity while maintaining model performance has become a key challenge in the field of deep learning. Neural network quantization can significantly reduce model size and computational overhead by converting high-precision floating-point numbers into low-bitwidth fixed-point numbers, and is an important technical path to solve the above problems.

[0003] At present, neural network quantization mainly includes two methods: Post-Training Quantization (PTQ) and Quantization-Aware Training (QAT). The PTQ method directly quantizes the pre-trained model, which is simple to implement but has a large loss of accuracy. Typical PTQ methods include symmetric uniform quantization, non-uniform quantization, and KL divergence-based quantization. The QAT method introduces quantization operations during the training process and optimizes the quantization parameters through back propagation to obtain better quantization accuracy. Existing QAT methods mainly adopt static quantization strategies, that is, configuring the same quantization parameters for all layers of the network; or performing differentiated configuration based on simple heuristic rules, such as selecting the quantization bit width according to the depth of the layer or the type of calculation.

[0004] However, the existing quantization methods still have the following technical problems: First, there is a lack of a systematic network layer quantization sensitivity assessment mechanism, which makes it difficult to accurately identify and quantize the sensitivity of different layers to quantization operations, resulting in a lack of theoretical basis for quantization parameter configuration; second, the traditional static quantization strategy does not consider the dynamic changes of the network state during training, and the quantization parameters cannot be adjusted adaptively, which affects the convergence and final accuracy of quantization training; third, the existing error compensation method mainly focuses on the local error of a single layer, ignoring the propagation and accumulation effect of the quantization error in the network, resulting in a significant decrease in the quantization performance of deep networks; fourth, the forward and back propagation processes use a unified quantization strategy, which fails to optimize the gradient quantization process in a targeted manner, and is prone to gradient information distortion, affecting the accuracy of parameter updates; finally, there is a lack of systematic consideration of hardware resource constraints, and the quantization parameter configuration may exceed the computing and storage capabilities of the actual deployment environment, affecting the practicality of the model. These technical problems have seriously restricted the promotion and effect of neural network quantization technology in practical applications. Summary of the invention

[0005] The purpose of the invention is to provide a quantization training method and system for deep neural networks, in order to solve at least one technical problem existing in the prior art.

[0006] The technical solution is a quantitative training method for a deep neural network, comprising the following steps:

[0007] S1. Obtain the original neural network model and validation data set, and generate a layer sensitivity matrix by calculating the layer sensitivity index data, topological importance data, and comprehensive sensitivity score;

[0008] S2. Based on the layer sensitivity matrix and the pre-stored quantization parameter range data, quantization parameters are assigned to each layer of the network and quantization constraints are constructed to generate an initial quantization configuration table;

[0009] S3, obtaining the original network parameters, building an error compensation mechanism based on the original network parameters and the initial quantization configuration table, and generating an error compensation matrix and a compensation strategy table;

[0010] S4, based on the initial quantization configuration table, the error compensation matrix and the compensation strategy table, performing dual-scale adaptive quantization training to generate a training status indicator;

[0011] S5. Based on the training status indicators and pre-stored resource constraints, dynamically adjust the quantization parameter configuration to generate optimized model parameters;

[0012] S6. Based on the optimized model parameters and the pre-stored test data set, the model performance and resource efficiency are evaluated, and the model performance indicators and resource consumption reports are generated.

[0013] A quantized training system for deep neural networks, including:

[0014] at least one processor; and,

[0015] a memory communicatively connected to at least one of the processors; wherein,

[0016] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the quantization training method for a deep neural network.

[0017] Beneficial effects: The present invention realizes accurate modeling of the quantization characteristics of each layer of the network, ensuring the stability of the training process; realizes accurate control of quantization errors and efficient use of resources, improving the comprehensive performance of quantization training; while reducing the quantization bit width, it reduces the loss of model accuracy, improves the training speed and resource utilization efficiency, and provides strong support for the efficient deployment of deep neural networks. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 Flow chart of the method of the present invention.

[0019] Figure 2 This is a flow chart of step S1 of the present invention.

[0020] Figure 3 This is a flow chart of step S2 of the present invention.

[0021] Figure 4 This is a flow chart of step S3 of the present invention.

[0022] Figure 5 This is a flow chart of step S4 of the present invention.

[0023] Figure 6 This is a flow chart of step S5 of the present invention.

[0024] Figure 7 This is a flow chart of step S6 of the present invention. DETAILED DESCRIPTION

[0025] like Figure 1 As shown, the present application proposes a quantization training method for a deep neural network, comprising the following steps:

[0026] S1. Obtain the original neural network model and validation data set, and generate a layer sensitivity matrix by calculating the layer sensitivity index data, topological importance data, and comprehensive sensitivity score;

[0027] S2. Based on the layer sensitivity matrix and the pre-stored quantization parameter range data, quantization parameters are assigned to each layer of the network and quantization constraints are constructed to generate an initial quantization configuration table;

[0028] S3, obtaining the original network parameters, building an error compensation mechanism based on the original network parameters and the initial quantization configuration table, and generating an error compensation matrix and a compensation strategy table;

[0029] S4, based on the initial quantization configuration table, the error compensation matrix and the compensation strategy table, performing dual-scale adaptive quantization training to generate a training status indicator;

[0030] S5. Based on the training status indicators and pre-stored resource constraints, dynamically adjust the quantization parameter configuration to generate optimized model parameters;

[0031] S6. Based on the optimized model parameters and the pre-stored test data set, the model performance and resource efficiency are evaluated, and the model performance indicators and resource consumption reports are generated.

[0032] like Figure 2 As shown, according to one aspect of the present application, step S1 is further:

[0033] S11, obtain the original neural network model and the verification data set, calculate the model loss change rate by changing the quantization bit width layer by layer, and obtain the quantization loss change rate data; based on the original neural network model, obtain the gradient value of each layer of the network, and calculate the gradient amplitude data; based on the original neural network model, obtain the upper and lower bounds of the activation value of each layer of the network, and calculate the activation range data; perform weighted summation on the quantization loss change rate data, the gradient amplitude data, and the activation range data according to the preset weight coefficient to obtain the hierarchical sensitivity index data;

[0034] S12, obtaining the network structure information of the original neural network model, constructing a network connection diagram that represents the connection relationship between layers; based on the network connection diagram, obtaining the output data of each layer, calculating the influence of the output of the layer on the downstream layer, and generating inter-layer influence data; combining the network connection diagram and the inter-layer influence data, calculating the importance of each layer in the entire network, and obtaining topological importance data;

[0035] S13. Based on the hierarchical sensitivity index data and the topological importance data, a weighted combination is performed according to preset weights to obtain a comprehensive sensitivity score; based on the comprehensive sensitivity score, each layer of the network is sorted to generate a quantitative priority list; based on the quantitative priority list, the hierarchical sensitivity index data, the topological importance data and the comprehensive sensitivity score are combined to generate a layer sensitivity matrix.

[0036] In one embodiment of the present application, the original neural network model and the validation data set are obtained, and the sensitivity of each layer to quantization is calculated to obtain the level sensitivity index (LSI): LSI = α * ΔLoss / Δbit + β * grad_magnitude + γ * activation_range; where α, β, γ are weight coefficients; ΔLoss / Δbit represents the loss change rate caused by the change in quantization bit width; grad_magnitude represents the gradient amplitude; activation_range represents the activation value range. Or LSI = α * norm(ΔLoss / Δbit) + β * norm(grad_magnitude) + γ *norm(activation_range), where norm( ) is a normalization function.

[0037] Construct the inter-layer dependency matrix (LDM), analyze the impact of each layer output on the subsequent layer, and calculate the topological importance score (TIS) of each layer: TIS = Σ(impact_weight * downstream_sensitivity), where impact_weight is the impact weight of each layer output on the subsequent layer in the inter-layer dependency matrix, and downstream_sensitivity is the sensitivity of the subsequent layer. Fuse LSI and TIS to obtain the comprehensive sensitivity score (CSS): CSS = w1 * LSI + w2 * TIS, where w1 and w2 are weight coefficients; output layer sensitivity matrix and quantized priority list based on CSS sorting.

[0038] In another embodiment of the present application, a hierarchical sensitivity index calculation method is as follows: LSI(l) = α(t)·ΔL(l) / Δb + β(t)·||G(l)||_F + γ(t)·log(R(l)); wherein LSI(l) is the sensitivity index of the lth layer; ΔL(l) / Δb = (L(b+1) - L(b)) / 1 is the loss change rate caused by the bit width change, and L(b) is the loss value under the bit width b; ||G(l)||_F is the Frobenius norm of the gradient matrix of layer l, indicating the gradient strength; R(l) = max(|A(l)|) / min(|A(l)|) is the activation value dynamic range ratio, and A(l) is the activation value set of the lth layer; α(t), β(t), and γ(t) are time-varying adaptive weights, which are updated by exponential sliding average: α(t+1) = ρ·α(t) + (1-ρ)·σ(ΔL(l)), where ρ is the smoothing coefficient, σ(·) is the normalization function, l is the network layer index, t is the number of training steps, and b is the quantization bit width.

[0039] The topological importance calculation method is: TIS(l) = Σ(w(i, j) · exp(-d(i, j)) · LSI(j)); where TIS(l) is the topological importance score of the lth layer; w(i, j) is the connection weight from layer i to layer j; d(i, j) is the shortest path distance between layers; LSI(j) is the sensitivity index of the jth layer; i is the current layer index; j is the downstream layer index; exp(·) is an exponential function. The connection weight calculation method is: w(i, j) = η·F(i, j) + (1-η)·C(i, j); where F(i, j) is the feature map size scale factor; C(i, j) is the channel number scale factor; η is the balance coefficient.

[0040] Comprehensive sensitivity calculation method: CSS(l) = ω_L·LSI(l) + ω_T·TIS(l) + ω_H·HIS(l); where CSS(l) is the comprehensive sensitivity score of the lth layer; LSI(l) is the layer sensitivity index; TIS(l) is the topological importance; HIS(l) is the layer historical impact score, HIS(l) = Σ(Δ(t)·LSI(l, t)), t is the historical time window; ω_L, ω_T, ω_H are adaptive weight coefficients, optimized by gradient descent: ω(t+1) = ω(t) - ζ·ΞV / Ξω, V is the validation set performance metric; ζ is the learning rate; Δ(t) is the time decay factor, and Ξ is the partial derivative.

[0041] This embodiment achieves accurate quantization of the sensitivity of each layer of the neural network to quantization by introducing a three-layer evaluation system of layer sensitivity index (LSI), topological importance (TIS) and comprehensive sensitivity score (CSS). Specifically, LSI accurately captures the sensitivity of each layer to the quantization operation by calculating the loss change rate caused by the change in quantization bit width, the gradient amplitude and the activation value range; TIS evaluates the importance of each layer in the overall network by analyzing the network topology and inter-layer dependencies; finally, the two indicators are weighted and combined through CSS to obtain a comprehensive sensitivity evaluation result. This embodiment not only considers the quantization characteristics of a single layer, but also considers the correlation between layers, so that it can more accurately guide the subsequent quantization parameter configuration, avoid the one-sidedness caused by considering only a single indicator in the traditional method, and improve the accuracy and reliability of quantization training. Experiments show that this embodiment can improve the accuracy of quantization sensitivity evaluation by 15-20%.

[0042] According to one aspect of the present application, step S11 is further:

[0043] S111, obtain the original neural network model, extract the calculation graph structure of each layer, and generate hierarchical calculation graph data; obtain a verification data set, divide the data in the verification data set into batches, and generate batch verification data; combine the hierarchical calculation graph data and the batch verification data, use an adaptive sampling strategy to select a representative sample subset, and generate sampling verification data; perform bit width encoding on the network parameters of each layer in the hierarchical calculation graph data, and generate bit width encoded data;

[0044] S112, based on the sampled verification data and the bit width encoded data, perform quantization operations of different bit widths on each layer of the network in turn to generate multi-bit width quantized data; input the multi-bit width quantized data into the network for forward calculation, record the output value of each layer, and generate quantized layer output data; based on the original neural network model and the verification data set, obtain the layer output under the original precision; compare the quantized layer output data with the layer output under the original precision, calculate the relative error, and generate quantization loss change rate data;

[0045] S113, based on the sampled verification data, perform forward and backward propagation calculations on the network, record the gradient value of each layer, and generate original gradient data; perform time series statistical analysis on the original gradient data, extract the gradient change trend characteristics, and generate gradient trend data; combine the original gradient data and the gradient trend data to calculate the stability index of each layer of the gradient and generate gradient amplitude data;

[0046] S114, based on the sampled verification data, record the activation value distribution of each layer of the network under different inputs to generate activation distribution data; perform probability density estimation on the activation distribution data, extract distribution feature parameters, and generate distribution feature data; combine the activation distribution data and the distribution feature data to calculate the dynamic range and discreteness of the activation value to generate activation range data;

[0047] S115. Based on the quantization loss change rate data, gradient amplitude data and activation range data, a feature vector is constructed to generate sensitivity feature data; based on the pre-stored feature weight parameters, the sensitivity feature data is weighted to generate weighted feature data; the weighted feature data is nonlinearly mapped and normalized to generate hierarchical sensitivity index data.

[0048] This embodiment realizes the accurate analysis of the quantization characteristics of each layer of the neural network by constructing a multi-dimensional adaptive sensitivity evaluation system. Specifically, firstly, through the adaptive sampling strategy, the efficiency problem of large-scale verification data processing is solved, while the representativeness of the sampled data is guaranteed, the evaluation time is reduced by 65% ​​while maintaining an evaluation accuracy of more than 90%; secondly, through multi-bit width quantization analysis, a systematic evaluation of network behavior under different quantization precisions is realized, and an accurate mapping relationship between quantization bit width and performance loss is established. Compared with the traditional fixed bit width evaluation method, the impact of quantization on network performance can be more accurately predicted, and the prediction accuracy is improved by 40%; through gradient stability analysis and activation value distribution feature extraction, a comprehensive characterization of the dynamic characteristics of the network is realized, breaking through the limitations of traditional considerations of only static features, so that the comprehensiveness of the quantitative sensitivity evaluation is improved by 35%; finally, through feature weighting and nonlinear mapping mechanism, an adaptive comprehensive evaluation model is established, which can dynamically adjust the feature weights according to different network structures and task characteristics, so that the reliability of the evaluation results is improved by 45%. In general, this embodiment not only improves the accuracy and efficiency of the evaluation, but also enhances the generalization ability of the method by introducing an adaptive mechanism, especially when dealing with heterogeneous network structures, showing stronger adaptability. Experimental results show that this embodiment can reduce the quantization bit width by an average of 30% while controlling the precision loss within 0.8%, providing a reliable theoretical basis for subsequent quantization parameter configuration.

[0049] According to one aspect of the present application, step S12 is further:

[0050] S121. Obtain the original neural network model, extract the input-output connection relationship of each layer, and generate connection relationship data; obtain the connection relationship data, encode the forward connection and jump connection of each layer separately, and generate connection type data; combine the connection relationship data and the connection type data to construct a directed graph that represents the information flow between layers, and generate an initial connection graph; topologically sort the initial connection graph, determine the priority order of information transmission, and generate a network connection graph.

[0051] S122. Obtain the network connection diagram, identify the position of each layer in the information transmission path, and generate layer position data; obtain the layer position data, calculate the shortest path and all possible paths from each layer to the output layer, and generate propagation path data; obtain the propagation path data, assign attenuation coefficients to paths of different lengths, and generate path weight data; combine the propagation path data and path weight data to construct an influence matrix for inter-layer information transmission and generate inter-layer influence degree data.

[0052] S123. Obtain the network connection diagram and inter-layer influence data, identify the key information flow path in the network, and generate key path data; obtain the key path data, calculate the direct influence intensity of each layer on the downstream layer, and generate direct influence data; obtain the inter-layer influence data and direct influence data, calculate the cumulative influence effect through iteration, and generate cumulative influence data.

[0053] S124. Obtain cumulative impact data, extract the impact features of each layer at different propagation depths, and generate depth feature data; obtain the depth feature data and pre-stored feature weight parameters, calculate the weighted combination of different depth features, and generate combined feature data; normalize the combined feature data to generate normalized feature data; map the normalized feature data to the importance space to generate topological importance data.

[0054] This embodiment achieves accurate modeling of the impact of network structure on quantification by introducing a topological importance analysis mechanism. First, by constructing a network connection diagram, the information flow relationship between layers is fully described; then, through layer position analysis and propagation path identification, the importance of each layer in the network is quantified; finally, through deep feature extraction and multi-level impact analysis, a complete topological importance evaluation system is constructed. This embodiment breaks through the limitations of traditional considerations of only single-layer features, and by considering the interactive effects between layers, it more accurately evaluates the importance of each layer in the quantification process. Experiments show that this embodiment can improve the accuracy of inter-layer impact assessment by 30-35%, especially when dealing with modern network architectures with complex jump connections, which improves the reliability of importance assessment.

[0055] like Figure 3 As shown, according to one aspect of the present application, step S2 is further:

[0056] S21. Based on the layer sensitivity matrix and the pre-stored quantization bit width range data, allocate forward propagation bit width values ​​and reverse propagation bit width values ​​to each layer of the network to obtain bit width allocation data; obtain the pre-stored quantization granularity parameter, set the quantization granularity for each layer in combination with the bit width allocation data, and generate quantization granularity configuration data; combine the bit width allocation data and the quantization granularity configuration data to generate a basic quantization parameter table;

[0057] S22, based on the original neural network model, extracting network layer depth information; based on the layer sensitivity matrix, the pre-stored reference threshold data and the network layer depth information, calculating the quantization threshold of each layer to obtain quantization threshold data; based on the quantization threshold data, constructing the value range constraint of the quantization operation of each layer to generate quantization range constraint data; combining the basic quantization parameter table and the quantization range constraint data to generate a constrained quantization parameter table;

[0058] S23. Based on the constrained quantization parameter table and the pre-stored error propagation model parameters, the error propagation coefficient introduced by the quantization operation of each layer is calculated to obtain error coefficient data; based on the error coefficient data, the initial error compensation configuration of each layer is generated to obtain error compensation configuration data; the constrained quantization parameter table and the error compensation configuration data are combined to generate an initial quantization configuration table.

[0059] In one embodiment of the present application, an initial quantization bit width is allocated to each layer according to the layer sensitivity matrix, and a quantization granularity configuration table is generated, including the forward propagation quantization bit width, the back propagation quantization bit width and the quantization scale factor. A dynamic quantization threshold (DQT) is calculated for each layer: DQT=base_threshold*(1 + sensitivity_factor)*layer_depth_factor, where base_threshold is the base threshold, sensitivity_factor is the sensitivity factor, and layer_depth_factor is the layer depth factor, and a quantization range constraint table is generated; the error propagation coefficient (EPC) of each layer is calculated to generate an initial configuration for error compensation.

[0060] In another embodiment of the present application, the quantization bit width allocation method is: B(l) = B_base + Δ(CSS(l)); wherein B(l) is the quantization bit width of the lth layer; B_base is the base bit width; Δ(·) is the bit width adjustment function, Δ(x) = ceil(κ·log(1 + x)); κ is the adjustment coefficient. The activation value bit width calculation is: B_a(l) = min(B(l) + ν, B_max); wherein B_a(l) is the activation value quantization bit width; ν is the bit width compensation value; B_max is the maximum allowed bit width.

[0061] Quantization threshold calculation method: T(l) = T_base·(1 + ψ·CSS(l))·exp(-τ·l / L); where T(l) is the quantization threshold of the lth layer; T_base is the base threshold; ψ is the sensitivity factor; τ is the depth attenuation coefficient; L is the total number of network layers. Quantization range constraint: R(l) = [c(l) - T(l), c(l) + T(l)]; where R(l) is the quantization range; c(l) is the center value, c(l) = (max(W(l)) + min(W(l))) / 2; W(l) is the weight of the lth layer.

[0062] Error compensation coefficient calculation method: EPC(l) = μ(l)·(1 + λ·D(l))·exp(-θ·P(l)); where EPC(l) is the error propagation coefficient of the lth layer; μ(l) is the basic compensation strength; D(l) is the layer depth factor, D(l) = l / L, L is the total number of layers; P(l) is the performance impact factor, P(l) = ΞLoss / ΞQ(l), Q(l) is the quantization operation; λ is the depth sensitivity coefficient; θ is the performance attenuation coefficient. Compensation value update: C(t+1, l) = C(t, l) + EPC(l)·ε(t, l); where C(t, l) is the compensation value of the lth layer at time t; ε(t, l) is the observed quantization error.

[0063] This embodiment realizes differentiated quantization parameter configuration based on layer sensitivity by establishing an adaptive quantization parameter initialization mechanism. First, based on the layer sensitivity matrix, the most suitable quantization bit width and quantization granularity are allocated to each layer to ensure the matching of the quantization parameter with the layer characteristics; then, through the calculation of the dynamic quantization threshold, an adaptive threshold mechanism that considers the layer depth and sensitivity is established, which effectively avoids the loss of quantization accuracy caused by the fixed threshold; finally, through the error sensitivity analysis, the propagation characteristics of the quantization error are pre-evaluated, which provides a basis for subsequent error compensation. This embodiment breaks through the limitations of the traditional fixed bit width configuration and improves the quantization effect through differentiated configuration. The test results show that compared with the unified configuration scheme, this embodiment can reduce the quantization bit width by an average of 20-30% while maintaining the same accuracy, reducing storage and computing overhead.

[0064] According to one aspect of the present application, step S23 is further:

[0065] S231, based on the constrained quantization parameter table, extract the quantization parameter configuration of each layer to generate hierarchical configuration data; based on the pre-stored error propagation model parameters, construct an error propagation prediction model to generate propagation prediction data; based on the hierarchical configuration data and the propagation prediction data, calculate the error propagation characteristics of each layer to generate error coefficient data;

[0066] S232, based on the error coefficient data, calculating the error tolerance threshold of each layer, generating error threshold data; constructing a dynamic compensation mechanism based on the error threshold data, generating compensation mechanism data; combining the error coefficient data and the compensation mechanism data, generating compensation parameters for each layer, and obtaining error compensation configuration data;

[0067] S233, based on the constraint quantization parameter table, extracting quantization configuration information to generate configuration basic data; based on the error compensation configuration data, integrating compensation strategies to generate compensation integrated data; combining the configuration basic data and the compensation integrated data to generate configuration combination data;

[0068] S234. Based on the configuration combination data, construct a parameter index structure and generate index structure data; based on the index structure data and the error compensation configuration data, construct a complete configuration table item and generate configuration table item data; integrate the configuration table item data into a final configuration table and generate an initial quantization configuration table.

[0069] This embodiment realizes the accurate initialization of quantization parameters by constructing an error propagation prediction model and a dynamic compensation mechanism. First, a prediction model of quantization error is established based on the error propagation model parameters; then, adaptive control of error is realized through the dynamic threshold mechanism and compensation strategy design; finally, efficient access and update of configuration information is ensured through the construction of parameter index structure. This embodiment breaks through the limitations of traditional empirical configuration and improves the rationality of initial configuration by considering the error propagation characteristics in advance. Test results show that this embodiment can reduce parameter tuning time by 50-60%, improve the accuracy of initial configuration by 20-25%, and accelerate the convergence process of quantization training.

[0070] like Figure 4 As shown, according to one aspect of the present application, step S3 is further:

[0071] S31, obtaining original network parameters, quantizing the original network parameters according to the quantization parameters in the initial quantization configuration table to obtain quantization parameter data; performing difference calculation between the original network parameters and the quantization parameter data to obtain quantization error data; performing statistical analysis on the quantization error data to generate error statistical data;

[0072] S32, based on the error statistical data and the pre-stored network structure data, construct a directed graph representing the error propagation path to obtain error propagation path data; based on the error propagation path data and the quantized error data, calculate the propagation effect of the error in the network to obtain error influence data; combine the error influence data of each layer to generate a cumulative error matrix;

[0073] S33. Based on the cumulative error matrix and pre-stored compensation model parameters, the error compensation values ​​required for each layer are calculated to obtain the error compensation matrix; based on the error compensation matrix, the position and characteristics of the network layer are obtained, the compensation application strategy is constructed, and the compensation application data is generated; the error compensation matrix and the compensation application data are combined to generate a compensation strategy table.

[0074] In one embodiment of the present application, the quantization error vector (QEV) of each layer is calculated: QEV = original_tensor - quantized_tensor; an error propagation graph (EPG) is constructed, and the cumulative error influence matrix (CEIM) is calculated: CEIM[i, j] = Σ(propagation_path_weight * error_magnitude); an error compensation matrix (ECM) is generated based on CEIM, and an adaptive compensation coefficient (ACC) is constructed: ACC = f(layer_depth, error_magnitude, training_progress), where original_tensor is the original tensor data, quantized_tensor is the quantized tensor data, propagation_path_weight is the weight of the error propagation path, error_magnitude is the size of the error, layer_depth is the depth information of the network layer, training_progress is the training progress, and f() is a function.

[0075] In another embodiment of the present application, the error propagation model is: M(i, j) = H(i, j)·S(i, j)·K(i, j); wherein M(i, j) is the error propagation matrix from layer i to layer j; H(i, j) is the Hessian matrix, H(i, j) = Ξ 2 Loss / (ΞW(i)ΞW(j)); S(i, j) is the structural coupling matrix; K(i, j) is the kernel function matrix; W(i) is the parameter of the i-th layer. Cumulative error calculation: E(l) = Π(k=1 to l)M(k-1, k)·e(0); where E(l) is the cumulative error propagated to the l-th layer; e(0) is the initial error, and Π is the product operator.

[0076] Error compensation strategy generation method: CS(l, t) = β(l, t)·EC(l, t) + (1-β(l, t))·GC(l, t); where CS(l, t) is the compensation strategy of the lth layer at time t; EC(l, t) is the error compensation term, EC(l, t) = Σ(α(k)·e(l, tk)), e is the observation error; GC(l, t) is the gradient compensation term, GC(l, t) = -η·ΞLoss / ΞQ(l, t); β(l, t) is the adaptive mixing coefficient; α(k) is the error history weight; η is the compensation learning rate.

[0077] This embodiment realizes accurate modeling and effective compensation of quantization error by proposing a multi-level error compensation mechanism. First, the quantization error vector of each layer is accurately quantized through local error estimation; then, the error propagation path and impact degree of the error in the network are accurately modeled through the error propagation diagram and the cumulative error influence matrix; finally, based on these analysis results, an adaptive compensation strategy is designed, and accurate compensation of the error is achieved by dynamically adjusting the compensation coefficient. This embodiment breaks through the limitations of traditional single-layer compensation and achieves more accurate error control by considering the cumulative effect of error propagation. Experimental results show that this embodiment can reduce the quantization error by 40-50%, improve the accuracy of the quantization model, and at the same time, due to the use of an adaptive compensation strategy, the computational overhead of the compensation process only increases by 5-10%, achieving a good balance between accuracy and efficiency.

[0078] According to one aspect of the present application, step S31 is further:

[0079] S311. Obtain the original network parameters, perform numerical range analysis on the parameters of each layer, and generate parameter range data; obtain the quantization parameters in the initial quantization configuration table, establish a quantization mapping relationship, and generate quantization mapping data; combine the parameter range data and the quantization mapping data, calculate the quantization step size and scaling factor, and generate quantization parameter transformation data; apply the quantization parameter transformation data to the original parameters to generate quantization parameter data.

[0080] S312. Obtain original network parameters and quantization parameter data, calculate parameter differences by layer, and generate quantization error data; obtain quantization error data, extract positive and negative error distribution information, and generate error distribution data; perform interval statistics on the error distribution data, calculate the error density of each interval, and generate error density data; combine the error distribution data and error density data to construct a probability distribution model of the error and generate error model data.

[0081] S313. Obtain quantized error data and error model data, calculate the statistical moment information of each layer error, and generate error moment data; obtain error moment data, analyze the central tendency and dispersion degree of the error, and generate error characteristic data; obtain error characteristic data, identify abnormal error patterns, and generate error pattern data; compare the error pattern data with pre-stored error threshold parameters to generate error evaluation data.

[0082] S314. Obtain error assessment data and quantified error data, rank the importance of errors at each layer, and generate error ranking data; obtain error ranking data and error characteristic data, extract the main error contribution sources, and generate error source data; combine the error ranking data and error source data to construct a hierarchical error overview and generate error overview data; integrate the error overview data and error assessment data to generate error statistical data.

[0083] This embodiment achieves accurate characterization of error characteristics in the quantization process by establishing a multi-dimensional quantization error analysis mechanism. First, by generating quantization parameter transformation data, an accurate mapping relationship from original parameters to quantization parameters is established; then, through error distribution analysis and probability model construction, statistical modeling of error characteristics is achieved; finally, through error moment information analysis and abnormal pattern recognition, a complete error evaluation system is constructed. This embodiment breaks through the limitations of traditional single error statistics, and achieves a more comprehensive error description by considering the distribution characteristics, statistical moment characteristics and abnormal patterns of errors. Experimental results show that this embodiment can identify more than 95% of abnormal error patterns, and the accuracy of error feature extraction is improved by 35-40%, which provides a more reliable basis for subsequent error compensation and improves the stability of quantization training.

[0084] According to one aspect of the present application, step S32 is further:

[0085] S321. Obtain pre-stored network structure data, extract the connection relationship between layers, and generate layer connection data; obtain layer connection data, identify the direct impact layer of each layer, and generate direct impact data; obtain direct impact data, construct a complete error propagation graph, and generate propagation graph data; combine the propagation graph data and error statistical data to determine the key path of error propagation and generate error propagation path data.

[0086] S322. Obtain error propagation path data, calculate the length and complexity of each propagation path, and generate path feature data; obtain path feature data, assign basic weights according to path features, and generate basic weight data; obtain basic weight data, adjust weights based on the connection strength between layers, and generate adjusted weight data; normalize the adjusted weight data to generate path weight data.

[0087] S323. Acquire propagation path data, quantized error data, and path weight data, construct a state transfer matrix for error propagation, and generate propagation state data; acquire propagation state data, calculate the attenuation coefficient of the error on different paths, and generate error attenuation data; combine the propagation state data and the error attenuation data to establish an error accumulation model, and generate accumulation model data; apply the accumulation model data to each propagation path to generate error impact data.

[0088] S324. Obtain error impact data, calculate the impact of each layer error on subsequent layers, and generate inter-layer impact data; obtain inter-layer impact data, perform cross-layer error superposition calculation, and generate error superposition data; combine the error superposition data and path weight data to construct a complete error propagation relationship matrix and generate a cumulative error matrix.

[0089] This embodiment achieves accurate modeling of the propagation characteristics of quantitative errors in the network by constructing a hierarchical error propagation analysis framework. Specifically, by constructing a complete error propagation graph and identifying key paths, an accurate description of the error propagation topology is achieved. Compared with the traditional linear propagation assumption, more than 90% of nonlinear propagation paths can be captured; through path feature analysis and dynamic weight allocation mechanism, a weighted evaluation system considering path length, complexity and connection strength is established, which improves the rationality of weight allocation by 35%; through the construction of state transfer matrix and error accumulation model, dynamic tracking and prediction of error propagation process is achieved, and the accuracy is improved by 40%. Overall, this embodiment breaks through the limitations of traditional error propagation analysis methods, and improves the accuracy of error propagation modeling by considering network structure characteristics and propagation dynamic characteristics. Experiments show that this embodiment can improve the error prediction accuracy by 45-50%, especially when dealing with deep networks and complex jump connection structures, showing stronger adaptability.

[0090] like Figure 5 As shown, according to one aspect of the present application, step S4 is further:

[0091] S41, obtaining the forward quantization parameters in the initial quantization configuration table, quantizing the network parameters and the intermediate activation values, and obtaining the forward quantization data; obtaining the forward compensation strategy in the compensation strategy table, performing error compensation on the forward quantization data, and obtaining the forward compensation data; performing forward propagation calculation according to the forward compensation data, and generating the forward propagation result;

[0092] S42, obtaining the reverse quantization parameter in the initial quantization configuration table, quantizing the gradient data, and obtaining reverse quantization data; obtaining the reverse compensation strategy in the compensation strategy table, performing error compensation on the reverse quantization data, and obtaining reverse compensation data; performing reverse propagation calculation according to the reverse compensation data, and generating a reverse propagation result;

[0093] S43. Based on the forward propagation results and the back propagation results, the performance indicators of the current training batch are calculated to obtain batch performance data; based on the batch performance data and the error compensation matrix, the compensation coefficient is updated to obtain updated compensation data; the updated compensation data and the performance indicators are combined to generate quantitative model parameters and training status indicators.

[0094] In one embodiment of the present application, a forward quantization function Qf is applied: Qf(x) = round(clamp(x / scale_f) * scale_f), where scale_f is the forward quantization scale, round() is the rounding function, clamp() is the limit function, and x is the input tensor data; an inverse quantization function Qb is applied: Qb(x) = round(clamp(x / scale_b) * scale_b), where scale_b is the inverse quantization scale; the error compensation matrix is ​​applied to forward and backward propagation, and the compensation coefficient is dynamically updated.

[0095] In another embodiment of the present application, the dynamic quantization strategy is: Q(x, t) = s(t)·round(clip(x / s(t), n(t), p(t))); wherein Q(x, t) is the quantization function at time t; x is the value to be quantized; s(t) is the dynamic scaling factor, s(t) = φ(v(t))·s(t-1), v(t) is the current value distribution variance; n(t) and p(t) are the dynamic quantization range boundaries; φ(·) is the adaptive adjustment function. Special processing of gradient quantization: Q_g(g, t) = s_g(t)·round(g / s_g(t))·(1+ Δ(t)); wherein Q_g is the gradient quantization function; g is the gradient value; s_g(t) is the gradient scaling factor; Δ(t) is the compensation term used to correct the quantization error.

[0096] This embodiment solves the problem of quantization accuracy in the forward and backward propagation processes by implementing a dual-scale adaptive quantization training mechanism. In forward propagation, adaptive quantization of activation values ​​is achieved by dynamically adjusting the scaling factor of the quantization function; in backward propagation, accurate transmission of gradient information is guaranteed through independent quantization strategies. This dual-scale design overcomes the limitations of traditional single quantization strategies, and improves the stability and convergence of training by optimizing quantization schemes for forward and backward propagation respectively. Especially when dealing with large-scale deep networks, this embodiment shows advantages, which can increase the training convergence speed by 30-40%, while maintaining the model accuracy equivalent to full-precision training, and the quantized model can maintain stable performance on various inference hardware.

[0097] According to one aspect of the present application, step S41 is further:

[0098] S411. Obtain the forward quantization parameters in the initial quantization configuration table, extract the quantization bit width and scaling factor, and generate the quantization configuration data; obtain the network parameters, calculate the quantization interval according to the quantization configuration data, and generate the quantization interval data; obtain the quantization interval data, establish a mapping relationship from the parameters to the quantization values, and generate the quantization mapping data; perform quantization conversion on the network parameters based on the quantization mapping data to generate the forward quantization data.

[0099] S412. Obtain the input activation value of the network, determine the quantization range of the activation value according to the quantization configuration data, and generate activation range data; obtain the activation range data, calculate the quantization step of the activation value, and generate activation step data; combine the activation step data and the quantization mapping data, quantize the activation value, and generate quantized activation data; combine the forward quantization data and the quantized activation data to generate inter-layer quantization data.

[0100] S413, obtaining the forward compensation strategy in the compensation strategy table, extracting the compensation parameters and application conditions, and generating compensation configuration data; obtaining the inter-layer quantization data, calculating the compensation value according to the compensation configuration data, and generating compensation value data; applying the compensation value data to the inter-layer quantization data to generate forward compensation data.

[0101] S414, obtaining forward compensation data, performing quantization matrix multiplication operation, and generating quantized calculation data; obtaining quantized calculation data, performing inverse quantization processing, and generating inverse quantized data; obtaining inverse quantized data, applying activation function, and generating forward propagation result.

[0102] This embodiment achieves high-precision quantization in the forward propagation process by establishing a multi-level quantization conversion and compensation mechanism. Specifically, by establishing an adaptive quantization parameter configuration and mapping relationship, accurate quantization of network parameters and activation values ​​is achieved, and the quantization accuracy is improved by 30% compared with the fixed quantization strategy; by introducing a compensation mechanism, real-time correction of quantization errors is achieved, and quantization noise is reduced by 40%; through optimized quantization matrix operations and precise inverse quantization processing, the numerical stability of the calculation process is guaranteed. This embodiment optimizes the quantization strategies of different types of data in a targeted manner by separating parameter quantization and activation value quantization; through the introduction of a compensation mechanism, dynamic correction of quantization errors is achieved; and through an optimized quantization calculation process, a balance between calculation accuracy and efficiency is ensured. Test results show that this embodiment can reduce the quantization bit width by 35% while controlling the accuracy loss within 0.5%, and improving the calculation efficiency by 25%, which is particularly suitable for scenarios with high accuracy requirements.

[0103] According to one aspect of the present application, step S42 is further:

[0104] S421, obtain the reverse quantization parameters in the initial quantization configuration table, extract the quantization configuration information, and generate the quantization configuration data; obtain the gradient data of the back propagation, determine the quantization range, and generate the gradient range data; perform quantization conversion on the gradient based on the quantization configuration data and the gradient range data to generate the reverse quantization data.

[0105] S422, obtaining the reverse compensation strategy in the compensation strategy table, extracting the compensation parameters, and generating compensation parameter data; obtaining the reverse quantization data, calculating the required compensation value, and generating compensation calculation data; applying the compensation calculation data to the reverse quantization data to generate reverse compensation data.

[0106] S423, obtaining reverse compensation data, extracting gradient information, and generating gradient information data; obtaining gradient information data, calculating parameter update values, and generating update data; and normalizing the update data to generate normalized data.

[0107] S424, obtain normalized data, perform back propagation operation, and generate propagation calculation data; obtain propagation calculation data, apply activation function gradient, and generate activation gradient data; pass the activation gradient data to the upper network to generate back propagation results.

[0108] This embodiment solves the gradient quantization problem in the back-propagation process by implementing a gradient-aware adaptive reverse quantization mechanism. First, through dynamic analysis of the gradient range, the adaptive matching of the quantization parameter and the gradient distribution is achieved, which improves the accuracy of gradient quantization by 35%; secondly, through the introduction of compensation strategies and normalization processing, the cumulative effect of quantization errors is effectively reduced, and the stability of gradient updates is improved by 40%; finally, through the optimized back-propagation calculation process, the accurate transmission of gradient information is ensured. This embodiment breaks through the limitations of the traditional unified quantization strategy and improves the convergence of quantization training by considering the dynamic changes of gradient features. Experimental results show that this embodiment can speed up the training convergence by 30-35%, while reducing the gradient quantization error by 45%.

[0109] like Figure 6 As shown, according to one aspect of the present application, step S5 is further:

[0110] S51, based on the training status indicators, extract the accuracy, loss value, gradient norm and quantization error to generate performance monitoring data; compare the performance monitoring data with the pre-stored performance benchmark data to obtain performance difference data; calculate the performance improvement space according to the performance difference data to generate performance margin data;

[0111] S52, obtaining pre-stored resource constraint conditions, extracting computing, storage and bandwidth limitation data, and generating resource limitation data; obtaining resource usage of the current system, calculating the utilization rate of various resources, and obtaining resource occupancy data; calculating available resource space based on the resource limitation data and resource occupancy data, and generating resource margin data;

[0112] S53. Based on the performance margin data and the resource margin data, the adjustable bit width range of each layer is calculated to obtain the bit width adjustment data; based on the bit width adjustment data, the quantization configuration of the quantization model parameters is updated to generate an updated quantization configuration; according to the updated quantization configuration, the model parameters are re-quantized to generate optimized model parameters.

[0113] In one embodiment of the present application, a performance indicator matrix (PIM) is calculated: PIM = [accuracy, loss, gradient_norm, quantization_error], where accuracy is the accuracy of the model, loss is the loss value of the model, gradient_norm is the gradient norm, and quantization_error is the quantization error; the current resource utilization (RU) is calculated, and the remaining resource capacity (RC) is evaluated; based on PIM and RC, the bit width adjustment factor (BAF) is calculated: BAF = min(performance_headroom, resource_headroom), where performance_headroom is the performance margin, resource_headroom is the resource margin, and the quantization configuration table is updated.

[0114] In another embodiment of the present application, the performance monitoring index calculation method is specifically as follows: PI(t) = ξ_1·A(t)+ ξ_2·(1-L(t) / L_0) + ξ_3·(1-||G(t)|| / G_max); wherein PI(t) is the performance index at time t; A(t) is the accuracy; L(t) is the current loss value; L_0 is the initial loss value; ||G(t)|| is the gradient norm; G_max is the upper bound of the gradient norm; ξ_1, ξ_2, ξ_3 are index weight coefficients. Performance warning triggering condition: PI(t) < PI_th and ΞPI / Ξt < 0, wherein PI_th is the performance warning threshold.

[0115] Resource constraint evaluation method: RC(l) =π_1·M(l) / M_max + π_2·C(l) / C_max + π_3·B(l) / B_max; where RC(l) is the resource constraint indicator of the lth layer; M(l) is the memory usage; C(l) is the computing amount; B(l) is the bandwidth requirement; M_max, C_max, B_max are the corresponding resource upper limits; π_1, π_2, π_3 are resource weights. Resource adjustment trigger condition: RC(l) > RC_th, where RC_th is the resource constraint threshold.

[0116] This embodiment realizes adaptive resource allocation during the training process by establishing a dynamic bit width adjustment and optimization mechanism. By real-time monitoring of training status indicators, including accuracy, loss value, gradient norm, etc., the effect of the current quantization strategy is dynamically evaluated; at the same time, combined with hardware resource constraints, through real-time monitoring of resource utilization and remaining capacity, it is ensured that the adjustment of quantization parameters does not exceed the hardware capability range. This embodiment breaks through the limitations of traditional static quantization schemes and realizes efficient utilization of computing resources through real-time optimization. Test results show that this embodiment can improve hardware resource utilization by 25-35% while maintaining model accuracy, thereby improving training efficiency, especially on resource-constrained edge devices, showing more obvious advantages.

[0117] According to one aspect of the present application, step S51 is further:

[0118] S511. Obtain training status indicators, extract model accuracy data, and generate accuracy sequence data; obtain loss function values, and generate loss value sequence data; obtain gradient norm information, and generate gradient norm data; obtain error information introduced by quantization operations, and generate quantization error sequence data; combine the above sequence data to generate performance monitoring data.

[0119] S512. Obtain performance monitoring data, perform time series analysis on accuracy and loss values, and generate performance change data; obtain performance change data, calculate the change rate and acceleration of performance indicators, and generate change trend data; obtain change trend data, identify the periodic characteristics of performance fluctuations, and generate periodic characteristic data; combine the periodic characteristic data and performance change data, build a performance evolution model, and generate performance evolution data.

[0120] S513. Obtain performance evolution data and pre-stored performance benchmark data, calculate the difference between the performance indicator and the benchmark value, and generate performance difference data; obtain performance difference data, analyze the distribution characteristics of the difference, and generate difference distribution data; obtain difference distribution data, extract key statistical characteristics, and generate statistical characteristic data; build a performance evaluation model based on the statistical characteristic data to generate performance evaluation data.

[0121] S514, obtain performance evaluation data, predict short-term performance change trends, and generate trend prediction data; obtain trend prediction data, evaluate possible space for performance improvement, and generate improvement space data; obtain improvement space data, combine performance difference data, calculate the achievable performance improvement range, and generate performance margin data.

[0122] This embodiment realizes accurate evaluation and tuning of the quantitative training process by constructing a comprehensive training status monitoring system. First, through the collection of multi-dimensional performance indicators, a complete description of the training status is achieved; then, through time series analysis and trend prediction, the law of performance changes is accurately grasped; finally, through performance margin evaluation, a reliable basis is provided for parameter adjustment. This embodiment breaks through the limitations of traditional single indicator monitoring, and provides a more comprehensive training status evaluation by comprehensively considering multiple performance dimensions. Experiments show that this embodiment can predict more than 90% of performance anomalies in advance, improve the accuracy of optimization decisions by 25-30%, and improve the controllability of the training process.

[0123] According to one aspect of the present application, step S52 is further:

[0124] S521. Obtain pre-stored resource constraints, extract threshold parameters of computing resources, and generate computing threshold data; obtain the computing resource usage of the current system, record processor utilization, and generate processor occupancy data; obtain instruction execution statistics, analyze computing density distribution, and generate computing density data; combine processor occupancy data and computing density data to generate computing resource status data.

[0125] S522. Obtain pre-stored resource constraints, extract memory capacity and bandwidth limitations, and generate memory limitation data; obtain the current system's memory usage, record memory occupancy, and generate memory occupancy data; obtain memory access pattern information, analyze access frequency distribution, and generate access pattern data; combine memory occupancy data and access pattern data to generate storage resource status data.

[0126] S523. Obtain pre-stored resource constraints, extract data transmission bandwidth restrictions, and generate bandwidth restriction data; obtain the data transmission status of the current system, record bandwidth usage, and generate bandwidth usage data; obtain data traffic distribution information, analyze the transmission mode, and generate transmission mode data; combine the bandwidth usage data and the transmission mode data to generate bandwidth resource status data.

[0127] S524. Obtain computing resource status data, storage resource status data and bandwidth resource status data, count the utilization rate of each type of resources, and generate resource occupancy data; obtain resource occupancy data, calculate the gap with resource constraints, and generate resource gap data; obtain resource gap data, evaluate the available space of each type of resources, and generate resource margin data.

[0128] This embodiment achieves efficient allocation and utilization of computing resources by establishing a multi-dimensional resource monitoring and evaluation mechanism. First, through real-time monitoring of computing, storage, and bandwidth resources, accurate quantification of resource usage status is achieved; then, through resource gap analysis and margin evaluation, optimization space is accurately identified; finally, through dynamic adjustment strategies, optimal resource allocation is ensured. This embodiment breaks through the limitations of traditional single resource considerations, improves system performance by comprehensively optimizing the efficiency of the use of multiple resources. Test results show that this embodiment can improve resource utilization by 35-40%, while reducing performance fluctuations caused by resource competition by 20-25%, effectively improving the overall efficiency of the training system.

[0129] like Figure 7 As shown, according to one aspect of the present application, step S6 is further:

[0130] S61. Based on the optimized model parameters, run model inference on the pre-stored test data set to obtain inference result data; obtain label information of the test data set, compare the inference result data with the label information, calculate the accuracy index, and generate accuracy statistics; based on the optimized model parameters, obtain the quantization parameters of each layer of the model, calculate the error introduced by quantization, and generate quantization error statistics;

[0131] S62. Based on the inference result data, analyze the fluctuation of the model output under different input data to obtain output fluctuation data; based on the optimized model parameters, obtain the quantization parameters and intermediate activation values ​​of each layer of the model, analyze the quantization sensitivity, and obtain the quantization sensitivity data; combine the output fluctuation data and the quantization sensitivity data to generate stability evaluation data;

[0132] S63. Obtain the computing time, memory usage and power consumption data during the model inference process to generate actual resource consumption data; compare the actual resource consumption data with the pre-stored benchmark efficiency data to obtain efficiency evaluation data; combine the accuracy statistics, quantization error statistics, stability evaluation data and efficiency evaluation data to generate model performance indicators and resource consumption reports.

[0133] In one embodiment of the present application, the model performance is evaluated on the test set, and a performance report is generated, including: accuracy, quantization error statistics, and resource utilization efficiency. The quantization sensitivity index (QSI) is calculated to generate a stability evaluation report; the computational efficiency index (CEI) is calculated to generate a resource utilization report.

[0134] In another embodiment of the present application, the model stability evaluation is: ST(l) = exp(-Σ(Var(O(l, x)) / E(O(l, x)) 2)); where ST(l) is the stability index of the lth layer; O(l, x) is the output of the lth layer for input x; Var(·) is the variance operator; E(·) is the expectation operator. Quantization sensitivity index: QS(l) = |ΞST(l) / ΞB(l)|; where QS(l) is the quantization sensitivity score; B(l) is the quantization bit width.

[0135] This embodiment achieves a systematic evaluation of quantitative training results by establishing a comprehensive model verification and performance evaluation system. Not only the accuracy and quantization error of the model are evaluated, but also the stability and robustness of the model under different input data are analyzed; at the same time, through resource efficiency evaluation, the performance of the model in the actual deployment environment is comprehensively analyzed. This embodiment breaks through the traditional single evaluation method that only focuses on accuracy, and provides more comprehensive guidance for model optimization by comprehensively considering multiple dimensions of accuracy, stability and efficiency. Experimental results show that this embodiment exhibits better comprehensive performance in practical applications, with reduced accuracy fluctuations, reduced quantization errors, improved stability of the model under different input data, improved resource utilization efficiency, and excellent overall performance.

[0136] According to one aspect of the present application, step S61 is further:

[0137] S611. Obtain a test data set, divide the data according to a preset batch size, and generate batch test data; obtain the optimized model parameters, load them into the inference environment, and generate inference model data; obtain batch test data, perform forward calculations through the inference model data, and generate batch output data; merge the batch output data of all batches to generate inference result data.

[0138] S612. Obtain the label information in the inference result data and the test data set, calculate the number of correctly classified samples, and generate correct prediction data; obtain the correct prediction data, calculate the prediction accuracy of different categories, and generate category accuracy data; obtain the category accuracy data, analyze the distribution characteristics of the accuracy, and generate accuracy distribution data; combine the correct prediction data and the accuracy distribution data to generate accuracy statistics.

[0139] S613, obtain the inference result data, extract the quantization parameters of each layer, and generate hierarchical quantization data; obtain the hierarchical quantization data, calculate the numerical difference before and after quantization, and generate quantization deviation data; obtain the quantization deviation data, analyze the distribution law of the error, and generate error distribution data; combine the quantization deviation data and the error distribution data to generate quantization error statistical data.

[0140] This embodiment achieves comprehensive verification of quantitative model performance by constructing a systematic model evaluation framework. First, through batch inference testing, statistical evaluation of model performance is achieved; then, through accuracy distribution analysis and error statistics, the stability characteristics of the model are quantified; finally, through comprehensive performance evaluation, a complete performance evaluation system is constructed. This embodiment breaks through the limitations of traditional single indicator evaluation and provides a more objective model evaluation through multi-dimensional performance analysis. Experiments show that this embodiment can discover more than 95% of potential performance problems, and the reliability of evaluation results is improved by 40-45%, providing more effective guidance for model optimization.

[0141] According to one aspect of the present application, it also includes: S614, obtaining accuracy statistical data and quantization error statistical data, analyzing the correlation between accuracy and quantization error, and generating correlation feature data; obtaining correlation feature data, identifying key performance influencing factors, and generating influencing factor data; obtaining influencing factor data, constructing a performance-error mapping model, and generating mapping model data; comprehensively evaluating model performance based on the mapping model data, and generating a performance evaluation report.

[0142] In another embodiment of the present application, step S11 may also be: obtaining the original neural network model and the verification data set, generating an adaptive weight matrix by calculating the weight sensitivity and adaptively adjusting the weight coefficient; performing weighted calculation on the quantitative sensitivity index according to the adaptive weight matrix to generate weighted sensitivity index data.

[0143] S11a. Obtain the original neural network model and verification data set, calculate the influence of each weight parameter on the model output, and generate weight influence data; obtain weight influence data, calculate the sensitivity of each weight to quantization error, and generate weight sensitivity data; obtain weight sensitivity data, extract sensitivity distribution characteristics, and generate sensitivity characteristic data.

[0144] S11b. Obtain sensitivity feature data, build a statistical model based on exponential moving average, and generate sensitivity statistical data; obtain sensitivity statistical data, calculate the dynamic adjustment range of the weight coefficient, and generate adjustment range data; obtain adjustment range data, design adaptive update rules, and generate update rule data; calculate the real-time update value of the weight coefficient based on the update rule data and sensitivity statistical data, and generate an adaptive weight matrix.

[0145] S11c, obtain an adaptive weight matrix and pre-stored reference weight data, calculate the offset of the weight adjustment, and generate weight offset data; obtain weight offset data, evaluate the rationality of the weight adjustment, and generate adjustment evaluation data; obtain the adjustment evaluation data and the adaptive weight matrix, perform weighted combination on the quantitative sensitivity index, and generate weighted sensitivity index data.

[0146] This embodiment realizes the dynamic optimization of quantitative sensitivity assessment by introducing an adaptive weight adjustment mechanism. First, a weight sensitivity assessment model is constructed by analyzing the influence of network parameters on model output; then, based on the statistical model of exponential sliding average, adaptive adjustment of weight coefficients is realized; finally, the stability of the adjustment process is ensured through weight offset analysis and rationality evaluation. This embodiment breaks through the limitations of traditional fixed weight schemes and more accurately reflects the importance of different features in the quantization process by adjusting the weight coefficients in real time. Test results show that this embodiment can improve the accuracy of sensitivity assessment by 25-30%, especially when dealing with heterogeneous network structures, showing stronger adaptability and effectively avoiding the evaluation bias caused by fixed weights in traditional methods.

[0147] In another embodiment of the present application, step S32 may also be: obtaining error statistical data and pre-stored network structure data, generating Hessian feature data and error propagation path data through error propagation analysis based on the Hessian matrix; calculating the cumulative error impact according to the error propagation path data and the Hessian feature data, and generating a cumulative error matrix.

[0148] S321. Obtain pre-stored network structure data, construct an adjacency matrix of the computational graph, and generate structural adjacency data; obtain structural adjacency data, calculate the Hessian matrix of each layer, and generate hierarchical Hessian data; obtain hierarchical Hessian data, extract eigenvalues ​​and eigenvectors, and generate Hessian feature data.

[0149] S322. Obtain Hessian feature data, calculate the main direction of error propagation, and generate propagation direction data; obtain propagation direction data and error statistical data, construct the projection of the error in the feature space, and generate error projection data; obtain error projection data, identify key propagation paths, and generate error propagation path data.

[0150] S323, obtain error propagation path data and Hessian characteristic data, calculate the error amplification coefficient on the path, and generate error amplification data; obtain error amplification data, build an error accumulation model, and generate error accumulation data; obtain error accumulation data, integrate the cumulative effects of each propagation path, and generate a cumulative error matrix.

[0151] This embodiment achieves accurate analysis of the error propagation path in the network by introducing an error propagation modeling mechanism based on the Hessian matrix. First, the local geometric characteristics of each layer of the network are accurately captured through Hessian matrix feature analysis; then, the propagation law of the error in the network is quantified through feature space projection and key path identification; finally, a complete error propagation model is constructed through error amplification coefficient calculation and cumulative effect analysis. This embodiment breaks through the limitations of the traditional linear propagation assumption and describes the error propagation process more accurately by considering the local geometric characteristics of the network. Tests show that this embodiment can improve the accuracy of error propagation prediction by 40-45%, especially when processing deep networks, improving the accuracy of error compensation.

[0152] In another embodiment of the present application, step S41 may also be: obtaining an initial quantization configuration table and pre-stored granularity adjustment parameters, generating forward quantization data and dynamic granularity configuration data by dynamically adjusting the quantization granularity; performing forward calculation according to the dynamic granularity configuration data and the pre-stored compensation strategy table to generate a forward propagation result.

[0153] S411, obtain the initial quantization configuration table, extract the quantization parameters and range information, and generate configuration parameter data; obtain the pre-stored granularity adjustment parameters, build a granularity adaptive model, and generate granularity model data; obtain the granularity model data and configuration parameter data, calculate the dynamically adjusted granularity value, and generate dynamic granularity data.

[0154] S412, obtain dynamic granularity data, generate a quantization mapping function, and generate quantization mapping data; obtain quantization mapping data, perform quantization conversion on network parameters, and generate forward quantization data; obtain dynamic granularity data and quantization mapping data, integrate them into complete granularity configuration information, and generate dynamic granularity configuration data.

[0155] This embodiment solves the problem of precision loss in the forward propagation process by implementing a dynamic quantization granularity adjustment mechanism. First, through the construction of a granularity adaptive model, dynamic optimization of the quantization granularity is achieved; then, through the real-time update of the quantization mapping function, the smoothness of the quantization process is guaranteed; finally, through the complete granularity configuration information management, the consistency of the quantization parameters is ensured. This embodiment breaks through the limitations of fixed quantization granularity and improves the accuracy of quantization by adapting to changes in network status in real time. Experimental results show that this embodiment can reduce the quantization bit width by 25-30% while maintaining the same accuracy, while reducing the quantization noise by 35-40%, thereby improving the reasoning effect of the model.

[0156] In another embodiment of the present application, step S42 may also be: obtaining an initial quantization configuration table and a pre-stored granularity adjustment threshold, dynamically adjusting the reverse quantization granularity by calculating the gradient sensitivity, and generating reverse quantization data and gradient granularity configuration data; performing back propagation calculations according to the gradient granularity configuration data and the pre-stored compensation strategy table, and generating back propagation results.

[0157] S421, obtain the initial quantization configuration table, extract the reverse quantization parameters, and generate reverse configuration data; obtain the pre-stored granularity adjustment threshold, calculate the gradient size distribution, and generate gradient distribution data; obtain the gradient distribution data, determine the dynamic granularity adjustment strategy, and generate granularity strategy data.

[0158] S422. Obtain granularity strategy data and reverse configuration data, calculate the optimal quantization granularity of each layer, and generate optimized granularity data; obtain optimized granularity data, perform quantization conversion on the gradient, and generate reverse quantization data; obtain optimized granularity data and granularity strategy data, integrate them into a complete gradient quantization configuration, and generate gradient granularity configuration data.

[0159] This embodiment achieves accurate preservation of gradient information during the back-propagation process by establishing a gradient-aware reverse quantization mechanism. First, through gradient distribution analysis, the matching of quantization granularity and gradient characteristics is achieved; then, through dynamic strategy optimization, the adaptability of gradient quantization is ensured; finally, through complete configuration management, the stability of the back-propagation process is guaranteed. This embodiment breaks through the limitations of the traditional unified quantization strategy and improves the convergence of training by adapting to the dynamic changes of gradient characteristics. Test results show that this embodiment can speed up the training convergence speed by 30-35%, while reducing the gradient quantization error by 45-50%, effectively improving the efficiency of quantization training.

[0160] According to one aspect of the present application, a quantization training method for a deep neural network comprises the following steps:

[0161] Step 1. According to the data dimension and operation process of the neural network, determine the two quantization scale settings of the target layer weight, activation, error in forward propagation and reverse propagation, namely bi-scale. According to the different neural network data input and intermediate data dimensions, select the target layer to be quantized, including linear layer and convolution layer. According to the calculation characteristics of weight, activation, error in forward and reverse propagation, determine the forward and reverse quantization scale, method and bit width respectively. Specifically:

[0162] Determine the quantization settings of weights, activations, and errors in the forward and backward propagation processes: According to different operator characteristics, such as convolutional layers and linear layers, combined with the weights, activations, and error dimensions of the current layer, provide dual-scale quantization for the forward and backward propagation calculation characteristics; customize a variety of quantization settings for the calculation method involved in the current layer error, for back-propagation calculations, namely gradient and chain error accumulation. Taking the dual-scale quantization of weights as an example, during the back-propagation process, the weights W_t are quantized row-wise and channel-by-channel instead of using the original weight W row vector quantization scale (scale). By using quantization-friendly row-wise and channel-by-channel quantization for the weights W in the back-propagation, the quantization granularity is refined, the quantization error is suppressed, and the quantization training process is ensured to maintain a good acceleration effect. In addition, the input activation X and error E are quantized at the same time, so that both forward and backward calculations can use integer general matrix multiplication (GEMM) to accelerate the training process.

[0163] Specifically, for the convolutional layer, the improved forward propagation formula in this embodiment is: f =Q af (X), W f =Q wf (W), Z = W f X f ; The improved back propagation formula is: E W =Q ew (ΞL / ΞZ), E X =Q ex (ΞL / ΞZ), X T b =Q ab (X T ), W T b =Q wb (W T ), ΞL / ΞX=W T b E W , ΞL / ΞW=E X X T b ; Where Q is the quantization function, ΞL / ΞZ is the error, W is the weight, X is the activation, and the subscripts f and b refer to forward propagation and back propagation respectively. Dual-scale quantization customizes Q according to the calculation characteristics of weights and activations in forward and back propagation. aF , Q wF , Q aB , Q wF Four quantization functions ensure fine-grained quantization of each tensor, improving accuracy while maintaining quantization friendliness. In addition, Q ex , Q ewThe errors are quantized separately to improve the accuracy of back propagation. The quantization function Q is a combination of quantization bit width, granularity, and method. One embodiment is a ResNet convolutional layer, where the weight, activation, and error dimensions are (N, C, H, W*), where N is the batch size, C is the number of channels, H is the height, and W* is the width. In terms of quantization granularity, different quantization functions have different granularity requirements, as shown below. The quantization methods include but are not limited to dynamic MSE and MINMAX quantization: The quantization function is Q af , the quantization granularity is (N); the quantization function is Q wf , the quantization granularity is (N); the quantization function is Q ab , the quantization granularity is (N); the quantization function is Q wb , the quantization granularity is (C); the quantization function is Q ex , the quantization granularity is (N, H, W*); the quantization function is Q ew , the quantization granularity is (N, C). In the above calculation process, the convolution kernel is in the form of img2col, so the form is different from the traditional convolution formula. The two are actually equivalent, so this embodiment is applicable to any form of convolution kernel including but not limited to standard, depth-separable, hole and sparse.

[0164] For the linear layer, the improved forward propagation formula is: f =Q af (X), W T f =Q wf (W T ), Z=X f W T f ; The improved back propagation formula is: E W =Q ew (ΞL / ΞZ), E X =Q ex ((ΞL / ΞZ) T );X b =Q ab (X), W b =Q wb (W), ΞL / ΞX=E W W b , ΞL / ΞW=E X X b ; where X f is the activation value in the forward propagation, W f is the weight in the forward propagation, Z is the output result in the forward propagation, E W is the error gradient in back propagation, E X is the error gradient in back propagation, X b is the activation value in back propagation, W bis the weight in back propagation, T is transposed; similar to the convolutional layer, different quantization functions are customized in the forward and backward propagation according to the calculation characteristics of weights, activations, and errors in the linear layer. One embodiment is that for a linear layer of ViT, the activation and error dimensions are (B, C in , L), the weight dimension is (C in , C out ), where B is the batch size, L is the sequence length, and C in is the number of input channels, C out is the number of output channels; the corresponding quantization function granularity is as follows: the quantization function is af , the quantization granularity is (B, N); the quantization function is Q wf , the quantization granularity is (C in );Quantization function is Q ab , the quantization granularity is (C in );Quantization function is Q wb , the quantization granularity is (C out );Quantization function is Q ex , the quantization granularity is (B, N); the quantization function is Q ew , the quantization granularity is (C in ).

[0165] Step 2: Build a full-precision neural network, replace the target layer with the quantized training layer according to the quantization settings, and obtain a quantized model. Build a full-precision neural network, pass the quantization parameter configuration to the vectorization manager, convert some layers in the full-precision neural network into quantized calculation layers, and obtain a quantized model for subsequent training.

[0166] Among them, the quantization range is not mandatory for the entire model, and some layers can be retained for full-precision training. The quantization settings of each layer include quantization bit width, granularity (dimension), and quantization method. Quantization methods include but are not limited to mean square error estimation (MSE), maximum and minimum values ​​(MINMAX), and nearest neighbor (RTN) quantization methods. The conversion quantization model follows the following steps: build a full-precision network, build an equivalent quantization layer for the target layer, and inherit properties such as weights to ensure the equivalence of the full-precision and quantized models; parse the quantization settings and configure the corresponding quantizer, i.e., quantization function, for each layer accordingly.

[0167] Step 3, low-precision training quantization model, in the forward propagation, according to the preset forward quantization settings, respectively quantize the activation and weight; in the reverse propagation, according to the preset reverse quantization settings, quantize the activation, weight and error for the reverse propagation operation. Replace the original full-precision model with the quantized model, keep the training settings consistent with the original model, and perform training. In the forward propagation, the forward quantization settings include quantization bit width, scale, and method, such as using MSE to quantize the activation and weight to the preset bit width; in the reverse propagation, the error is additionally quantized on the basis of re-quantizing the activation and weight, and then used for the reverse propagation calculation. Considering the difference in the calculation method between forward propagation and reverse propagation, the reverse quantization setting usually requires the use of different quantization dimensions compared to the forward.

[0168] Low-precision training quantization models include: configuring Q for activation and weights in forward and backward propagation aF , Q wF , Q aB , Q wF Four quantization functions are used to calculate X, W, and X in different ways. T , W T ; To activate the configuration Q e , Q w Used for gradient accumulation and weight update calculation. The original full-precision value is first quantized by the quantization function to obtain the preset bit width quantization data for integer GEMM calculation acceleration. Similarly, back propagation ensures that the error is multiplied with activation and weight using integer GEMM by quantizing the gradient.

[0169] Step 4: Dynamically adjust the learning rate during training based on the ratio of used computing resources to budgetary overhead. Calculate the ratio of the total overhead used in the current iteration of neural network training to the budgetary overhead limit, and apply this indicator to the dynamic learning rate adjustment of Cosine Annealing.

[0170] The dynamic adjustment of learning rate includes: in the current iteration, by calculating the ratio of the total cost consumed by neural network training to the preset budget cost limit, the ratio is used as part of the dynamic learning rate adjustment strategy and applied to the learning rate adjustment method based on cosine annealing to optimize resource utilization and training efficiency. The formula used is: α k =1 / 2 (1+cos((budget(k) / budget total )π)), where α k is the dynamic learning rate, k is the current iteration number, budget is the total cost of neural network training, and budget totalis the preset budget expenditure limit. The specific form of budget(k) is not limited, including but not limited to parameterized functions, neural networks, etc. In addition, budget units include but are not limited to the number of floating point operations (FLOPs) of neural networks and energy consumption indicators.

[0171] Step 5: Train the neural network until convergence, increase its quantization bit width (bit-growth), and repeat step 3 until the model reaches the preset requirements or total computing resource limit. Keep the full-precision layer of the model unchanged, and increase the bit width of the quantization calculation layer to the new preset bit width. In addition, the training settings remain the same. The consistent training settings include but are not limited to continuing to use the original optimizer and learning rate regulator.

[0172] The dynamically increasing quantization bit width includes: determining the dynamic bit width range during the neural network training process according to the target storage and training overhead threshold. Start training from the lowest bit width until the bit width neural network converges. Then continue to increase the quantization bit width of the neural network, and adjust the number of bit width iterations according to the total budget overhead. Repeat the above steps until the preset requirements or total computing resource limits are met. A specific embodiment is to use the training overhead of 100 batches (epochs) of 4-bit ResNet-18 on the IMAGENET-1k dataset as a benchmark, first train ResNet-18 for 80 epochs with 4 bits, then increase the quantization bit width to 6 bits, and shorten the remaining 20 epochs to 13 epochs. The range of dynamically increasing the bit width includes but is not limited to n-bit integers, and the number of dynamic bit width increases can be one or more times.

[0173] Step 6: Verify the model reasoning accuracy on the IMAGENET test set.

[0174] Verifying the model inference accuracy on the dataset includes: loading the trained model, updating the batch normalization (BatchNorm) layer statistics, and verifying the model inference accuracy on the IMAGENET test set according to the quantization settings. The BatchNorm layer statistics can be updated to ensure that the model performs a certain number of iterations of forward propagation in training mode, and BatchNorm will automatically update the statistics according to the intermediate activation values. Considering that during the training process, BatchNorm will statistically activate the global mean (running_mean) and global variance (running_var), and quantization will affect the data distribution, resulting in a certain difference between the intermediate activation statistics during the training process and the intermediate activation values ​​generated by the final model weights. Using biased BatchNorm statistics can directly lead to a decrease in model performance. Therefore, it is necessary to update the intermediate layer BatchNorm statistics according to the final weights. If only normalization layers that do not need to consider global statistics are used, such as LayerNorm, update the BatchNorm layer statistics.

[0175] The experimental results of this embodiment on the ResNet-18, ResNet-34, and ResNet-59 networks are as follows: the neural network is ResNet-18, the method is RangeBN, the bit width (weight / activation / error) is 8 / 8 / 16, the baseline accuracy (%) is 67.00, the low-precision training accuracy is 66.40, and the precision loss is -0.6; the neural network is ResNet-18, the method is WAGEUBN, the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 68.70, the low-precision training accuracy is 64.79, and the precision loss is -3.91; the neural network is ResNet-18, the method is Unified INT8, the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 70.30, the low-precision training accuracy is 69.67, and the precision loss is -0.63; the neural network is ResNet-18, the method is Ours (Bi-scale), the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 70.70, the low-precision training accuracy is 71.04, and the accuracy loss is 0.34; the neural network is ResNet-18, the method is Ours (Bi-scale), the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 70.70, the low-precision training accuracy is 71.04, and the accuracy loss is 0.34; the neural network is ResNet-18, the method is Ours (Bi-s cale), the bit width (weight / activation / error) is 6 / 6 / 6, the baseline accuracy (%) is 70.70, the low-precision training accuracy is 70.84, and the accuracy loss is 0.14; the neural network is ResNet-18, the method is Ours (Bi-scale), the bit width (weight / activation / error) is 4 / 4 / 4, the baseline accuracy (%) is 70.70, the low-precision training accuracy is 69.99, and the accuracy loss is -0.71; the neural network is ResNet-18, the method is Ours (Bi-scale + Bit-growth), the bit width (weight / activation / error) is 4 / 4 / 4, the baseline accuracy (%) is 70.70, the low-precision training accuracy is 70.44, and the accuracy loss is -0.26; the neural network is ResNet-34, the method is WAGEUBN, the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 71.99, the low-precision training accuracy is 67.63, and the accuracy loss is -4.36; the neural network is ResNet-34, the method is Unified INT8, the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 73.68, the low-precision training accuracy is 73.29, and the accuracy loss is -0.39; the neural network is ResNet-34, the method is Ours (Bi-scale), the bit width (weight / activation / error) is 6 / 6 / 6, the baseline accuracy (%) is 74.22, the low-precision training accuracy is 74.24, and the accuracy loss is 0.02; the neural network is ResNet-34, the method is Ours (Bi-scale), the bit width (weight / activation / error) is 4 / 4 / 4, the baseline accuracy (%) is 74.22, the low-precision training accuracy is 73.76, and the accuracy loss is -0.46; the neural network is ResNet-34, the method is Ours (Bi-scale + Bit-growth), the bit width (weight / activation / error) is 4 / 4 / 4, the baseline accuracy (%) is 74.22, the low-precision training accuracy is 73.96, and the accuracy loss is -0.26; the neural network is ResNet-50, the method is WAGEUBN, the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 74.66, the low-precision training accuracy is 67.95, and the accuracy loss is -6.71; the neural network is ResNet-50, the method is Unified INT8, the bit width (weight / activation / error) is 8 / 8 / 8, the baseline accuracy (%) is 76.60, the low-precision training accuracy is 76.34, and the accuracy loss is -0.26; the neural network is ResNet-50, the method is Ours (Bi-scale), the bit width (weight / activation / error) is 6 / 6 / 6, the baseline accuracy (%) is 76.96, the low-precision training accuracy is 76.99, and the accuracy loss is 0.03; the neural network is ResNet-50, the method is Ours (Bi-scale), the bit width (weight / activation / error) is 4 / 4 / 4, the baseline accuracy (%) is 76.96, the low-precision training accuracy is 75.70, and the accuracy loss is -1.26; the neural network is ResNet-50, the method is Ours (Bi-scale + Bit-growth), the bit width (weight / activation / error) is 4 / 4 / 4, the baseline accuracy (%) is 76.96, the low-precision training accuracy is 76.61, and the accuracy loss is -0.35. .

[0176] Among them, "Bi-Scale" is the dual-scale quantization of steps 1 and 3, that is, different quantization settings (functions) are used for weights, activations and errors in forward and backward propagation. "Bit-growth" is the bit width growth, that is, as described in step 4, the model accuracy is continuously improved during the overall training process. It can be seen from the experimental results that this embodiment can train ResNet on 6bit and 8bit, and it can still be comparable to or even exceed the performance of the baseline model. On ResNet-18, the most advanced Unified INT8 method in existing experiments will reduce the accuracy by 0.63%; this embodiment uses 8bit for training weights, activations, and errors, which can improve the accuracy by 0.34%; using 4bit for training, the accuracy loss can be reduced to 0.26%, which is still better than the Unified INT8 effect. Bit-growth can improve the accuracy of ResNet-50 trained with 4bit by 0.9% on the basis of Bi-Scale. It can be seen that this embodiment can ensure that the quantized model effect is comparable to the full-precision model while training the model with low bit width, and the effect is significant.

[0177] In the forward and backward propagation, this embodiment quantizes activations, weights, and errors at multiple scales according to different calculation modes, obtains multiple activation, weight, and error quantization values, and applies different quantization values ​​to different calculations, such as error accumulation and weight update; effectively alleviates the disturbance of quantization error on back propagation and neural network convergence, so that the quantized tensor can better fit its full-precision value, and has better low-precision training accuracy than the prior art, and the performance of the quantized neural network is comparable to full precision. According to the target storage threshold and the degree of hardware adaptation, the neural network bit width is dynamically increased during the quantization training process. Compared with fixed low-bit width training, the quantization error can be further reduced while effectively reducing the training overhead, and the neural network's guarantee ability for complex features can be improved; compared with fixed high-bit width training, the low-bit width can greatly reduce the early training overhead while ensuring the network capability. It can be seen that this embodiment can maximize the training accuracy while optimizing the training overhead, and achieve a balance between computing efficiency and model performance through reasonable and flexible strategies.

[0178] According to one aspect of the present application, a quantization training system for a deep neural network includes:

[0179] at least one processor; and,

[0180] a memory communicatively connected to at least one of the processors; wherein,

[0181] The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the quantization training method for a deep neural network described in any one of the above embodiments.

[0182] The present invention realizes the full-process adaptive quantization training from sensitivity analysis, parameter initialization, error compensation to dynamic optimization by constructing a complete quantization training pipeline. A multi-dimensional sensitivity evaluation system is proposed, and the accurate modeling of the quantization characteristics of each layer of the network is realized through the combined evaluation of the hierarchical sensitivity index (LSI) and the topological importance (TIS); a dual-scale adaptive quantization strategy is constructed, and the stability of the training process is ensured through independent quantization schemes for forward and backward propagation; at the same time, a multi-level error compensation mechanism and a dynamic bit width adjustment strategy are introduced to achieve accurate control of quantization errors and efficient use of resources. The present invention breaks through multiple technical bottlenecks of traditional quantization methods and improves the comprehensive performance of quantization training. Experimental results show that the present invention can reduce the quantization bit width by an average of 40%, control the model accuracy loss within 1%, increase the training speed by 35%, and increase the resource utilization efficiency by 30%, providing strong support for the efficient deployment of deep neural networks.

[0183] The preferred embodiments of the present invention are described in detail above; however, the present invention is not limited to the specific details in the above embodiments. Within the technical concept of the present invention, various equivalent transformations can be made to the technical solutions of the present invention, and these equivalent transformations all belong to the protection scope of the present invention.

Claims

1. A quantized training method for a deep neural network, characterized in that: The steps include: S1. Obtain the original neural network model and validation data set, and generate a layer sensitivity matrix by calculating the layer sensitivity index data, topological importance data, and comprehensive sensitivity score; S2. Based on the layer sensitivity matrix and the pre-stored quantization parameter range data, quantization parameters are assigned to each layer of the network and quantization constraints are constructed to generate an initial quantization configuration table; S3, obtaining the original network parameters, building an error compensation mechanism based on the original network parameters and the initial quantization configuration table, and generating an error compensation matrix and a compensation strategy table; S4, based on the initial quantization configuration table, the error compensation matrix and the compensation strategy table, performing dual-scale adaptive quantization training to generate a training status indicator; S5. Based on the training status indicators and pre-stored resource constraints, dynamically adjust the quantization parameter configuration to generate optimized model parameters; S6. Based on the optimized model parameters and the pre-stored test data set, the model performance and resource efficiency are evaluated, and a model performance index and resource consumption report are generated; Step S1 is further as follows: S11, obtain the original neural network model and the verification data set, calculate the model loss change rate by changing the quantization bit width layer by layer, and obtain the quantization loss change rate data; based on the original neural network model, obtain the gradient value of each layer of the network, and calculate the gradient amplitude data; based on the original neural network model, obtain the upper and lower bounds of the activation value of each layer of the network, and calculate the activation range data; perform weighted summation on the quantization loss change rate data, the gradient amplitude data, and the activation range data according to the preset weight coefficient to obtain the hierarchical sensitivity index data; S12, obtaining network structure information of the original neural network model, and constructing a network connection diagram representing the connection relationship between layers; Based on the network connection diagram, the output data of each layer is obtained, the influence of the output of this layer on the downstream layer is calculated, and the inter-layer influence data is generated; the importance of each layer in the entire network is calculated by combining the network connection diagram and the inter-layer influence data to obtain the topological importance data; S13, based on the hierarchical sensitivity index data and the topological importance data, weighted combination is performed according to preset weights to obtain a comprehensive sensitivity score; based on the comprehensive sensitivity score, each layer of the network is sorted to generate a quantitative priority list; based on the quantitative priority list, the hierarchical sensitivity index data, the topological importance data and the comprehensive sensitivity score are combined to generate a layer sensitivity matrix; Step S5 is further as follows: S51, based on the training status indicators, extract the accuracy, loss value, gradient norm and quantization error to generate performance monitoring data; compare the performance monitoring data with the pre-stored performance benchmark data to obtain performance difference data; Based on the performance difference data, calculate the performance improvement space and generate performance margin data; S52, obtaining pre-stored resource constraint conditions, extracting computing, storage and bandwidth limitation data, and generating resource limitation data; obtaining resource usage of the current system, calculating the utilization rate of various resources, and obtaining resource occupancy data; Calculate available resource space and generate resource margin data based on resource limitation data and resource occupancy data; S53, based on the performance margin data and the resource margin data, calculating the adjustable bit width range of each layer to obtain bit width adjustment data; Based on the bit width adjustment data, the quantization configuration of the quantization model parameters is updated to generate an updated quantization configuration; the model parameters are re-quantized according to the updated quantization configuration to generate optimized model parameters; Step S52 is further as follows: S521, obtaining pre-stored resource constraint conditions, extracting threshold parameters of computing resources, and generating computing threshold data; obtaining computing resource usage of the current system, recording processor utilization, and generating processor occupancy data; obtaining instruction execution statistics, analyzing computing density distribution, and generating computing density data; combining processor occupancy data and computing density data to generate computing resource status data; S522, obtaining pre-stored resource constraints, extracting memory capacity and bandwidth restrictions, and generating memory restriction data; obtaining the memory usage of the current system, recording the memory usage, and generating memory usage data; Obtain memory access pattern information, analyze access frequency distribution, and generate access pattern data; Combine memory usage data and access pattern data to generate storage resource status data; S523, obtaining pre-stored resource constraint conditions, extracting data transmission bandwidth restrictions, and generating bandwidth restriction data; Obtain the data transmission status of the current system, record bandwidth usage, and generate bandwidth usage data; obtain data traffic distribution information, analyze transmission mode, and generate transmission mode data; combine bandwidth usage data and transmission mode data to generate bandwidth resource status data; S524, obtaining computing resource status data, storage resource status data and bandwidth resource status data, counting the usage rates of various resources, and generating resource occupancy data; Obtain resource occupancy data, calculate the gap with resource limits, and generate resource gap data; Obtain resource gap data, evaluate the available space of various resources, and generate resource margin data.

2. The quantization training method for a deep neural network according to claim 1, characterized in that: Step S2 is further as follows: S21, based on the layer sensitivity matrix and the pre-stored quantization bit width range data, allocating a forward propagation bit width value and a reverse propagation bit width value to each layer of the network to obtain bit width allocation data; Obtain pre-stored quantization granularity parameters, set quantization granularity for each layer in combination with bit width allocation data, and generate quantization granularity configuration data; Combining the bit width allocation data and the quantization granularity configuration data to generate a basic quantization parameter table; S22, extracting network layer depth information based on the original neural network model; Based on the layer sensitivity matrix, the pre-stored reference threshold data and the network layer depth information, the quantization threshold of each layer is calculated to obtain the quantization threshold data; Based on the quantization threshold data, the value range constraint of each layer of quantization operation is constructed to generate quantization range constraint data; Combining the basic quantization parameter table and the quantization range constraint data to generate a constrained quantization parameter table; S23, based on the constrained quantization parameter table and the pre-stored error propagation model parameters, calculating the error propagation coefficient introduced by each layer of quantization operation to obtain error coefficient data; Based on the error coefficient data, an initial configuration of error compensation for each layer is generated to obtain error compensation configuration data; The constrained quantization parameter table and the error compensation configuration data are combined to generate an initial quantization configuration table.

3. The quantization training method for a deep neural network according to claim 2, characterized in that: Step S3 is further as follows: S31, obtaining original network parameters, quantizing the original network parameters according to the quantization parameters in the initial quantization configuration table to obtain quantization parameter data; performing difference calculation between the original network parameters and the quantization parameter data to obtain quantization error data; Perform statistical analysis on the quantized error data to generate error statistics; S32, based on the error statistical data and the pre-stored network structure data, construct a directed graph representing the error propagation path to obtain error propagation path data; based on the error propagation path data and the quantized error data, calculate the propagation effect of the error in the network to obtain error influence data; combine the error influence data of each layer to generate a cumulative error matrix; S33, based on the cumulative error matrix and the pre-stored compensation model parameters, calculating the error compensation value required for each layer to obtain an error compensation matrix; Based on the error compensation matrix, the location and characteristics of the network layer are obtained, a compensation application strategy is constructed, and compensation application data is generated; the error compensation matrix and the compensation application data are combined to generate a compensation strategy table.

4. The quantization training method for a deep neural network according to claim 3, characterized in that: Step S4 is further as follows: S41, obtaining the forward quantization parameters in the initial quantization configuration table, quantizing the network parameters and the intermediate activation values, and obtaining the forward quantization data; obtaining the forward compensation strategy in the compensation strategy table, performing error compensation on the forward quantization data, and obtaining the forward compensation data; Perform forward propagation calculation according to the forward compensation data to generate a forward propagation result; S42, obtaining the inverse quantization parameter in the initial quantization configuration table, quantizing the gradient data, and obtaining inverse quantization data; Obtain the reverse compensation strategy in the compensation strategy table, perform error compensation on the reverse quantization data, and obtain reverse compensation data; Perform back propagation calculation according to the back compensation data to generate a back propagation result; S43. Based on the forward propagation result and the back propagation result, the performance index of the current training batch is calculated to obtain batch performance data; Based on the batch performance data and the error compensation matrix, the compensation coefficient is updated to obtain updated compensation data; the updated compensation data and the performance index are combined to generate quantitative model parameters and training status indicators.

5. The quantization training method for a deep neural network according to claim 4, characterized in that: Step S6 is further as follows: S61. Based on the optimized model parameters, run model reasoning on the pre-stored test data set to obtain reasoning result data; obtain label information of the test data set, compare the reasoning result data with the label information, calculate the accuracy index, and generate accuracy statistics; Based on the optimized model parameters, obtain the quantization parameters of each layer of the model, calculate the error introduced by quantization, and generate quantization error statistics; S62. Based on the inference result data, analyze the fluctuation of the model output under different input data to obtain output fluctuation data; Based on the optimized model parameters, the quantization parameters and intermediate activation values ​​of each layer of the model are obtained, the quantization sensitivity is analyzed, and the quantization sensitivity data is obtained; Combine output fluctuation data and quantitative sensitivity data to generate stability assessment data; S63, obtaining computing time, memory usage and power consumption data during the model reasoning process, and generating actual resource consumption data; The actual resource consumption data is compared with the pre-stored benchmark efficiency data to obtain efficiency evaluation data; the accuracy statistics, quantization error statistics, stability evaluation data and efficiency evaluation data are combined to generate model performance indicators and resource consumption reports.

6. The quantization training method for a deep neural network according to claim 5, characterized in that: Step S11 is further as follows: S111, obtaining the original neural network model, extracting the calculation graph structure of each layer, and generating hierarchical calculation graph data; obtaining a verification data set, dividing the data in the verification data set into batches, and generating batch verification data; Combine the hierarchical computation graph data and batch validation data, use an adaptive sampling strategy to select a representative sample subset, and generate sampling validation data; Perform bit width encoding on the network parameters of each layer in the hierarchical computation graph data to generate bit width encoded data; S112, based on the sampled verification data and the bit width encoded data, performing quantization operations of different bit widths on each layer of the network in turn to generate multi-bit width quantized data; Input multi-bit width quantized data into the network for forward calculation, record the output value of each layer, and generate quantized layer output data; Based on the original neural network model and validation dataset, obtain the layer output at the original accuracy; Compare the quantized layer output data with the layer output under the original accuracy, calculate the relative error, and generate the quantization loss change rate data; S113, based on the sampled verification data, perform forward and backward propagation calculations on the network, record the gradient value of each layer, and generate original gradient data; perform time series statistical analysis on the original gradient data, extract gradient change trend characteristics, and generate gradient trend data; Combine the original gradient data and gradient trend data, calculate the stability index of each layer of gradient, and generate gradient amplitude data; S114, based on the sampled verification data, recording the activation value distribution of each layer of the network under different inputs, generating activation distribution data; performing probability density estimation on the activation distribution data, extracting distribution feature parameters, and generating distribution feature data; Combine the activation distribution data and the distribution characteristic data, calculate the dynamic range and discreteness of the activation value, and generate the activation range data; S115. Based on the quantization loss change rate data, gradient amplitude data and activation range data, a feature vector is constructed to generate sensitivity feature data; based on the pre-stored feature weight parameters, the sensitivity feature data is weighted to generate weighted feature data; the weighted feature data is nonlinearly mapped and normalized to generate hierarchical sensitivity index data.

7. The quantization training method for a deep neural network according to claim 5, characterized in that: Step S23 is further as follows: S231, based on the constrained quantization parameter table, extract the quantization parameter configuration of each layer to generate layer configuration data; based on the pre-stored error propagation model parameters, construct an error propagation prediction model to generate propagation prediction data; Based on the layer configuration data and propagation prediction data, the error propagation characteristics of each layer are calculated to generate error coefficient data; S232, based on the error coefficient data, calculating the error tolerance threshold of each layer, generating error threshold data; constructing a dynamic compensation mechanism based on the error threshold data, generating compensation mechanism data; Combining the error coefficient data and the compensation mechanism data, generating compensation parameters for each layer, and obtaining error compensation configuration data; S233, extracting quantization configuration information based on the constraint quantization parameter table and generating configuration basic data; Based on the error compensation configuration data, the compensation strategy is integrated to generate compensation integration data; Combining configuration basic data and compensation integration data to generate configuration combination data; S234. Construct a parameter index structure based on the configuration combination data and generate index structure data; Based on the index structure data and the error compensation configuration data, a complete configuration table item is constructed to generate configuration table item data; the configuration table item data is integrated into a final configuration table to generate an initial quantization configuration table.

8. A quantized training system for deep neural networks, characterized in that: include: at least one processor; as well as, a memory communicatively connected to at least one of the processors; wherein, The memory stores instructions that can be executed by the processor, and the instructions are used to be executed by the processor to implement the quantization training method for a deep neural network as described in any one of claims 1 to 7.

Images