Quantization-aware training method and device of neural network model, electronic equipment and storage medium

Abstract

The application relates to a quantization-aware training method and device of a neural network model, electronic equipment and a storage medium. The method comprises the following steps: acquiring a first neural network model to be trained; identifying an operator pair and a non-module operator in the first neural network model, wherein the non-module operator is an operation operation not based on a module class implementation, and the operator pair comprises a connected convolution operator and a batch normalization operator; encapsulating the non-module operator into a module operator and encapsulating the operator pair into a new convolution operator to obtain a second neural network model, wherein the new convolution operator runs a fusion process of the convolution operator and the batch normalization operator in forward propagation, and parameters of the convolution operator and the batch normalization operator are updated respectively in backward propagation; and performing quantization-aware training on the second neural network model. The method can guarantee the inference accuracy of a quantization model while improving the deployment efficiency of the quantization model.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a quantitative perception training method, apparatus, electronic device, and storage medium for a neural network model. Background Technology

[0002] With the rapid development of artificial intelligence technology, deep learning models, represented by convolutional neural networks, have continuously improved inference accuracy, but model size and computational complexity have also increased simultaneously. While high-precision floating-point models (such as FP32 / FP16) can guarantee inference performance, they suffer from high computational resource consumption and high inference latency, making them unsuitable for deployments requiring low computing power and low latency in applications such as automotive terminals and embedded devices. To balance model accuracy and deployment efficiency, Quantization-Aware Training (QAT) technology is typically used to optimize models for low-bit quantization. This involves inserting pseudo-quantization and dequantization processes during the training phase to simulate low-bit (such as INT8) quantization errors, which can reduce model storage usage and computational overhead while maintaining accuracy.

[0003] However, existing quantization-aware training schemes still have significant limitations: First, they do not provide comprehensive support for quantization of operators in the model, making it difficult to achieve full coverage when facing different types of operators. This results in unquantized computational paths in the model, frequent switching between integer and floating-point formats during inference, increasing format conversion and memory access overhead, and affecting the consistency of quantization computation. Second, to adapt to deployment requirements, convolution and batch normalization operators are merged during training, which loses the balancing effect of batch normalization on data distribution during training, leading to a decrease in model training stability. Third, when statistically quantizing key parameters (such as scaling factor and zero point), the scope of data utilization is narrow, making it susceptible to the limited amount of training data and large data fluctuations, which in turn interferes with the stability of training. This makes it difficult for the final quantized model to achieve the expected inference accuracy, ultimately affecting the inference accuracy and deployment efficiency of the quantized model. Summary of the Invention

[0004] Based on this, it is necessary to address the aforementioned technical problems by providing a quantization-aware training method, device, electronic device, and storage medium for neural network models. This method can achieve full operator quantization coverage to ensure the continuity of quantization calculations. During the training phase, it obtains quantization information adapted to deployment requirements through the fused new convolution operators while retaining the balancing effect of the batch normalization operator on data distribution to maintain model training stability. Furthermore, the full operator quantization coverage constructs a coherent quantization calculation chain, enabling the statistics of key quantization parameters during quantization-aware training to be based on the full-chain data. This expands the scope of data utilization and reduces data fluctuation interference, thereby improving the deployment efficiency of the quantization model while ensuring its inference accuracy.

[0005] According to a first aspect of certain exemplary embodiments of the present disclosure, a quantization-aware training method for a neural network model is provided, comprising: acquiring a first neural network model to be trained; identifying operator pairs and non-module operators in the first neural network model, wherein the non-module operators are operations not implemented based on module classes, and the operator pairs include connected convolution operators and batch normalization operators; encapsulating the non-module operators into module operators and encapsulating the operator pairs into new convolution operators to obtain a second neural network model, wherein the new convolution operators perform a fusion process of convolution operators and batch normalization operators during forward propagation, and update the parameters of convolution operators and batch normalization operators respectively during back propagation; and performing quantization-aware training on the second neural network model.

[0006] According to certain exemplary embodiments of the present disclosure, identifying operator pairs in a first neural network model includes: identifying operator pairs in a first neural network model using a computation graph search-based approach; and / or, identifying operator pairs in a first neural network model using a rule matching-based approach.

[0007] According to certain exemplary embodiments of the present disclosure, identifying operator pairs in a first neural network model includes: if a computation graph is constructed based on the first neural network model, then identifying operator pairs in the first neural network model using a computation graph search method; if a computation graph cannot be constructed based on the first neural network model, then identifying operator pairs in the first neural network model using a rule matching method.

[0008] According to certain exemplary embodiments of this disclosure, an operator pair in a first neural network model is identified using a computation graph search-based method, including: constructing a computation graph of the first neural network model and locating multiple convolution operator nodes marked as convolution operators in the computation graph; traversing the downstream connection relationships of each convolution operator node, and if it is identified that the downstream connection of any convolution operator node is only a batch normalization operator, then determining that the convolution operator of any convolution operator node and the corresponding batch normalization operator constitute an operator pair; and / or, an operator pair in the first neural network model is identified using a rule matching method, including: obtaining the naming information of each operator in the first neural network model; and determining, according to a preset naming convention and the naming information of each operator, that the convolution operator whose name satisfies the matching relationship and the corresponding batch normalization operator constitute an operator pair.

[0009] According to certain exemplary embodiments of this disclosure, quantization-aware training of a second neural network model includes: in iterative training of quantization-aware training, acquiring the tensors of floating-point numbers output by each network layer of the second neural network model; calculating the maximum and minimum values ​​of the floating-point number tensors using an exponential averaging method, and calculating a scaling factor and zero point based on the maximum and minimum values, and quantizing the floating-point number to an integer based on the scaling factor and zero point; or, obtaining the maximum and minimum values ​​of the tensors of floating-point numbers output by each network layer of the second neural network model in multiple iterative training of quantization-aware training, performing aggregated statistical processing on the maximum and minimum values ​​within the window using a sliding window statistical method, calculating a scaling factor and zero point, and quantizing the floating-point number to an integer based on the scaling factor and zero point.

[0010] According to certain exemplary embodiments of this disclosure, a quantization-aware training method for a neural network model further includes: after quantization-aware training is completed, acquiring first output data of each network layer in the second neural network model that has not been processed by pseudo-quantization operators and dequantization operators, and second output data of each network layer after processing by pseudo-quantization operators and dequantization operators; calculating quantization error index values ​​of each network layer in the second neural network model based on the first output data and the second output data, wherein the quantization error index values ​​include one or more of mean square error, mean absolute error, cosine similarity, signal-to-noise ratio, and saturation rate, and the saturation rate is the proportion of values ​​falling on the boundary of the quantization interval in the output data of the corresponding network layer after processing by pseudo-quantization operators; determining the network layer whose quantization error index value exceeds a preset quantization error threshold as a quantization-sensitive layer; and performing quantization accuracy backoff processing on the quantization-sensitive layer, and then performing a new round of quantization-aware training on the second neural network model after the quantization accuracy backoff processing.

[0011] According to certain exemplary embodiments of this disclosure, a method for quantization perception training of a neural network model further includes: when performing quantization perception training on a second neural network model, using a first neural network model as a teacher model and keeping the model parameters of the teacher model unchanged, using the second neural network model as a student model, and performing noise matching on the student model through the teacher model, so that the quantization noise distribution generated by each network layer in the student model after processing by pseudo-quantization operators and inverse quantization operators tends to be consistent with the output noise distribution of the teacher model.

[0012] According to certain exemplary embodiments of this disclosure, a quantization-aware training method for a neural network model further includes: during the quantization-aware training and quantization precision rollback process of a second neural network model, recording the quantization parameters, performance index values, and precision rollback history of each network layer in the second neural network model; wherein, the quantization parameters include one or more of the following: scaling factor, zero point, maximum and minimum values ​​of floating-point tensors, and update history of each network layer; the update history includes the timestamp corresponding to the quantization parameter adjustment and / or the iteration training number during each iteration training; the performance index values ​​include one or more of the following of each network layer: signal-to-noise ratio, saturation rate, mean square error, and cosine similarity; and the precision rollback history includes one or more of the following: the network layer identifier of the quantization-sensitive layer, the execution time of the quantization precision rollback process, the quantization precision before rollback, the quantization precision after rollback, and the reason for triggering the quantization precision rollback process.

[0013] According to a second aspect of certain exemplary embodiments of the present disclosure, a quantization-aware training apparatus for a neural network model is provided, characterized in that the apparatus comprises: a first acquisition module for acquiring a first neural network model to be trained; an identification module for identifying operator pairs and non-module operators in the first neural network model, wherein the non-module operators are operations not implemented based on module classes, and the operator pairs include connected convolution operators and batch normalization operators; an encapsulation module for encapsulating non-module operators into module operators and encapsulating operator pairs into new convolution operators to obtain a second neural network model, wherein the new convolution operators perform a fusion process of convolution operators and batch normalization operators during forward propagation, and update the parameters of convolution operators and batch normalization operators respectively during back propagation; and a training module for performing quantization-aware training on the second neural network model.

[0014] According to a third aspect of certain exemplary embodiments of the present disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above methods.

[0015] According to a fourth aspect of certain exemplary embodiments of the present disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0016] The aforementioned quantization-aware training method, apparatus, electronic device, and storage medium for a neural network model acquire a first neural network model to be trained; identify operator pairs and non-modular operators in the first neural network model, wherein non-modular operators are operations not implemented based on module classes, and operator pairs include connected convolution operators and batch normalization operators; encapsulate non-modular operators into modular operators and operator pairs into new convolution operators to obtain a second neural network model, wherein the new convolution operators perform a fusion process of convolution operators and batch normalization operators during forward propagation, and update the parameters of convolution operators and batch normalization operators respectively during back propagation; and perform quantization-aware training on the second neural network model.

[0017] Therefore, by encapsulating non-modular operators into modular operators, full operator quantization coverage is achieved, avoiding frequent format conversions and memory access overhead during inference and ensuring the continuity of quantization computation. Furthermore, by encapsulating the operator pair consisting of convolution operators and batch normalization operators into a new convolution operator, this new convolution operator performs a fusion process during the forward propagation of quantization-aware training to obtain quantization information suitable for deployment, while retaining the batch normalization operator parameter updates during backpropagation to maintain training stability. Simultaneously, full operator quantization coverage constructs a coherent quantization computation chain, allowing the statistics of key quantization parameters during quantization-aware training to be based on the entire chain of data, expanding the scope of data utilization, reducing data fluctuation interference, and further ensuring training stability and parameter accuracy. Ultimately, this approach effectively reduces model storage and computational overhead while ensuring the inference accuracy of the quantized model (i.e., the second neural network model after quantization-aware training), thereby improving the deployment efficiency of the quantized model. Attached Figure Description

[0018] Figure 1 This is a schematic diagram illustrating the application environment of a quantitative perception training method for a neural network model, as shown in some exemplary embodiments of this disclosure.

[0019] Figure 2 This is a flowchart illustrating a quantitative perception training method for a neural network model, as shown in some exemplary embodiments of this disclosure.

[0020] Figure 3 This is a schematic diagram of the quantization precision rollback process in some exemplary embodiments of this disclosure;

[0021] Figure 4 This is a structural block diagram of a quantization perception training device for a neural network model in some other exemplary embodiments of the present disclosure;

[0022] Figure 5 This is a diagram illustrating the internal structure of an electronic device in some other exemplary embodiments of this disclosure. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] The following detailed descriptions are provided to aid the reader in gaining a comprehensive understanding of the methods, apparatus, electronic devices, storage media, and / or computer program products described herein. However, upon understanding the disclosure of this disclosure, various changes, modifications, and equivalents of the methods, apparatus, storage media, and / or computer program products described herein will become apparent. For example, the order of operations described herein is merely illustrative and is not limited to those orders set forth herein, but may be changed as will become clear upon understanding the disclosure of this disclosure, except for operations that must occur in a specific order. Furthermore, for clarity and conciseness, descriptions of features known in the art may be omitted.

[0025] The features described herein may be implemented in different forms and should not be construed as limited to the examples described herein. Rather, the examples described herein are provided only to illustrate some of the many feasible ways of implementing the methods, electronic devices, and / or storage media described herein, many of which will become clear upon understanding this disclosure.

[0026] The terminology used herein is for the purpose of describing various examples only and is not intended to limit disclosure. Unless the context clearly indicates otherwise, the singular form is intended to include the plural form as well. The terms “comprising,” “including,” and “having” indicate the presence of the described features, quantities, operations, components, elements, and / or combinations thereof, but do not exclude the presence or addition of one or more other features, quantities, operations, components, elements, and / or combinations thereof. Unless otherwise stated, “ / ” means “or,” for example, A / B can mean A or B; “and / or” in the text is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can mean: A alone, A and B simultaneously, and B alone. Furthermore, in the description of embodiments of the invention, “multiple” means two or more.

[0027] Unless otherwise defined, all terms used herein (including technical and scientific terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains upon understanding this disclosure. Unless expressly defined herein, terms (such as those defined in a general dictionary) shall be interpreted as having a meaning consistent with their meaning in the context of the relevant field and in this disclosure, and shall not be interpreted in an idealized or overly formalistic manner.

[0028] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in some of the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0029] Furthermore, in the description of the examples, detailed descriptions of well-known related structures or functions will be omitted when it is believed that such detailed descriptions would lead to a vague interpretation of this disclosure.

[0030] In the following description, embodiments will be described in detail with reference to the accompanying drawings. However, embodiments may be implemented in various forms and are not limited to the examples described herein.

[0031] The abbreviations and key terms in this disclosure are explained as follows:

[0032] 1. Quantization-Aware Training (QAT): One of the core technologies for low-bit quantization optimization of deep learning models. Its core idea is to insert pseudo-quantization operators and dequantization operators in advance during the training phase for high-precision floating-point models (such as FP32 and FP16 models). During training, the pseudo-quantization operator simulates the numerical conversion process of low-bit integer (such as INT8) quantization, mapping floating-point data to analog integer format to introduce quantization error. The dequantization operator then converts the analog integer back to floating-point format, ensuring that the model can continue to complete subsequent training iterations with floating-point precision, and finally outputs a low-bit quantized model that can be directly deployed.

[0033] 2. Convolutional Operator: Also known as Conv operator, it is the core computational unit of Convolutional Neural Network (CNN). Essentially, it performs sliding convolution calculations on the input feature map through a pre-defined convolution kernel (Filter) to extract local features from data such as images and speech.

[0034] 3. Batch Normalization Operator (BN): This is a commonly used operator in deep learning models to improve model training stability and accelerate convergence. Its core function is to normalize the feature dimensions of a single batch of data.

[0035] 4. Functional Operator: Also known as a Function Operator, it is an operator in a deep learning model that implements computational logic in the form of function calls. These operators do not have independent instantiation objects and do not contain learnable parameters. Their computation process is entirely determined by the input data and function parameters. Common examples include basic operations such as addition, multiplication, and tensor concatenation (Concat).

[0036] 5. Scale factor: This is a key parameter for ensuring quantization accuracy in low-bit quantization. It needs to be calculated separately for each network layer and is used to establish the mapping relationship between floating-point data and integer data. Essentially, it is the ratio of the floating-point data range to the integer data range.

[0037] 6. Zero Point: Abbreviated as Zero Point, it is an auxiliary parameter used in low-bit quantization to handle asymmetric quantization scenarios. Essentially, it is the integer corresponding to the "0 value" of floating-point data in integer data, ensuring that the "floating-point 0 value" can be accurately mapped to an integer during the quantization process, reducing quantization error.

[0038] In some exemplary embodiments of this disclosure, such as Figure 1 The diagram illustrates an application environment for a quantization-aware training method for a neural network model. The quantization-aware training system 200 executes this method. Specifically, it acquires a first neural network model to be trained; identifies operator pairs and non-modular operators in the first neural network model, where non-modular operators are operations not implemented based on module classes, and operator pairs include connected convolution operators and batch normalization operators; encapsulates non-modular operators into modular operators and operator pairs into new convolution operators to obtain a second neural network model. During forward propagation, the new convolution operators perform a fusion process of the convolution operators and batch normalization operators, and during backpropagation, update the parameters of the convolution operators and batch normalization operators respectively; and performs quantization-aware training on the second neural network model. The quantization-aware trained second neural network model can be deployed on a terminal device 100, which can be an in-vehicle terminal, a mobile phone, a tablet computer, etc.

[0039] In some exemplary embodiments of this disclosure, such as Figure 2 As shown, a quantization-based perceptual training method for a neural network model is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0040] Step 201: Obtain the first neural network model to be trained.

[0041] Specifically, a first neural network model to be trained is obtained. This first neural network model can be built based on a deep learning framework (such as PyTorch or TensorFlow) and includes basic network structures such as convolutional layers, batch normalization layers, and activation layers. All layer operations are performed in high-precision floating-point format (such as FP32 or FP16), which is suitable for deep learning tasks such as image classification and object detection. However, it has the problems of high computational resource consumption and high inference latency. It needs to be adapted to low-computing-power deployment scenarios (such as vehicle terminals and embedded devices) through quantization optimization.

[0042] Step 202: Identify operator pairs and non-modular operators in the first neural network model.

[0043] Among them, non-module operators are operations that are not implemented based on module classes, and operator pairs include convolution operators and batch normalization operators.

[0044] Specifically, by scanning and analyzing the structure of the first neural network model, operator pairs and non-modular operators in the model were identified. Non-modular operators mainly include functional operators and some low-level operations not encapsulated in modular form. These operators typically participate in computation through direct function calls, such as addition, multiplication, and concatenation. Simultaneously, specific operator pairs tightly connected to convolution and batch normalization operators were identified in the model. This accurate identification of operator pairs and non-modular operators in the first neural network model lays the foundation for subsequent differentiated processing of operator pairs and non-modular operators.

[0045] In one example, the specific implementation of identifying operator pairs in the first neural network model may include the following steps: identifying operator pairs in the first neural network model using a computation graph search-based method; and / or, identifying operator pairs in the first neural network model using a rule matching-based method.

[0046] Specifically, operator pairs in the first neural network model can be identified through computation graph search and / or rule matching. The core of computation graph search is to analyze the computation graph structure of the model and identify directly related operator combinations by tracing the data flow connections between operators. Rule matching, on the other hand, relies on pre-defined specifications during model training and construction (such as operator naming conventions and module partitioning rules) to filter operator pairs that meet the association criteria by matching operator attribute information (such as name and module identifier). These two methods can be used independently or in combination, flexibly adapting to neural network models with different structures and construction specifications to efficiently identify operator pairs.

[0047] In one example, the specific implementation of identifying operator pairs in the first neural network model may include the following steps: if a computation graph is constructed based on the first neural network model, then the operator pairs in the first neural network model are identified using a computation graph search method; if a computation graph cannot be constructed based on the first neural network model, then the operator pairs in the first neural network model are identified using a rule matching method.

[0048] Specifically, to comprehensively and accurately identify all potential operator pairs in the first neural network model and avoid omissions due to the limitations of a single identification method, a progressive identification strategy prioritizing computational graph search and supplementing with rule matching can be adopted. If a computational graph can be constructed based on the first neural network model, computational graph search is prioritized to identify one or more operator pairs with clear connections in the model. These operator pairs can be quickly located using the node connection information of the computational graph. However, if a computational graph cannot be constructed based on the first neural network model, identification is performed using rule matching. For example, based on preset rules such as operator naming conventions followed during model construction (e.g., operators within the same functional module sharing the same sequence number or prefix) and module partitioning logic, the attribute information of operators (such as name and module identifier) ​​can be matched to identify operator pairs that conform to preset association rules. In some embodiments, when some operator pairs in the first neural network model are not identified by computation graph search due to the complexity of the computation graph structure of the first neural network model, the omission of edge scenes during graph analysis, or the existence of indirect relationships between operators, a rule-based matching method can be used for supplementary identification. This combined method of identifying operator pairs can efficiently identify explicitly related operator pairs by leveraging the structured analysis of computation graph search, and can also cover related scenes under complex structures through rule matching, ultimately achieving complete identification of operator pairs in the model.

[0049] Based on the above embodiments, the specific implementation methods for identifying operator pairs in the first neural network model using a computation graph search-based approach and / or a rule-matching-based approach are further described. Specifically, identifying operator pairs in the first neural network model using a computation graph search-based approach includes: constructing a computation graph of the first neural network model and locating multiple convolution operator nodes marked as convolution operators in the computation graph; traversing the downstream connections of each convolution operator node; if any convolution operator node's downstream connection is identified as only a batch normalization operator, then determining that the convolution operator of any convolution operator node and its corresponding batch normalization operator constitute an operator pair; and / or, identifying operator pairs in the first neural network model using a rule-matching-based approach includes: obtaining the naming information of each operator in the first neural network model; and, according to a preset naming convention and the naming information of each operator, determining that convolution operators whose names satisfy the matching relationship and their corresponding batch normalization operators constitute an operator pair.

[0050] Specifically, when identifying operator pairs using a computation graph search approach, the computation graph of the first neural network model is first constructed using tools such as torch.jit.trace. This computation graph contains different operator nodes and the connections between them. All nodes labeled as convolution operators are accurately located within the computation graph. Then, the downstream connections of each convolution operator node are traversed. If the output of any convolution operator node is found to be connected to only a single operator node, and the connected operator node is a batch normalization operator, then the convolution operator and its corresponding connected batch normalization operator constitute an operator pair. When identifying operator pairs using a rule-based matching approach, the naming information of each operator in the first neural network model is first obtained, including complete hierarchical naming and type identification. Then, according to preset naming conventions (such as convolution operators and batch normalization operators having the same sequence number or sharing a parent module prefix), convolution operators and batch normalization operators that satisfy the matching relationship are selected from the operator naming information to form operator pairs. Therefore, the computation graph search approach, by tracing the data flow and operator connection relationships, can rigorously identify continuously deployed operator pairs, ensuring the rigor of the identification logic; the rule matching approach, through named association, enables rapid filtering, flexibly handling complex scenarios of computation graph parsing and ensuring the efficiency of the identification process. The two approaches identify operator pairs from different dimensions, jointly ensuring the comprehensiveness and accuracy of the identification results.

[0051] Step 203: Encapsulate the non-modular operators into modular operators and encapsulate the operator pairs into new convolution operators to obtain the second neural network model.

[0052] The new convolution operator performs the fusion process of the convolution operator and the batch normalization operator during forward propagation, and updates the parameters of the convolution operator and the batch normalization operator respectively during back propagation.

[0053] Specifically, the operator pairs and non-modular operators identified in the above steps are encapsulated separately. During the encapsulation of non-modular operators in the first neural network model, corresponding modular classes need to be constructed based on the original computational logic of the non-modular operators, such as the numerical accumulation rules of the addition operator and the dimension concatenation order of the Concat operator. For example, the torch.add function can be encapsulated into the AddModule class, and the torch.cat function into the ConcatModule class. The computational logic of the encapsulated modular type operators is exactly the same as that of the corresponding non-modular type operators. That is, for the same input data, the output result of the encapsulated modular operators is completely consistent with the original non-modular operators, thus ensuring that the model functionality is not affected before and after the conversion.

[0054] Simultaneously, the encapsulated modular operators need to provide standardized quantization interfaces, including parameter storage structures and forward hooks, to facilitate the subsequent insertion of pseudo-quantization or dequantization operators at appropriate locations. This step is crucial for achieving Quantization-Aware Training (QAT), enabling the integration of functional operators that originally did not support quantization into a unified quantization system. This allows them to possess configurable quantization and dequantization capabilities, similar to existing modular operators such as convolution and batch normalization. By encapsulating non-modular operators into modular operators, full operator quantization coverage can be achieved, avoiding additional format conversion and memory access overhead during model inference due to the presence of unquantized operators. This ensures the continuity of quantization computation and improves the deployment efficiency of quantized models.

[0055] Furthermore, this step requires encapsulating the identified operator pairs into new convolution operators. During forward propagation, these encapsulated new convolution operators perform a fusion operation of convolution computation and batch normalization, completing the entire process of feature extraction and data distribution normalization in one go. During backward propagation, these encapsulated new convolution operators maintain an independent update mechanism for the parameters of the original convolutional and batch normalization layers, calculating and updating the parameters of the original convolution operators and the original batch normalization operators separately. This design, while gaining the advantage of accelerated forward computation, fully preserves the balancing effect of the batch normalization layer on data distribution during training, thus ensuring that the stability and convergence of model training are not affected. Simultaneously, full operator quantization coverage constructs a coherent quantization computation chain, enabling the statistical analysis of key quantization parameters during subsequent quantization-aware training to be based on the full-chain data, expanding the scope of data utilization, reducing data fluctuation interference, and further ensuring the stability of training and the accuracy of parameters.

[0056] Step 204: Perform quantization perception training on the second neural network model.

[0057] Specifically, when performing quantization-aware training on the second neural network model, pseudo-quantization operators and dequantization operators are first deployed in pairs in the connection paths of adjacent module operators and / or new convolution operators to construct the second neural network model for quantization-aware training. The pseudo-quantization operator converts the floating-point format output data of module type operators or fused convolution operators into floating-point format data of analog integers according to a preset quantization rule, and the dequantization operator converts the floating-point format data of analog integers into floating-point format data according to a preset dequantization rule. For example, the preset quantization rule and the preset dequantization rule can be expressed by the following formula:

[0058] ,

[0059] ;

[0060] In this context, Equation (1) is the formula for the preset quantization rule, Equation (2) is the formula for the preset inverse quantization rule, q is the fixed-point integer after quantization, r is the original floating-point format data output by the module type operator or the fusion convolution operator, S is the quantization scaling factor, Z is the zero point, and round represents the rounding operation.

[0061] Then, based on the second neural network model constructed above, the quantization perception training process is initiated. Specifically, in each training iteration, the model first calculates the output results of the module operators and / or new convolution operators of each network layer after processing by pseudo-quantization and dequantization operators through forward propagation. Then, based on the output results, the model parameters (such as model weights, parameters of convolution operators and batch normalization operators in new convolution operators) and quantization parameters (such as scaling factor S, zero point Z) of each network layer are dynamically updated during backpropagation to achieve collaborative optimization of model parameters and quantization parameters. Among them, during the forward propagation process, the pseudo-quantization operator and dequantization operator will simulate the quantization and dequantization process according to the above formulas (1) and (2). First, based on the original floating-point data r output by the module operators and / or new convolution operators, it is converted into floating-point format data of simulated integers by the pseudo-quantization operator according to formula (1). Then, the dequantization operator restores it to floating-point format data according to formula (2) and continues to pass it backward. For example, assuming the current quantization mode is symmetric quantization in INT8 format (e.g., integer range [-128, 127], corresponding floating-point range [-5.0, 5.0], zero point Z is 0), the quantization scaling factor can be calculated. Taking the original floating-point data r output by the module operator and / or the new convolution operator as 2.3 as an example, in the above INT8 symmetric quantization scenario, the pseudo-quantization operator processes the original floating-point data r according to equation (1) to calculate the quantized INT8 integer value q as 58, that is... Then, q is converted into floating-point data 58.0 of analog integers. By actively introducing quantization error, the pseudo-quantization operator allows the model to adapt to the numerical characteristics of low-bit computation in advance during the training phase, avoiding performance degradation due to sudden changes in precision during subsequent quantization deployment. Next, the dequantization operator, based on the same set of S and Z, reversely converts the floating-point data q of analog integers into high-precision floating-point data r according to equation (2), that is, While the result shows a slight error compared to the original FP32 data 2.3, it fully preserves the numerical characteristics of the quantization process. The dequantization operator, through format conversion, allows subsequent operators to still perform calculations with FP32 precision, avoiding gradient truncation or precision loss caused by integer data directly participating in gradient propagation, thus ensuring the continuity of the gradient flow during training. At the same time, by preserving quantization error, the model can specifically optimize its robustness to quantization noise when updating parameters, ultimately achieving the goal of being aware of the impact of quantization during training and directly using low-bit computation during deployment.

[0062] Simultaneously, during the backpropagation phase, based on the total loss (including task loss and quantization-related loss) and the output results of the module operators and / or new convolution operators of each network layer after processing by pseudo-quantization and dequantization operators, the model parameters (such as model weights, parameters of convolution operators and batch normalization operators in new convolution operators) and quantization parameters (such as scaling factor S and zero point Z) of each network layer are calculated and updated layer by layer. In this way, the second neural network model continuously learns the compensation mechanism for quantization errors during quantization-aware training, ultimately achieving an optimal balance between model parameters and quantization parameters (S, Z), ensuring that the quantized model maintains high accuracy while meeting the efficiency requirements of low-bit integer operations during deployment.

[0063] In one example, the specific implementation of quantization-aware training of the second neural network model may include the following steps: In the iterative training of quantization-aware training, the tensors of floating-point numbers output by each network layer of the second neural network model are collected; the maximum and minimum values ​​of the floating-point tensors are calculated using an exponential averaging method, and the scaling factor and zero point are calculated based on the maximum and minimum values, and the floating-point number is quantized to an integer based on the scaling factor and zero point; or, the maximum and minimum values ​​of the tensors of floating-point numbers output by each network layer of the second neural network model are obtained in multiple iterations of quantization-aware training, the maximum and minimum values ​​within the window are aggregated and statistically processed using a sliding window statistical method, the scaling factor and zero point are calculated, and the floating-point number is quantized to an integer based on the scaling factor and zero point.

[0064] In this embodiment, the core of the iterative training of the second neural network model for quantization perception training is to accurately statistically analyze the distribution of data and dynamically calculate quantization parameters, allowing the model to adapt to quantization constraints during training. Specifically, each iteration first collects the floating-point tensors output by each network layer of the second neural network model, using this as the basis for data statistics. Then, according to a preset strategy, the maximum and minimum values ​​of the floating-point tensors are calculated using exponential averaging or sliding window statistics. Based on the maximum and minimum values, the scaling factor and zero point are calculated, providing a basis for quantization parameter calculation. For example, if the exponential averaging method is used, firstly, the floating-point tensors output by the batch data of the current iteration training after passing through each network layer of the second neural network model are collected. Combined with the maximum and minimum value data of the floating-point tensors of the same layer stored in the historical iterations, according to the preset decay coefficient (usually 0.9-0.999, such as 0.99), for each network layer, the maximum and minimum values ​​of the floating-point tensors are obtained by weighted fusion of the maximum floating-point value statistically obtained in the current iteration training and the maximum exponential average value accumulated in the historical iteration training, and by weighted fusion of the minimum floating-point value statistically obtained in the current iteration training and the minimum exponential average value accumulated in the historical iteration training. For example, if the historical average maximum value of any network layer is 8.5, the current batch maximum value is 9.2, and the decay coefficient is 0.99, then the maximum value of the updated floating-point tensor is 0.99 × 8.5 + (1 - 0.99) × 9.2 = 8.507; if the historical average minimum value is -3.2, and the current batch minimum value is -3.5, then the minimum value of the updated floating-point tensor is 0.99 × (-3.2) + (1 - 0.99) × (-3.5) = -3.203. Alternatively, if a sliding window statistical method is used, a buffer with a window size of N (N is a positive integer greater than 1, such as 8, 16, etc., which can be adjusted according to the batch characteristics of model training) needs to be set to store the maximum and minimum values ​​of the floating-point tensors output by each network layer in the most recent N iterations of training. In each iteration of training, floating-point tensors of each network layer in the current batch are first collected and stored in the corresponding buffer according to the network layer. If the number of iterations stored in the buffer has reached N, the maximum and minimum values ​​of the floating-point tensor from the earliest iteration are removed before adding the data from the current batch. This ensures that the buffer always retains only the iteration data from the most recent N iterations, achieving dynamic window sliding. Furthermore, for each network layer, a global aggregation and statistical analysis is performed on all floating-point tensors within the window in the buffer: all tensor data from N iterations are traversed, and the maximum and minimum values ​​are compared and selected.For example, if the window size N=8 for any network layer, and the maximum floating-point tensor values ​​for the 8 iterations within the window are 6.3, 6.5, 6.7, 6.4, 6.6, 6.8, 6.5, and 6.9 respectively, then the maximum floating-point tensor value within the window after aggregation is 6.9; and the minimum floating-point tensor values ​​for the 8 iterations are -1.2, -1.3, -1.1, -1.4, -1.2, -1.5, -1.3, and -1.2 respectively, then the minimum floating-point tensor value within the window after aggregation is -1.5. The maximum and minimum floating-point tensor values ​​calculated using the exponential averaging or sliding window statistical methods avoid statistical biases caused by small data volumes and large distribution fluctuations when relying solely on the current batch of data for statistical quantization. This provides a more comprehensive reflection of the overall data distribution characteristics of each network layer during training, offering a stable and reliable basis for subsequent quantization parameter calculations, thereby further ensuring the stability of the training process.

[0065] Finally, based on the maximum and minimum values ​​of the floating-point tensors calculated using the exponential averaging or sliding window statistical methods described above, the scaling factor and zero point in the current iteration of training are calculated, thereby quantizing the floating-point numbers to integers according to the scaling factor and zero point. Specifically, firstly, based on the quantization bit depth (e.g., 8-bit, 16-bit), quantization mode (e.g., symmetric or asymmetric quantization), and the integer types supported by the hardware, the preset integer format range for each network layer is determined, i.e., the maximum and minimum values ​​of the discrete data in the preset integer format for each network layer. For example, for INT8 symmetric quantization, to avoid overflow of positive data, the integer range is [-127, 127], while for INT8 asymmetric quantization, the full range [-128, 127] is used. If the hardware has special limitations (e.g., only supporting [-64, 63]), the hardware constraints prevail. Then, based on the maximum and minimum values ​​of the floating-point tensors calculated using the exponential averaging or sliding window statistical methods described above, and the maximum and minimum values ​​of the discrete data in the preset integer format for the corresponding network layer, the scaling factor for each network layer in the current iteration of training is calculated. For example, the formula for calculating the scaling factor is as follows:

[0066] ;

[0067] Where S is the scaling factor for any network layer, r max r represents the maximum value of the floating-point tensor of the corresponding network layer. min q represents the minimum value of the floating-point tensor of the corresponding network layer. max q represents the maximum value of discrete data in a preset integer format for the corresponding network layer. min The minimum value of discrete data in a preset integer format for the corresponding network layer.

[0068] The scaling factor of each network layer in the current iteration of training can be calculated using the above formula (3). For example, if any network layer uses the above sliding window statistical method to obtain the maximum value r of the floating-point tensor... max =5.8, minimum value r min =-2.5, and this layer uses INT8 asymmetric quantization (the preset integer range is [-128, 127], i.e., q max =127, q min =-128), then the scaling factor of this network layer can be calculated according to the formula. Furthermore, the zeros of each network layer in the current iteration of training can be calculated based on the calculated scaling factor, the minimum value of the floating-point tensor of each network layer, and the minimum value of the discrete data in the preset integer format of the corresponding network layer. For example, the formula for calculating the zeros is as follows:

[0069] ;

[0070] Where Z represents the zero of any network layer, q min r is the minimum value of discrete data in integer format for the corresponding network layer. min is the minimum value of the floating-point tensor of the corresponding network layer, S is the scaling factor of the corresponding network layer, and round is the rounding operation.

[0071] According to the above formula (4), the zeros of each network layer in the current iteration of training can be calculated. For example, for the above INT8 asymmetric quantized network layer, the minimum value r of its floating-point tensor is... min =-2.5, the minimum value q of discrete data in preset integer format. min =-128, the calculated scaling factor S≈0.0325, substituting into the zero-point calculation formula, we get It should be noted that in some embodiments, if the calculated zero point Z exceeds the preset integer format range of the corresponding network layer (such as [-128, 127]), the zero point needs to be truncated, that is, the zero point is adjusted to the minimum value of the integer range, -128, to avoid the hardware being unable to represent the value, which would cause the quantization logic to fail. Finally, the scaling factor and zero point calculated in the above manner can be used to quantize the integer of the floating-point number output by each network layer using the above formula (1). Furthermore, the scaling factor and zero point calculated during iterative training will be dynamically updated with each iteration, which can provide a precise numerical mapping basis for the quantization perception training of the neural network, so as to facilitate the subsequent deployment of the quantization model.

[0072] It is understood that this embodiment only exemplifies one method for calculating the scaling factor and zero point in the quantization parameters. In some embodiments, the quantization parameters can also be calculated using other formulas. For example, the formula for calculating the zero point can also be... This does not constitute a specific limitation.

[0073] The aforementioned quantization-aware training method for a neural network model achieves full operator quantization coverage by encapsulating non-modular operators into modular operators, avoiding frequent format conversions and memory access overhead during inference and ensuring the continuity of quantization computation. It further encapsulates operator pairs composed of convolution operators and batch normalization operators into new convolution operators, allowing these new operators to perform a fusion process during the forward propagation of quantization-aware training to obtain quantization information suitable for deployment, while retaining batch normalization operator parameter updates during backpropagation to maintain training stability. Simultaneously, full operator quantization coverage constructs a coherent quantization computation chain, enabling the statistical analysis of key quantization parameters during quantization-aware training to be based on data from the entire chain, expanding the scope of data utilization, reducing interference from single-round data fluctuations, and further ensuring training stability and parameter accuracy. Ultimately, this method effectively reduces model storage usage and computational overhead while ensuring the inference accuracy of the quantized model (i.e., the second neural network model after quantization-aware training), thereby improving the deployment efficiency of the quantized model.

[0074] In some exemplary embodiments of this disclosure, based on the above embodiments, further descriptions are provided, such as... Figure 3 As shown, a quantization-based perceptual training method for a neural network model may further include the following steps:

[0075] Step 301: After the quantization perception training is completed, obtain the first output data of each network layer in the second neural network model before it has been processed by the pseudo-quantization operator and the dequantization operator, and the second output data of each network layer after it has been processed by the pseudo-quantization operator and the dequantization operator.

[0076] Step 302: Calculate the quantization error index value of each network layer in the second neural network model based on the first output data and the second output data.

[0077] Among them, the quantization error index values ​​include one or more of the following: mean square error, mean absolute error, cosine similarity, signal-to-noise ratio, and saturation rate. The saturation rate is the proportion of values ​​in the output data of the corresponding network layer after processing by the pseudo-quantization operator that fall within the boundary of the quantization interval.

[0078] Step 303: The network layer whose quantization error index value exceeds the preset quantization error threshold is identified as a quantization sensitive layer.

[0079] Step 304: After performing quantization precision rollback processing on the quantization sensitive layer, a new round of quantization perception training is performed on the second neural network model after the quantization precision rollback processing.

[0080] In this embodiment, hierarchical quantization error evaluation and quantization accuracy optimization can be performed on the second neural network model trained with quantization perception. Specifically, firstly, the data of each network layer in the second neural network model trained with quantization perception before and after quantization are obtained, namely, the first output data without pseudo-quantization and dequantization operator processing, and the second output data of each network layer after pseudo-quantization and dequantization operator processing. The first output data completely retains the original floating-point calculation results of each network layer and is not affected by any quantization distortion; while the second output data is the floating-point format data after quantization processing in actual deployment. Therefore, the difference between the second output data and the first output data can directly reflect the degree of interference of the quantization operation on the layer output, thereby calculating the quantization error index value. The quantization error index value can include multiple dimensions, such as one or more of the following: mean square error, mean absolute error, cosine similarity, signal-to-noise ratio, and saturation rate. Among them, mean square error and / or mean absolute error can intuitively quantify the degree of deviation at the numerical level, cosine similarity can assess the consistency of data features, signal-to-noise ratio can assess the fidelity of data features, and saturation rate can detect whether the data has excessive overflow due to the limitation of quantization range. This multi-dimensional evaluation method avoids the one-sidedness of a single indicator and can comprehensively capture the quantization adaptation problem of different network layers.

[0081] Furthermore, by comparing the calculated quantization error index value with a preset quantization error threshold, precise selection of quantization-sensitive layers is achieved. The preset quantization error threshold can be dynamically configured based on the model's application scenario and accuracy requirements. For example, for models with high fidelity requirements, the cosine similarity threshold can be set to 0.98 and the saturation threshold to 10%; while for models with moderate accuracy requirements, the mean squared error threshold can be relaxed to 0.01 and the cosine similarity threshold to 0.95, etc. By using the preset threshold, quantization-sensitive layers with low tolerance to quantization operations, such as core feature extraction layers and attention weight layers, are distinguished from ordinary layers with higher tolerance to quantization. This allows for targeted quantization accuracy rollback processing of quantization-sensitive layers, avoiding accuracy loss caused by indiscriminate optimization. For example, for quantization-sensitive layers that are prone to overflow and excessive saturation due to large numerical ranges, the original low-bit quantization precision (such as INT8) can be rolled back to a higher-bit quantization precision (such as INT16). By expanding the range of integer representation, data overflow can be reduced and the saturation rate can be lowered. For sensitive layers that have low cosine similarity due to asymmetric feature distribution and mapping offset, the asymmetric quantization mode can be switched to the symmetric quantization mode, or the quantization interval boundary can be readjusted to make the quantization parameters more closely match the data distribution characteristics and improve feature consistency. For sensitive layers that have high numerical precision requirements and excessive mean square error or mean absolute error, some of their floating-point calculation characteristics can be retained, or a mixed-precision quantization strategy can be adopted to specifically reduce numerical deviation.

[0082] Finally, after completing the precision rollback processing of the quantization-sensitive layer, a new round of quantization-aware training is required for the adjusted second neural network model to ensure that the model parameters are adapted to the new quantization configuration. In this new round of quantization-aware training, the optimizer (such as Adam or SGD), learning rate scheduling strategy, and training dataset of the original training framework can be reused. The focus is on updating the quantization parameter information for the quantization-sensitive layer whose quantization precision has been rolled back, and dynamically optimizing its scaling factor S and zero point Z. This allows the quantization parameters of the rollback layer to gradually adapt to a higher precision quantization mode. For non-sensitive layers, the optimal quantization parameters determined in the first round of quantization-aware training are directly reused, eliminating the need for repeated statistics and optimization, thus reducing computational redundancy and improving training efficiency. Through this embodiment, the quantization error of the quantization-sensitive layer after training can be effectively controlled within a preset threshold, while the non-sensitive layer maintains high compression efficiency. This solves the problem of local precision loss caused by unified quantization in traditional schemes and avoids the increased storage and computational costs associated with high-precision quantization of the entire model. Ultimately, it achieves the goal of effectively reducing model storage and computational overhead while ensuring the inference accuracy and performance of the quantization model, thereby improving the deployment efficiency of the quantization model.

[0083] In some exemplary embodiments of this disclosure, based on the above embodiments, a method for quantization perception training of a neural network model may further include: when performing quantization perception training on a second neural network model, using a first neural network model as a teacher model and keeping the model parameters of the teacher model unchanged, using the second neural network model as a student model, and performing noise matching on the student model through the teacher model, so that the quantization noise distribution generated by each network layer in the student model after processing by the pseudo-quantization operator and the inverse quantization operator tends to be consistent with the output noise distribution of the teacher model.

[0084] In this embodiment, during the quantization-aware training of the second neural network model, a teacher-student distillation quantization mechanism can be introduced. The high-precision format (e.g., FP32 / FP16) of the first neural network model is used as the teacher model, keeping its parameters unchanged. The second neural network model is used as the student model. Noise matching is performed on the student model using the teacher model, allowing the student model to better fit the teacher distribution even with quantization noise. Specifically, key network layers are first selected, and the output representations of the corresponding layers of the teacher model (without quantization noise interference) and the corresponding layers of the student model (raw floating-point outputs without pseudo-quantization operator processing) are obtained. The difference between the two is calculated using the mean squared error loss function (MSE) or the cosine loss function, i.e., the distillation loss function L. KD Among them, the mean squared error loss focuses on accurate matching at the numerical level, while the cosine loss function focuses on the consistency of feature directions; both can be flexibly selected according to model type and quantization requirements. Furthermore, the discretization noise ε introduced by the pseudo-quantization operator is addressed.s , can be represented as (Q is the pseudo-quantization operator, x) s For the network layer input of the student model, on the one hand, a noise suppression loss function is designed to directly reduce the discretization error in the quantization process and reduce the interference of noise on feature transmission by minimizing the difference norm between the pseudo-quantization operator input and output (such as L1 norm, L2 norm). On the other hand, a distribution alignment loss function is designed, using the mean square error loss function or cosine loss function to calculate the difference between the student quantization noise distribution and the teacher output noise distribution, ensuring that the output distribution of the student model can still approximate the teacher model even with quantization noise. Then, the target network layer in the student model with a signal-to-noise ratio (SNR) lower than a preset threshold (such as 15dB) or a saturation rate higher than a preset threshold (such as 30%) is given a higher target weight coefficient when calculating the noise matching loss. This coefficient is greater than the weight coefficient of the non-target network layer. Then, the calculated noise suppression loss and distribution alignment loss are weighted and fused to obtain the noise matching loss function, which can be expressed as: , where L noise L represents the noise matching loss function. noise_suppress L represents the noise suppression loss function. dist_align Let w1 and w2 represent the distribution alignment loss function, respectively, and their values ​​can be adjusted according to the actual application scenario. Finally, by applying the above noise matching loss function L... noise Distillation loss function L KD and the task loss function L task (For example, the cross-entropy loss function for classification tasks and the MSE loss function for regression tasks) are weighted and summed to obtain the total loss function L for quantized perceptual training. total The formula can be expressed as: Where α represents the distillation weight and β represents the noise matching weight, the values ​​of α and β can be dynamically adjusted during the training process. Based on the total loss function L... total By synchronously updating the weight parameters, bias parameters, and quantization parameters (scaling factor S, zero point Z) of each network layer in the student model through the backpropagation algorithm, the statistical characteristics of the quantization noise gradually approach the output noise of the teacher model while the student model learns the task knowledge. This effectively reduces the accuracy loss caused by low-bit quantization noise and improves the stability of model training and the accuracy of final quantization deployment.

[0085] Based on the above embodiments, the quantization method for identifying the model may further include: during the quantization-aware training and quantization precision backoff processing of the second neural network model, recording the quantization parameters, performance index values, and precision backoff history of each network layer in the second neural network model; wherein, the quantization parameters include one or more of the following: the scaling factor, zero point, maximum and minimum values ​​of the floating-point tensor, and update history of each network layer; the update history includes the timestamp corresponding to the adjustment of the quantization parameters during each iteration of training and / or the iteration training number; the performance index values ​​include one or more of the following of each network layer: signal-to-noise ratio, saturation rate, mean square error, and cosine similarity; and the precision backoff history includes one or more of the following: the network layer identifier of the quantization-sensitive layer, the execution time of the quantization precision backoff processing, the quantization precision before backoff, the quantization precision after backoff, and the reason for triggering the quantization precision backoff processing.

[0086] In this embodiment, by recording the quantization parameters, performance metrics, and accuracy fallback history of each network layer during quantization-aware training and accuracy fallback, end-to-end traceability of the quantization process is provided. When the model experiences accuracy fluctuations or anomalies, the root cause of the problem (such as a surge in saturation rate due to an abnormal scaling factor) can be quickly identified by recording the layers with abrupt changes in positioning parameters and layers with performance exceeding limits, thereby significantly reducing troubleshooting costs. At the same time, recording the accuracy fallback history provides a reusable reference template for quantizing similar models, thereby continuously optimizing the fallback strategy and ensuring model performance and deployment safety.

[0087] It should be understood that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order constraint on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0088] In some exemplary embodiments of this disclosure, such as Figure 4As shown, a quantization-aware training device for a neural network model is provided, including a first acquisition module 401, an identification module 402, an encapsulation module 403, and a training module 404. The first acquisition module 401 is used to acquire a first neural network model to be trained; the identification module 402 is used to identify operator pairs and non-module operators in the first neural network model, wherein non-module operators are operations not implemented based on module classes, and operator pairs include connected convolution operators and batch normalization operators; the encapsulation module 403 is used to encapsulate non-module operators into module operators and operator pairs into new convolution operators to obtain a second neural network model, wherein the new convolution operators perform a fusion process of convolution operators and batch normalization operators during forward propagation, and update the parameters of convolution operators and batch normalization operators respectively during backpropagation; the training module 404 is used to perform quantization-aware training on the second neural network model.

[0089] In one embodiment of this disclosure, the identification module 402 is specifically used to: identify operator pairs in the first neural network model using a computation graph search-based method; and / or to identify operator pairs in the first neural network model using a rule matching-based method.

[0090] In one embodiment of this disclosure, the identification module 402 is specifically used to: if a computation graph is constructed based on the first neural network model, identify the operator pairs in the first neural network model using a computation graph search method; if a computation graph cannot be constructed based on the first neural network model, identify the operator pairs in the first neural network model using a rule matching method.

[0091] In one embodiment of this disclosure, operator pairs in a first neural network model are identified using a computation graph search-based method, including: constructing a computation graph of the first neural network model and locating multiple convolution operator nodes marked as convolution operators in the computation graph; traversing the downstream connection relationships of each convolution operator node, and if it is identified that the downstream connection of any convolution operator node is only a batch normalization operator, then determining that the convolution operator of any convolution operator node and the corresponding batch normalization operator constitute an operator pair; and / or, operator pairs in the first neural network model are identified using a rule-based matching method, including: obtaining the naming information of each operator in the first neural network model; and determining, according to a preset naming convention and the naming information of each operator, that convolution operators whose names satisfy the matching relationship and the corresponding batch normalization operators constitute an operator pair.

[0092] In one embodiment of this disclosure, the training module 404 is specifically used to: collect the floating-point tensors output by each network layer of the second neural network model during iterative training of quantization-aware training; calculate the maximum and minimum values ​​of the floating-point tensors using an exponential averaging method, and calculate the scaling factor and zero point based on the maximum and minimum values, and quantize the floating-point number into an integer based on the scaling factor and zero point; or, obtain the maximum and minimum values ​​of the floating-point tensors output by each network layer of the second neural network model during multiple iterative training of quantization-aware training, perform aggregate statistical processing on the maximum and minimum values ​​within the window using a sliding window statistical method, calculate the scaling factor and zero point, and quantize the floating-point number into an integer based on the scaling factor and zero point.

[0093] In one embodiment of this disclosure, the apparatus further includes: a second acquisition module, configured to acquire, after quantization-aware training is completed, first output data of each network layer in the second neural network model that has not been processed by pseudo-quantization operators and dequantization operators, and second output data of each network layer after processing by pseudo-quantization operators and dequantization operators; a calculation module, configured to calculate the quantization error index value of each network layer in the second neural network model based on the first output data and the second output data, wherein the quantization error index value includes one or more of mean square error, mean absolute error, cosine similarity, signal-to-noise ratio, and saturation rate, and the saturation rate is the proportion of values ​​falling on the boundary of the quantization interval in the output data of the corresponding network layer after processing by pseudo-quantization operators; a determination module, configured to determine the network layer whose quantization error index value exceeds a preset quantization error threshold as a quantization-sensitive layer; and a quantization accuracy backoff processing module, configured to perform quantization accuracy backoff processing on the quantization-sensitive layer and then perform a new round of quantization-aware training on the second neural network model after quantization accuracy backoff processing.

[0094] In one embodiment of this disclosure, the apparatus further includes: a noise matching module, used to, when performing quantization perception training on the second neural network model, use the first neural network model as the teacher model and keep the model parameters of the teacher model unchanged, use the second neural network model as the student model, and perform noise matching on the student model through the teacher model, so that the quantization noise distribution generated by each network layer in the student model after processing by pseudo-quantization operators and dequantization operators tends to be consistent with the output noise distribution of the teacher model.

[0095] In one embodiment of this disclosure, the apparatus further includes: a recording module, used to record the quantization parameters, performance index values, and precision fallback history of each network layer in the second neural network model during the quantization-aware training and quantization precision fallback processing of the second neural network model; wherein, the quantization parameters include one or more of the following: the scaling factor, zero point, maximum and minimum values ​​of the floating-point tensor, and update history of each network layer; the update history includes the timestamp corresponding to the quantization parameter adjustment and / or the iteration training number during each iteration training; the performance index values ​​include one or more of the following of each network layer: signal-to-noise ratio, saturation rate, mean square error, and cosine similarity; and the precision fallback history includes one or more of the following: the network layer identifier of the quantization-sensitive layer, the execution time of the quantization precision fallback processing, the quantization precision before fallback, the quantization precision after fallback, and the reason for triggering the quantization precision fallback processing.

[0096] Specific limitations regarding the quantization perception training device for a neural network model can be found in the limitations of the quantization perception training method for a neural network model described above, and will not be repeated here. Each module in the aforementioned quantization perception training device for a neural network model can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in an electronic device, or stored in the memory of an electronic device in software form, so that the processor can call and execute the operations corresponding to each module.

[0097] In some exemplary embodiments of this disclosure, an electronic device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, this electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The database stores code and related data for a quantitative perception training method for a neural network model. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a quantitative perception training method for a neural network model.

[0098] Those skilled in the art will understand that Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0099] In some exemplary embodiments of this disclosure, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a quantitative perception training method for a neural network model as described in any of the exemplary embodiments above.

[0100] In some exemplary embodiments of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of a quantization perception training method for a neural network model in any of the exemplary embodiments described above.

[0101] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0102] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0103] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention patent. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this patent application should be determined by the appended claims.

Claims

1. A quantitative perception training method for a neural network model, characterized in that, Applied to vehicle-mounted terminals, the method includes: Obtain a first neural network model to be trained, which performs deep learning tasks such as image classification and object detection. Identify operator pairs and non-modular operators in the first neural network model, wherein the non-modular operators are operations not implemented based on module classes, and the operator pairs include connected convolution operators and batch normalization operators; The non-modular operators are encapsulated into modular operators and the operator pairs are encapsulated into new convolution operators to obtain a second neural network model. The new convolution operator performs the fusion process of the convolution operator and the batch normalization operator during forward propagation and updates the parameters of the convolution operator and the batch normalization operator respectively during back propagation. The second neural network model is subjected to quantization perception training, wherein the second neural network model after quantization perception training is deployed on the vehicle terminal to perform the deep learning task of target detection. The step of identifying operator pairs in the first neural network model includes: identifying operator pairs in the first neural network model using a computation graph search-based method; and / or, identifying operator pairs in the first neural network model using a rule matching-based method. The step of performing quantization-aware training on the second neural network model includes: during the iterative training of the quantization-aware training, collecting the tensors of floating-point numbers output by each network layer of the second neural network model; calculating the maximum and minimum values ​​of the floating-point number tensors using an exponential averaging method, and calculating a scaling factor and a zero point based on the maximum and minimum values, and quantizing the floating-point number into an integer based on the scaling factor and the zero point; or, obtaining the maximum and minimum values ​​of the tensors of floating-point numbers output by each network layer of the second neural network model during multiple iterations of the quantization-aware training, performing aggregated statistical processing on the maximum and minimum values ​​within the window using a sliding window statistical method, calculating a scaling factor and a zero point, and quantizing the floating-point number into an integer based on the scaling factor and the zero point.

2. The method according to claim 1, characterized in that, The identification of operator pairs in the first neural network model includes: If a computational graph is constructed based on the first neural network model, then the operator pairs in the first neural network model are identified using the computational graph search method. If a computational graph cannot be constructed based on the first neural network model, then the operator pairs in the first neural network model are identified using the rule-based matching method.

3. The method according to claim 1, characterized in that, The method of identifying operator pairs in the first neural network model using computation graph search includes: Construct a computation graph of the first neural network model, and locate multiple convolution operator nodes marked as convolution operators in the computation graph; Traverse the downstream connection relationships of each convolution operator node. If it is identified that the downstream connection of any convolution operator node is only the batch normalization operator, then it is determined that the convolution operator of any convolution operator node and the corresponding batch normalization operator constitute the operator pair. And / or, The step of identifying operator pairs in the first neural network model using a rule-based matching method includes: Obtain the naming information of each operator in the first neural network model; Based on the preset naming convention and the naming information of each operator, the convolution operator whose name satisfies the matching relationship and the corresponding batch normalization operator constitute the operator pair.

4. The method according to claim 1, characterized in that, The method further includes: After the quantization perception training is completed, the first output data of each network layer in the second neural network model before being processed by the pseudo-quantization operator and the dequantization operator, and the second output data of each network layer after being processed by the pseudo-quantization operator and the dequantization operator are obtained. The quantization error index value of each network layer in the second neural network model is calculated based on the first output data and the second output data. The quantization error index value includes one or more of the following: mean square error, mean absolute error, cosine similarity, signal-to-noise ratio and saturation rate. The saturation rate is the proportion of the values ​​that fall on the boundary of the quantization interval in the output data of the corresponding network layer after being processed by the pseudo-quantization operator. Network layers whose quantization error index values ​​exceed a preset quantization error threshold are identified as quantization-sensitive layers. After performing quantization precision rollback processing on the quantization-sensitive layer, a new round of quantization-aware training is performed on the second neural network model after the quantization precision rollback processing.

5. The method according to claim 1, characterized in that, The method further includes: When training the second neural network model for quantization perception, the first neural network model is used as the teacher model and the model parameters of the teacher model are kept unchanged. The second neural network model is used as the student model. The noise matching of the student model is performed through the teacher model so that the quantization noise distribution generated by each network layer in the student model after processing by pseudo-quantization operator and dequantization operator tends to be consistent with the output noise distribution of the teacher model.

6. The method according to claim 4, characterized in that, The method further includes: During the quantization perception training and quantization accuracy back-off process of the second neural network model, the quantization parameters, performance index values ​​and accuracy back-off history of each network layer in the second neural network model are recorded. The quantization parameters include one or more of the following: scaling factor, zero point, maximum and minimum values ​​of floating-point tensors, and update history for each network layer. The update history includes the timestamp corresponding to the quantization parameter adjustment during each training iteration and / or the iteration training number. The performance index values ​​include one or more of the following: signal-to-noise ratio, saturation rate, mean square error, and cosine similarity for each network layer. The precision fallback history includes one or more of the following: network layer identifier of the quantization-sensitive layer, execution time of the quantization precision fallback process, quantization precision before fallback, quantization precision after fallback, and the reason for triggering the quantization precision fallback process.

7. A quantization-sensing training device for a neural network model, characterized in that, The device, applied to an in-vehicle terminal, includes: The first acquisition module is used to acquire a first neural network model to be trained, which performs deep learning tasks such as image classification and object detection. The identification module is used to identify operator pairs and non-module operators in the first neural network model, wherein the non-module operators are operations that are not implemented based on module classes, and the operator pairs include connected convolution operators and batch normalization operators. An encapsulation module is used to encapsulate the non-modular operators into modular operators and to encapsulate the operator pairs into new convolution operators to obtain a second neural network model. The new convolution operator performs a fusion process of the convolution operator and the batch normalization operator during forward propagation and updates the parameters of the convolution operator and the batch normalization operator respectively during back propagation. The training module is used to perform quantitative perception training on the second neural network model, wherein the second neural network model after quantitative perception training is deployed on the vehicle terminal to perform the deep learning task of target detection. The step of identifying operator pairs in the first neural network model includes: identifying operator pairs in the first neural network model using a computation graph search-based method; and / or, identifying operator pairs in the first neural network model using a rule matching-based method. The step of performing quantization-aware training on the second neural network model includes: during the iterative training of the quantization-aware training, collecting the tensors of floating-point numbers output by each network layer of the second neural network model; calculating the maximum and minimum values ​​of the floating-point number tensors using an exponential averaging method, and calculating a scaling factor and a zero point based on the maximum and minimum values, and quantizing the floating-point number into an integer based on the scaling factor and the zero point; or, obtaining the maximum and minimum values ​​of the tensors of floating-point numbers output by each network layer of the second neural network model during multiple iterations of the quantization-aware training, performing aggregated statistical processing on the maximum and minimum values ​​within the window using a sliding window statistical method, calculating a scaling factor and a zero point, and quantizing the floating-point number into an integer based on the scaling factor and the zero point.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images