Model order quantization method and system based on channel balance and related device

Abstract

The application discloses a model layer sequence quantization method and system based on channel balance and related equipment, and belongs to the technical field of model compression. The method obtains the output feature maps of the floating point model and the quantization model under the same input for the target operator in the quantization model; each channel of the two feature maps is independently normalized, the similarity between the corresponding channels after normalization is calculated and aggregated, and a channel balance evaluation index representing quantization fidelity is obtained; if the index does not reach a preset threshold, the quantization precision of the layer is improved and the model is updated, and then the model is updated to execute the above evaluation process again; the process is iterated layer by layer in the network topology order until all layers are processed. Through the channel level evaluation and dynamic error suppression of the quantization effect, the application can more accurately allocate the quantization bit width, improve the hardware resource utilization efficiency while ensuring the model quantization precision.

Description

Technical Field

[0001] This application relates to the field of model compression technology, and in particular to a model hierarchical quantization method, system and related equipment based on channel balance. Background Technology

[0002] To deploy complex deep neural network models on edge devices with limited computing resources, model quantization techniques are widely used. The core of this technique is to convert floating-point parameters and activation values ​​(i.e., magnitudes) in the model into low-bit integer representations to reduce model storage size and computational overhead.

[0003] Currently, when making mixed-precision quantization decisions (such as simultaneously using 8-bit integers (INT8) and 16-bit integers (INT16)), models typically rely on evaluating the overall similarity (e.g., cosine similarity) between the quantized model and the original floating-point model in terms of operator output. If the overall similarity is higher than a preset threshold, the layer is considered to be able to use lower precision. However, in network architectures such as Lightweight Convolutional Neural Networks (MobileNet) and Transformers, the semantic information and numerical distribution carried by different output channels within the same operator differ significantly (channel heterogeneity). Some channels contain key features but have small numerical ranges, while other channels may contain redundant information but have large dynamic ranges. Existing evaluation methods based on overall output similarity implicitly assume that all channels contribute uniformly to the final result, which can easily lead to quantization distortion where high-amplitude channels mask low-amplitude but critical channels. Specifically, this will cause two main problems: 1. Critical channels may be incorrectly assigned too low a bit width because the overall similarity meets the standard, resulting in a significant decrease in model accuracy; 2. In order to meet the overall similarity requirements, the system may also assign high bit widths to a large number of insensitive redundant channels, resulting in a waste of hardware computing and storage resources.

[0004] Therefore, there is an urgent need for a model quantization scheme that can sense channel heterogeneity and dynamically allocate quantization precision, thereby maximizing hardware efficiency while ensuring model accuracy. Summary of the Invention

[0005] This application provides a model sequence quantization method, system, and related equipment based on channel balance, relating to the field of model compression technology. Through channel-level precise evaluation using dual-end channel-by-channel normalization and sequence iterative feedback control based on the evaluation results, the quantization width and quantization error suppression of mixed quantization precision in the quantization model are determined. The specific technical solution of this application is as follows:

[0006] Firstly, a model hierarchical quantization method based on channel balance is provided, comprising the following steps: inputting a calibration dataset into a pre-trained floating-point model and a quantization model preset to a first quantization bit width; obtaining a first output feature map of the first target operator output in the floating-point model and a second output feature map of the first target operator output in the quantization model; determining a channel balance evaluation index characterizing the quantization fidelity of the first target operator based on the normalization processing of the channels in the first and second output feature maps; when the channel balance similarity is less than a similarity threshold, iteratively updating the quantization precision of the first target operator in the quantization model to a second quantization bit width, wherein the quantization precision of the second quantization bit width is higher than that of the first quantization bit width, so that the updated quantization model has a channel balance similarity greater than or equal to the similarity threshold; after the quantization bit width of the first target operator is confirmed, determining the second target operator in the next layer of the quantization model according to the layer order of the network topology in the quantization model, repeating the aforementioned steps to obtain the quantization bit width of the second target operator, thereby determining the quantization bit width of each layer in the quantization model layer by layer.

[0007] In conjunction with the first aspect, based on the normalization processing of the channels in the first output feature map and the channels in the second output feature map, a channel balance evaluation index characterizing the quantization fidelity of the first target operator is determined. Specifically, this includes: normalizing each channel corresponding to a position in the first output feature map and the second output feature map, calculating the similarity of each pair of normalized corresponding channels, and aggregating the similarities of each pair of corresponding channels to obtain the channel balance similarity.

[0008] In conjunction with the first aspect, in some embodiments of the first aspect, normalization processing is performed on each channel corresponding to a position in the first output feature map and the second output feature map, specifically including: for each pair of channels corresponding to a position, calculating a first statistic of the channel in the first output feature map and a second statistic of the channel in the second output feature map, wherein both the first statistic and the second statistic include the mean and standard deviation of the corresponding channel; normalizing the corresponding channel in the first output feature map based on the first statistic, and normalizing the corresponding channel in the second output feature map based on the second statistic.

[0009] In conjunction with the first aspect, the similarity of each pair of normalized corresponding channels is calculated, and the similarity of each pair of corresponding channels is aggregated to obtain the channel balance similarity. Specifically, this includes: calculating the cosine similarity of each pair of normalized corresponding channels, and calculating the mean based on the cosine similarity of each pair of corresponding channels to obtain the channel balance similarity.

[0010] In conjunction with the first aspect, the quantization precision of the first target operator in the quantization model is iteratively updated to the second quantization bit width. Specifically, this includes: adjusting the quantization precision of the first target operator in the quantization model from the first quantization bit width to the second quantization bit width; updating the quantization model according to the adjusted quantization precision; and based on the updated quantization model, re-acquiring the second output feature map of the first target operator and iteratively calculating the channel balance similarity until the channel balance similarity is greater than or equal to the similarity threshold.

[0011] In conjunction with the first aspect, when the channel balance similarity is greater than or equal to the similarity threshold, the first target operator maintains the current quantization bit width.

[0012] In conjunction with the first aspect, after determining the quantization bit width of each layer in the quantization model layer by layer, the method further includes: generating a mixed-precision quantization configuration file based on the quantization bit width of each layer in the quantization model; the quantizer generates the quantization model through the mixed-precision quantization configuration file, which includes the layer name, index, and quantization bit width in the quantization model; adjusting the similarity threshold based on the overall quantization precision of the quantization model on the validation dataset; and using the adjusted similarity threshold to redetermine the quantization bit width of each layer in the quantization model to obtain the optimized quantization model.

[0013] It should be noted that, in the absence of conflict, the features in the various embodiments of the first aspect can be combined with each other, and any combination of features in different embodiments is also within the protection scope of this application. That is to say, the various embodiments described above can also be arbitrarily combined according to actual needs.

[0014] Secondly, a model sequence quantization system based on channel balance is provided for implementing the method as described in the first aspect or any of the embodiments in the first aspect, including:

[0015] The feature acquisition module is used to input the calibration dataset into the pre-trained floating-point model and the quantization model with a preset first quantization bit width, and to acquire the first output feature map of the first target operator output in the floating-point model and the second output feature map of the first target operator output in the quantization model.

[0016] The normalization module is used to determine the channel balance evaluation index that characterizes the quantization fidelity of the first target operator based on the normalization processing performed on the channels in the first output feature map and the channels in the second output feature map.

[0017] The evaluation module is used to compare the channel balance similarity with a preset similarity threshold.

[0018] The decision and adjustment module is used to make decisions and adjustments based on the comparison results of the evaluation module: when the channel balance similarity is less than the similarity threshold, the quantization precision of the first target operator in the quantization model is iteratively updated to the second quantization bit width. The quantization precision of the second quantization bit width is higher than that of the first quantization bit width, so that the updated quantization model can calculate the channel balance similarity as greater than or equal to the similarity threshold. When the channel balance similarity is greater than or equal to the similarity threshold, the first target operator maintains the current quantization bit width.

[0019] The iterative control module is used to determine the second target operator in the next layer of the quantization model according to the layer order of the network topology in the quantization model after the quantization bit width of the first target operator is confirmed, and repeat the aforementioned steps to obtain the quantization bit width of the second target operator, thereby determining the quantization bit width of each layer in the quantization model layer by layer.

[0020] Thirdly, a computer device is provided, including one or more memories and one or more processors; the memory is coupled to the one or more processors, the memory is used to store computer program code, the computer program code including computer instructions, and the one or more processors call the computer instructions to cause the computer device to implement the method as described in the first aspect or any of the embodiments of the first aspect.

[0021] Fourthly, a computer-readable storage medium is provided that stores computer instructions thereon, which, when executed by a processor, implement the method as described in the first aspect or any of the embodiments in the first aspect.

[0022] In the embodiments of this application, the method provided by this application has the following beneficial effects.

[0023] 1. Improved accuracy of quantization bit width allocation. By normalizing the channels in the model, interference caused by differences in amplitude scale between different channels in similarity assessment was eliminated, allowing the evaluation metrics to more fairly and accurately reflect the information fidelity of each channel after quantization. This helps to accurately identify key channels that are sensitive to quantization and avoid their over-compression.

[0024] 2. Effectively suppresses the propagation of quantization errors. The quantization width is determined through layer-by-layer iteration, and quantization verification and real-time correction are performed layer by layer in the forward propagation order. If a layer becomes distorted in quantization, its accuracy is improved and it is re-evaluated based on a new model, forming a local feedback loop. This approach can block and correct single-layer quantization errors, preventing their accumulation and amplification to downstream networks, thus improving the robustness of the overall quantization model.

[0025] 3. It achieves refined utilization of hardware resources. The combination of the above assessment and dynamic decision-making mechanism ensures that only layers or channels that do not meet accuracy requirements are allocated higher quantization bit widths (such as 16-bit integers (INT16)), while 8-bit integer (INT8) quantization bit widths are reserved for the majority of channels that meet accuracy requirements. This helps to maximize the utilization efficiency of hardware resources and reduce deployment costs while meeting model accuracy targets.

[0026] 4. An adaptive optimization path is provided. By introducing a feedback mechanism that adjusts the internal similarity threshold based on model accuracy feedback, the entire quantization process becomes adaptive, automatically finding the optimal balance between accuracy and efficiency for different models and tasks. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 This is a schematic diagram of a system framework for model sequence quantization based on channel balance provided in an embodiment of this application;

[0029] Figure 2 This is an overall flowchart of a quantization method for a quantization model with mixed quantization width provided in an embodiment of this application;

[0030] Figure 3 This is a schematic diagram of a process for evaluating and deciding on the channel balance of a target operator, provided in an embodiment of this application.

[0031] Figure 4 This is a schematic diagram of a model sequence quantization system based on channel balance provided in an embodiment of this application;

[0032] Figure 5 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application;

[0033] Figure 6 This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0035] It should be understood that "multiple" as mentioned in this application refers to two or more. In the description of this application, unless otherwise stated, " / " indicates "or," for example, A / B can mean A or B; "and / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, to facilitate a clear description of the technical solutions of this application, the terms "first," "second," etc., are used to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or execution order, and that "first," "second," etc., do not necessarily imply differences.

[0036] The terms "one embodiment" or "some embodiments" used in this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this application do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. Furthermore, the terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0037] The following three embodiments describe the model hierarchical quantization method, system, and related devices based on channel balance provided in this application. Embodiment 1 describes the system architecture of the model hierarchical quantization method based on channel balance. Embodiment 2 describes the method flow for implementing model hierarchical quantization based on channel balance. Embodiment 3 describes the system module solution structure, computer equipment hardware structure, and computer-readable storage medium for executing the model hierarchical quantization method based on channel balance.

[0038] Example 1

[0039] Figure 1 This is a schematic diagram of a system framework for model sequence quantization based on channel balance, provided in an embodiment of this application. For example... Figure 1 As shown, the system framework 100 can be divided into four levels: system entry and resource management layer 110, process scheduling and hierarchical control layer 120, quantitative processing and decision-making layer 130, and output generation and system optimization layer 140. From top to bottom, it constitutes a technical solution implementation system from initial configuration to core processing and then to output optimization.

[0040] In this embodiment, the system entry and resource management layer 110 is used for data initialization and resource allocation. The system entry and resource management layer 110 includes a data loading unit 111 and a model initialization unit 112. The data loading unit 111 reads in a pre-trained floating-point neural network model and a calibration dataset, providing a benchmark and input for subsequent quantization evaluation. The model initialization unit 112 creates an initial quantization model based on the floating-point neural network model, where all layers are configured with a uniform low-precision first quantization bit width (e.g., 8-bit integer (INT8)). The system entry and resource management layer 110 provides a stable initial state for the entire system.

[0041] In this embodiment, the process scheduling and sequence control layer 120 defines the processing order from the input layer to the output layer based on the network topology of the quantization model. The process scheduling and sequence control layer 120 includes a sequence control unit 121, which obtains the first target operator in the quantization model to determine the quantization width. After the processing of the current first target operator is completed, it obtains the second target operator in the next layer of the quantization model and repeats the previous processing flow to obtain the quantization bit width of the second target operator. In some embodiments, the sequence control unit 121 includes a global state judge, which can continuously monitor the processing progress of the quantization model and coordinate the system to exit the loop and enter the result generation stage after the processing of the last layer of the quantization model is completed. The process scheduling and sequence control layer 120 can ensure that the process of determining the quantization width of the quantization model proceeds sequentially along the network topology.

[0042] In this embodiment, the quantization processing and decision-making layer 130 includes a feature map acquisition module 131 for performing target operator quantization evaluation and decision-making, a dual-channel normalization module 132, a channel balance similarity calculation module 133, and a decision-making and feedback adjustment module 134. For a floating-point model and a quantization model with the same network topology, firstly, the feature map acquisition module 131 acquires the first output feature map of the floating-point model output by the first target operator, and the second output feature map of the quantization model output by the first target operator. Next, the dual-channel normalization module 132 performs statistical analysis (calculates the mean and standard deviation) on each channel corresponding to the position in the first and second output feature maps and completes normalization to eliminate amplitude differences between different channels and achieve fair comparison. Then, the channel balance similarity calculation module 133 calculates the similarity (such as cosine similarity) for each pair of normalized corresponding channels and aggregates the similarities of each corresponding channel into a comprehensive index characterizing the quantization fidelity of the first target operator, namely, the channel balance similarity. Finally, the decision-making and feedback adjustment module 134 compares the channel balance similarity with a preset similarity threshold: if it meets the threshold, it confirms the current quantization accuracy and instructs the process scheduling and hierarchical control layer 120 to process the second target operator of the next layer; if it does not meet the threshold, it performs a feedback loop to improve the quantization accuracy of the first target operator, updates the state of the quantization model, and returns to the feature map acquisition step of this layer for a new round of evaluation until the channel balance similarity meets the requirements. In this way, based on the evaluation and feedback mechanism of the quantization processing and decision-making layer 130, the quantization accuracy of the quantization model can be dynamically updated and accurately determined.

[0043] In this embodiment, the output generation and system optimization layer 140 is responsible for recording and inputting the quantization precision determined layer by layer. The output generation and system optimization layer 140 includes a configuration file generation module 141, which collects the final determined quantization precision and parameters for each layer of the quantization model and generates a structured mixed-precision quantization configuration file. The quantizer can use this mixed-precision quantization configuration file to generate the final deployed quantization model. In some embodiments, the system also performs global optimization on the generated quantization model: after the initial quantization model is generated, the system can evaluate its overall task precision using a validation dataset. If the expected precision is not achieved, the similarity threshold used in the quantization processing and decision layer 130 is adjusted, and the system is driven to re-execute the entire process from the system entry and resource management layer 110, thereby achieving collaborative optimization of the overall performance of the quantization model.

[0044] In this embodiment, the layers within the system framework 100 can interact through data interfaces and control signals. Upper layers provide execution context and resources to lower layers, while lower layers feed back processing results and status to upper layers. In particular, the feedback loop within the quantization processing and decision-making layer 130 and the global optimization loop within the system optimization layer 140 enable the adaptive adjustment of the system framework 100, allowing it to accurately complete the task of mixing quantization precision for the quantization model.

[0045] It is understood that the functional division between various levels and modules in the system framework 100 provided in this application embodiment is only illustrative and does not constitute a functional limitation of the system framework 100. In other embodiments of this application, the system framework 100 may also adopt different levels and modules than those in the above embodiments, or implement the functions in the system framework 100 through a combination of multiple levels and modules.

[0046] Example 2

[0047] Figure 2 This is an overall flowchart of a quantization method for mixing quantization widths in a quantization model, provided in an embodiment of this application, and is applied to, for example... Figure 1 The system framework 100 shown specifically includes:

[0048] S101. Data Preprocessing and Quantization Code Framework Construction.

[0049] In this embodiment, a pre-trained floating-point model (such as a deep neural network model) to be quantized and its corresponding calibration dataset (e.g., a small subset of the training set, including 100 to 1000 samples) are loaded. A quantization model is initialized, with the initial configuration of setting all layers in the network topology to the first quantization bit width (e.g., using the default INT8 quantization format). The quantization bit width of this quantization model will be dynamically updated during the process.

[0050] In this embodiment, data preprocessing includes performing conventional preprocessing operations such as standardization, size alignment, and normalization on the input data to ensure consistent feature distribution. Meanwhile, the quantization code framework is built upon mainstream deep learning frameworks (such as PyTorch and ONNX) to create an extensible quantization code framework.

[0051] S102. Channel-by-channel normalization of double-ended operators.

[0052] In this embodiment, for each target operator to be quantized (such as convolutional layers, linear layers, etc. in a network topology), the output feature maps of the floating-point model corresponding to the target operator and the current quantization model are obtained under the same calibration dataset input, and then split into feature sub-maps of individual channels according to the channel dimension. The feature map obtained from the floating-point model of the current first target operator's output is called the first output feature map, and the feature map obtained from the quantization model of the current first target operator's output is called the second output feature map. Using the same calibration dataset input ensures that subsequent comparisons are performed under identical input stimuli.

[0053] In this embodiment of the application, for each channel, its own statistics (such as mean and standard deviation) are calculated. Based on the statistics of each channel, normalization is performed according to the following formula (1) to obtain the normalized channel feature sub-map:

[0054]

[0055] in, is the feature value of the c-th channel in the first output feature map after normalization, where c is a positive integer; This represents the feature value of the c-th channel in the normalized second output feature map. The feature value of the c-th channel in the first output feature map; This refers to the feature value of the c-th channel in the second output feature map; and , where is the first statistic, and are the mean and standard deviation of the c-th channel in the first output feature map, respectively; and The second statistic is represented by the mean and standard deviation of the c-th channel in the second output feature map, respectively. It is a numerical stability constant, and its value ranges from 1 to 10. arrive between.

[0056] In the embodiments of this application, the normalization operation of the floating-point model feature map and the normalization operation of the quantized model feature map are performed independently. By normalizing channel by channel, the deviation caused by the difference in numerical range (i.e. amplitude) between different channels (such as the activation value range of some channels being 0 to 0.1, and the activation value range of other channels being 0 to 10, etc.) on the subsequent similarity calculation is eliminated, so that all channels participate in the comparison on the same scale, which can improve the reliability of similarity assessment.

[0057] S103. Channel-level similarity calculation.

[0058] In this embodiment of the application, after step S102, the normalized first output feature map and the second output feature map are first processed by analyzing each pair of channel feature values. and Calculate their similarity (e.g., cosine similarity), and then aggregate the similarities of each corresponding channel (e.g., calculate the mean of all channel similarities) to obtain the channel balance similarity. This channel balance similarity can reflect the overall fidelity of the current first target operator after quantization, avoiding the dominance of a few high-amplitude channels.

[0059] In the embodiments of this application, and The cosine similarity is calculated as shown in the following formula (2):

[0060]

[0061] in, Let be the cosine similarity of the c-th channel, and N be the total number of pixels in the channel space.

[0062] In this embodiment, the cosine similarity of each corresponding channel is aggregated, and the resulting channel balance similarity is shown in the following formula (3):

[0063]

[0064] in, Let k be the channel balance similarity of the k-th objective operator (i.e., the first objective operator) in the quantization model, where k is a positive integer; The total number of channels in the target operator.

[0065] S104. Iterative update of the hierarchical sequence.

[0066] In this embodiment, step S104 is the core of dynamic feedback control, used to form a closed-loop structure for judgment and adjustment. The result calculated in step S103... The comparison is made with a preset similarity threshold T, and the action is selected from the following two options based on the comparison result. The similarity threshold T can range from 0 to 1, with a default value of, for example, 0.99. All operators in the quantization model can be traversed layer by layer in the order of the network topology (e.g., from the input layer to the output layer).

[0067] like If the quantization precision (i.e., the first quantization bit width) of the current first target operator is considered sufficient, this configuration is retained. The subsequent process replaces the current first target operator with the second target operator in the next layer of the quantization model, i.e., sets k = k + 1, preparing to process the next layer of the network topology in the quantization model. In each iteration of the loop, one target operator is processed. After updating the current target operator, it is necessary to determine whether all layers in the quantization model have been processed (e.g., whether k is greater than K, where K is the total number of layers in the quantization model): if yes, the loop in step S104 is exited, and the process proceeds to step S105; if not, the processing flow for the current target operator is returned, i.e., steps S102 to S104 are re-executed.

[0068] like If the quantization precision (i.e., the first quantization bit width) of the first target operator is insufficient, the quantization precision is iteratively updated to a second quantization bit width that is higher than the first quantization bit width, ensuring that the updated quantization model has a similarity to the channel balance that is greater than or equal to the similarity threshold. Specifically, this includes the following closed-loop operations:

[0069] 1. First, increase the quantization precision of the first target operator from the first quantization bit width to the second quantization bit width, for example, from INT8 to INT16, or, if it supports setting the quantization bit width for each channel, from 8-bit integer per tensor (per_tensor_INT8) to 8-bit integer per channel (per_channel_INT8).

[0070] 2. Then, the quantization model is updated according to the new precision setting of the first target operator to obtain a temporary mixed precision model. Based on this mixed precision model, the aforementioned steps S102 and S103 are executed again to re-acquire the second output feature map and channel balance similarity of the current first target operator, and the evaluation calculation of step S104 is executed again until the channel balance similarity is greater than or equal to the similarity threshold.

[0071] In the embodiments of this application, based on the closed-loop operation of the target operator in the quantization model described above, it can be ensured that the quantization error of each processed layer has been effectively controlled before the next layer of the network topology is processed.

[0072] In this embodiment, the layer-by-layer iterative update method shown in step S104 can achieve error perception and local correction of the quantization model. When a layer suffers quantization distortion due to channel imbalance, the quantization accuracy of that layer can be increased and re-evaluated to prevent quantization errors from propagating downwards. Simultaneously, this layer-by-layer iterative update method can use a second quantization bit width with higher quantization accuracy only for necessary layers, while maximizing the efficiency advantage of the first quantization bit width.

[0073] S105. Generation of hierarchical bit width allocation and mixed precision quantization configuration files.

[0074] In this embodiment, after traversing all layers in the quantization model as described in step S104, the system can record the final determined quantization bit width for each layer in the quantization model and generate a structured mixed-precision quantization configuration file. This mixed-precision quantization configuration file includes: the layer name, index, quantization bit width, and quantization parameters (such as scaling factor and zero-point offset). The system can synchronize this mixed-precision quantization configuration file to the quantizer to generate a hardware-deployable mixed-precision quantization model.

[0075] S106. Model validation and optimization closed loop.

[0076] In this embodiment of the application, in order to further improve the robustness of the solution, step S106 can provide an external global optimization closed loop. That is, after generating the quantized model, its overall task accuracy is evaluated on an independent validation dataset. For example, end-to-end inference can be performed on the generated quantized model, and key performance indicators (such as Top-1 accuracy, Mean Average Precision (MAP)) can be evaluated on the validation set.

[0077] If the accuracy loss meets the preset tolerance threshold, the final quantization model is determined; if the accuracy is insufficient, the similarity threshold can be increased, and steps S102-S105 can be re-executed using the adjusted similarity threshold until the obtained quantization model simultaneously meets the hierarchical balance requirements (i.e., ...). This allows for effective coordination between local evaluation and global performance of the quantized model, as well as the required global accuracy of the model.

[0078] Based on the method described in steps S101-S106 above, this application achieves several beneficial effects by constructing a complete closed loop of synergistic effect between dual-end channel-by-channel normalization evaluation and hierarchical iterative feedback: First, by performing independent channel-by-channel normalization on the feature map, the interference of amplitude differences between channels on the evaluation is eliminated, so that the calculated channel balance similarity can accurately reflect the quantization fidelity of each channel. This solves the problem that in traditional global evaluation indicators, key channels are incorrectly compressed or redundant channels are overprotected in channel heterogeneity scenarios, thus improving the accuracy of bit width allocation; Second, the feedback closed loop for the quantization bit width of the target operator can ensure that the quantization model... The targeted operator improves quantization accuracy and re-verifies when quantization distortion occurs, thus suppressing quantization errors in a timely manner. This effectively blocks the propagation and accumulation of quantization errors along the network topology layer sequence to subsequent layers, enhancing the overall robustness of the model. Finally, the combination of the above evaluation and dynamic control allows the system to increase the layer accuracy from INT8 to higher precisions such as INT16 only when necessary. This maximizes the utilization of low-precision computing units while ensuring that the quantization model meets the task accuracy requirements. At the same time, the global optimization closed loop can adaptively adjust the similarity threshold to balance the quantization accuracy and efficiency of the quantization model, thereby achieving refined utilization that takes into account computing, storage, and bandwidth resources.

[0079] It should be understood that, as mentioned above Figure 2 The steps in the flowcharts are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are performed; they can be executed in other orders. Furthermore, as mentioned above... Figure 2 The flowchart may include at least some steps or stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0080] Figure 3 This is a flowchart illustrating a channel balance evaluation and decision-making process for a target operator, provided in an embodiment of this application, applicable to, for example... Figure 1 The system framework 100 shown includes the following specific steps:

[0081] S201. Target layer identification and context establishment.

[0082] In this embodiment, the system can obtain the target operator to be processed from the k-th layer (i.e., the target layer) of the network structure in the quantization model, initialize the context environment for each round of iteration calculation, and load the structural information of the target layer (e.g., a convolutional layer, a linear layer, or a composite layer containing an activation function). Step S201 is used to focus the global layer order processing of the quantization model on the current specific target layer.

[0083] S202. Construction of intra-layer evaluation sequences.

[0084] In this embodiment, the system can analyze the internal structure of the target layer to generate an evaluation scheme for the quantization model, specifically including a strategy for determining the activation function. To improve evaluation efficiency and hardware compatibility, the system can logically fuse the linear activation function with preceding linear computation operators (such as convolution, fully connected layers, etc.) to form a larger evaluation unit. Based on this optimization strategy, the system can generate an intra-layer evaluation sequence for the target layer. This intra-layer evaluation sequence contains all the basic units (i.e., target operators) that require independent channel balance evaluation, and can be used for subsequent refined evaluation.

[0085] S203. Intra-layer evaluation loop control.

[0086] In this embodiment, step S203 is used to determine whether the intra-layer evaluation sequence obtained in step S202 is empty: if it is not empty, it means that there are still unevaluated or unmet target operators in the current target layer, and the process will continue to perform evaluation and decision-making, i.e., proceed to step S204. If it is empty, it indicates that all basic units in the current target layer have met the requirements, and the process will return as follows. Figure 2 Step S104 shows the preparation for processing the next layer of the network structure topology in the quantization model.

[0087] S204. Perform channel balance assessment.

[0088] In this embodiment of the application, the channel balance assessment may also refer to the foregoing. Figure 2 Steps S102 to S103 include: using the same batch of calibration data, performing parallel forward propagation through the floating-point model and the current version of the quantization model to capture the output of the target evaluation unit, and obtaining the first output feature map (from the floating-point model) and the second output feature map (from the quantization model) respectively; for the two feature maps, normalization can be performed independently on each channel by referring to the aforementioned formula (1); for each pair of normalized corresponding channels, cosine similarity can be calculated by referring to the aforementioned formula (2), and the similarity of each corresponding channel can be aggregated, and the index of the evaluation unit (i.e., channel balance similarity) can be obtained by using the aforementioned formula (3).

[0089] S205. Fidelity judgment.

[0090] In the embodiments of this application, reference is made to the foregoing Figure 2 In step S104, the channel balance similarity obtained from step S204 is compared with a preset configurable similarity threshold, and the comparison result determines the two execution methods.

[0091] If the channel balance similarity is greater than or equal to the similarity threshold, it indicates that the current evaluation unit meets the performance requirements at the existing quantitative accuracy (e.g., INT8), and the system can remove the currently passed evaluation unit from the in-layer evaluation sequence. Simultaneously, the system returns to the aforementioned S203 step to check the next unit to be processed in the in-layer evaluation sequence. This allows for sequential processing and rapid passage of units within the target layer.

[0092] If the channel balance similarity is less than the similarity threshold, it indicates that the current quantization accuracy is causing unacceptable distortion. The system can improve the quantization accuracy by optimizing the closed-loop operation, increasing the quantization accuracy of the current evaluation unit from a low-precision quantization bit width to a high-precision quantization bit width. Simultaneously, the system updates the parameters and configuration of the corresponding unit in the working quantization model, returns to step S204, and re-executes feature map acquisition, normalization, and similarity calculation based on the updated quantization model. This iterative calculation continues until the calculated channel balance similarity meets the similarity threshold requirement.

[0093] S206. Current layer processing completed and state synchronized.

[0094] In this embodiment of the application, if the judgment result of step S203 shows that the evaluation sequence within the layer is empty, it indicates that all units in the current target layer have been processed and the state synchronization of the quantization model is performed.

[0095] Specifically, the final quantization precision of all units in the current target layer (i.e., the k-th layer) is fixed in the configuration of the quantization model, which is used to output a stable quantization model. The system can execute actions such as... Figure 2 The S104 step shown indicates that the target operator of the k-th layer has been processed. Let k = k + 1, and prepare to process the next layer of the network structure topology in the quantization model.

[0096] Based on the method shown in steps S201-S206 above, in steps S204-S206, by evaluating the channel balance, dynamically updating the model and re-verifying, it can be ensured that quantization errors are identified and corrected in a timely manner when they occur, thereby minimizing the problem of quantization error accumulation and propagation and realizing an efficient and reliable mixed-precision quantization model.

[0097] It should be understood that, as mentioned above Figure 3The steps in the flowcharts are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise explicitly stated herein, there is no strict order in which these steps are performed; they can be executed in other orders. Furthermore, as mentioned above... Figure 3 The flowchart may include at least some steps or stages. These steps or stages are not necessarily completed at the same time, but may be executed at different times. The execution order of these steps or stages is not necessarily sequential, but may be executed in turn or alternately with other steps or at least some of the steps or stages in other steps.

[0098] Example 3

[0099] Figure 4 This is a schematic diagram of a model sequence quantization system based on channel balance provided in an embodiment of this application. For example... Figure 4 As shown, the model sequence quantization system 400 based on channel balance specifically includes the following modules:

[0100] The feature acquisition module 410 is used to input the calibration dataset into the pre-trained floating-point model and the quantization model preset to the first quantization bit width, and to acquire the first output feature map of the first target operator output in the floating-point model and the second output feature map of the first target operator output in the quantization model.

[0101] The normalization module 420 is used to obtain similarity by normalizing the channels in the first output feature map and the channels in the second output feature map, and to determine the channel balance evaluation index that characterizes the quantization fidelity of the first target operator through the similarity. Specifically, it includes: normalizing each channel corresponding to the position in the first output feature map and the second output feature map, calculating the similarity of each pair of normalized corresponding channels, and aggregating the similarity of each corresponding channel to obtain the channel balance similarity that characterizes the quantization fidelity of the first target operator.

[0102] Evaluation module 430 is used to compare channel balance similarity with a preset similarity threshold.

[0103] The decision and adjustment module 440 is used to make decisions and adjustments based on the comparison results of the evaluation module: when the channel balance similarity is less than the similarity threshold, the quantization precision of the first target operator in the quantization model is iteratively updated to the second quantization bit width, and the quantization precision of the second quantization bit width is higher than that of the first quantization bit width, so that the updated quantization model calculates the channel balance similarity to be greater than or equal to the similarity threshold; when the channel balance similarity is greater than or equal to the similarity threshold, the first target operator maintains the current quantization bit width.

[0104] The iterative control module 450 is used to determine the second target operator in the next layer of the quantization model according to the layer order of the network topology in the quantization model after the quantization bit width of the first target operator is confirmed, and obtain the quantization bit width of the second target operator, thereby determining the quantization bit width of each layer in the quantization model layer by layer.

[0105] It is understood that the functional division between the modules illustrated in the embodiments of this application is merely illustrative and does not constitute a functional limitation on the model hierarchical quantization system 400 based on channel balance. In other embodiments of this application, the model hierarchical quantization system 400 based on channel balance may also employ different modules than those in the above embodiments, or a combination of multiple modules, to implement the functions of the model hierarchical quantization system 400 based on channel balance.

[0106] Figure 5 This is a schematic diagram of the hardware structure of a computer device provided in an embodiment of this application. The computer device 500 may include the aforementioned... Figure 1 The system framework 100 shown and as described above Figure 4 The illustrated model is a hierarchical quantization system 400 based on channel balance. (Example:) Figure 5 As shown, the computer device 500 includes: a processor 501, a memory 502, a communication module 504, and a computer program 503 stored in the memory 502 and executable on the processor 501. When the processor 501 executes the computer program 503, it implements the aforementioned... Figures 2 to 3 The execution steps are shown. For example, the computer program 503 described above can be divided into one or more units / modules, which are stored in the memory 502 and executed by the processor 501 to complete this application.

[0107] The aforementioned one or more units / modules may be a series of computer program instruction segments capable of performing a specific function. These instruction segments describe the execution process of the aforementioned computer program 503 within the aforementioned computer device 500. For example, the aforementioned computer program 503 may be used to perform actions such as... Figure 2 The specific functions or mechanisms of the methods shown in steps S101-S106 have been described in the above embodiments and will not be repeated here.

[0108] Those skilled in the art will understand that Figure 5 This is merely an example of computer device 500 and does not constitute a limitation on computer device 500. It may include more or fewer components than shown, or combine certain components, or different components. For example, the computer device 500 described above may also include input / output devices, network access devices, buses, etc.

[0109] The processor 501 mentioned above can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0110] In some embodiments, the processor 501 may include one or more interfaces. These interfaces may include: an internal integrated circuit I2C interface, an integrated circuit built-in audio bus I2S interface, a pulse code modulation (PCM) interface, a universal asynchronous transceiver (URAT) interface, a mobile industry processor MIPI interface, a general purpose input / output (GPIO) interface, an on-board diagnostic (OBD) interface, and / or a universal serial bus (USB) interface, etc. It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the computer device 500. In other embodiments of this application, the computer device 500 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0111] In some embodiments, the computer device 500 can connect internal devices and modules through one or more interfaces. The aforementioned memory 502 can be an internal storage unit of the computer device 500, such as a hard disk or RAM. The aforementioned memory 502 can also include both internal storage units and external storage devices. The aforementioned memory 502 is used to store the aforementioned computer program and other programs and data required by the computer device 500. The aforementioned memory 502 can also be used to temporarily store data that has been output or will be output.

[0112] Communication module 504 can provide solutions for wireless communication applications on computer device 500, including Wireless Local Area Network (WLAN), Bluetooth (BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), and Infrared (IR). Communication module 504 can be one or more communication devices integrating at least one communication processing module. It receives electromagnetic waves via an antenna, demodulates and filters the electromagnetic wave signals, and sends the processed signals to processor 501. Communication module 504 can also receive signals to be transmitted from processor 501, frequency modulate and amplify them, and then convert them into electromagnetic waves for radiation via the antenna.

[0113] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is used as an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the above equipment can be divided into different functional units or modules to complete all or part of the functions described above.

[0114] The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of software functional units.

[0115] In the embodiments of this application, the specific names of each functional unit and module are only for easy distinction and are not intended to limit the scope of protection of this application. It should be understood that each step in the above-described method embodiments provided in this application can be completed by the integrated logic circuits in the processor hardware or by instructions in software form. The method steps disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or being executed by a combination of hardware and software modules in the processor.

[0116] This application also provides a computer program product, which includes: a computer program (also referred to as code or instructions) that, when run, causes a computer to execute the model hierarchical quantization method based on channel balance in the above embodiments.

[0117] The various embodiments of this application can be combined arbitrarily to achieve different technical effects.

[0118] In the embodiments provided in this application, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented, in whole or in part, in the form of a computer program product.

[0119] The computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device.

[0120] This application also provides a computer-readable storage medium storing a computer program (also referred to as code or instructions). When the computer program is run, it causes the computer to perform the method executed by the computer device in any of the foregoing embodiments.

[0121] Figure 6This is a schematic diagram of a computer-readable storage medium provided in an embodiment of this application. For example... Figure 6 As shown, the computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means.

[0122] The computer-readable storage medium can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. The available medium can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Universal Optical Discs, DVDs), or semiconductor media (e.g., solid-state drives, SSDs), etc.

[0123] Those skilled in the art will understand that implementing all or part of the processes in the foregoing embodiments can be accomplished by a computer program instructing related hardware. This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the foregoing method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM or RAM, magnetic disks, or optical disks.

[0124] In summary, the above description is merely an embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made based on the disclosure of this application should be included within the scope of protection of this application.

Claims

1. A model hierarchical quantization method based on channel balance, characterized in that, Includes the following steps: The calibration dataset is input into a pre-trained floating-point model and a quantization model with a preset first quantization bit width. The first output feature map of the first target operator output in the floating-point model and the second output feature map of the first target operator output in the quantization model are obtained. Normalize each channel corresponding to the position in the first output feature map and the second output feature map respectively, calculate the similarity of each pair of normalized corresponding channels, and aggregate the similarity of each pair of corresponding channels to obtain the channel balance similarity. If the channel balance similarity is less than the similarity threshold, the quantization precision of the first target operator in the quantization model is iteratively updated to the second quantization bit width. The quantization precision of the second quantization bit width is higher than that of the first quantization bit width, so that the updated quantization model can calculate the channel balance similarity to be greater than or equal to the similarity threshold. After the quantization bit width of the first target operator is confirmed, the second target operator in the next layer of the quantization model is determined according to the layer order of the network topology in the quantization model. The aforementioned steps are repeated to obtain the quantization bit width of the second target operator, thereby determining the quantization bit width of each layer in the quantization model layer by layer.

2. The method according to claim 1, characterized in that, The normalization process for each channel corresponding to the position in the first output feature map and the second output feature map specifically includes: For each pair of positions corresponding to the channel, a first statistic of the channel in the first output feature map and a second statistic of the channel in the second output feature map are calculated respectively. Both the first statistic and the second statistic include the mean and standard deviation of the corresponding channel. The corresponding channels in the first output feature map are normalized based on the first statistic, and the corresponding channels in the second output feature map are normalized based on the second statistic.

3. The method according to claim 1 or 2, characterized in that, The calculation of the similarity of each pair of normalized corresponding channels, and the aggregation of the similarities of each pair of corresponding channels to obtain the channel balance similarity, specifically includes: Calculate the cosine similarity of each pair of normalized corresponding channels, and calculate the mean based on the cosine similarity of each pair of corresponding channels to obtain the channel balance similarity.

4. The method according to claim 1, characterized in that, The step of iteratively updating the quantization precision of the first target operator in the quantization model to the second quantization bit width specifically includes: The quantization precision of the first target operator in the quantization model is adjusted from the first quantization bit width to the second quantization bit width; The quantization model is updated based on the adjusted quantization precision. Based on the updated quantization model, the second output feature map of the first target operator is re-acquired, and the channel balance similarity is iteratively calculated until the channel balance similarity is greater than or equal to the similarity threshold.

5. The method according to claim 1 or 4, characterized in that, If the channel balance similarity is greater than or equal to the similarity threshold, the first target operator maintains the current quantization bit width.

6. The method according to claim 1, characterized in that, After determining the quantization bit width of each layer in the quantization model layer by layer, the method further includes: A mixed-precision quantization configuration file is generated based on the quantization bit width of each layer in the quantization model. The quantizer generates the quantization model through the mixed-precision quantization configuration file. The mixed-precision quantization configuration file includes the layer name, index, and quantization bit width in the quantization model. Based on the overall quantization accuracy of the quantization model on the validation dataset, the similarity threshold is adjusted. Using the adjusted similarity threshold, the quantization bit width of each layer in the quantization model is redetermined to obtain the optimized quantization model.

7. A model hierarchical quantization system based on channel balance, characterized in that, For implementing the method as described in any one of claims 1 to 6, comprising: The feature acquisition module is used to input the calibration dataset into a pre-trained floating-point model and a quantization model preset to a first quantization bit width, and to acquire a first output feature map of the first target operator output in the floating-point model and a second output feature map of the first target operator output in the quantization model. The normalization module is used to normalize each channel corresponding to the position in the first output feature map and the second output feature map respectively, calculate the similarity of each pair of normalized corresponding channels, and aggregate the similarity of each pair of corresponding channels to obtain the channel balance similarity. An evaluation module is used to compare the channel balance similarity with a preset similarity threshold; The decision and adjustment module is used to make decisions and adjustments based on the comparison results of the evaluation module: when the channel balance similarity is less than the similarity threshold, the quantization precision of the first target operator in the quantization model is iteratively updated to the second quantization bit width, the quantization precision of the second quantization bit width is higher than the quantization precision of the first quantization bit width, so that the updated quantization model calculates the channel balance similarity to be greater than or equal to the similarity threshold. When the channel balance similarity is greater than or equal to the similarity threshold, the first target operator maintains the current quantization bit width. The iterative control module is used to determine the second target operator in the next layer of the quantization model according to the layer order of the network topology in the quantization model after the quantization bit width of the first target operator is confirmed, and repeat the aforementioned steps to obtain the quantization bit width of the second target operator, thereby determining the quantization bit width of each layer in the quantization model layer by layer.

8. A computer device, characterized in that, The device includes one or more memories and one or more processors; the memories are coupled to the one or more processors, the memories are used to store computer program code, the computer program code including computer instructions, and the one or more processors invoke the computer instructions to cause the computer device to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing computer instructions thereon, characterized in that, When the computer instructions are executed by the processor, they implement the method of any one of claims 1 to 6.

Images