Hybrid quantization processing method and system for neural network model

Abstract

The application provides a hybrid quantization processing method and system of a neural network model, and relates to the technical field of neural network models. The method comprises the following steps: identifying and classifying each layer operator of the neural network model, and dividing the network into multiple blocks according to the classification result; in the multiple divided blocks, an unfixed block is selected to be quantized by using multiple bit widths, and other unfixed blocks are quantized by using a preset bit width; evaluating the quantized quantization combination model to obtain evaluation data; selecting the optimal quantization model according to the evaluation data, and fixing the corresponding block quantization result in the model; judging whether the improvement of the current network model compared with the network model selected last time reaches a preset threshold value; if the preset threshold value is not reached, the processing is stopped; if the preset threshold value is reached, the above steps are iteratively processed until the improvement of the current network model compared with the network model selected last time does not reach the preset threshold value.

Description

Technical Field

[0001] This invention relates to the field of neural network model technology, and more particularly to a hybrid quantization processing method and system for neural network models. Background Technology

[0002] This section is intended to provide background or context for the embodiments of the invention set forth in the claims. The description herein is not an admission that it is prior art simply because it is included in this section.

[0003] In existing technologies, quantization is a crucial method for compressing neural network models and a fundamental step in deploying inference models for edge computing chips. The appropriateness of quantization directly affects the deployment performance and accuracy of neural network models on the chip. Common quantization is 8-bit quantization, but in some cases, 8-bit quantization results in significant accuracy loss, requiring higher bit quantization to ensure the accuracy of the quantized model. Furthermore, in some applications, 8-bit quantization offers almost no accuracy loss, and to accelerate inference and conserve chip power, lower bit quantization, such as 4-bit, is chosen. In certain applications, to balance model accuracy, speed, power consumption, and time, mixed bit quantization can also be selected.

[0004] Hybrid quantization models are inherently non-differential optimization problems and cannot be optimized using traditional gradient descent. Another approach, exhaustively searching to calculate the inference time and speed of each combination, is extremely time-consuming. Other methods propose using reinforcement learning to find the optimal hybrid quantization combination. However, reinforcement learning requires training the agent each time, and each training session is only effective for specific networks. For other networks, retraining is required, resulting in low overall efficiency and poor performance of hybrid quantization.

[0005] In summary, there is an urgent need for a technical solution that can overcome the above-mentioned shortcomings and find the optimal combination of hybrid quantization neural network models. Summary of the Invention

[0006] To address the problems existing in the prior art, this invention proposes a hybrid quantization processing method and system for neural network models. This method can find the optimal combination of hybrid quantization neural network models, enabling high-precision, high-efficiency, and low-power operation of neural networks on chips. This invention performs quantization bit-width sensitivity analysis on the model's operators, divides the operators into multiple blocks through identification and classification, and applies hybrid quantization to each block. Simultaneously, it combines feedback information from the chip hardware and accuracy loss to perform iterative selection of high and low bit quantization, resulting in a network model with the minimum accuracy loss and the best hardware performance.

[0007] In a first aspect of the present invention, a hybrid quantization processing method for a neural network model is proposed, comprising:

[0008] The operators of each layer of the neural network model are identified and classified, and the network is divided into multiple blocks according to the classification results;

[0009] Among the multiple blocks, one unfixed block is selected for quantization using multiple bit widths, while the other unfixed blocks are quantized using a preset bit width.

[0010] The quantified combination model is evaluated to obtain evaluation data;

[0011] The optimal quantization model is selected based on the evaluation data, and the corresponding block quantization results in the model are fixed.

[0012] Determine whether the improvement of the current network model compared to the previously selected network model has reached a preset threshold;

[0013] If the preset threshold is not reached, the processing stops; if the preset threshold is reached, the above steps are repeated iteratively until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold.

[0014] In a second aspect of the present invention, a hybrid quantization processing system for a neural network model is proposed, comprising:

[0015] Network partitioning units are used to identify and classify the operators of each layer of the neural network model, and divide the network into multiple blocks based on the classification results;

[0016] The quantization unit is used to select one unfixed block from the multiple blocks to quantize using multiple bit widths, while other unfixed blocks are quantized using a preset bit width.

[0017] The evaluation unit is used to evaluate the quantified combination model and obtain evaluation data.

[0018] The block quantization result fixing unit is used to select the optimal quantization model based on the evaluation data and fix the corresponding block quantization results in the model.

[0019] The judgment unit is used to determine whether the improvement of the current network model compared to the previously selected network model has reached a preset threshold.

[0020] If the preset threshold is not reached, the processing stops; if the preset threshold is reached, the above steps are repeated iteratively until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold.

[0021] In a third aspect of the present invention, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement a hybrid quantization processing method for a neural network model.

[0022] In a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program, which, when executed by a processor, implements a hybrid quantization processing method for a neural network model.

[0023] In a fifth aspect of the present invention, a computer program product is provided, the computer program product comprising a computer program that, when executed by a processor, implements a hybrid quantization processing method for a neural network model.

[0024] The hybrid quantization processing method and system for neural network models proposed in this invention can identify, classify, and block the quantizable layers according to preset rules. Each block is then quantized using all supported bit widths. The optimal quantized network is then evaluated using a corresponding chip compiler and simulator. This optimal quantized network exhibits high accuracy, fast inference speed, and low power consumption. Through iterative iteration, it fully utilizes the structural information of the underlying chip hardware, minimizing the use of unreasonable models and hybrid quantization strategies, thus achieving high-precision, high-efficiency, and low-power operation of the neural network on the chip. This invention can find different hybrid quantization strategies for different hardware architectures, fully utilizing the computing and storage resources of the underlying chip, providing strong technical support for neural network research. Attached Figure Description

[0025] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 This is a schematic diagram of the hybrid quantization processing method for a neural network model according to an embodiment of the present invention.

[0027] Figure 2 This is a detailed flowchart illustrating the identification, classification, and network partitioning process of a specific embodiment of the present invention.

[0028] Figure 3 This is a detailed flowchart illustrating the evaluation quantification combination model according to a specific embodiment of the present invention.

[0029] Figure 4 This is a schematic diagram of the hybrid quantization process of a neural network model according to a specific embodiment of the present invention.

[0030] Figure 5 This is a schematic diagram of the block division relationship in a specific embodiment of the present invention.

[0031] Figure 6 This is a schematic diagram of the relationship of hybrid quantization in a specific embodiment of the present invention.

[0032] Figure 7 This is a schematic diagram of the relationship after the first round of optimization in a specific embodiment of the present invention.

[0033] Figure 8 This is a schematic diagram of the relationship after the second round of optimization in a specific embodiment of the present invention.

[0034] Figure 9 This is a schematic diagram of the relationship between the optimal quantized combination obtained through iterative processing in a specific embodiment of the present invention.

[0035] Figure 10 This is a schematic diagram of the hybrid quantization processing system architecture of a neural network model according to an embodiment of the present invention.

[0036] Figure 11 This is a schematic diagram of a computer device structure according to an embodiment of the present invention. Detailed Implementation

[0037] The principles and spirit of the invention will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement the invention, and are not intended to limit the scope of the invention in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0038] Those skilled in the art will recognize that embodiments of the present invention can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0039] According to embodiments of the present invention, a hybrid quantization processing method and system for neural network models is proposed, relating to the field of neural network model technology. This invention identifies and classifies operators at each layer of the neural network model, categorizing operators at different levels based on their sensitivity to quantization bit width. Blocks are then divided according to the quantization sensitivity levels of each layer's operators. Based on the block division information, each block is quantized for all bits supported by the chip. The quantized model is then compiled and its performance evaluated on the deployed chip, and accuracy loss is tested. The optimal quantization network is comprehensively evaluated, and the quantization result of the corresponding block from the previous iteration is fixed based on the optimal quantization network. If a significant improvement is observed, iterative processing continues, quantizing other blocks for all supported bit widths, evaluating performance and accuracy, and repeating this process until the selected optimal network shows no significant improvement in accuracy or performance compared to the network model selected in the previous iteration. The overall solution can improve the running efficiency of neural network models on chips, ensure the accuracy of neural network models, and reduce chip power consumption.

[0040] The principles and spirit of the present invention will be explained in detail below with reference to several representative embodiments.

[0041] Figure 1 This is a schematic flowchart of a hybrid quantization processing method for a neural network model according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0042] S101, identify and classify the operators of each layer of the neural network model, and divide the network into multiple blocks according to the classification results;

[0043] S102, among the multiple blocks, select one unfixed block and quantize it using multiple bit widths, while the other unfixed blocks are quantized using a preset bit width;

[0044] S103, Evaluate the quantified combination model to obtain evaluation data;

[0045] S104, Select the optimal quantization model based on the evaluation data, and fix the corresponding block quantization results in the model;

[0046] S105, determine whether the improvement of the current network model compared to the previously selected network model has reached a preset threshold;

[0047] If the preset threshold is not reached, (S106) the processing will stop.

[0048] If the preset threshold is reached, the above steps are used for iterative processing until the improvement of the current network model compared with the previously selected network model does not reach the preset threshold (S106), then the processing stops.

[0049] To provide a clearer explanation of the hybrid quantization processing method for the aforementioned neural network model, each step will be described in detail below.

[0050] refer to Figure 2 This is a detailed flowchart illustrating the identification, classification, and network partitioning process according to a specific embodiment of the present invention. Figure 2 As shown, in S101, the specific process of identifying and classifying the operators of each layer of the neural network model, and dividing the network into multiple blocks based on the classification results, is as follows:

[0051] S201, Based on the operations of each layer of operators in the neural network model, identify and classify the operators according to their sensitivity levels;

[0052] For example, convolution operations are of medium quantization sensitivity; operations such as addition, concatenation, max, and flat pooling are of low quantization sensitivity; and non-linear operations, such as squaring or exponential operations, are of high quantization sensitivity.

[0053] In this embodiment, low-sensitivity operators can be independently partitioned or aggregated into high-sensitivity operators.

[0054] S202, based on the recognition and classification results, the network is divided into multiple blocks; each block is quantized with a uniform bit width.

[0055] In S102, the specific process of selecting one unfixed block from the multiple blocks for quantization using multiple bit widths, and quantizing the other unfixed blocks using a preset bit width, is as follows:

[0056] Among the multiple blocks, one unfixed block is selected to quantize all bit widths supported by the chip, while the other unfixed blocks maintain the quantization of the highest bit width supported by the chip.

[0057] For a selected block, quantization can be performed using multiple bit widths supported by the chip, such as 4-bit, 8-bit, 16-bit, or other unfixed blocks, using a single bit width. In practical applications, other unfixed blocks can use the highest 16-bit quantization supported by the chip. Multiple quantization models can be obtained in this way.

[0058] refer to Figure 3 This is a detailed flowchart illustrating the evaluation quantification combination model according to a specific embodiment of the present invention. Figure 3 As shown in S103, the specific process for evaluating the quantified combination model and obtaining the evaluation data is as follows:

[0059] S301 performs accuracy tests on multiple quantized combined models to obtain accuracy loss data, and deploys compilation simulation of the target chip to obtain performance index data;

[0060] Among them, the performance metrics data obtained from the compilation simulation of the target chip include at least: inference speed and power consumption.

[0061] S302, based on the accuracy loss data and performance index data, evaluates the quantization combination model to obtain evaluation data.

[0062] Then, S104 is executed to select the optimal quantization model based on the evaluation data and fix the quantization results of the corresponding blocks in the model; through the above process, the quantization results of the block can be fixed.

[0063] In S105, it is determined whether the improvement of the current network model compared to the previously selected network model has reached a preset threshold.

[0064] If the preset threshold is not reached, the processing stops; if the preset threshold is reached, iterative processing is performed, and mixed quantization processing is carried out in accordance with the above steps (S102-S106).

[0065] Specifically, during iterative processing, one unfixed block is selected from the unfixed blocks and quantized using multiple bit widths, while other unfixed blocks are quantized using a preset bit width. The quantized combination model is evaluated, and the optimal quantization model is selected based on the evaluation data. The quantization results of the corresponding blocks in the model are fixed, and it is determined whether the improvement of the current network model compared to the previously selected network model reaches a preset threshold. If the preset threshold is reached, the above steps are repeated until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold, at which point the iterative processing stops.

[0066] Through the above process, the quantization of the neural network can be adjusted sequentially to optimize the performance of the neural network model.

[0067] To provide a clearer explanation of the hybrid quantization processing method for the aforementioned neural network model, a specific embodiment will be used as an example below.

[0068] refer to Figure 4 This is a schematic diagram illustrating the hybrid quantization process of a neural network model according to a specific embodiment of the present invention. (See reference...) Figure 4 The specific process is as follows:

[0069] S401 identifies and classifies the operator operations of each layer of the network model.

[0070] In practical applications, convolution operations are of medium quantization sensitivity; operations such as addition, concatenation, max, and flat pooling are of low quantization sensitivity to bit width; and nonlinear operations, such as squaring or exponential operations, are of high quantization sensitivity.

[0071] The network is divided into blocks based on sensitivity levels, with different sensitivity levels assigned to different blocks. For details, please refer to [link / reference]. Figure 5 This is a schematic diagram of block partitioning according to a specific embodiment of the present invention. Figure 5 In the image, the left side shows the original network result, and the right side shows the network structure after being divided into blocks, with each block being quantized to a uniform bit.

[0072] S402 performs quantization on each block for all bit widths supported by the chip, while other blocks maintain the highest bit width supported by the chip.

[0073] For example, the selected block can be quantized with various bit widths such as 4-bit, 8-bit, and 16-bit, while other blocks can maintain the highest 16-bit quantization supported by the chip, thus generating multiple quantization models.

[0074] refer to Figure 6 This is a schematic diagram illustrating the relationship of hybrid quantization in a specific embodiment of the present invention. Figure 6 As shown, each block is quantized using three methods: 4-bit, 8-bit, and 16-bit. When adjusting the quantization bit width of a certain block (the special quantization bit width in the figure is in bold), other blocks are quantized according to the 16-bit quantization supported by the chip.

[0075] For example, the first block (Convolution, 128*1000) is quantized using 4-bit, 8-bit, and 16-bit methods respectively, while other blocks are quantized using the 16-bit method supported by the chip. The second block (Convolution, 128*128) is also quantized using 4-bit, 8-bit, and 16-bit methods respectively, while other blocks are quantized using the 16-bit method supported by the chip. All blocks are quantized using the above methods in a mixed quantization manner.

[0076] S403 tests the accuracy of all quantized combination models and performs compilation simulation on the target chip to obtain relevant index data such as inference speed and power consumption.

[0077] Taking inference time as an example, t0 represents the inference time of the highest 16-bit quantized model supported by the chip in the chip simulator, and t represents the inference time of the current iteration model in the simulator; a0 represents the test accuracy of the full floating-point model, and a represents the accuracy of the current iteration quantized model on the test data; according to formula (1), the optimal quantization combination is selected. In order to avoid a0 and a being equal, the values ​​of both can be accurate to four or five decimal places.

[0078]

[0079] The target maximum value (funtion) is obtained through calculation.

[0080] S404: Based on the optimal quantization combination selected by S403, fix the quantization results of the corresponding modules in S403, and then proceed to the next round of iterative optimization.

[0081] In the next round of optimization, the quantization result of the fixed block remains unchanged, while all bits of the other blocks are quantized again.

[0082] by Figure 6 Taking the network structure in the example, if the optimal combination found by S403 is that the first block is 8 bits, in the next round of optimization, the quantization result of this block is fixed, and so on. Figure 7 The way of transformation, in Figure 7 In the diagram, the region within solid box 701 represents the optimal quantization combination found by S403. The first block is quantized to a fixed 8 bits (dashed box 702), and then the other blocks are quantized to all supported bits.

[0083] Furthermore, all quantization combination models are compiled and simulated to test accuracy, and compiled and simulated on the target chip to obtain relevant index data such as inference speed and power consumption. The optimal quantization combination is selected according to the S403 formula. If the target maximum value of this iteration is significantly improved compared with the previous result, the next round of optimization is continued; if the target maximum value of this iteration is not significantly improved compared with the previous result, the iteration is stopped.

[0084] If the target maximum value in this iteration is significantly improved compared to the previous result, then the next round of optimization continues; for example, if the optimal combination found in the previous iteration was that the third block was 4 bits, then in the next round of optimization, the quantization result of that block is fixed, and so on. Figure 8 The way of transformation, in Figure 8 In the diagram, the solid box 801 represents the currently found optimal quantization combination, where the first block's convolutional quantization is fixed at 8 bits, and the third block's convolutional quantization is fixed at 4 bits (in the dashed box 802, the first and third blocks are blocks with fixed quantization results). Then, the other blocks are quantized to all supported bits.

[0085] Then, all the quantization combination models are compiled and simulated to test their accuracy, and the compilation and simulation of the target chip are deployed to obtain relevant index data such as inference speed and power consumption. The optimal quantization combination is selected according to the formula of S403.

[0086] The iteration continues until the target maximum value in the current iteration is no longer significantly higher than the previous result, at which point the iteration stops. For example, after iterative processing, the final result can be obtained... Figure 9 The optimal quantization combination.

[0087] This invention analyzes the sensitivity of the operator to quantization bit precision and combines feedback information from the chip hardware to perform iterative selection of high and low bit quantization, ultimately obtaining a network model with minimal precision loss and the best hardware performance.

[0088] It should be noted that although the operation of the method of the present invention has been described in a specific order in the above embodiments and figures, this does not require or imply that the operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0089] After introducing the method of exemplary embodiments of the present invention, the following references are made. Figure 10 A hybrid quantization processing system for a neural network model according to an exemplary embodiment of the present invention will be described.

[0090] The implementation of the hybrid quantization processing system for neural network models can refer to the implementation of the methods described above, and will not be repeated here. The terms "module" or "unit" used below can refer to a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0091] Based on the same inventive concept, this invention also proposes a hybrid quantization processing system for neural network models, such as... Figure 10 As shown, the system includes:

[0092] The network partitioning unit 1010 is used to identify and classify the operators of each layer of the neural network model, and divide the network into multiple blocks according to the classification results;

[0093] The quantization unit 1020 is used to select one unfixed block from the multiple blocks to quantize using multiple bit widths, and to quantize the other unfixed blocks using a preset bit width.

[0094] Evaluation unit 1030 is used to evaluate the quantified combination model and obtain evaluation data.

[0095] The block quantization result fixing unit 1040 is used to select the optimal quantization model based on the evaluation data and fix the corresponding block quantization results in the model.

[0096] The judgment unit 1050 is used to determine whether the improvement of the current network model compared to the previously selected network model has reached a preset threshold.

[0097] If the preset threshold is not reached, the processing stops; if the preset threshold is reached, the above steps are repeated iteratively until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold.

[0098] In one embodiment, the network partitioning unit 1010 is specifically used for:

[0099] Based on the operations of each layer of operators in the neural network model, the operators are identified and classified according to their sensitivity levels;

[0100] Based on the identification and classification results, the network is divided into multiple blocks; each block is quantized with a uniform bit width.

[0101] In one embodiment, the quantization unit 1020 is specifically used for:

[0102] Among the multiple blocks, one unfixed block is selected for quantization using all bit widths supported by the chip, while the other unfixed blocks are quantized using the highest bit width supported by the chip.

[0103] In one embodiment, the evaluation unit 1030 is specifically used for:

[0104] Accuracy tests were performed on multiple quantized combined models to obtain accuracy loss data, and performance index data was obtained by compiling and simulating the target chip.

[0105] The quantization combination model is evaluated based on the accuracy loss data and performance index data to obtain evaluation data.

[0106] In one embodiment, the performance metrics data obtained from the compilation simulation of the target chip include at least: inference speed and power consumption.

[0107] In one embodiment, during iterative processing, a quantization unit 1020, an evaluation unit 1030, a block quantization result fixing unit 1040, and a judgment unit 1050 are invoked. One unfixed block is selected from the unfixed blocks and quantized using multiple bit widths. Other unfixed blocks are quantized using preset bit widths. The quantized combination model is evaluated, and the optimal quantization model is selected based on the evaluation data. The corresponding block quantization results in the model are fixed. It is determined whether the improvement of the current network model compared to the previously selected network model reaches a preset threshold. If the preset threshold is reached, the above steps are repeated until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold, at which point the iterative processing stops.

[0108] It should be noted that although several modules of the hybrid quantization processing system for neural network models have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in a single module. Conversely, the features and functions of a single module described above can be further divided and embodied by multiple modules.

[0109] Based on the aforementioned inventive concept, such as Figure 11 As shown, the present invention also proposes a computer device 1100, including a memory 1110, a processor 1120, and a computer program 1130 stored in the memory 1110 and executable on the processor 1120. When the processor 1120 executes the computer program 1130, it implements the aforementioned hybrid quantization processing method for the neural network model.

[0110] Based on the aforementioned inventive concept, the present invention proposes a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned hybrid quantization processing method for the neural network model.

[0111] Based on the aforementioned inventive concept, the present invention proposes a computer program product, which includes a computer program that, when executed by a processor, implements a hybrid quantization processing method for a neural network model.

[0112] The hybrid quantization processing method and system for neural network models proposed in this invention can identify, classify, and block the quantizable layers according to preset rules. Each block is then quantized using all supported bit widths. The optimal quantized network is then evaluated using a corresponding chip compiler and simulator. This optimal quantized network exhibits high accuracy, fast inference speed, and low power consumption. Through iterative iteration, it fully utilizes the structural information of the underlying chip hardware, minimizing the use of unreasonable models and hybrid quantization strategies, thus achieving high-precision, high-efficiency, and low-power operation of the neural network on the chip. This invention can find different hybrid quantization strategies for different hardware architectures, fully utilizing the computing and storage resources of the underlying chip, providing strong technical support for neural network research.

[0113] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0114] This invention is described with reference to flowchart illustrations and / or block diagrams of methods and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0115] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0116] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0117] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A hybrid quantization processing method for a neural network model, characterized in that, include: The operators of each layer of the neural network model are identified and classified, and the network is divided into multiple blocks according to the classification results; Among the multiple blocks, one unfixed block is selected for quantization using multiple bit widths, while the other unfixed blocks are quantized using a preset bit width. The quantified combination model is evaluated to obtain evaluation data; The optimal quantization model is selected based on the evaluation data, and the corresponding block quantization results in the model are fixed. Determine whether the improvement of the current network model compared to the previously selected network model has reached a preset threshold; If the preset threshold is not reached, the processing stops; if the preset threshold is reached, the above steps are repeated until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold. This involves identifying and classifying the operators at each layer of the neural network model, and dividing the network into multiple blocks based on the classification results, including: Based on the operations of each layer of the neural network model, the operators are identified and classified according to their sensitivity levels. Among them, convolution operations are classified as medium-level quantization sensitivity; operations such as addition, concatenation, max, and average pooling are classified as low-level quantization sensitivity; and non-linear operations such as squaring or exponential operations are classified as high-level quantization sensitivity. Based on the identification and classification results, the network is divided into multiple blocks; each block is quantized with a uniform bit width. Among the multiple blocks, one unfixed block is selected for quantization using multiple bit widths, while the other unfixed blocks are quantized using preset bit widths, including: Among the multiple blocks, one unfixed block is selected for quantization using all bit widths supported by the chip, while the other unfixed blocks are quantized using the highest bit width supported by the chip.

2. The method according to claim 1, characterized in that, The quantified combination model was evaluated to obtain evaluation data, including: Accuracy tests were performed on multiple quantized combined models to obtain accuracy loss data, and performance index data was obtained by compiling and simulating the target chip. The quantization combination model is evaluated based on the accuracy loss data and performance index data to obtain evaluation data.

3. The method according to claim 2, characterized in that, The performance metrics data obtained from the compilation simulation of the target chip should include at least: inference speed and power consumption.

4. The method according to claim 2, characterized in that, If the preset threshold is reached, the above steps are iteratively performed until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold. This also includes: During iterative processing, one unfixed block is selected from the unfixed blocks and quantized using multiple bit widths, while other unfixed blocks are quantized using preset bit widths. The quantized combination model is evaluated, and the optimal quantization model is selected based on the evaluation data. The quantization results of the corresponding blocks in the model are fixed, and it is determined whether the improvement of the current network model compared to the previously selected network model reaches a preset threshold. If the preset threshold is reached, the above steps are repeated until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold, at which point the iterative processing stops.

5. A hybrid quantization processing system for a neural network model, characterized in that, include: Network partitioning units are used to identify and classify the operators of each layer of the neural network model, and divide the network into multiple blocks based on the classification results; The quantization unit is used to select one unfixed block from the multiple blocks to quantize using multiple bit widths, while other unfixed blocks are quantized using a preset bit width. The evaluation unit is used to evaluate the quantified combination model and obtain evaluation data. The block quantization result fixing unit is used to select the optimal quantization model based on the evaluation data and fix the corresponding block quantization results in the model. The judgment unit is used to determine whether the improvement of the current network model compared to the previously selected network model has reached a preset threshold. If the preset threshold is not reached, the processing stops; if the preset threshold is reached, the above steps are repeated until the improvement of the current network model compared to the previously selected network model does not reach the preset threshold. Specifically, the network partitioning unit is used for: Based on the operations of each layer of the neural network model, the operators are identified and classified according to their sensitivity levels. Among them, convolution operations are classified as medium-level quantization sensitivity; operations such as addition, concatenation, max, and average pooling are classified as low-level quantization sensitivity; and non-linear operations such as squaring or exponential operations are classified as high-level quantization sensitivity. Based on the identification and classification results, the network is divided into multiple blocks; each block is quantized with a uniform bit width. Specifically, the quantization unit is used for: Among the multiple blocks, one unfixed block is selected for quantization using all bit widths supported by the chip, while the other unfixed blocks are quantized using the highest bit width supported by the chip.

6. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 4.

8. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 4.

Images