Hybrid precision quantification method of deep neural network and related device

Abstract

The invention belongs to the technical field of mixing precision quantification, and discloses a mixing precision quantification method of a deep neural network and a related device. The mixing precision quantification method comprises the following steps: carrying out diversity processing on a population by using a variation strategy in a differential evolution algorithm to obtain a variation population; recombining the sub-components of the variation individuals in the variation population with the sub-components of the target individuals in the current population by using binomial crossover operation to obtain a plurality of test individuals; performing hybrid precision quantification on the deep neural network based on the test individual and the target individual, and selecting a better individual from the test individual and the target individual; and based on the selected better individual, judging whether the differential evolution algorithm converges or not, if so, obtaining the deep neural network after final mixing precision quantification, and otherwise, selecting the obtained better individual as a population of the next generation. The method can realize mixing precision quantification of the deep neural network, and has the advantages of relatively low calculation complexity and relatively high adaptability.

Description

Technical Field

[0001] The present invention belongs to the technical field of mixed precision quantization, and in particular relates to a mixed precision quantization method and related devices for a deep neural network. Background Art

[0002] In deep learning, convolutional neural networks (CNN) are usually used to solve object classification tasks; to explain, CNN is a variant of the deep neural network (DNN) architecture, which is widely used in machine vision tasks such as image classification and object detection, and has achieved great success. Neural networks are computationally intensive and memory intensive, with the characteristics of large storage space occupation and high power consumption, and are difficult to deploy on embedded systems with limited hardware resources. In view of the above situation, deep neural networks need to be lightweight to reduce the memory occupation and the required amount of calculation, so that they can be deployed in embedded devices and truly implement powerful classification and recognition performance in scientific research and practical application scenarios; therefore, the lightweight of deep neural networks is of great significance to promote the development of deep neural network technology from theory to practical application and improve the conversion rate of scientific research results.

[0003] Explanatorily, quantization is a technique for optimizing deep learning models by converting model parameters and / or intermediate computational results from 32-bit floating-point numbers (single precision) into lower-bit integer representations to reduce the model's storage requirements and computational costs. Common quantization techniques include binary quantization, ternary quantization, quaternary quantization, and fixed-point quantization. These techniques can greatly reduce the model's storage space and computational resource requirements, making it possible to deploy deep learning models on resource-constrained devices. Further explanatorily, mixed-precision quantization is a specific form of quantization technology. Compared with the above-mentioned common quantization techniques, the mixed-precision quantization method assigns different quantization bit widths to each layer of neural network parameters, taking advantage of the inconsistent quantization sensitivity of the parameters at each layer.

[0004] The existing mixed-precision quantization technology still has the defects of high computational complexity and poor adaptability. A new neural network mixed-precision quantization technology is urgently needed to solve the above problems. Specifically, the existing scheme of using reinforcement learning to perform mixed-precision quantization of deep neural networks requires a lot of training and interaction. The existing scheme of using the feature information of the model to perform mixed-precision quantization depends on the specific structure and type of the neural network and has poor adaptability. Summary of the invention

[0005] The purpose of the present invention is to provide a mixed precision quantization method and related device for deep neural networks to solve one or more of the above-mentioned technical problems. In the technical solution provided by the present invention, by exploring the influence of weight distribution on quantization error, the differential evolution algorithm is used to find the optimal combination of setting different bit width quantization for different layers, so as to realize mixed precision quantization of deep neural networks, which has the advantages of relatively low computational complexity and strong adaptability.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a mixed precision quantization method for a deep neural network, comprising the following steps:

[0008] Step 1, based on the differential evolution algorithm, randomly generate multiple initial quantization models of the deep neural network to be mixed precision quantized, and use the multiple initial quantization models and elite vectors as individuals to obtain an initialized population; wherein the elite vector is based on the skewness and kurtosis of the weights of each layer of the deep neural network to be mixed precision quantized, and the obtained quantization bit width dictionary is determined; the value of each subcomponent in each individual in the initialized population represents the quantization bit width of each layer of the deep neural network;

[0009] Step 2: Use the mutation strategy in the differential evolution algorithm to diversify the population and obtain a mutant population;

[0010] Step 3, using a binomial crossover operation, recombining the subcomponents of the mutant individuals in the mutant population with the subcomponents of the target individuals in the current population to obtain multiple test individuals;

[0011] Step 4: Based on the test individuals and the target individuals, the deep neural network is quantized with mixed precision to obtain the deep neural network after mixed precision quantization; the test accuracy of the deep neural network after mixed precision quantization on the test set and the weight compression ratio of the deep neural network after mixed precision quantization are used as the fitness function of the differential evolution algorithm to select the better individual from the test individuals and the target individuals;

[0012] Step 5: Based on the selected better individuals, determine whether the differential evolution algorithm converges. If it converges, the final deep neural network after mixed precision quantization is obtained. Otherwise, the selected better individuals are used as the next generation population, and steps 2 to 5 are repeated.

[0013] A further improvement of the present invention is that

[0014] Based on the skewness and kurtosis of the weights of each layer of the deep neural network to be quantized with mixed precision, the steps of determining to obtain a quantization bit width dictionary include:

[0015] The kurtosis and skewness of each layer weight of the deep neural network are calculated in layers, and the kurtosis and skewness of each layer weight are mapped to a preset interval using different mapping methods; the mapped values ​​of the kurtosis and skewness of each layer weight are added in proportion to obtain the quantitative sensitivity weight value of each layer;

[0016] Divide the interval between the maximum value and the minimum value of the quantization sensitivity weight value of each layer into multiple intervals according to the number of quantization bit width values, and each of the divided intervals corresponds to a quantization bit width value;

[0017] The quantization bit width values ​​represented by the intervals to which the quantization sensitivity weight values ​​of each layer belong are allocated to the corresponding layers of the deep neural network to obtain a quantization bit width dictionary.

[0018] A further improvement of the present invention is that

[0019] For the kth layer of the deep neural network, the calculation, mapping, and proportional addition expressions of kurtosis and skewness are:

[0020]

[0021]

[0022] Ku′ k =((Ku k -Ku min ) / (Ku max -Ku min ))×(b-α)+a;

[0023] Sk′ k =b-(|Sk k |-|Sk| min ) / (|Sk| max -|Sk| min )×(ba);

[0024] M k =α×Ku′ k +β×Sk′ k ;

[0025] In the formula, Ku k is the kurtosis of the kth layer; x i is the i-th weight value of the k-th layer, μ is the mean weight value of the k-th layer, N is the total number of weight values ​​in the kth layer; δ is the standard deviation of the N weight values ​​in the kth layer; Sk k is the skewness of the kth layer; a is the minimum quantization bit width value that the deep neural network can quantize to, and b is the maximum quantization bit width value that the deep neural network can quantize to; Ku′ kis the value after kurtosis mapping of the kth layer; Ku min is the minimum value of the kurtosis value of each layer; Ku max is the maximum value of the kurtosis value of each layer; Sk′ k is the value after skewness mapping of the kth layer; |Sk| min is the minimum value among the absolute values ​​of the skewness values ​​of each layer; |Sk| max is the maximum value of the absolute value of the skewness of each layer; M k is the quantization sensitivity weight value of the kth layer, α and β are the proportional coefficients of kurtosis and skewness respectively.

[0026] A further improvement of the present invention is that

[0027] The steps of obtaining the initialized population with multiple initial quantization models and elite vectors as individuals include:

[0028] Assign the value of the elite vector to the first individual in the population;

[0029] Excluding the first individual, the j-th subcomponent value of the d-th individual is expressed as:

[0030] X d，j =a+rand(0,1)×(ba);

[0031] Where, X d,j is the jth subcomponent value of the dth individual; rand(0,1) represents a random real number between (0,1); a is the minimum quantization bit width value that the deep neural network can quantize to, and b is the maximum quantization bit width value that the deep neural network can quantize to.

[0032] A further improvement of the present invention is that

[0033] Step 4 specifically includes the following steps:

[0034] Step 4.1: The obtained experimental individuals and target individuals The sub-component values ​​of are used as the quantization bit widths of each layer of the deep neural network. The deep neural network is quantized with mixed precision using an asymmetric quantization method to obtain the experimental individual and target individuals Mixed precision quantized deep neural network; where G is the number of iterations;

[0035] Step 4.2, obtain the test individuals and target individuals Weight compression ratio of deep neural networks with mixed precision quantization And the test accuracy on the test set

[0036] Step 4.3, based on the obtained weight compression ratio And the test accuracy on the test set For test individuals and target individuals Make a selection and take the individuals that meet the selection rules as the better individuals.

[0037] A further improvement of the present invention is that

[0038] The selection rule is:

[0039] When the test accuracy of the test set corresponding to the test individual and the target individual both meet the requirements of the deep neural network for accuracy loss, and the weight compression ratio corresponding to the test individual is larger, the test individual is selected as the better individual;

[0040] When the test accuracy of the test set corresponding to the test individual meets the requirements of the deep neural network for accuracy loss, and the test accuracy of the test set corresponding to the target individual does not meet the requirements of the deep neural network for accuracy loss, the test individual is selected as the better individual;

[0041] When the test accuracy of the test set corresponding to the test individual and the target individual does not meet the requirements of the deep neural network for accuracy loss, and the weight compression ratio corresponding to the test individual is larger, the test individual is selected as the better individual;

[0042] In other cases, the target individual is selected as the better individual.

[0043] A further improvement of the present invention is that

[0044] In step 3, the crossover factor of the binomial crossover operation is set to 0.9.

[0045] In a second aspect of the present invention, a mixed precision quantization system for a deep neural network is provided, which is used to implement the mixed precision quantization method for a deep neural network according to the first aspect of the present invention. The mixed precision quantization system for a deep neural network comprises:

[0046] A population initialization module is used to randomly generate multiple initial quantization models of a deep neural network to be quantized with mixed precision based on a differential evolution algorithm, and to obtain an initialized population using multiple initial quantization models and elite vectors as individuals; wherein the elite vector is a quantization bit width dictionary obtained by determining the skewness and kurtosis of the weights of each layer of the deep neural network to be quantized with mixed precision; the value of each subcomponent in each individual in the initialized population represents the quantization bit width of each layer of the deep neural network;

[0047] The mutant population acquisition module is used to use the mutation strategy in the differential evolution algorithm to process the population diversity and obtain the mutant population;

[0048] A test individual acquisition module is used to use a binomial crossover operation to recombine the subcomponents of the mutant individuals in the mutant population with the subcomponents of the target individuals in the current population to obtain multiple test individuals;

[0049] A better individual acquisition module is used to perform mixed precision quantization on the deep neural network based on the test individuals and the target individuals to obtain the deep neural network after mixed precision quantization; the test accuracy of the deep neural network after mixed precision quantization on the test set and the weight compression ratio of the deep neural network after mixed precision quantization are used as the fitness function of the differential evolution algorithm to select better individuals from the test individuals and the target individuals;

[0050] The convergence judgment module is used to judge whether the differential evolution algorithm converges based on the selected better individuals. If converged, the final deep neural network after mixed precision quantization is obtained. Otherwise, the better individuals selected are used as the next generation of population, and the steps in the variant population acquisition module, the trial individual acquisition module, the better individual acquisition module and the convergence judgment module are repeatedly executed.

[0051] According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the mixed precision quantization method for a deep neural network as described in any one of the first aspects of the present invention is implemented.

[0052] In a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the mixed precision quantization method for a deep neural network as described in any one of the first aspects of the present invention is implemented.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] In the mixed precision quantization method of deep neural network provided by the present invention, in view of the computational complexity problem brought by the existing mixed precision quantization technology based on reinforcement learning, a differential evolution algorithm (Differential Evolution, DE) is used to search for the optimal quantization bit width of each layer of the neural network. Compared with the complex policy network training, environment modeling and a large number of exploration processes of reinforcement learning, the technical solution of the present invention only needs to obtain the accuracy of the quantization bit width corresponding to the individual in the differential evolution algorithm on the test set and the compression multiple brought to the neural network, which can reduce the computational complexity to a certain extent; in addition, in view of the existing mixed precision quantization technology with the help of neural network model feature information, the present invention obtains the distribution information of the weights of each layer of the neural network, and searches for the optimal quantization bit width with the help of this distribution information. Although it uses the feature information of the model network to a certain extent, the subsequent search for the optimal quantization bit width with the help of the differential evolution algorithm can make the technical solution of the present invention show strong adaptability when facing different neural network models, better explore the appropriate quantization configuration, and reduce the accuracy loss caused by quantization. In summary, the present invention explores the influence of weight distribution on quantization error and uses the differential evolution algorithm to find the optimal combination of setting different bit width quantization for different layers, so as to realize mixed precision quantization of deep neural networks; compared with some existing methods such as reinforcement learning, the present invention has the advantage of relatively low computational complexity; compared with some existing mixed precision quantization techniques that rely on feature information of neural network models, the present invention does not depend on the specific structure and type of neural network, and can show strong adaptability when facing different neural network models, and can be applied to various complex models with unique structures and characteristics. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below; obviously, the drawings described below are some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0056] Figure 1 It is a flow chart of a mixed precision quantization method for a deep neural network in an embodiment of the present invention.

[0057] Figure 2 It is a schematic diagram of the flow of the differential evolution algorithm in an embodiment of the present invention.

[0058] Figure 3 It is a schematic diagram of a mixed precision quantization system for a deep neural network in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0060] See also Figure 1 and Figure 2 In an embodiment of the present invention, a mixed precision quantization method for a deep neural network is provided, specifically a mixed precision quantization method for a deep neural network based on a differential evolution algorithm, and the specific steps are as follows:

[0061] Step 1, based on the differential evolution algorithm, randomly generate multiple initial quantization models of the deep neural network to be mixed precision quantized, and use the multiple initial quantization models and elite vectors as individuals to obtain an initialized population; wherein the elite vector is based on the skewness and kurtosis of the weights of each layer of the deep neural network to be mixed precision quantized, and the obtained quantization bit width dictionary is determined; the value of each subcomponent in each individual in the initialized population represents the quantization bit width of each layer of the deep neural network;

[0062] A further exemplary scheme includes: first, based on the weight distribution of each layer of the deep neural network to be mixed precision quantized (exemplarily, it can be a trained deep neural network model), the skewness and kurtosis of the weights of each layer of the deep neural network are obtained; secondly, based on the skewness and kurtosis of the weights of each layer of the deep neural network, a better quantization bit width dictionary is determined and used as an elite vector; then, multiple initial quantization models are added with the elite vector to form an initialized population corresponding to the deep neural network;

[0063] Step 2: Use the mutation strategy in the differential evolution algorithm to diversify the population and obtain a mutant population;

[0064] Step 3, using a binomial crossover operation, the mutant individuals in the mutant population are recombined with the subcomponents of the target individuals in the current population to obtain multiple test individuals; in a specific exemplary scheme, the crossover factor of the binomial crossover operation is 0.9;

[0065] Step 4: Based on the test individuals and the target individuals, the deep neural network is quantized with mixed precision to obtain the deep neural network after mixed precision quantization; the test accuracy of the deep neural network after mixed precision quantization on the test set and the weight compression ratio of the deep neural network after mixed precision quantization are used as the fitness function of the differential evolution algorithm to select better individuals from the test individuals and the target individuals;

[0066] Step 5: Based on the selected better individuals, determine whether the differential evolution algorithm converges. If it converges, the final deep neural network after mixed precision quantization is obtained. Otherwise, the better individuals obtained by selection are used as the next generation population, and steps 2 to 5 are repeated.

[0067] In the technical solution provided by the embodiment of the present invention, by exploring the influence of weight distribution on quantization error, the differential evolution algorithm is used to find the optimal combination of setting different bit width quantization for different layers, thereby realizing mixed precision quantization of deep neural networks; compared with some existing methods such as reinforcement learning, the present invention only needs to iterate the results of the differential evolution algorithm until convergence in a specific technical solution, and does not need to perform repeated trial and error and learning processes like reinforcement learning, thereby reducing the computational complexity; compared with some existing mixed precision quantization technologies that rely on feature information of neural network models, the technical solution of the present invention does not require special adjustment and optimization for each deep neural network, does not depend on the specific structure and type of the neural network, and can achieve mixed precision quantization by exploring weight distribution and using differential evolution algorithms when facing different neural network models. It has a wider adaptability and can be applied to various complex models with unique structures and characteristics.

[0068] In one embodiment of the present invention, in step 1, based on the differential evolution algorithm, a plurality of initial quantization models of the deep neural network to be mixed precision quantized are randomly generated, and the steps of obtaining the initialized population using the plurality of initial quantization models and elite vectors as individuals specifically include:

[0069] The deep neural network to be quantized with mixed precision is an N f A deep neural network F of layers;

[0070] Based on the skewness and kurtosis of the weights of each layer of the deep neural network F, the steps of determining to obtain the quantization bit width dictionary are as follows: hierarchically calculating the kurtosis and skewness of the weights of each layer of the deep neural network F, mapping the kurtosis and skewness of the weights of each layer to a preset [a, b] interval using different mapping methods, and adding the mapped values ​​of the kurtosis and skewness in proportion to obtain the quantization sensitivity weight value of each layer;

[0071] Divide the interval between the maximum value and the minimum value of the quantization sensitivity weight value of each layer into multiple intervals according to the number of quantization bit width values, and each of the divided intervals corresponds to a quantization bit width value;

[0072] Assigning the quantization bit width value represented by the interval to which the quantization sensitivity weight value of each layer belongs to the corresponding layers of the deep neural network to obtain a quantization bit width dictionary;

[0073] Specifically and exemplarily, for the kth layer of the deep neural network F, the calculation expression is:

[0074]

[0075]

[0076] Ku′ k =((Ku k -Ku min ) / (Ku max -Ku min ))×(ba)+a;

[0077] Sk′ k =b-(|Sk k |-|Sk| min ) / (|Sk| max -|Sk| min )×(b-α);

[0078] M k =α×Ku′ k +β×Sk′ k ;

[0079] In the formula, Ku k is the kurtosis of the kth layer; x i is the i-th weight value of the k-th layer, μ is the mean weight value of the k-th layer, N is the total number of weight values ​​in the kth layer; δ is the standard deviation of the N weight values ​​in the kth layer; Sk k is the skewness of the kth layer; a is the minimum quantization bit width value that the deep neural network F can be quantized to, and b is the maximum quantization bit width value that the deep neural network F can be quantized to; Ku′ k is the value after kurtosis mapping of the kth layer; Ku min is the minimum value of the kurtosis value of each layer; Ku max is the maximum value of the kurtosis value of each layer; Sk′ k is the value after skewness mapping of the kth layer; |Sk| min is the minimum value among the absolute values ​​of the skewness values ​​of each layer; |Sk| max is the maximum value of the absolute value of the skewness of each layer; M kis the quantization sensitivity weight value of the kth layer, α and β are the proportional coefficients of kurtosis and skewness, respectively, which are used to balance the influence of kurtosis and skewness;

[0080] In the step of obtaining an initialized population using multiple initial quantization models and elite vectors as individuals,

[0081] The size NP of the initialized population is set to 100, the dimension D of the individual is equal to the number of layers of the deep neural network, the upper bound of the search space is b (interpretatively, the maximum quantization bit width is b bits; in the exemplary scheme, the value of b can be taken as 8), and the lower bound of the search space is a (interpretatively, the minimum quantization bit width is a bits; in the exemplary scheme, the value of a can be taken as 2; the value of the elite vector is assigned to the first individual, and the values ​​of the remaining individuals are randomly generated. For the dth individual, the value of the jth dimension is randomly generated between a and b, and then each sub-component value is rounded to ensure that each sub-component value of the individual is an integer, expressed as:

[0082] X d，j =a+rand(0,1)×(ba);

[0083] Where, X d，j is the jth subcomponent value of the dth individual; rand(0,1) represents a random real number between (0,1);

[0084] For each set of d and j, rand(0,1) will regenerate new values, independent of each other; after initialization, the population will perform mutation, crossover, and selection operations in sequence until the algorithm meets the termination condition.

[0085] In one embodiment of the present invention, step 4 specifically includes:

[0086] The experimental individuals and target individuals The sub-component values ​​of are used as the quantization bit widths of each layer of the deep neural network. The deep neural network is quantized with mixed precision using an asymmetric quantization method to obtain the experimental individual and target individuals Mixed precision quantized deep neural network; where G is the number of iterations;

[0087] Obtaining test individuals and target individuals Weight compression ratio of deep neural networks with mixed precision quantization And the test accuracy on the test set

[0088] According to the obtained weight compression ratio and the test accuracy on the test set, the test individuals are and target individuals Select individuals with good weight compression ratio and test accuracy on the test set to enter the next generation of population;

[0089] Among them, for the dth individual The selection rule is expressed as:

[0090] when and and hour,

[0091] When the test accuracies on the test sets corresponding to the test individual and the target individual meet the requirements of the deep neural network for accuracy loss and the weight compression ratio corresponding to the test individual is larger, the test individual is selected to enter the next generation.

[0092] when and hour,

[0093] When the test accuracy of the test set corresponding to the test individual meets the requirements of the deep neural network for accuracy loss, but the test accuracy of the test set corresponding to the target individual does not meet the requirements of the deep neural network for accuracy loss, the test individual is selected to enter the next generation.

[0094] when and and hour,

[0095] When the test accuracy of the test sets corresponding to the test individual and the target individual does not meet the requirements of the deep neural network for accuracy loss, but the corresponding weight compression ratio of the test individual is larger, the test individual is selected to enter the next generation.

[0096] In other cases, select the target individual to enter the next generation

[0097] In the formula, acc F is the original accuracy of the trained deep neural network before quantization on the test set, Δ acc is the maximum acceptable accuracy loss. Representatives are selected to be individuals in the next generation population.

[0098] In summary, the embodiment of the present invention specifically discloses a mixed precision quantization scheme for a deep neural network based on a differential evolution algorithm, which uses the distribution information of the weights of each layer of the deep neural network to obtain a better individual (elite vector) with quantization bit width information of each layer, and randomly generates multiple initial quantization models for the deep neural network, and obtains an initialized population based on the combination of the elite vector and multiple initial quantization models; the population is processed for diversity using mutation and crossover operators, and the test accuracy of the quantized deep neural network model on the test set and the weight compression ratio of the quantized deep neural network model after compression are used as the objective function of the differential evolution algorithm, in order to find a better individual to guide the quantization of the neural network. The above method of the embodiment of the present invention reduces the calculation and storage requirements of the network model, while improving the operating efficiency of the model in resource-constrained environments such as embedded devices, and can realize edge intelligence and mobile applications.

[0099] The feasibility of the technical solution of the embodiment of the present invention is explained in principle. By analyzing the properties related to quantization of different layers of the neural network, it is found that in a deep neural network, the structures of different layers are different, and the properties related to quantization are also different. Therefore, performing the same quantization on all layers is not the optimal result. Trying to set different bit widths for quantization of different layers can obtain better results. The technical solution of the embodiment of the present invention searches for the optimal quantization bit width of each layer of the neural network by utilizing the distribution state of the weights of each layer of the neural network and the differential evolution algorithm, so that the neural network model can significantly compress the model while meeting the accuracy requirements of the test set.

[0100] In a specific embodiment of the present invention, based on the Windows operating system and the Pytoch framework, deep neural networks such as LeNet5 and ResNet20 are used to test the Mnist dataset and the CIFAR-10 dataset respectively; in this embodiment of the present invention, the quantization bits used for weights are abbreviated as "w bits", the test accuracy is abbreviated as "Acc", and the compression ratio of weights is abbreviated as "W-Comp". The experimental results are shown in Tables 1 and 2.

[0101] Table 1. Quantization results of LeNet-5 on Mnist and quantization bit width of each layer

[0102]

[0103]

[0104] In the embodiment of the present invention, the accuracy of the LeNet-5 original model provided by Pytorch on the Mnist data set reaches 99.14%; as can be seen from Table 1, the neural network quantization (DEQ) method based on the differential evolution algorithm proposed in the present invention achieves a good compression effect, and the quantization of the neural network weights reaches a compression ratio of 11.53×, and the accuracy rate only decreases by 0.05%, which shows that the quantization method proposed in the present invention is effective and achieves a relatively high compression ratio with an extremely low accuracy decrease.

[0105] Table 2. Comparison of quantization results of ResNet20 on Cifar-10

[0106] W-Comp Accuracy decrease (%) DEQ 10.31× 2.63 4-bit integer quantization 8.00× 3.83

[0107] In the embodiment of the present invention, the accuracy of the ResNet20 original model provided by Pytorch on the Cifar-10 dataset reaches 92.02%; it can be seen from Table 2 that the neural network quantization (DEQ) method based on the differential evolution algorithm proposed in the present invention achieves a good compression effect on the ResNet20 model, and the quantization of the neural network weights reaches a compression ratio of 10.31×, and the accuracy rate only decreases by 2.63%. Compared with the 4-bit integer quantization method, the method of the embodiment of the present invention is more effective and achieves a greater compression ratio and model accuracy than it.

[0108] The following are device embodiments of the present invention, which can be used to implement the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0109] See also Figure 3 In an embodiment of the present invention, a mixed precision quantization system for a deep neural network is provided, which is used to implement the mixed precision quantization method for a deep neural network described above in an embodiment of the present invention. The mixed precision quantization system for a deep neural network includes:

[0110] A population initialization module is used to randomly generate multiple initial quantization models of a deep neural network to be quantized with mixed precision based on a differential evolution algorithm, and to obtain an initialized population using multiple initial quantization models and elite vectors as individuals; wherein the elite vector is a quantization bit width dictionary obtained by determining the skewness and kurtosis of the weights of each layer of the deep neural network to be quantized with mixed precision; the value of each subcomponent in each individual in the initialized population represents the quantization bit width of each layer of the deep neural network;

[0111] The mutant population acquisition module is used to use the mutation strategy in the differential evolution algorithm to process the population diversity and obtain the mutant population;

[0112] A test individual acquisition module is used to use a binomial crossover operation to recombine the subcomponents of the mutant individuals in the mutant population with the subcomponents of the target individuals in the current population to obtain multiple test individuals;

[0113] A better individual acquisition module is used to perform mixed precision quantization on the deep neural network based on the test individuals and the target individuals to obtain the deep neural network after mixed precision quantization; the test accuracy of the deep neural network after mixed precision quantization on the test set and the weight compression ratio of the deep neural network after mixed precision quantization are used as the fitness function of the differential evolution algorithm to select better individuals from the test individuals and the target individuals;

[0114] The convergence judgment module is used to judge whether the differential evolution algorithm converges based on the selected better individuals. If converged, the final deep neural network after mixed precision quantization is obtained. Otherwise, the better individuals selected are used as the next generation of population, and the steps in the variant population acquisition module, the trial individual acquisition module, the better individual acquisition module and the convergence judgment module are repeatedly executed.

[0115] In one embodiment of the present invention, a computer device is provided, the computer device including a processor and a memory, the memory is used to store a computer program, the computer program includes program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, which are suitable for implementing one or more instructions, and are specifically suitable for loading and executing one or more instructions in a computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used to perform the operation of the mixed precision quantization method of the deep neural network.

[0116] In one embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here may include both a built-in storage medium in a computer device and an extended storage medium supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by a processor are also stored in the storage space, and these instructions may be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here may be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor may load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the mixed precision quantization method for deep neural networks in the above-mentioned embodiment.

[0117] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0118] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0119] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0120] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A mixed precision quantization method for deep neural networks, characterized in that: The following steps are involved: Step 1, based on the differential evolution algorithm, randomly generate multiple initial quantization models of the deep neural network to be mixed precision quantized, and use the multiple initial quantization models and elite vectors as individuals to obtain an initialized population; wherein the elite vector is based on the skewness and kurtosis of the weights of each layer of the deep neural network to be mixed precision quantized, and the obtained quantization bit width dictionary is determined; the value of each subcomponent in each individual in the initialized population represents the quantization bit width of each layer of the deep neural network; Step 2: Use the mutation strategy in the differential evolution algorithm to diversify the population and obtain a mutant population; Step 3, using a binomial crossover operation, recombining the subcomponents of the mutant individuals in the mutant population with the subcomponents of the target individuals in the current population to obtain multiple test individuals; Step 4: Based on the test individuals and the target individuals, the deep neural network is quantized with mixed precision to obtain the deep neural network after mixed precision quantization; the test accuracy of the deep neural network after mixed precision quantization on the test set and the weight compression ratio of the deep neural network after mixed precision quantization are used as the fitness function of the differential evolution algorithm to select the better individual from the test individuals and the target individuals; Step 5: Based on the selected better individuals, determine whether the differential evolution algorithm converges. If it converges, the final deep neural network after mixed precision quantization is obtained. Otherwise, the selected better individuals are used as the next generation population, and steps 2 to 5 are repeated.

2. The mixed precision quantization method for a deep neural network according to claim 1, characterized in that: Based on the skewness and kurtosis of the weights of each layer of the deep neural network to be quantized with mixed precision, the steps of determining to obtain a quantization bit width dictionary include: The kurtosis and skewness of each layer weight of the deep neural network are calculated in layers, and the kurtosis and skewness of each layer weight are mapped to a preset interval using different mapping methods; the mapped values ​​of the kurtosis and skewness of each layer weight are added in proportion to obtain the quantitative sensitivity weight value of each layer; Divide the interval between the maximum value and the minimum value of the quantization sensitivity weight value of each layer into multiple intervals according to the number of quantization bit width values, and each of the divided intervals corresponds to a quantization bit width value; The quantization bit width values ​​represented by the intervals to which the quantization sensitivity weight values ​​of each layer belong are allocated to the corresponding layers of the deep neural network to obtain a quantization bit width dictionary.

3. A mixed precision quantization method for a deep neural network according to claim 2, characterized in that: For the kth layer of the deep neural network, the calculation, mapping, and proportional addition expressions of kurtosis and skewness are: You' k =((You k -You min ) / (You max -You min ))×(ba)+a; Sk′ k =b-(|Sk k |-|Sk| min ) / (|Sk| max -|Sk| min )×(ba); M k =α×Ku′ k +β×Sk′ k ; In the formula, Ku k is the kurtosis of the kth layer; x i is the i-th weight value of the k-th layer, μ is the mean weight value of the k-th layer, N is the total number of weight values ​​in the kth layer; δ is the standard deviation of the N weight values ​​in the kth layer; Sk k is the skewness of the kth layer; a is the minimum quantization bit width value that the deep neural network can quantize to, and b is the maximum quantization bit width value that the deep neural network can quantize to; Ku′ k is the value after kurtosis mapping of the kth layer; Ku min is the minimum value of the kurtosis value of each layer; Ku max is the maximum value of the kurtosis value of each layer; Sk′ k is the value after skewness mapping of the kth layer; |Sk| min is the minimum value among the absolute values ​​of the skewness values ​​of each layer; |Sk| max is the maximum value of the absolute value of the skewness of each layer; M k is the quantization sensitivity weight value of the kth layer, α and β are the proportional coefficients of kurtosis and skewness respectively.

4. The mixed precision quantization method for a deep neural network according to claim 1, characterized in that: The steps of obtaining the initialized population with multiple initial quantization models and elite vectors as individuals include: Assign the value of the elite vector to the first individual in the population; Excluding the first individual, the j-th subcomponent value of the d-th individual is expressed as: X d,j =a+rand(0,1)×(b-a); In the formula, X d,j is the jth subcomponent value of the dth individual; rand(0,1) represents a random real number between (0,1); a is the minimum quantization bit width value that the deep neural network can quantize to, and b is the maximum quantization bit width value that the deep neural network can quantize to.

5. The mixed precision quantization method for a deep neural network according to claim 1, characterized in that: Step 4 specifically includes the following steps: Step 4.1: The obtained experimental individuals and target individuals The sub-component values ​​of are used as the quantization bit widths of each layer of the deep neural network. The deep neural network is quantized with mixed precision using an asymmetric quantization method to obtain the experimental individual and target individuals Mixed precision quantized deep neural network; where G is the number of iterations; Step 4.2, obtain the test individuals and target individuals Weight compression ratio of deep neural networks with mixed precision quantization And the test accuracy on the test set Step 4.3, based on the obtained weight compression ratio And the test accuracy on the test set For test individuals and target individuals Make a selection and take the individuals that meet the selection rules as the better individuals.

6. A mixed precision quantization method for a deep neural network according to claim 5, characterized in that: The selection rule is: When the test accuracy of the test set corresponding to the test individual and the target individual both meet the requirements of the deep neural network for accuracy loss, and the weight compression ratio corresponding to the test individual is larger, the test individual is selected as the better individual; When the test accuracy of the test set corresponding to the test individual meets the requirements of the deep neural network for accuracy loss, and the test accuracy of the test set corresponding to the target individual does not meet the requirements of the deep neural network for accuracy loss, the test individual is selected as the better individual; When the test accuracy of the test set corresponding to the test individual and the target individual does not meet the requirements of the deep neural network for accuracy loss, and the weight compression ratio corresponding to the test individual is larger, the test individual is selected as the better individual; In other cases, the target individual is selected as the better individual.

7. The mixed precision quantization method for a deep neural network according to claim 1, characterized in that: In step 3, the crossover factor of the binomial crossover operation is set to 0.

9.

8. A mixed precision quantization system for a deep neural network, characterized in that: A mixed precision quantization method for a deep neural network according to claim 1, wherein the mixed precision quantization system for the deep neural network comprises: A population initialization module is used to randomly generate multiple initial quantization models of a deep neural network to be quantized with mixed precision based on a differential evolution algorithm, and to obtain an initialized population using multiple initial quantization models and elite vectors as individuals; wherein the elite vector is a quantization bit width dictionary obtained by determining the skewness and kurtosis of the weights of each layer of the deep neural network to be quantized with mixed precision; the value of each subcomponent in each individual in the initialized population represents the quantization bit width of each layer of the deep neural network; The mutant population acquisition module is used to use the mutation strategy in the differential evolution algorithm to process the population diversity and obtain the mutant population; A test individual acquisition module is used to use a binomial crossover operation to recombine the subcomponents of the mutant individuals in the mutant population with the subcomponents of the target individuals in the current population to obtain multiple test individuals; A better individual acquisition module is used to perform mixed precision quantization on the deep neural network based on the test individuals and the target individuals to obtain the deep neural network after mixed precision quantization; the test accuracy of the deep neural network after mixed precision quantization on the test set and the weight compression ratio of the deep neural network after mixed precision quantization are used as the fitness function of the differential evolution algorithm to select better individuals from the test individuals and the target individuals; The convergence judgment module is used to judge whether the differential evolution algorithm converges based on the selected better individuals. If converged, the final deep neural network after mixed precision quantization is obtained. Otherwise, the better individuals selected are used as the next generation of population, and the steps in the variant population acquisition module, the trial individual acquisition module, the better individual acquisition module and the convergence judgment module are repeatedly executed.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the mixed precision quantization method for the deep neural network according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the mixed precision quantization method for the deep neural network according to any one of claims 1 to 7 is implemented.

Images