A method and apparatus for obtaining an activation threshold of neural network quantization

Abstract

Embodiments of the present application provide a method and device for obtaining an activation threshold of neural network quantization, the method comprising: inputting a preset number of calibration images into a preset neural network; obtaining all initial activation values of each initial network layer of the preset neural network; generating an initial statistical histogram of each initial network layer according to all initial activation values of each initial network layer; determining a candidate activation threshold of each initial network layer according to a preset quantization bit number, each initial statistical histogram and a corresponding initial activation threshold; and selecting a target activation threshold from all candidate activation thresholds, the similarity of the candidate activation threshold being determined according to a Wasserstein distance between a reference distribution and a quantization distribution generated according to the candidate activation threshold. The present application uses the Wasserstein distance to calculate the similarity between the reference distribution and the quantization distribution, which has a small calculation amount, a short time consumption, can determine the activation threshold suitable for each initial layer of the preset neural network, and reduces the precision loss of the preset neural network caused by quantization.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular, to a method and device for obtaining an activation threshold of neural network quantization. BACKGROUND

[0002] Neural networks have achieved great success in many fields such as image processing and computer vision, but their expensive storage and computing costs have seriously hindered their application on a wide range of resource-constrained devices, especially in real-time applications. In order to solve this problem, in recent years, quantization technology has emerged as a very promising network compression scheme, which can reduce storage and speed up the inference of neural networks. Generally, low-bit quantization of weight parameters and activations is required to achieve quantization of neural networks, and in the process of low-bit quantization of activations, a suitable activation threshold needs to be selected to enable the quantized low-bit space to both well identify the entire activation feature space and have high recognition accuracy.

[0003] In the process of neural network image processing, a suitable activation threshold can be selected by perceptual quantization training in the prior art, but a large amount of quantization training dataset is required, which is difficult and time-consuming. A suitable activation threshold can also be selected by post-training quantization methods such as KL divergence-based methods, but the KL divergence calculation process contains a logarithmic function, and there are many large mutation values in the calculated KL divergence values, which affects the selection of the activation threshold and increases the calculation density and time consumption. Therefore, there is an urgent need for a method for obtaining an activation threshold of neural network quantization to quickly determine a suitable activation threshold to achieve quantization of neural networks and reduce the loss of neural network quantization. SUMMARY

[0004] Therefore, one of the purposes of the present application is to provide a method and device for obtaining an activation threshold of neural network quantization to at least solve some of the above technical problems.

[0005] In a first aspect, the embodiments of the present application provide a method for obtaining an activation threshold of neural network quantization, which comprises:

[0006] inputting a preset number of calibration images into a preset neural network;

[0007] obtaining all initial activation values of each initial network layer of the preset neural network;

[0008] generating an initial statistical histogram corresponding to each initial network layer according to all initial activation values of each initial network layer, wherein an abscissa of the initial statistical histogram is divided into initial activation value intervals, a boundary value of each initial activation value interval represents an initial activation threshold value, an ordinate represents a number of initial activation values of the initial network layer falling into each initial activation value interval, and the initial statistical histogram includes at least two initial activation value intervals;

[0009] determining at least two candidate activation threshold values of each initial network layer according to a preset quantization bit number, each initial statistical histogram, and each initial activation threshold value corresponding to the initial statistical histogram;

[0010] selecting a candidate activation threshold value with the highest similarity from all candidate activation threshold values of each initial network layer as a target activation threshold value corresponding to the initial network layer, wherein the similarity of the candidate activation threshold value is determined according to a Wasserstein distance between a reference distribution corresponding to the candidate activation threshold value and a quantization distribution.

[0011] Optionally, the step of determining at least two candidate activation threshold values of each initial network layer according to a preset quantization bit number, each initial statistical histogram, and each initial activation threshold value corresponding to the initial statistical histogram includes:

[0012] calculating a quantization parameter according to the preset quantization bit number;

[0013] determining at least two first target intervals according to the quantization parameter and a number of internal intervals of a target statistical histogram, wherein the quantization parameter is smaller in value than the number of internal intervals of the target statistical histogram, and the target statistical histogram is any initial statistical histogram in all initial statistical histograms;

[0014] determining a candidate activation threshold value according to each first target interval and a boundary value on a neighboring interval of each first target interval.

[0015] Optionally, the step of generating a reference distribution corresponding to the candidate activation threshold value includes:

[0016] identifying a first target interval in which the candidate activation threshold value is located;

[0017] determining at least one second target interval according to the first target interval in which the candidate activation threshold value is located and all intervals of the initial statistical histogram in which the candidate activation threshold value is located;

[0018] generating a first array according to initial activation values on all second target intervals;

[0019] generating a second array according to initial activation values on intervals other than all second target intervals, and splicing the first array and the second array to obtain a spliced first array.

[0020] The first array after splicing is normalized to obtain a reference distribution corresponding to the candidate activation threshold.

[0021] Optionally, the generating of the quantized distribution corresponding to the candidate activation threshold comprises:

[0022] The first array is quantized to obtain a third array;

[0023] The third array is subjected to an expansion mapping to obtain a fourth array;

[0024] The fourth array is normalized to obtain the quantized distribution corresponding to the candidate activation threshold.

[0025] Optionally, the calculating of the similarity comprises:

[0026] The Wasserstein distance between the reference distribution and the quantized distribution corresponding to each candidate activation threshold is calculated according to a preset calculation rule;

[0027] The similarity between the reference distribution and the quantized distribution corresponding to each candidate activation threshold is determined according to the Wasserstein distance, wherein the candidate activation threshold with the highest similarity is the candidate activation threshold corresponding to the minimum Wasserstein distance between the reference distribution and the quantized distribution.

[0028] Optionally, after the step of calculating the Wasserstein distance between the reference distribution and the quantized distribution corresponding to each candidate activation threshold according to the preset calculation rule, the method further comprises:

[0029] The Wasserstein distance between the reference distribution and the quantized distribution corresponding to each candidate activation threshold is filtered according to a preset filter to filter out abnormal Wasserstein distances greater than a first preset distance and / or smaller than a second preset distance, wherein the first preset distance is greater than the second preset distance, and the preset filter comprises at least one of an arithmetic average filter, a recursive average filter, and a weighted moving average filter.

[0030] In a second aspect, an embodiment of the present application provides an acquisition device for a quantized activation threshold of a neural network, wherein the quantized device of the neural network comprises:

[0031] A first processing module is configured to input a preset number of calibration images into a preset neural network;

[0032] An acquisition module is configured to acquire all initial activation values of each initial network layer of the preset neural network;

[0033] The second processing module is configured to generate an initial statistical histogram corresponding to each initial network layer according to all initial activation values of each initial network layer, wherein an abscissa of the initial statistical histogram is divided into initial activation value intervals, a boundary value of each initial activation value interval represents an initial activation threshold value, an ordinate represents a quantity of initial activation values of the initial network layer falling into each initial activation value interval, and the initial statistical histogram includes at least two initial activation value intervals.

[0034] The computing module is configured to determine at least two candidate activation threshold values of each initial network layer according to the preset quantization bit number, each initial statistical histogram, and each initial activation threshold value corresponding to the initial statistical histogram.

[0035] The third processing module is configured to select a candidate activation threshold value with the highest similarity from all candidate activation threshold values of each initial network layer as a target activation threshold value of the corresponding initial network layer, wherein the similarity of the candidate activation threshold value is determined according to a Wasserstein distance between a reference distribution corresponding to the candidate activation threshold value and a quantization distribution.

[0036] Optionally, the computing module is further configured to calculate a quantization parameter according to the preset quantization bit number.

[0037] The at least two first target intervals are determined according to the quantization parameter and a quantity of internal intervals of the target statistical histogram, wherein the quantization parameter is smaller in value than the quantity of internal intervals of the target statistical histogram, and the target statistical histogram is any initial statistical histogram in the initial statistical histograms.

[0038] The candidate activation threshold value is determined according to each first target interval and a boundary value on a neighboring interval of each first target interval.

[0039] In a third aspect, an embodiment of the present application provides a computer device, which includes a memory and a processor, and the memory stores a computer program, and the computer program is executed by the processor to implement the method for obtaining the activation threshold value of the quantized neural network provided in the first aspect.

[0040] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by one or more processors to implement the method for obtaining the activation threshold value of the quantized neural network provided in the present application.

[0041] The method for obtaining the activation threshold of the neural network quantization provided by the embodiment of the application comprises the following steps: generating an initial statistical histogram of each initial network layer according to all initial activation values of each initial network layer of a preset neural network; and confirming a candidate activation threshold of each initial network layer through the initial statistical histogram. Then, the similarity between a reference distribution corresponding to each candidate activation threshold and a quantization distribution is calculated, and the candidate activation threshold with the highest similarity is selected from all candidate activation thresholds of each initial network layer as the target activation threshold of the corresponding initial network layer. The calculation process in the middle has a small amount of calculation and a short time consumption, and can determine the activation threshold suitable for each initial layer of the preset neural network, thereby reducing the precision loss caused by the quantization of the preset neural network. BRIEF DESCRIPTION OF DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some of the embodiments of the application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0043] Figure 1 A structural schematic diagram of a computer device provided by an embodiment of the application is shown;

[0044] Figure 2 A method flowchart of a method for obtaining an activation threshold of neural network quantization provided by an embodiment of the application is shown;

[0045] Figure 3 A functional module schematic diagram of an apparatus for obtaining an activation threshold of neural network quantization provided by an embodiment of the application is shown. DETAILED DESCRIPTION

[0046] In order to make the objects, technical solutions and advantages of the embodiments of the application clearer, the following will combine the drawings in the embodiments of the application to clearly and completely describe the technical solutions in the embodiments of the application. Obviously, the described embodiments are some of the embodiments of the application, but not all the embodiments. The components of the embodiments of the application described and shown in the drawings can be arranged and designed in various different configurations.

[0047] Therefore, the following detailed description of the embodiments of the application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the application. Based on the embodiments in the application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the application.

[0048] It should be noted that like reference numerals and letters refer to like items throughout the attached drawings, and once an item is defined in one drawing, further definition and explanation of the item in a subsequent drawing is unnecessary.

[0049] In the description of the present application, it should be noted that if the terms "upper", "lower", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship when the product of the application is usually placed, and are only for the convenience of describing the application and simplifying the description, and therefore cannot be understood as indicating or implying that the device or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application.

[0050] In addition, if the terms "first", "second" and the like appear, they are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0051] It should be noted that the features in the embodiments of the present application can be combined with each other without conflict.

[0052] Please refer to Figure 1 , the provided computer device 100 can be a computer device with data processing ability, for example, a personal computer, a server, etc., mainly including a neural network quantized activation threshold acquisition device 110, a memory 120 and a processor 130. Among them, the computer device 100 can be used as an execution device of the following neural network quantized activation threshold acquisition method, and the elements of the memory 120 and the processor 130 are directly or indirectly electrically connected with each other to realize the transmission or interaction of data. For example, these elements can be electrically connected with each other through one or more communication buses or signal lines. The computer device 100 includes at least one software function module in the form of software or firmware (Firmware) stored in the memory 120 or solidified in the operating system (Operating System, OS) of the computer device 100. The processor 130 is used to execute the executable modules stored in the memory 120, such as software function modules and computer programs included in the neural network quantized activation threshold acquisition device 110.

[0053] The memory 120 can be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), etc. The memory 120 is configured to store a program. The processor 130 is configured to execute the program after receiving an execution instruction.

[0054] The processor 130 can be an integrated circuit chip having a processing capability of a signal. The processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc. The processor can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The processor can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present application. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0055] Please refer to Figure 2 , Figure 2 A method flowchart of the method for obtaining an activation threshold of a neural network quantization provided in the embodiments of the present application is provided. Each step of the method for obtaining an activation threshold of a neural network quantization will be described in detail below.

[0056] In S210, a preset number of calibration images are input into a preset neural network.

[0057] The preset neural network in the embodiments has certain image processing and recognition functions, and is suitable for the field of image recognition. The preset number of calibration images can be used as a calibration data set of the preset neural network. In this embodiment, tens to hundreds of training images can be selected as the preset number of calibration images. It is not necessary to select a large number of training images. When the preset neural network processes the preset number of calibration images, the selection of the activation threshold in the subsequent embodiments can be quickly completed.

[0058] S220, obtaining all initial activation values of each initial network layer of the preset neural network.

[0059] In the embodiment, the preset neural network generally comprises an input layer, multiple intermediate layers and an output layer, which are all referred to as initial network layers in the embodiment. The initial network layers of the preset neural network can use the same activation function or different activation functions, but the activation functions used all have corresponding activation values, which are all referred to as initial activation values in the embodiment.

[0060] Specifically, for the activation, that is, the input feature map of the next initial network layer, there is generally a long-tail distribution, in which case a suitable activation threshold needs to be selected so that the low-bit space after quantization has high precision and can well represent the entire activation feature space.

[0061] S230, generating an initial statistical histogram corresponding to each initial network layer according to all initial activation values of each initial network layer, wherein the horizontal axis of the initial statistical histogram is divided into initial activation value intervals, the boundary values of each initial activation value interval represent initial activation thresholds, the vertical axis represents the number of initial activation values of the initial network layer falling into each initial activation value interval, and the initial statistical histogram comprises at least two initial activation value intervals.

[0062] In the embodiment, all initial activation values of each initial network layer can be represented by a statistical histogram, that is, the initial statistical histogram in the embodiment. Using the initial statistical histogram to represent the distribution of initial activation values can significantly reduce the amount of calculation and more intuitively show the distribution of initial activation values. The boundary values of each initial activation value interval can be determined as initial activation thresholds.

[0063] In the embodiment, multiple initial statistical histograms can be generated for one preset neural network, and one initial statistical histogram is generated for each initial network layer of the preset neural network.

[0064] The method is used in the symmetric linear quantization neural network in the embodiment to illustrate the selection of the activation threshold of the neural network quantization. Those skilled in the art can easily think that the method can also be used in symmetric, asymmetric, linear, nonlinear quantization and combinations thereof of the neural network, which are all within the protection scope of the application.

[0065] In the embodiment, for symmetric quantization, absolute value processing can be performed on all activation values, that is, the activation values on the negative half of the number axis are folded to the positive half, so that the best activation threshold of the initial network layer can be selected at one time. The initial statistical histogram is denoted as H, which has a minimum value of 0 and a maximum value of the maximum absolute value V of all initial activation values of the layer.M , preset interval number is G; the number of initial statistical histogram H is recorded as R i , the absolute value of the initial activation value of the current layer falls into the interval R i , the number of statistics is recorded as Ni, i∈[0,G-1];

[0066] S240, according to the preset quantization bit number, each initial statistical histogram and each initial activation threshold corresponding to the initial statistical histogram, determine at least two candidate activation thresholds of each initial network layer.

[0067] In order to reduce the number of calculations and determine the most suitable target activation threshold for each initial network layer, the initial activation threshold of each initial network layer is screened according to the quantization parameter calculated by the preset quantization bit number, and each initial network layer can obtain at least two candidate activation thresholds. In actual application, the candidate activation threshold of each initial network layer is much more than two, in order to determine the most suitable target activation threshold, the candidate activation threshold of each network layer after screening is much more than two.

[0068] In one possible implementation, the step of "determining at least two candidate activation thresholds of each initial network layer according to the preset quantization bit number, each initial statistical histogram and each initial activation threshold corresponding to the initial statistical histogram", includes:

[0069] Calculate the quantization parameter according to the preset quantization bit number;

[0070] Determine at least two first target intervals according to the quantization parameter and the number of intervals in the target statistical histogram, wherein the quantization parameter is smaller in value than the number of intervals in the target statistical histogram, and the target statistical histogram is any initial statistical histogram in all initial statistical histograms;

[0071] Determine the candidate activation threshold according to each first target interval and the boundary value on the adjacent interval of each first target interval.

[0072] Among them, considering that the symmetric linear quantization formula can be selected to realize the quantization of the preset neural network, the available symmetric linear quantization formula is formula (1):

[0073] S=T / 2^(nbit-1) Formula (1)

[0074] Wherein, S represents the quantization factor or precision, which can represent the data size represented by each part of the quantization space, nbit represents the number of bits of quantization, that is, the quantization bit number, T represents the activation threshold, and the initial activation threshold, the candidate activation threshold in the embodiment and the target activation threshold in the following embodiment are applicable to formula (1).

[0075] In addition, the number of intervals of the initial statistical histogram in the above embodiment can be greater than or equal to 2^(nbit-1).

[0076] Correspondingly, the preset number of quantization bits in the embodiment can be nbit in formula (1), and 8-bit quantization, 4-bit quantization, and 2-bit quantization are often performed in low-bit quantization processing. If 8-bit quantization is performed, the corresponding preset number of quantization bits is 8 bits, the quantization parameter L = 2^(8-1) = 128, and the preset number of intervals of the initial statistical histogram G = 6001; a plurality of first target intervals can be determined according to the quantization parameter L and the preset number of intervals of the initial statistical histogram G. In the embodiment, the number of intervals of the initial statistical histogram is much greater than the quantization parameter.

[0077] Then, the initial activation threshold on each first target interval can be taken as a candidate activation threshold. Considering the case that a certain interval in the histogram has two boundary values, the candidate activation threshold on the first target interval in the embodiment can select the initial activation threshold on the side of the interval close to the coordinate axis origin, or can select the initial activation threshold on the side of the interval away from the coordinate axis origin.

[0078] S250, selecting a candidate activation threshold with the highest similarity from all candidate activation thresholds of each initial network layer as a target activation threshold corresponding to the initial network layer, wherein the similarity of the candidate activation threshold is a similarity determined according to the Wasserstein distance between the reference distribution and the quantization distribution corresponding to the candidate activation threshold.

[0079] In order to further determine the suitable target activation threshold for each initial network layer, the embodiment determines the target activation threshold of each initial network layer through the Wasserstein distance between the reference distribution and the quantization distribution corresponding to each candidate activation threshold.

[0080] Optionally, the similarity calculation step comprises:

[0081] calculating the Wasserstein distance between the reference distribution and the quantization distribution corresponding to each candidate activation threshold according to a preset calculation rule;

[0082] judging the similarity between the reference distribution and the quantization distribution corresponding to each candidate activation threshold according to the Wasserstein distance, wherein the candidate activation threshold with the highest similarity is the candidate activation threshold corresponding to the minimum Wasserstein distance between the reference distribution and the quantization distribution.

[0083] Specifically, the similarity between the two objects can be represented by calculating the Wasserstein distance between the two objects, the smaller the Wasserstein distance, the higher the similarity, the larger the Wasserstein distance, the lower the similarity, and the Wasserstein distance equal to 0, which means that the two objects are completely coincident. In this embodiment, the Wasserstein distance between the reference distribution and the quantization distribution corresponding to each candidate activation threshold can be calculated, and the candidate activation threshold with the smallest Wasserstein distance is selected from all the candidate activation thresholds of each initial network layer as the target activation threshold of the corresponding initial network layer, and the optimal activation threshold of each initial network layer is determined. Moreover, compared with the kld-based method, no logarithmic operation is performed in the process of determining the target activation threshold of each initial network layer, which can greatly reduce the time required for quantizing the preset neural network.

[0084] From the above analysis, the method for obtaining the activation threshold of the neural network quantization provided by the embodiments of the present application generates an initial statistical histogram for each initial network layer according to all the initial activation values of each initial network layer of the preset neural network, and then determines the candidate activation threshold of each initial network layer through the initial statistical histogram. Then, the Wasserstein distance between the reference distribution and the quantization distribution corresponding to each candidate activation threshold is calculated, and the candidate activation threshold with the smallest Wasserstein distance is selected from all the candidate activation thresholds of each initial network layer as the target activation threshold of the corresponding initial network layer. The intermediate calculation process has small calculation amount and short time consumption. Since the Wasserstein distance can effectively measure the similarity when two distributions are very different and have no overlap, the activation threshold suitable for each initial layer of the preset neural network can be determined, thereby reducing the precision loss caused by quantizing the preset neural network.

[0085] In order to determine the target activation threshold most suitable for each initial network layer, optionally, after the step of calculating the Wasserstein distance between the reference distribution and the quantization distribution corresponding to each candidate activation threshold according to the preset calculation rule, the method further comprises:

[0086] According to the preset filter, the Wasserstein distance corresponding to each candidate activation threshold is filtered to filter out abnormal Wasserstein distance greater than a first preset distance and / or less than a second preset distance, wherein the first preset distance is greater than the second preset distance, and the preset filter includes at least one of an arithmetic average filter, a recursive average filter and a weighted moving average filter.

[0087] The Wasserstein distance between the reference distribution and the quantized distribution corresponding to each candidate activation threshold is filtered by a preset filter, and abnormal Wasserstein distances are filtered out to obtain smoother Wasserstein distances, thereby improving the accuracy of selecting the target activation threshold.

[0088] The filter used for filtering can be at least one of the arithmetic average filter, the recursive average filter and the weighted moving average filter in the above embodiments.

[0089] In a possible implementation, the generating of the reference distribution corresponding to the candidate activation threshold comprises:

[0090] Identifying a first target interval in which the candidate activation threshold is located;

[0091] Determining at least one second target interval according to the first target interval in which the candidate activation threshold is located and all intervals of the initial statistical histogram in which the candidate activation threshold is located;

[0092] Generating a first array according to the initial activation values in all second target intervals;

[0093] Generating a second array according to the initial activation values in intervals other than all second target intervals, and splicing the first array and the second array to obtain a spliced first array;

[0094] Normalizing the spliced first array to obtain the reference distribution corresponding to the candidate activation threshold.

[0095] In the embodiment, each candidate activation threshold is an interval boundary value on the side close to the coordinate origin of each first target interval in the above embodiment. Each integer j in the range of [L, G-1] is sequentially selected as the quantization parameter L in the above embodiment, i.e., j∈[L, G-1], and the left boundary value of the jth interval R j of the above initial statistical histogram H is the j-Lth candidate activation threshold T j-L , and the first array is [N0, N1, N2, N3, …, N j-1 , which can be represented by P j-L ; and the second array is [N j , N j+1 , N j+2 , N j+3 , …, N G-1 , which can be represented by O j-L .

[0096] The splicing of the first array and the second array can be the accumulation of the second array to the last element of the first array, and the specific calculation is as shown in formula (2):

[0097]

[0098] The first array after splicing is normalized to obtain the j-Lth candidate threshold T j-L The reference distribution p j-L is calculated as formula (3):

[0099]

[0100] In one possible implementation, the generation of the quantization distribution corresponding to the candidate activation threshold includes:

[0101] The first array is quantized to obtain a third array;

[0102] The third array is expanded to obtain a fourth array;

[0103] The fourth array is normalized to obtain the quantization distribution corresponding to the candidate activation threshold.

[0104] Specifically, the left boundary value of the jth interval R j of the initial statistical histogram H in the above embodiment, that is, the candidate activation threshold T j-L , is selected in turn to generate the j-Lth quantization statistical histogram H j-L corresponding to the initial network layer;

[0105] The minimum value of the above quantization statistical histogram H j-L is 0, the maximum value is T j-L , the preset interval number is L, that is, quantized to L parts of space; the kth interval is denoted as D k , k∈[0,L-1].

[0106] Optionally, the boundary value of the above quantization statistical histogram interval D k may be rounded to the boundary value of the interval R i , that is, to ensure that the interval D k contains an integer number of R i intervals. The number of absolute values of all initial activation values of the initial network layer falling into the kth interval D k is denoted as M k ; since the interval size of the above quantization statistical histogram H j-L is not equal, it is not a statistical histogram in the mathematical sense.

[0107] Each element Q1 j-L [k]=M k is constructed to obtain a third array;

[0108] Then the quantization distribution Q1 j-L is mapped to the j-level space, denoted as Q j-L , to obtain a fourth array; at this time, each quantization distribution element Q j-L [i] and the reference distribution element P j-L [i] are in one-to-one correspondence, wherein the expansion mapping is calculated in the manner of formula (5):

[0109]

[0110] wherein C k represents that the kth interval of the above-mentioned quantization statistical histogram contains the above-mentioned initial statistical histogram interval R i , and R i corresponds to the number of intervals in which P j-L [i]≠0.

[0111] The fourth array Q j-L is normalized to obtain the quantization distribution q j-L of the j-Lth candidate activation threshold T j-L , and the specific calculation formula is formula (4):

[0112]

[0113] Optionally, the calculation formula of the Wasserstein distance W in the above-mentioned embodiment can be formula (5) as follows:

[0114]

[0115] wherein P i and Q i represent the ith element of any reference distribution P and the corresponding quantization distribution Q respectively, the initial δ=0, and δ represents the calculation cost.

[0116] The following is real data obtained from relevant experiments according to the method for obtaining the activation threshold of the quantized neural network in the embodiments of the present application.

[0117] Since no logarithmic function is used, the Wasserstein distance is more stable than the KL divergence in calculating the similarity between the reference distribution and the quantized distribution, and the operation speed is faster; when the two distributions are very different and have no overlap, the Wasserstein distance can still effectively measure their similarity, so that the neural network model has less precision loss when quantized. See Table 1 below for the precision loss of four classical models based on the above two methods (except for the activation threshold selection method, all other aspects are consistent; based on imageNet2012-1K, 200 pictures in the calibration set and 30,000 pictures in the verification set). From the table below, it can be seen that compared with using KL divergence, using Wasserstein distance to quantize the neural network model, the four models have a certain degree of precision improvement, and the improvement for small models is more significant (in the field of neural network quantization, small models generally have more precision loss than large models, and quantization is more difficult).

[0118] Table 1

[0119]

[0120] Among them, compared with KL divergence, the Wasserstein distance has no complex logarithmic function operation, so the running speed is faster, and in practical application, the model quantization time can be significantly shortened. Take p j-L and q j-L as an example, the time consumption of calculating KL divergence and Wasserstein distance is tested (environment: Intel(R) Xeon(R) CPU E5-2650v4 @ 2.20GHz, Ubuntu 16.04), as shown in the following table (other conditions are the same). From Table 2 below, it can be seen that under the same conditions, the average calculation of the Wasserstein distance is about 1.65 times faster than the calculation of the KL divergence.

[0121] Table 2

[0122] Experiment No. Based on KL divergence calculation (s) Based on Wasserstein distance (s) Speed improvement about (times) 1 2.62 1.59 1.647799 2 2.75 1.64 1.676829 3 2.72 1.69 1.609467 4 2.65 1.59 1.666667 5 2.66 1.68 1.583333 6 2.73 1.61 1.695652 7 2.63 1.60 1.64375 8 2.63 1.60 1.64375 9 2.63 1.60 1.64375 10 2.64 1.60 1.65 Average 2.67 1.62 1.65

[0123] In summary, the method for obtaining an activation threshold of a quantized neural network provided in the embodiments of the present application generates an initial statistical histogram for each initial network layer according to all initial activation values of the initial network layer of a preset neural network, and then determines candidate activation thresholds for each initial network layer according to the initial statistical histograms. Then, the similarity between a reference distribution and a quantization distribution corresponding to each candidate activation threshold is calculated, and the candidate activation threshold corresponding to the minimum Wasserstein distance is selected from all candidate activation thresholds of each initial network layer as the target activation threshold of the corresponding initial network layer. The calculation amount is small, and the time consumption is short. By determining the activation threshold suitable for each initial layer of the preset neural network, the precision loss caused by the quantization of the preset neural network is reduced.

[0124] Corresponding to the method embodiments described above, the present application also provides a device 300 for obtaining an activation threshold of a quantized neural network. The device 300 for obtaining an activation threshold of a quantized neural network can be the device 110 for obtaining an activation threshold of a quantized neural network in the embodiments described above.

[0125] Please refer to Figure 3 , Figure 3 A functional module diagram of a device for obtaining an activation threshold of a quantized neural network provided in the embodiments of the present application is shown in FIG. 3. The device 300 for obtaining an activation threshold of a quantized neural network includes:

[0126] A first processing module 310 is configured to input a preset number of training images into a preset neural network.

[0127] An obtaining module 320 is configured to obtain all initial activation values of each initial network layer of the preset neural network.

[0128] A second processing module 330 is configured to generate an initial statistical histogram for each initial network layer according to all initial activation values of each initial network layer. The horizontal axis of the initial statistical histogram is divided into initial activation value intervals, the boundary values of each initial activation value interval represent initial activation thresholds, the vertical axis represents the number of initial activation values of the initial network layer falling into each initial activation value interval, and the initial statistical histogram includes at least two initial activation value intervals.

[0129] A calculation module 340 is configured to determine at least two candidate activation thresholds for each initial network layer according to a preset quantization bit number, each initial statistical histogram, and each initial activation threshold corresponding to the initial statistical histogram.

[0130] The third processing module 350 is configured to select a candidate activation threshold with the highest similarity from all candidate activation thresholds of each initial network layer as a target activation threshold of the corresponding initial network layer, where the similarity of the candidate activation threshold is determined according to the Wasserstein distance between the reference distribution and the quantized distribution corresponding to the candidate activation threshold.

[0131] The obtaining device for the activation threshold of the neural network quantization provided by the embodiment of the present application generates the initial statistical histogram of each initial network layer according to all initial activation values of each initial network layer of the preset neural network, and then determines the candidate activation threshold of each initial network layer through the initial statistical histogram. Then, the similarity between the reference distribution and the quantized distribution corresponding to each candidate activation threshold is calculated, and the candidate activation threshold with the minimum Wasserstein distance is selected from all candidate activation thresholds of each initial network layer as the target activation threshold of the corresponding initial network layer. The intermediate calculation process has small calculation amount and short time consumption; the activation threshold suitable for each initial layer of the preset neural network can be determined, and the precision loss caused by the quantization of the preset neural network is reduced.

[0132] Optionally, the calculation module is further configured to calculate the quantization parameter according to the preset quantization bit number.

[0133] determine at least two first target intervals according to the quantization parameter and the number of intervals in the target statistical histogram, where the quantization parameter is smaller in value than the number of intervals in the target statistical histogram, and the target statistical histogram is any initial statistical histogram in all initial statistical histograms;

[0134] determine the candidate activation threshold according to each first target interval and the boundary value on the adjacent interval of each first target interval.

[0135] The embodiment of the present application further discloses a computer readable storage medium, and the computer readable storage medium stores a computer program. When the computer program is executed by a processor, the obtaining method for the activation threshold of the neural network quantization is implemented.

[0136] The specific implementation process of the obtaining method, device, computer device, and computer readable storage medium for the activation threshold of the neural network quantization provided by the embodiment of the present application can be referred to the specific implementation process of the obtaining method for the activation threshold of the neural network quantization, which will not be repeated here.

[0137] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited to this. Any changes or replacements that can be easily thought of by those skilled in the art within the technical range disclosed by the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for obtaining an activation threshold of neural network quantization, characterized in that, The method comprises: inputting a preset number of calibration images into a preset neural network; obtaining all initial activation values of each initial network layer of the preset neural network; generating an initial statistical histogram corresponding to each initial network layer according to all initial activation values of each initial network layer, wherein an abscissa of the initial statistical histogram is divided into initial activation value intervals, a boundary value of each initial activation value interval represents an initial activation threshold value, an ordinate represents a number of initial activation values of the initial network layer falling into each initial activation value interval, and the initial statistical histogram comprises at least two initial activation value intervals; determining at least two candidate activation threshold values of each initial network layer according to a preset quantization bit number, each initial statistical histogram and each initial activation threshold value corresponding to the initial statistical histogram; selecting a candidate activation threshold value with the highest similarity from all candidate activation threshold values of each initial network layer as a target activation threshold value of the corresponding initial network layer, wherein the similarity of the candidate activation threshold value is determined according to a Wasserstein distance between a reference distribution corresponding to the candidate activation threshold value and a quantized distribution. The generating step of the reference distribution corresponding to the candidate activation threshold value comprises: identifying a first target interval in which the candidate activation threshold value is located; determining at least one second target interval according to the first target interval in which the candidate activation threshold value is located and all intervals of the initial statistical histogram in which the candidate activation threshold value is located; generating a first array according to initial activation values on all second target intervals; generating a second array according to initial activation values on intervals other than all second target intervals, and splicing the first array and the second array to obtain a spliced first array; and performing normalization processing on the spliced first array to obtain the reference distribution corresponding to the candidate activation threshold value.

2. The method of claim 1, wherein, The step of determining at least two candidate activation threshold values of each initial network layer according to a preset quantization bit number, each initial statistical histogram and each initial activation threshold value corresponding to the initial statistical histogram comprises: calculating a quantization parameter according to the preset quantization bit number; determining at least two first target intervals according to the quantization parameter and a number of intervals in a target statistical histogram, wherein the quantization parameter is smaller in value than the number of intervals in the target statistical histogram, and the target statistical histogram is any one of all initial statistical histograms; determining a candidate activation threshold value according to each first target interval and boundary values on adjacent intervals of each first target interval.

3. The method of claim 1, wherein, The generating step of the quantized distribution corresponding to the candidate activation threshold value comprises: performing quantization processing on the first array to obtain a third array; performing extension mapping processing on the third array to obtain a fourth array; performing normalization processing on the fourth array to obtain the quantized distribution corresponding to the candidate activation threshold value.

4. The method of claim 1, wherein, The calculating step of the similarity comprises: calculating a Wasserstein distance between the reference distribution corresponding to each candidate activation threshold value and the quantized distribution according to a preset calculation rule. The similarity between the reference distribution and the quantized distribution corresponding to each candidate activation threshold is determined according to the Wasserstein distance, and the candidate activation threshold with the highest similarity is the candidate activation threshold corresponding to the minimum Wasserstein distance between the reference distribution and the quantized distribution.

5. The method of claim 4, wherein, After the step of calculating the Wasserstein distance between the reference distribution and the quantized distribution corresponding to each candidate activation threshold according to the pre-designed calculation rule, the method further comprises: The Wasserstein distances between the reference distribution and the quantized distribution corresponding to each candidate activation threshold are filtered according to a preset filter to filter out abnormal Wasserstein distances greater than a first preset distance and / or less than a second preset distance, wherein the first preset distance is greater than the second preset distance, and the preset filter comprises at least one of an arithmetic average filter, a recursive average filter and a weighted moving average filter.

6. A device for obtaining the activation threshold of a neural network quantization, characterized in that, The quantization device of the neural network comprises: The first processing module is configured to input a preset number of calibration images into a preset neural network. The acquisition module is configured to acquire all initial activation values of each initial network layer of the preset neural network. The second processing module is configured to generate an initial statistical histogram corresponding to each initial network layer according to all initial activation values of each initial network layer, wherein the horizontal axis of the initial statistical histogram is divided into initial activation value intervals, the boundary value of each initial activation value interval represents an initial activation threshold, and the vertical axis represents the number of initial activation values of the initial network layer falling into each initial activation value interval, and the initial statistical histogram comprises at least two initial activation value intervals. The calculation module is configured to determine at least two candidate activation thresholds of each initial network layer according to a preset quantization bit number, each initial statistical histogram and each initial activation threshold corresponding to the initial statistical histogram. The third processing module is configured to select a candidate activation threshold with the highest similarity from all candidate activation thresholds of each initial network layer as a target activation threshold corresponding to the initial network layer, wherein the similarity of the candidate activation threshold is determined according to the Wasserstein distance between the reference distribution and the quantized distribution corresponding to the candidate activation threshold. The generation of the reference distribution corresponding to the candidate activation threshold comprises: identifying a first target interval in which the candidate activation threshold is located; determining at least one second target interval according to the first target interval in which the candidate activation threshold is located and all intervals of the initial statistical histogram in which the candidate activation threshold is located; generating a first array according to initial activation values on all second target intervals; generating a second array according to initial activation values on intervals other than all second target intervals, and splicing the first array and the second array to obtain a spliced first array; and performing normalization processing on the spliced first array to obtain the reference distribution corresponding to the candidate activation threshold.

7. The apparatus for acquiring a quantized activation threshold of a neural network according to claim 6, wherein, The calculation module is further configured to calculate a quantization parameter according to the preset quantization bit number. determining at least two first target intervals according to the quantization parameter and a number of intervals in a target statistical histogram, wherein the quantization parameter is smaller in value than the number of intervals in the target statistical histogram, and the target statistical histogram is any of the initial statistical histograms; determining a candidate activation threshold according to each first target interval and a boundary value on a neighboring interval of each first target interval.

8. A computer device, comprising: The computer device comprises a memory and a processor, and the memory stores a computer program, and the computer program, when executed by the processor, implements the method for obtaining the activation threshold of the neural network quantization in any one of claims 1-5.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program, when executed by one or more processors, implements the method for obtaining the activation threshold of the neural network quantization in any one of claims 1-5.

Images