Sub-interval quantization method and device based on neural network weight distribution

Abstract

The application discloses a sub-interval quantization method and device based on neural network weight distribution, and relates to the field of weight quantization, and comprises the following steps: acquiring weight data to be quantized in a neural network, and counting distribution characteristics thereof; dividing the weight data into multiple sub-intervals according to the distribution characteristics; fine-tuning the range of each sub-interval based on the principle of minimum mean square error; determining a quantization parameter for each sub-interval, and quantizing the weight data, wherein the quantization parameter comprises a scaling factor and a zero-point offset. The main technical scheme and effect are as follows: the statistical characteristics that the weight data presents an approximate normal distribution, a dense center and rapid attenuation on both sides after training are fully utilized. The sub-interval division based on data distribution overcomes the defects that the traditional method has a serious precision loss under uneven data distribution, especially under a multimodal distribution.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of weight quantization, and in particular to a sub-interval quantization method and device based on neural network weight distribution. BACKGROUND

[0002] In recent years, with the rapid development of deep learning technology, neural networks have been widely used in image recognition, speech recognition, speech synthesis and other audio and video processing tasks. In order to obtain higher accuracy and better performance, academia and industry generally use larger and deeper neural network models. However, the increase in model size results in a sharp increase in computational and storage requirements, making it difficult to deploy such models on resource-constrained embedded devices or mobile terminals.

[0003] In order to solve the above problems, the industry usually adopts model quantization (Quantization) technology, that is, mapping floating-point weights and activation values to low-bit fixed-point values to reduce storage and computing overhead. The common quantization methods currently include:

[0004] Per-Tensor Quantization: Assign a uniform quantization parameter to the entire tensor (such as a layer of weight matrix), which is simple to calculate and friendly to hardware, but when the weight distribution is significantly different, the small value part will be ignored, resulting in serious precision loss.

[0005] Per-Channel Quantization: Assign a separate quantization parameter to each channel of the convolutional or fully connected layer, which can improve accuracy to some extent, but still cannot completely avoid errors caused by uneven data distribution.

[0006] Group Quantization: By dividing the weights into multiple groups for separate quantization, the parameter expression is refined, and the accuracy is improved compared to the first two methods, but when facing multi-modal or strong asymmetric distribution, there will still be a large error.

[0007] In summary, the existing quantization techniques do not fully consider the statistical characteristics of neural network weight distribution. When the weight distribution has obvious unevenness or multi-modal characteristics, traditional quantization methods often cannot guarantee the quantization accuracy, and may even cause the network model to fail to meet the accuracy requirements in actual deployment. SUMMARY

[0008] The purpose of the present application is to solve the technical problems in the prior art and provide a sub-interval quantization method and device based on neural network weight distribution.

[0009] This specification relates to one or more embodiments of a sub-interval quantization device based on neural network weight distribution, an electronic device, a computer-readable storage medium, and a computer program product, in order to address the technical deficiencies existing in the prior art.

[0010] Technical solution:

[0011] Firstly, this application proposes a sub-interval quantization method based on the weight distribution of a neural network, comprising:

[0012] Obtain the weight data to be quantized in the neural network and statistically analyze its distribution characteristics;

[0013] Based on the distribution characteristics of the weighted data, it is divided into multiple sub-intervals;

[0014] Based on the principle of minimizing mean square error, the range of each sub-interval is finely adjusted;

[0015] A quantization parameter is determined for each sub-interval, and the weighted data is quantized. The quantization parameter includes a scaling factor and a zero offset.

[0016] Preferably, the weight data to be quantized in the neural network is obtained, and its distribution characteristics are statistically analyzed, including:

[0017] The distribution characteristics of the weighted data are fitted into a normal distribution probability density function using histogram statistics.

[0018] Preferably, the weighted data is divided into multiple sub-intervals using confidence levels or the three sigma principle, including:

[0019] Obtain the weight data to be quantized in the neural network and the preset confidence level to obtain the weight data range [x]. min ,x max and a set of weighted data {x0, x1, x2, ..., x} N−1};

[0020] The mean of the weighted data is calculated. and standard deviation ;

[0021] The endpoints for dividing multiple sub-intervals are calculated using confidence level or the three sigma principle;

[0022] Through the endpoints and weighted data ranges of multiple sub-intervals [x] min ,x max The intervals are divided into multiple ranges based on their size.

[0023] Preferably, the endpoints for dividing multiple sub-intervals are obtained through confidence level calculation, including:

[0024] The end points of the plurality of sub-intervals are calculated according to at least one determined confidence level, wherein one confidence level can obtain the end points of two sub-intervals;

[0025] The relationship between the number of confidence levels required for dividing the plurality of sub-intervals is:

[0026] ;

[0027] Wherein n is the number of sub-intervals, and k is the number of confidence levels required to be preset.

[0028] Preferably, the end points of the plurality of sub-intervals are obtained by the three-sigma principle, including:

[0029] The overall distribution is divided into sub-intervals by taking =1, 2, 3... as the boundary end points of the sub-intervals (wherein =1, 2, 3...).

[0030] Preferably,

[0031] The mean value and the standard deviation in the weight data are calculated, and the formula is as follows:

[0032] ;

[0033] Wherein, is the i-th weight, is the total number of weights.

[0034] Preferably, the end points of the plurality of sub-intervals are calculated according to at least one determined confidence level, including the following formula:

[0035] ;

[0036] Wherein, the end points of the sub-intervals of one confidence level are and , z is a normal distribution variable, and α is a significance level for determining a confidence level parameter, and the formula is CL=1−α, Φ(z) is a probability distribution function, and z α / 2 is a critical value for determining the end points of the confidence interval.

[0037] Preferably, based on the principle of minimum mean square error, the range of each sub-interval is fine-tuned, including defining the following objective function formula:

[0038] ;

[0039] Wherein, is a sub-interval fine-tuning parameter, is a floating point calculation result of a network layer. quantized calculation result, data size of network layer calculation result;

[0040] adjusting the sub-interval fine-tuning parameter θ using an iterative algorithm to solve the minimum value of the objective function J(θ);

[0041] the sub-interval fine-tuning parameter value θ obtained after iterative optimization as the fine-tuned sub-interval boundary parameter.

[0042] Preferably, a quantization parameter is determined for each sub-interval, and the weight data is quantized, the quantization parameter including a scaling factor and a zero-point offset, including:

[0043] mapping the floating-point number range of the sub-interval to the target fixed-point number range [Q min ,Q max] ;

[0044] calculating the scaling factor of the sub-interval:

[0045] ;

[0046] calculating the zero-point offset of the sub-interval;

[0047] ;

[0048] wherein, for any floating-point weight value belonging to the sub-interval, its corresponding quantized value is calculated by the following formula:

[0049] .

[0050] In a second aspect, an embodiment of the present application provides a sub-interval quantization device based on neural network weight distribution, comprising:

[0051] an acquisition unit configured to acquire weight data to be quantized in a neural network and to count its distribution characteristics;

[0052] a generation unit configured to divide the weight data into multiple sub-intervals according to its distribution characteristics;

[0053] an adjustment unit configured to fine-tune the range of each sub-interval based on the principle of minimum mean square error;

[0054] a quantization unit configured to determine a quantization parameter for each sub-interval and to quantize the weight data, the quantization parameter including a scaling factor and a zero-point offset.

[0055] In a third aspect, an electronic device is provided, comprising a processor and a memory. The memory is configured to store one or more computer programs. When the one or more computer programs stored in the memory are executed by the processor, the electronic device is enabled to implement the method of any possible design of the first aspect.

[0056] In a fourth aspect, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the method of any one of the above embodiments is implemented.

[0057] In a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device is enabled to perform the method of any possible design of any one of the above aspects.

[0058] Beneficial effects: Unlike the existing tensor quantization, channel quantization and group quantization, the present application does not simply determine the quantization parameter according to the position of the weight in the network structure, but fully utilizes the statistical characteristics that the weight data presents approximate normal distribution, center dense and rapid decay on both sides after training. By dividing the sub-interval based on data distribution, the defects of serious precision loss of traditional methods in uneven data distribution, especially in multi-modal distribution, are overcome.

[0059] The present application first divides the weights into several sub-intervals according to the confidence interval of the normal distribution, and then refines the boundaries by combining methods such as the "three sigma principle", so that each sub-interval can more accurately reflect the distribution characteristics of the weights in different intervals, thereby providing a more scientific basis for quantization.

[0060] On the basis of preliminary sub-interval division, the present application introduces the principle of minimum mean square error to dynamically fine-tune the range of each sub-interval, so as to ensure that the deviation between the quantized fixed-point value and the floating-point value is minimized. This mechanism effectively reduces the quantization error and maintains the stability and precision of the model in the inference process.

[0061] Through the above strategies, the present application can significantly compress the network model parameter quantity and computational complexity without significantly sacrificing accuracy, so that large-scale neural networks can also meet the accuracy and performance requirements in resource-limited embedded devices, greatly improving the landing nature of practical applications.

[0062] This method is not only suitable for speech synthesis (such as FastSpeech2) models, but also can be extended to various deep learning networks such as image recognition and speech recognition, providing a unified quantization optimization idea for different task scenarios, and has strong universality and promotion value. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1A method framework diagram is provided for the present application.

[0064] Figure 2 A current weight value diagram is provided for the present application.

[0065] Figure 3 A normal distribution fitting diagram of the current weight value is provided for the present application.

[0066] Figure 4 A device structure block diagram provided by an embodiment of the present application is shown.

[0067] Figure 5 An electronic device structure block diagram provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0068] To make the technical solutions of the present application clearer, the following will further describe the present application in detail with specific embodiments combined with the accompanying drawings.

[0069] Embodiment 1

[0070] To make the purpose, technical solutions and advantages of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application combined with the drawings of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application. Unless otherwise defined, the technical terms or scientific terms used herein should be understood as the usual meanings understood by those skilled in the art in the field of the present application. The words such as “include” and similar words used herein mean that the elements or objects before the words cover the elements or objects listed after the words and their equivalents, without excluding other elements or objects.

[0071] In the process of quantizing the weights of a neural network, through a large amount of statistical analysis and observation, it is found that the weight values of a fully trained neural network generally exhibit a significant leptokurtic and approximately symmetric distribution characteristic. Specifically, most weight values are highly concentrated and distributed in a very small interval around the value zero, forming a sharp peak. As the absolute value increases, the frequency of weight occurrence decreases sharply, but in the tail interval far from zero on both sides, there are still a small amount of non-negligible large positive and negative values, constituting the so-called “thick tail” phenomenon.

[0072] To specifically illustrate this statistical law, the current widely used speech synthesis network FastSpeech2 (such as Figure 2As an example, the weight tensor of a typical linear layer (or convolutional layer) of a fully trained neural network is analyzed in depth. The histogram statistical method is used to preliminarily explore the numerical distribution of the weights of the layer: first, all the weight values of the layer are unfolded into a one-dimensional vector; then, the value range of the data is divided by selecting a suitable bin width, and the number of weights falling into each interval (frequency) is counted.

[0073] Figure 3 The statistical distribution of the weight values of the trained neural network is presented in the form of a histogram, where the horizontal coordinate represents the weight value, and the vertical coordinate represents the corresponding probability density. The curve is the normal distribution probability density curve superimposed on the histogram to fit the distribution trend of the actual weight data.

[0074] From the comparison of the histogram and the fitting curve, it can be seen that the weight data as a whole presents a typical unimodal approximately normal distribution characteristic, that is, most of the data are highly concentrated near the center, and the distribution peak position is close to zero. There are only a small number of weight data points in the larger positive value interval and the smaller negative value interval far from the center. This result further confirms that the neural network weights have the distribution characteristics of significant central concentration and rapid decay on both sides, which provides a statistical basis for the selection of weight quantization and compression algorithms. The normal distribution probability density function is:

[0075] ;

[0076] The mean is The standard deviation is

[0077] In view of the problems existing in the prior art, such as Figure 1 As shown in the prior art, a sub-interval quantization method based on neural network weight distribution is proposed, including:

[0078] Step S101, obtaining weight data to be quantized in a neural network, and counting the distribution characteristics thereof;

[0079] In some specific embodiments, obtaining weight data to be quantized in a neural network, and counting the distribution characteristics thereof, includes:

[0080] The distribution characteristics of the weight data are fitted into a normal distribution probability density function by a histogram statistical method.

[0081] The histogram of the weights of each layer of the trained network is counted, and the probability density is normalized and fitted to a normal distribution to obtain the mean μ and the standard deviation σ. The skewness and kurtosis can also be supplemented as the basis for subsequent division.

[0082] Step S102, dividing the weight data into a plurality of sub-intervals according to the distribution characteristics thereof;

[0083] According to the confidence interval or "three sigma principle", the weight space is divided into several sub-intervals (central dense region and two side tail intervals) according to the numerical interval, and the number of sub-intervals and boundaries are initially set and adaptively selected as needed, so as to decouple the "high density small value region" and "sparse large value tail", and avoid mutual restraint of single quantization parameter considering extreme and common values.

[0084] Step S103, based on the principle of minimum mean square error, fine-tuning the range of each sub-interval;

[0085] Iterative fine-tuning of each sub-interval boundary (coordinate search / small range grid / greedy adjustment can be used) until a better boundary configuration is obtained, converting the "statistical initial division" into a "minimum error division for task-oriented and reasoning closed loop", and minimizing the cumulative quantization error under the real forward calculation at the level and operator level.

[0086] Step S104, determining the quantization parameter for each sub-interval, quantizing the weight data, the quantization parameter including a scaling factor and a zero-point offset, using asymmetric fixed-point mapping for each sub-interval, calculating the scaling factor scale and the zero-point offset zero-point to realize reversible approximate mapping between floating-point and fixed-point; uniform quantization can be selected in each sub-interval to facilitate hardware implementation. In each sub-interval, make full use of the dynamic range to obtain finer granularity resolution for small values in the center and sufficient coverage for large values in the tail, and reduce overflow and resolution unevenness as a whole.

[0087] In some specific embodiments, the weight data is divided into multiple sub-intervals by confidence or three sigma principle, including:

[0088] Obtain the weight data to be quantized in the neural network and the preset confidence, obtain the range [x min ,x max ] of the weight data and the set {x0, x1, x2, …, x N−1} composed of the weight data; obtain the range [x min ,x max ] of the weight data, collect the weight set {x0, x1, x2, …, x N−1} of each layer after the network training is completed, and set the required confidence level (such as 95%, 99%) to provide data basis and division basis.

[0089] Calculate the mean and standard deviation of the weight data, calculate the mean μ and the standard deviation σ, which are used to describe the central tendency and dispersion of the data distribution, the mean is used to locate the distribution center, and the standard deviation is used to quantify the fluctuation range of the weight data;

[0090] The endpoints of the multiple sub-intervals are calculated by confidence or three-sigma principle;

[0091] If confidence is used, the data range within the confidence interval is calculated based on μ and σ, for example, 95% confidence corresponds to [μ−1.96σ, μ+1.96σ].

[0092] If the three-sigma principle is used, for example, [μ−3σ, μ+3σ] is used as the main interval, and the rest of the tail data is used as the extreme interval.

[0093] The endpoints of the multiple sub-intervals and the weight data range [x min ,x max ] are combined according to size to form multiple intervals, and [x min ,x max ].

[0094] The endpoints obtained in the previous step are merged, and a number of sub-intervals are formed in order of numerical size.

[0095] In some specific embodiments, the endpoints of the multiple sub-intervals are calculated by confidence, including:

[0096] The endpoints of the multiple sub-intervals are calculated according to at least one determined confidence, wherein one confidence can obtain the endpoints of two sub-intervals;

[0097] The relationship between the number of confidences required to divide the multiple sub-intervals is:

[0098] ;

[0099] wherein n is the number of sub-intervals divided, and k is the number of confidences required to be preset.

[0100] In some specific embodiments, the endpoints of the multiple sub-intervals are calculated according to at least one determined confidence, including the following formula:

[0101] ;

[0102] wherein the endpoints of the sub-intervals of one confidence are and , z is a normal distribution variable, α is a significance level that determines the confidence parameter, the formula is CL=1−α, Φ(z) is a probability distribution function, and z α / 2 is a critical value that determines the confidence interval boundary endpoint.

[0103] Specifically, when one confidence is taken (three intervals are obtained):

[0104] The first endpoint e− and the second endpoint e+ are calculated;

[0105] respectively ;

[0106] ;

[0107] Interval construction by endpoints, combined with weight data range [x min ,x max ], respectively, to build three spaces:

[0108] [x min ,e − ), [e − ,e + ), [e + ,x max ]; left closed right open or other endpoint conventions can be unified.

[0109] Specific examples

[0110] μ=0.3, σ=0.5, [x min ,x max ]=[−2.0, 2.0], CL(confidence)=0.95, α=0.05.

[0111] Look up table z α / 2 =z 0.025 ≈1.960;

[0112] e − =0.3−1.960×0.5=−0.680;

[0113] e + =0.3+1.960×0.5=1.280.

[0114] Get interval:

[0115] [−2.0,−0.680), [−0.680, 1.280), [1.280, 2.0].

[0116] Specifically, when setting two confidence levels (getting 5 sub-intervals)

[0117] Choose two sets of confidence levels CL1=1−α1 and CL2=1−α2 (usually CL2>CL1⇒α2<α1).

[0118] The four endpoints are calculated as:

[0119] e1 − =μ− σ;

[0120] e1 + =μ+ σ;

[0121] e2 − =μ+ σ;

[0122] e2 + =μ+ σ;

[0123] By constructing intervals through endpoints, combined with weight data range [x min ,x max ], respectively, five spaces are constructed as follows:

[0124] [x min ,e2 − ),[e2 − ,e1 − ),[e1 − ,e1 + ),[e1 + ,e2 + ),[e2 + ,x max ];

[0125] Wherein, > ; so that e 2 − <e 1 − <e 1 + <e 2 + ;

[0126] Numerical example:

[0127] μ=0.3,σ=0.5,[x min ,x max ]=[−2.0,2.0];

[0128] CL1 (confidence level) = 0.90, α = 0.05, we get =z 0.05 ≈1.645.

[0129] CL2 (confidence level) = 0.99, α = 0.01, =z 0.005 ≈2.576.

[0130] e1 − =0.3−1.645×0.5=−0.5225(≈−0.523);

[0131] e2 − =0.3−2.576×0.5=−0.988;

[0132] e1 + = 0.3 + 1.645 x 0.5 = 1.1225 (≈ 1.123);

[0133] e2 + = 0.3 + 2.576 x 0.5 = 1.588;

[0134] Five intervals:

[0135] [−2.0,−0.988), [−0.988,−0.523), [−0.523, 1.123), [1.123, 1.588), [1.588, 2.0].

[0136] In some specific embodiments, the endpoints of dividing multiple sub-intervals are obtained by the three-sigma principle, including:

[0137] As the boundary endpoints of sub-intervals (where = 1, 2, 3,...), the overall distribution is divided into = 1, 2, 3,...).

[0138] Specifically, use μ ± hσ as the boundary endpoints (where h = 1, 2, 3,...).

[0139] The entire weight distribution interval [x min , x max ] is divided into 2h+1 sub-intervals.

[0140] For example:

[0141] When h = 1, 3 sub-intervals are obtained;

[0142] When h = 2, 5 sub-intervals are obtained;

[0143] When h = 3, 7 sub-intervals are obtained;

[0144] Exemplary:

[0145] Suppose the network weight calculation of a certain layer is obtained;

[0146] μ = 0.3, σ = 0.5, [x min , x max ] = [−2.0, 2.0]

[0147] When h = 1: endpoints [−0.2, 0.8], intervals [−2.0,−0.2), [−0.2, 0.8), [0.8, 2.0];

[0148] When h = 2: endpoints [−0.7,−0.2, 0.8, 1.3], intervals 5 segments.

[0149] ​When h = 3: endpoints [-1.2, -0.7, -0.2, 0.8, 1.3, 1.8], 7 segments.

[0150] In some specific embodiments, the mean value in the weight data is calculated and the standard deviation , the formula is as follows:

[0151] ;

[0152] wherein, is the ith weight, is the total number of weights.

[0153] Specifically, the mean value μ describes the center position of the weight data, that is, the average value of all weights;

[0154] When the weight distribution is approximately normal, μ is close to the peak position of the distribution.

[0155] The standard deviation σ measures the dispersion of the weight data relative to the mean value; the smaller σ is, the more concentrated the weights are around μ; the larger σ is, the more dispersed the weight distribution is.

[0156] The calculation of μ and σ is the basis for interval division, and subsequent generation of sub-interval endpoints based on confidence intervals or three-sigma principle;

[0157] Exemplarily:

[0158] Suppose the weights of a certain layer of neural network are N = 6:

[0159] {x0, x1, x2, x3, x4, x5} = {0.2, -0.1, 0.5, 0.3, -0.4, 0.6};

[0160] The mean value μ ≈ 0.183;

[0161] The standard deviation σ ≈ 0.35.

[0162] In some specific embodiments, based on the principle of minimum mean square error, the range of each sub-interval is fine-tuned, including defining the following objective function formula:

[0163] ;

[0164] wherein, is the sub-interval fine-tuning parameter, is the floating point calculation result of the network layer, is the quantized calculation result, is the data size of the network layer calculation result;

[0165] The minimum value of the objective function J(θ) is obtained by adjusting the fine-tuning parameter θ of the sub-interval using an iterative algorithm.

[0166] The sub-interval fine-tuning parameter value θ obtained after iterative optimization is used as the boundary parameter of the fine-tuned sub-interval.

[0167] Specifically, by introducing a parameter θ to represent the adjustment of the sub-interval boundary or scaling factor, the optimization objective is to minimize the mean square error (MSE) between the quantization result and the floating-point result. Iterative algorithms (such as gradient descent, coordinate search, or greedy algorithm) are used to continuously adjust θ. At each step, the error between the quantization result and the floating-point result is calculated until the parameter θ that minimizes the objective function J(θ) is found. The optimized θ is used as the boundary parameter of the fine-tuned sub-interval. The quantization output is closer to the floating-point output, thereby reducing the precision loss introduced by quantization.

[0168] Suppose a certain layer has an initial weight interval of [−0.7, 0.8].

[0169] The error J between the quantized output and the floating-point output calculated using this interval is 0.015;

[0170] Adjust the endpoints to [−0.72, 0.80] and recalculate J = 0.007;

[0171] This indicates that this new interval is better and should be used as the final subinterval.

[0172] In some specific embodiments, quantization parameters are determined for each sub-interval, and the weighted data is quantized. The quantization parameters include a scaling factor and a zero-point offset, including:

[0173] Sub-intervals The floating-point range is mapped to the target fixed-point range [Q] min Q max] ;

[0174] Calculate the scaling factor for this sub-interval:

[0175] ;

[0176] Calculate the zero offset of this sub-interval;

[0177] ;

[0178] For any floating-point weight value belonging to this sub-interval Its corresponding quantization value Calculated by the following formula:

[0179] .

[0180] Specifically, the scaling factor si determines the proportional relationship between the floating-point interval and the integer interval, and the zero-point offset z i : ensures that the interval lower bound ai can be mapped to the integer interval Q min , avoiding overall offset;

[0181] Quantization value Q i Map floating-point numbers to the integer domain for storage and hardware calculation;

[0182] Dequantization value : Map from integer to floating-point approximation value, used for model forward inference.

[0183] Each sub-interval independently determines the scaling factor and zero point, which better fits the distribution characteristics of the weights in the interval; compared with a single scaling factor, the global quantization error is reduced, and the accuracy in the central decimal value area is higher;

[0184] Exemplary:

[0185] Suppose a sub-interval range is [a i ,b i ]=[−0.5,0.7], and the bit width is 8bit signed integer, that is, [Q min ,Q max ]=[−128,127].

[0186] Scaling factor:

[0187] ;

[0188] Zero-point drift:

[0189] ;

[0190] Quantization value of a certain weight xi=0.3:

[0191] ;

[0192] Approximate value obtained by dequantization:

[0193] ;

[0194] Final result: ≈0.301, with an error of only 0.001 from the original value x i =0.3, which is relatively small.

[0195] In some embodiments, the present application proposes a sub-interval quantization device based on neural network weight distribution, which combines Figure 4 , comprising:

[0196] Acquisition unit 201 is used to acquire the weight data to be quantized in the neural network and to statistically analyze its distribution characteristics;

[0197] The generation unit 202 is used to divide the weighted data into multiple sub-intervals according to the distribution characteristics of the weighted data;

[0198] The adjustment unit 203 is used to fine-tune the range of each sub-interval based on the principle of minimizing mean square error;

[0199] The quantization unit 204 is used to determine quantization parameters for each sub-interval and to quantize the weighted data. The quantization parameters include a scaling factor and a zero offset.

[0200] In other embodiments of the present invention, an electronic device 400 is disclosed, such as... Figure 5 As shown, the electronic device may include: one or more processors 401; a memory 402; a display 403; one or more application programs (not shown); and one or more computer programs 404. These devices can be connected via one or more communication buses 405. The one or more computer programs 404 are stored in the memory 402 and configured to be executed by the one or more processors 401. The one or more computer programs 404 include instructions that can be used to perform actions such as... Figures 1 to 2 And the steps in the corresponding embodiments.

[0201] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0202] In the various embodiments of this invention, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0203] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application or the entire or part of the technical solutions that make essential contributions to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a flash memory, a mobile hard disk, a read-only memory, a random access memory, a magnetic disk or an optical disk, and various media that can store program codes.

[0204] The above is only a specific implementation of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any change or replacement within the technical scope disclosed by the embodiments of the present application should be covered within the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A sub-interval quantization method based on neural network weight distribution, applied to image recognition, characterized in that, include: Obtain the weight data to be quantized in the neural network and statistically analyze its distribution characteristics; Based on the distribution characteristics of the weighted data, it is divided into multiple sub-intervals; Based on the principle of minimizing mean square error, the range of each sub-interval is finely adjusted; Quantization parameters are determined for each sub-interval, and the weighted data is quantized. The quantization parameters include a scaling factor and a zero-point offset. The weighted data is divided into multiple sub-intervals using confidence levels or the three Sigma principle, including: Obtain the weight data to be quantized in the neural network and the preset confidence level to obtain the weight data range [x]. min ,x max ]; The mean of the weighted data is calculated. and standard deviation ; The endpoints for dividing multiple sub-intervals are calculated using confidence level or the three sigma principle; Through the endpoints and weighted data ranges of multiple sub-intervals [x] min ,x max The intervals are divided into multiple ranges based on their size.

2. The sub-interval quantization method based on neural network weight distribution according to claim 1, characterized in that, Obtain the weight data to be quantized in the neural network and statistically analyze its distribution characteristics, including: The distribution characteristics of the weighted data are fitted into a normal distribution probability density function using histogram statistics.

3. The sub-interval quantization method based on neural network weight distribution according to claim 2, characterized in that, The endpoints for dividing multiple sub-intervals are obtained through confidence level calculation, including: The endpoints of multiple sub-intervals are calculated based on at least one determined confidence level, wherein one confidence level can obtain the endpoints of two sub-intervals; The relationship between the number of confidence levels required to divide the data into multiple sub-intervals is as follows: ； Where n is the number of sub-intervals, and k is the number of confidence levels that need to be preset.

4. The sub-interval quantization method based on neural network weight distribution according to claim 2, characterized in that, The endpoints for dividing multiple sub-intervals are obtained using the Three Sigma principle, including: by As the boundary endpoints of the sub-intervals, where =1,2,3..., dividing the overall distribution into... Sub-intervals.

5. The sub-interval quantization method based on neural network weight distribution according to claim 1, characterized in that, The mean of the weighted data is calculated. and standard deviation The formula is as follows: ； in, For the i-th weight, This represents the total number of weights.

6. The sub-interval quantization method based on neural network weight distribution according to claim 1 or 3, characterized in that, The endpoints for dividing multiple sub-intervals are calculated based on at least one determined confidence level, including the following formula: ； Wherein, the endpoints of the sub-interval of a confidence level are as well as Let z be a normally distributed variable, α be the significance level that determines the confidence parameter, and the formula be CL = 1 − α. Let Φ(z) be the probability distribution function. α / 2 To determine the critical values ​​at the endpoints of the confidence interval.

7. The sub-interval quantization method based on neural network weight distribution according to claim 1, characterized in that, Based on the principle of minimizing mean square error, the range of each sub-interval is fine-tuned, including defining the following objective function formula: ； in, For fine-tuning parameters in sub-intervals, The floating-point calculation results for the network layers. The result is the quantized calculation. The data size of the network layer calculation results; The minimum value of the objective function J(θ) is found by adjusting the sub-interval fine-tuning parameter θ using an iterative algorithm. The fine-tuning parameter θ of the sub-interval obtained after iterative optimization is used as the boundary parameter of the fine-tuned sub-interval.

8. The sub-interval quantization method based on neural network weight distribution according to claim 1, characterized in that, For each sub-interval, quantization parameters are determined, and the weighted data is quantized. These quantization parameters include a scaling factor and a zero-point offset, and include: Sub-intervals The floating-point range is mapped to the target fixed-point range [Q] min Q max] ; Calculate the scaling factor for this sub-interval: ； Calculate the zero offset of this sub-interval; ； For any floating-point weight value belonging to this sub-interval Its corresponding quantization value Calculated by the following formula: 。 9. A sub-interval quantization device based on neural network weight distribution, applied to image recognition, characterized in that, include: The acquisition unit is used to acquire the weight data to be quantized in the neural network and to statistically analyze its distribution characteristics. The generation unit is used to divide the weighted data into multiple sub-intervals based on the distribution characteristics of the weighted data; The adjustment unit is used to fine-tune the range of each sub-interval based on the principle of minimizing mean square error; A quantization unit is used to determine quantization parameters for each sub-interval and to quantize the weighted data. The quantization parameters include a scaling factor and a zero-point offset. The weighted data is divided into multiple sub-intervals using confidence levels or the three Sigma principle, including: Obtain the weight data to be quantized in the neural network and the preset confidence level to obtain the weight data range [x]. min ,x max ]; The mean of the weighted data is calculated. and standard deviation ; The endpoints for dividing multiple sub-intervals are calculated using confidence level or the three sigma principle; Through the endpoints and weighted data ranges of multiple sub-intervals [x] min ,x max The intervals are divided into multiple ranges based on their size.

Images