Subinterval quantization method and device based on neural network weight distribution

Abstract

The invention discloses a sub-interval quantization method and device based on neural network weight distribution, and relates to the field of weight quantization, and the method comprises the steps: obtaining to-be-quantized weight data in a neural network, and carrying out the statistics of the distribution characteristics of the to-be-quantized weight data; dividing the weight data into a plurality of sub-intervals according to the distribution characteristics of the weight data; on the basis of a mean square error minimum principle, performing fine adjustment on the range of each subinterval; a quantization parameter is determined for each subinterval, the weight data is quantized, and the quantization parameters include a scaling factor and a zero offset. According to the main technical scheme and effects, the statistical characteristics of approximate normal distribution, dense center and rapid attenuation on two sides of weight data after training are fully utilized. The subinterval division is carried out based on data distribution, so that the defect that the traditional method is uneven in data distribution and particularly serious in precision loss under multi-modal distribution is overcome.

Description

Technical Field

[0001] This invention relates to the field of weight quantization, and in particular to a sub-interval quantization method and apparatus based on the weight distribution of a neural network. Background Technology

[0002] In recent years, with the rapid development of deep learning technology, neural networks have been widely used in various audio and video processing tasks such as image recognition, speech recognition, and speech synthesis. To achieve higher accuracy and better performance, academia and industry have generally adopted larger-scale and deeper neural network models. However, the increased model size leads to a dramatic increase in computational and storage requirements, making it extremely difficult to deploy such models on resource-constrained embedded devices or mobile terminals.

[0003] To address these issues, the industry typically employs model quantization techniques, which map floating-point weights and activation values ​​to low-to-low ratio specific point values ​​to reduce storage and computational overhead. Common quantization methods currently include:

[0004] Per-Tensor Quantization: Assigns uniform quantization parameters to the entire tensor (such as a single-layer weight matrix). It is computationally simple and hardware-friendly, but when the weight distribution varies greatly, small values ​​are ignored, resulting in a significant loss of accuracy.

[0005] Per-Channel Quantization: Assigning quantization parameters to each channel of a convolutional or fully connected layer can improve accuracy to some extent, but it still cannot completely avoid errors caused by uneven data distribution.

[0006] Group quantization refines the parameter representation by dividing the weights into multiple groups and quantizing them separately. Compared with the previous two methods, it improves accuracy, but it still produces large errors when dealing with multimodal or strongly asymmetric distributions.

[0007] In summary, current quantization techniques do not fully consider the statistical characteristics of neural network weight distribution. When the weight distribution exhibits significant non-uniformity or multimodal characteristics, traditional quantization methods often struggle to guarantee quantization accuracy, and may even lead to network models failing to meet accuracy requirements in practical deployments. Summary of the Invention

[0008] Purpose of the invention: The purpose of this invention is to solve the technical problems in the prior art and provide a sub-interval quantization method, device, equipment and medium based on neural network weight distribution.

[0009] This specification relates to one or more embodiments of a sub-interval quantization device based on neural network weight distribution, an electronic device, a computer-readable storage medium, and a computer program product, in order to address the technical deficiencies existing in the prior art.

[0010] Technical solution:

[0011] Firstly, this application proposes a sub-interval quantization method based on the weight distribution of a neural network, comprising:

[0012] Obtain the weight data to be quantized in the neural network and statistically analyze its distribution characteristics;

[0013] Based on the distribution characteristics of the weighted data, it is divided into multiple sub-intervals;

[0014] Based on the principle of minimizing mean square error, the range of each sub-interval is finely adjusted;

[0015] A quantization parameter is determined for each sub-interval, and the weighted data is quantized. The quantization parameter includes a scaling factor and a zero offset.

[0016] Preferably, the weight data to be quantized in the neural network is obtained, and its distribution characteristics are statistically analyzed, including:

[0017] The distribution characteristics of the weighted data are fitted into a normal distribution probability density function using histogram statistics.

[0018] Preferably, the weighted data is divided into multiple sub-intervals using confidence levels or the three sigma principle, including:

[0019] Obtain the weight data to be quantized in the neural network and the preset confidence level to obtain the weight data range [x]. min ,x max and a set of weighted data {x0,x1,x2,…, };

[0020] The mean of the weighted data is calculated. and standard deviation ;

[0021] The endpoints for dividing multiple sub-intervals are calculated using confidence level or the three sigma principle;

[0022] Through the endpoints and range of values ​​of multiple sub-intervals [x min ,x max The intervals are divided into multiple ranges based on their size.

[0023] Preferably, the endpoints for dividing multiple sub-intervals are obtained through confidence level calculation, including:

[0024] The endpoints of multiple sub-intervals are calculated based on at least one determined confidence level, wherein one confidence level can obtain the endpoints of two sub-intervals;

[0025] The relationship between the number of confidence levels required to divide the data into multiple sub-intervals is as follows:

[0026] ;

[0027] Where n is the number of sub-intervals, and k is the number of confidence levels that need to be preset.

[0028] Preferably, the endpoints for dividing multiple sub-intervals are obtained using the three Sigma principle, including:

[0029] by As the boundary endpoints of the sub-interval (where =1,2,3...), dividing the overall distribution into A range.

[0030] Preferred,

[0031] The mean of the weighted data is calculated. and standard deviation The formula is as follows:

[0032] ;

[0033] in, For the i-th weight, This represents the total number of weights.

[0034] Preferably, the endpoints for dividing multiple sub-intervals are calculated based on at least one determined confidence level, including the following formula:

[0035] ;

[0036] Among them, the endpoint of one of the confidence levels of the sub-interval is as well as Let z be a normally distributed variable, and α be the significance level that determines the confidence parameter. The formula is: Φ(z) is the probability distribution function, z α / 2 To determine the critical values ​​at the endpoints of the confidence interval.

[0037] Preferably, based on the principle of minimizing mean square error, the range of each sub-interval is fine-tuned, including defining the following objective function formula:

[0038] ;

[0039] in, For fine-tuning parameters in sub-intervals, The floating-point calculation results for the network layers. The result is the quantized calculation. The data size of the network layer calculation results;

[0040] The optimization variable θ is adjusted using an iterative algorithm to find the minimum value of the objective function J(θ);

[0041] The optimized variable value θ obtained after iterative optimization is used as the fine-tuned sub-interval boundary parameter.

[0042] Preferably, quantization parameters are determined for each sub-interval, and the weighted data is quantized. The quantization parameters include a scaling factor and a zero-point offset, including:

[0043] Sub-intervals The floating-point range is mapped to the target fixed-point range [Q] min Q max] ;

[0044] Calculate the scaling factor for this sub-interval:

[0045] ;

[0046] Calculate the zero offset of this sub-interval;

[0047] ;

[0048] For any floating-point weight value belonging to this sub-interval Its corresponding quantization value Calculated by the following formula:

[0049] .

[0050] Secondly, embodiments of the present invention provide a sub-interval quantization device based on neural network weight distribution, comprising:

[0051] The acquisition unit is used to acquire the weight data to be quantized in the neural network and to statistically analyze its distribution characteristics.

[0052] The generation unit is used to divide the weighted data into multiple sub-intervals based on the distribution characteristics of the weighted data;

[0053] The adjustment unit is used to fine-tune the range of each sub-interval based on the principle of minimizing mean square error;

[0054] A quantization unit is used to determine quantization parameters for each sub-interval and to quantize the weighted data. The quantization parameters include a scaling factor and a zero-point offset.

[0055] Thirdly, embodiments of the present invention provide an electronic device, including a processor and a memory. The memory stores one or more computer programs; when the one or more computer programs stored in the memory are executed by the processor, the electronic device is able to implement any of the possible design methods described in the first aspect.

[0056] Fourthly, the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method as described in any of the above embodiments.

[0057] Fifthly, embodiments of the present invention also provide a computer program product that, when run on an electronic device, causes the electronic device to perform any possible design method of any of the above aspects.

[0058] Beneficial effects: Unlike existing tensor-based quantization, channel-based quantization, and group quantization, this invention does not simply determine the quantization parameters based on the position of the weights in the network structure. Instead, it fully utilizes the statistical characteristics of weight data after training, which exhibits an approximately normal distribution, dense distribution at the center, and rapid decay at both sides. By dividing the data into sub-intervals based on the data distribution, it overcomes the shortcomings of traditional methods that suffer severe accuracy loss under uneven data distribution, especially under multimodal distribution.

[0059] This invention first divides the weights into several sub-intervals according to the confidence interval of the normal distribution, and then refines the boundaries by combining methods such as the "three sigma principle" so that each sub-interval can more accurately reflect the distribution characteristics of the weights in different intervals, thereby providing a more scientific basis for quantification.

[0060] Based on the initial sub-interval division, this invention introduces the principle of minimizing mean square error, dynamically fine-tuning the range of each sub-interval to ensure that the deviation between the quantized fixed-point value and the floating-point value is minimized. This mechanism effectively reduces quantization error and maintains the stability and accuracy of the model during inference.

[0061] Through the above strategy, the present invention can significantly compress the number of network model parameters and computational complexity without significantly sacrificing accuracy, so that large-scale neural networks can meet the accuracy and performance requirements in resource-constrained embedded devices, greatly improving the feasibility of practical applications.

[0062] This method is not only applicable to speech synthesis (such as FastSpeech2) models, but can also be extended to various deep learning networks such as image recognition and speech recognition. It provides a unified quantitative optimization approach for different task scenarios and has strong universality and promotion value. Attached Figure Description

[0063] Figure 1A schematic diagram of the method framework for this invention is provided;

[0064] Figure 2 This is a schematic diagram of the existing weight values ​​of the present invention;

[0065] Figure 3 This is a schematic diagram of fitting the existing weight values ​​to a normal distribution according to the present invention;

[0066] Figure 4 This is a block diagram of a device structure provided in one embodiment of this application;

[0067] Figure 5 This is a block diagram of an electronic device structure provided in one embodiment of this application. Detailed Implementation

[0068] To make the technical solution of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0069] Example 1

[0070] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions in the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without inventive effort are within the scope of protection of this invention. Unless otherwise defined, the technical or scientific terms used herein should have the ordinary meaning understood by those skilled in the art. The terms "comprising" and similar expressions used herein mean that the element or object preceding the word covers the element or object listed following the word and its equivalents, but do not exclude other elements or objects.

[0071] In the process of quantifying the weights of neural networks, extensive statistical analysis and observation revealed that the weight values ​​of fully trained neural networks generally exhibit a significant leptokurtic and approximately symmetrical distribution. Specifically, the vast majority of weight values ​​are highly concentrated in a very small interval near the zero point, forming sharp peaks; while as the absolute value increases, the frequency of weight occurrence decreases sharply, but in the tail intervals far from the zero point, there are still a small number of relatively large positive and negative values ​​that cannot be ignored, constituting the so-called "thick tail" phenomenon.

[0072] To illustrate this statistical pattern, we will use the widely used speech synthesis network FastSpeech2 (e.g., Figure 2Taking a typical linear layer (or convolutional layer) as an example, we conducted an in-depth empirical analysis of its weight tensor after sufficient training. We used histogram statistics to explore the numerical distribution of the layer's weights: first, all weight values ​​in the layer were expanded into a one-dimensional vector; then, an appropriate bin width was selected to divide the data range, and the number (frequency) of weights falling into each bin was counted.

[0073] Figure 3 The statistical distribution of weight values ​​after network training is presented in the form of a histogram, where the horizontal axis represents the weight value, the vertical axis represents the corresponding probability density, and the curve is a normal distribution probability density curve superimposed on the histogram to fit the distribution trend of the actual weight data.

[0074] The comparison between the histogram and the fitted curve reveals that the weight data exhibits a typical unimodal approximate normal distribution, meaning that the vast majority of data is highly concentrated near the center, with the peak value close to zero. In contrast, only a small number of weight data points exist in the larger positive and smaller negative value ranges, which are farther from the center. This result further confirms that neural network weights have a significant distribution characteristic of being densely concentrated at the center and rapidly decaying at both sides, providing a statistical basis for the selection of subsequent weight quantization and compression algorithms. The normal distribution probability density function is:

[0075] ;

[0076] The mean, The standard deviation is denoted as .

[0077] In response to the problems existing in the current technology, such as Figure 1 As shown, a sub-interval quantization method based on neural network weight distribution is proposed, including:

[0078] Step S101: Obtain the weight data to be quantized in the neural network and statistically analyze its distribution characteristics;

[0079] In some specific embodiments, the weight data to be quantized in the neural network is obtained, and its distribution characteristics are statistically analyzed, including:

[0080] The distribution characteristics of the weighted data are fitted into a normal distribution probability density function using histogram statistics.

[0081] Histogram statistics are performed on the weights of each layer of the trained network. Probability density normalization and normality fitting are then performed to obtain the mean μ and standard deviation σ. Supplementary statistics such as skewness / kurtosis can be added for subsequent partitioning.

[0082] Step S102: Divide the weighted data into multiple sub-intervals according to the distribution characteristics of the weighted data;

[0083] Based on the confidence interval or the "three sigma principle", the weight space is divided into several sub-intervals (central dense region and two tail regions on both sides) according to the numerical interval. If necessary, the number of sub-intervals and the boundary are initially set and adaptively selected to decouple the "high-density small value region" and the "sparse large value tail" to avoid a single quantization parameter from simultaneously taking into account extreme and common values ​​and thus mutually constraining each other.

[0084] Step S103: Based on the principle of minimizing mean square error, fine-tune the range of each sub-interval;

[0085] The boundaries of each sub-interval are iteratively fine-tuned (using coordinate search / small-scale grid / greedy adjustment) until convergence to a better boundary configuration, transforming the "statistical initial partitioning" into "minimum error partitioning oriented towards task and inference closed loop", minimizing the accumulation of quantization error under hierarchical and operator-level real forward computation.

[0086] Step S104: Determine quantization parameters for each sub-interval and quantize the weighted data. The quantization parameters include a scaling factor and a zero-point offset. Asymmetric fixed-point mapping is applied to each sub-interval, and the scaling factor (scale) and zero-point offset (zero-point) are calculated to achieve a reversible approximate mapping between floating-point and fixed-point data. Uniform quantization can be selected within the sub-interval to facilitate hardware implementation. The dynamic range is fully utilized within each sub-interval to achieve finer-grained resolution for smaller central values ​​and sufficient coverage for larger tail values, thereby reducing overall overflow and resolution unevenness.

[0087] In some specific embodiments, the weight data is divided into multiple sub-intervals using confidence levels or the three sigma principle, including:

[0088] Obtain the weight data to be quantized in the neural network and the preset confidence level to obtain the weight data range [x]. min ,x max and a set of weighted data {x0,x1,x2,…, }; The range of the weighted data [x] was obtained. min ,x max Collect the weight set {x0, x1, x2, ..., x3} of each layer after the network training is completed. } and set the required confidence level (such as 95%, 99%) to provide a data basis and a basis for division.

[0089] The mean of the weighted data is calculated. and standard deviation The mean μ and standard deviation σ are calculated to characterize the central tendency and dispersion of the data distribution. The mean is used to locate the distribution center, and the standard deviation is used to quantify the fluctuation range of the weighted data.

[0090] The endpoints for dividing multiple sub-intervals are calculated using confidence level or the three sigma principle;

[0091] If confidence levels are used, the range of data within the confidence interval is calculated based on μ and σ. For example, a 95% confidence level corresponds to […]. ,μ+1.96σ].

[0092] If the "Three Sigma principle" is adopted, then for example [ [μ+3σ] is used as the main interval, and the remaining tail data is used as the extreme interval.

[0093] Through the endpoints and range of values ​​of multiple sub-intervals [x min ,x max Based on the size, divide [x] into multiple intervals. min ,x max ].

[0094] Merge with the endpoints obtained in the previous step, and form several sub-intervals in numerical order.

[0095] In some specific embodiments, the endpoints for dividing multiple sub-intervals are obtained through confidence level calculation, including:

[0096] The endpoints of multiple sub-intervals are calculated based on at least one determined confidence level, wherein one confidence level can obtain the endpoints of two sub-intervals;

[0097] The relationship between the number of confidence levels required to divide the data into multiple sub-intervals is as follows:

[0098] ;

[0099] Where n is the number of sub-intervals, and k is the number of confidence levels that need to be preset.

[0100] In some specific embodiments, the endpoints for dividing multiple sub-intervals are calculated based on at least one determined confidence level, including the following formula:

[0101] ;

[0102] Among them, the endpoint of one of the confidence levels of the sub-interval is as well as Let z be a normally distributed variable, and α be the significance level that determines the confidence parameter. The formula is: Φ(z) is the probability distribution function, z α / 2 To determine the critical values ​​at the endpoints of the confidence interval.

[0103] Specifically: When taking a confidence level (resulting in three intervals):

[0104] By calculating the first endpoint The second endpoint e+;

[0105] They are respectively ;

[0106] ;

[0107] Constructing intervals using endpoints, combined with the range [x] min ,x max They are constructed into three spaces respectively:

[0108] , ,[e + ,x max Left-closed, right-open, or other endpoint conventions can be standardized.

[0109] Specific examples;

[0110] μ=0.3, σ=0.5, [x min ,x max ]= With CL (confidence level) = 0.95, we can obtain α = 0.05.

[0111] Find z from the table α / 2 =z 0.025 ≈1.960;

[0112] ;

[0113] e + =0.3+1.960×0.5=1.280.

[0114] The interval is obtained:

[0115] .

[0116] Specifically, when two confidence levels are set (5 sub-intervals are obtained);

[0117] Choose two confidence levels and (generally ).

[0118] The endpoints are calculated to yield four endpoints:

[0119] ;

[0120] e1 + =μ+ σ;

[0121] ;

[0122] e2 + =μ+ σ;

[0123] Constructing intervals using endpoints, combined with the range [x] min ,x max They are constructed into five spaces respectively:

[0124] ,[e1 + e2 + ),[e2 + ,x max ];

[0125] in, > ; to make ;

[0126] Numerical examples:

[0127] μ=0.3, σ=0.5, [x min ,x max ]= ;

[0128] With CL1 (confidence level) = 0.90, we can obtain α = 0.05. =z 0.05 ≈1.645.

[0129] With CL2 (confidence level) = 0.99, we can obtain α = 0.01. =z 0.005 ≈2.576.

[0130]

[0131] ;

[0132] e1 + =0.3 + 1.645 × 0.5 = 1.1225 (≈1.123);

[0133] e2 + =0.3 + 2.576 × 0.5 = 1.588;

[0134] Five intervals:

[0135] ,[1.123,1.588),[1.588,2.0].

[0136] In some specific embodiments, the endpoints for dividing multiple sub-intervals are obtained using the three Sigma principle, including:

[0137] by As the boundary endpoints of the sub-interval (where =1,2,3...), dividing the overall distribution into A range.

[0138] Specifically, μ±hσ is used as the boundary endpoints (where h=1,2,3,…).

[0139] The entire weight distribution interval [x min ,x max It is divided into 2h+1 sub-intervals.

[0140] For example:

[0141] When h=1, we get 3 sub-intervals;

[0142] When h=2, we get 5 sub-intervals;

[0143] When h=3, we get 7 sub-intervals;

[0144] For example:

[0145] Assume the weights of a certain layer of the network are calculated as follows;

[0146] μ=0.3, σ=0.5, [x min ,x max ]=

[0147] When h=1: the endpoint is The interval is ,[0.8,2.0];

[0148] When h=2: Endpoint The interval is divided into 5 segments.

[0149] When h=3: Endpoint The interval consists of 7 segments.

[0150] In some specific embodiments, the mean of the weighted data is calculated. and standard deviation The formula is as follows:

[0151] ;

[0152] in, For the i-th weight, This represents the total number of weights.

[0153] Specifically, the mean μ describes the central location of the weighted data, that is, the average of all weights;

[0154] When the weight distribution is approximately normal, μ is close to the peak position of the distribution.

[0155] The standard deviation σ measures the degree of dispersion of the weighted data relative to the mean; the smaller the σ, the more concentrated the weights are around μ; the larger the σ, the more dispersed the weight distribution.

[0156] The calculation of μ and σ is the basis for interval division. Whether confidence intervals or the three sigma principle are used, the endpoints of the sub-intervals must be generated based on these two statistics.

[0157] For example:

[0158] Suppose that a certain layer of a neural network has a total of N=6 weights:

[0159] {x0,x1,x2,x3,x4,x5}= ;

[0160] The mean μ ≈ 0.183;

[0161] The standard deviation σ ≈ 0.35.

[0162] In some specific embodiments, based on the principle of minimizing mean square error, the range of each sub-interval is fine-tuned, including defining the following objective function formula:

[0163] ;

[0164] in, For fine-tuning parameters in sub-intervals, The floating-point calculation results for the network layers. The result is the quantized calculation. The data size of the network layer calculation results;

[0165] The optimization variable θ is adjusted using an iterative algorithm to find the minimum value of the objective function J(θ);

[0166] The optimized variable value θ obtained after iterative optimization is used as the fine-tuned sub-interval boundary parameter.

[0167] Specifically, by introducing a parameter θ to represent the adjustment of the sub-interval boundary or scaling factor, the optimization objective is to minimize the mean square error (MSE) between the quantization result and the floating-point result. Iterative algorithms (such as gradient descent, coordinate search, or greedy algorithm) are used to continuously adjust θ. At each step, the error between the quantization result and the floating-point result is calculated until the parameter θ that minimizes the objective function J(θ) is found. The optimized θ is used as the boundary parameter of the fine-tuned sub-interval. The quantization output is closer to the floating-point output, thereby reducing the precision loss introduced by quantization.

[0168] Suppose a certain layer has an initial weight interval of... ,

[0169] The error J between the quantized output and the floating-point output calculated using this interval is 0.015;

[0170] Adjust endpoint to Recalculate J = 0.007;

[0171] This indicates that this new interval is better and should be used as the final subinterval.

[0172] In some specific embodiments, quantization parameters are determined for each sub-interval, and the weighted data is quantized. The quantization parameters include a scaling factor and a zero-point offset, including:

[0173] Sub-intervals The floating-point range is mapped to the target fixed-point range [Q] min Q max] ;

[0174] Calculate the scaling factor for this sub-interval:

[0175] ;

[0176] Calculate the zero offset of this sub-interval;

[0177] ;

[0178] For any floating-point weight value belonging to this sub-interval Its corresponding quantization value Calculated by the following formula:

[0179] .

[0180] Specifically, the scaling factor si determines the ratio between the floating-point interval and the integer interval, and the zero offset z i : Ensure that the lower bound of the interval ai can be mapped to the integer interval Q min To avoid overall offset;

[0181] Quantization value Q i Mapping floating-point numbers to integer fields facilitates storage and hardware computation;

[0182] Inverse quantization value : Maps integers back to floating-point approximations for forward inference of the model.

[0183] Each sub-interval independently determines the scaling factor and zero point, better matching the weight distribution characteristics of that interval; compared with a single scaling factor, the global quantization error is reduced, especially the accuracy is higher in the central small value region;

[0184] For example:

[0185] Suppose that the range of a certain subinterval is [ai ,b i ]= 8-bit signed integer, i.e., [Q min Q max ]= .

[0186] Scaling factor:

[0187] ;

[0188] Zero drift amount:

[0189] ;

[0190] The quantized value of a certain weight xi=0.3:

[0191] ;

[0192] Dequantization yields an approximate value:

[0193] ;

[0194] Final result: ≈0.301, compared to the original value x i The error of 0.3 is only 0.001, which is relatively small.

[0195] In some embodiments, this application proposes a sub-interval quantization device based on neural network weight distribution, combined with Figure 4 ,include:

[0196] Acquisition unit 201 is used to acquire the weight data to be quantized in the neural network and to statistically analyze its distribution characteristics;

[0197] The generation unit 202 is used to divide the weighted data into multiple sub-intervals according to the distribution characteristics of the weighted data;

[0198] The adjustment unit 203 is used to fine-tune the range of each sub-interval based on the principle of minimizing mean square error;

[0199] The quantization unit 204 is used to determine quantization parameters for each sub-interval and to quantize the weight data. The quantization parameters include a scaling factor and a zero offset.

[0200] In other embodiments of the present invention, an electronic device 400 is disclosed, such as... Figure 5As shown, the electronic device may include: one or more processors 401; a memory 402; a display 403; one or more application programs (not shown); and one or more computer programs 404. These devices can be connected via one or more communication buses 405. The one or more computer programs 404 are stored in the memory 402 and configured to be executed by the one or more processors 401. The one or more computer programs 404 include instructions that can be used to perform actions such as... Figures 1 to 2 And the various steps in the corresponding embodiments.

[0201] Through the above description of the embodiments, those skilled in the art will clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device, and unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0202] In the various embodiments of this invention, the functional units can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0203] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as flash memory, portable hard disk, read-only memory, random access memory, magnetic disk, or optical disk.

[0204] The above description is merely a specific implementation of the embodiments of the present invention, but the protection scope of the embodiments of the present invention is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of the present invention should be covered within the protection scope of the embodiments of the present invention. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims

1. A sub-interval quantization method based on neural network weight distribution, characterized in that, include: Obtain the weight data to be quantized in the neural network and statistically analyze its distribution characteristics; Based on the distribution characteristics of the weighted data, it is divided into multiple sub-intervals; Based on the principle of minimizing mean square error, the range of each sub-interval is finely adjusted; A quantization parameter is determined for each sub-interval, and the weighted data is quantized. The quantization parameter includes a scaling factor and a zero offset.

2. The sub-interval quantization method based on neural network weight distribution according to claim 1, characterized in that, Obtain the weight data to be quantized in the neural network and statistically analyze its distribution characteristics, including: The distribution characteristics of the weighted data are fitted into a normal distribution probability density function using histogram statistics.

3. The sub-interval quantization method based on neural network weight distribution according to claim 2, characterized in that, The weighted data is divided into multiple sub-intervals using confidence levels or the three Sigma principle, including: Obtain the weight data to be quantized in the neural network and the preset confidence level to obtain the weight data range [x]. min ,x max ]; The mean of the weighted data is calculated. and standard deviation ; The endpoints for dividing multiple sub-intervals are calculated using confidence level or the three sigma principle; Through the endpoints and range of values ​​of multiple sub-intervals [x min ,x max The intervals are divided into multiple ranges based on their size.

4. The sub-interval quantization method based on neural network weight distribution according to claim 2, characterized in that, The endpoints for dividing multiple sub-intervals are obtained through confidence level calculation, including: The endpoints of multiple sub-intervals are calculated based on at least one determined confidence level, wherein one confidence level can obtain the endpoints of two sub-intervals; The relationship between the number of confidence levels required to divide the data into multiple sub-intervals is as follows: ； Where n is the number of sub-intervals, and k is the number of confidence levels that need to be preset.

5. The sub-interval quantization method based on neural network weight distribution according to claim 2, characterized in that, The endpoints for dividing multiple sub-intervals are obtained using the Three Sigma principle, including: by As the boundary endpoints of the sub-intervals, where =1,2,3..., dividing the overall distribution into... Sub-intervals.

6. The sub-interval quantization method based on neural network weight distribution according to claim 3, characterized in that, The mean of the weighted data is calculated. and standard deviation The formula is as follows: ； in, For the i-th weight, This represents the total number of weights.

7. The sub-interval quantization method based on neural network weight distribution according to claim 3 or 4, characterized in that, The endpoints for dividing multiple sub-intervals are calculated based on at least one determined confidence level, including the following formula: ； Among them, the endpoint of one of the confidence levels of the sub-interval is as well as Let z be a normally distributed variable, and α be the significance level that determines the confidence parameter. The formula is: Φ(z) is the probability distribution function, z α / 2 To determine the critical values ​​at the endpoints of the confidence interval.

8. The sub-interval quantization method based on neural network weight distribution according to claim 3, characterized in that, Based on the principle of minimizing mean square error, the range of each sub-interval is fine-tuned, including defining the following objective function formula: ； in, For fine-tuning parameters in sub-intervals, The floating-point calculation results for the network layers. The result is the quantized calculation. The data size of the network layer calculation results; The minimum value of the objective function J(θ) is found by adjusting and optimizing the variable θ using an iterative algorithm. The optimized variable value θ obtained after iterative optimization is used as the fine-tuned sub-interval boundary parameter.

9. The sub-interval quantization method based on neural network weight distribution according to claim 1, characterized in that, For each sub-interval, quantization parameters are determined, and the weighted data is quantized. These quantization parameters include a scaling factor and a zero-point offset, and include: Sub-intervals The floating-point range is mapped to the target fixed-point range [Q] min Q max] ; Calculate the scaling factor for this sub-interval: ； Calculate the zero offset of this sub-interval; ； For any floating-point weight value belonging to this sub-interval Its corresponding quantization value Calculated by the following formula: 。 10. A sub-interval quantization device based on neural network weight distribution, characterized in that, include: The acquisition unit is used to acquire the weight data to be quantized in the neural network and to statistically analyze its distribution characteristics. The generation unit is used to divide the weighted data into multiple sub-intervals based on the distribution characteristics of the weighted data; The adjustment unit is used to fine-tune the range of each sub-interval based on the principle of minimizing mean square error; A quantization unit is used to determine quantization parameters for each sub-interval and to quantize the weighted data. The quantization parameters include a scaling factor and a zero-point offset.

Images