Quantization method and device based on neural network, chip, storage medium and program product
By generating initialization and simulation data in the neural network model, calculating the loss of the batch normalization layer, and iterating the simulated data, the model quantization problem without training set is solved, and efficient model quantization and performance improvement is achieved.
Patent Information
- Application Number
- CN202411593515.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-08
- Publication Date
- 2025-03-28
AI Technical Summary
In actual scenarios such as medical and finance, due to commercial secrets or personal privacy issues, the training set may not be obtained, resulting in the existing model quantization method being inapplicable and the model being unable to quantify, which greatly limits the performance of the model.
A quantization method based on neural network is proposed. By generating initialized data and simulated data, using the model to be quantized to process these data to determine the relaxation distribution coefficient of the batch normalization layer and the mean and variance of the output data, the total loss is calculated, and the simulation data is iterated until the total loss meets the preset conditions, and the target data for quantization is generated.
In the absence of data, effective target data for quantifying the model is generated, which improves the quantitative effect, thereby improving the model performance, and improving the generalization ability and training efficiency of the model to be quantified.
Smart Images
Figure CN119721142A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the technical field of quantization of neural network models, and in particular, relates to a quantization method, a quantization device, a chip, a computer-readable storage medium, and a computer program product based on a neural network. Background Art
[0002] At present, when quantizing a model, data is generally collected from the model's training set as a calibration set, and the model is quantized through the calibration set to obtain a quantized model. Model quantization can alleviate the problems of large number of parameters, large amount of calculation, and high memory usage of existing convolutional neural networks, and has "potential" advantages such as compressing parameters for neural networks, improving speed, and reducing memory usage.
[0003] However, in many practical scenarios such as medicine and finance, training sets may not be available due to commercial confidentiality or personal privacy issues. Due to the lack of training sets, existing model quantization methods are no longer applicable, resulting in the inability to quantify the model, which greatly limits the performance of the model. Summary of the invention
[0004] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a quantization method, a quantization device, a chip, a computer-readable storage medium and a computer program product based on a neural network, which can generate target data for model quantization in the absence of data, thereby quantizing the model and improving the model performance.
[0005] In a first aspect, the present application provides a quantization method, comprising generating initialization data and simulation data; processing the initialization data through a model to be quantized to determine a relaxation distribution coefficient of each batch normalization layer, the model to be quantized comprising multiple batch normalization layers, the relaxation distribution coefficient being determined based on a first mean and a first variance of output data of each batch normalization layer; processing the simulation data respectively through the model to be quantized to determine a second mean and a second variance of output data of each batch normalization layer; determining a total loss of the batch normalization layer, the total loss of the batch normalization layer being determined based on the relaxation distribution coefficient and the second mean and the second variance corresponding to each batch normalization layer; iterating the simulation data based on the total loss of the batch normalization layer until determining target data that makes the total loss of the batch normalization layer meet a preset condition, the target data being used to quantize the model to be quantized to generate a quantization model.
[0006] In a second aspect, the present application provides a quantization device, the quantization device comprising:
[0007] A generation module, used to generate initialization data and simulation data;
[0008] A first processing module, configured to process the initialization data through a model to be quantized to determine a relaxation distribution coefficient of each batch normalization layer, wherein the model to be quantized includes a plurality of batch normalization layers, and the relaxation distribution coefficient is determined based on a first mean and a first variance of output data of each batch normalization layer;
[0009] A second processing module, used for processing the simulated data respectively through the model to be quantized to determine a second mean and a second variance of the output data of each batch normalization layer;
[0010] a loss determination module, configured to determine a total loss of the batch normalization layer, wherein the total loss of the batch normalization layer is determined based on the relaxation distribution coefficient, and the second mean and the second variance corresponding to each of the batch normalization layers;
[0011] An iteration module is used to iterate the simulation data based on the total loss of the batch normalization layer until the target data that makes the total loss of the batch normalization layer meet the preset conditions is determined, and the target data is used to quantize the model to be quantized to generate a quantized model.
[0012] In a third aspect, the present application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which implements the above-mentioned quantization method when executed by a processor.
[0013] In a fourth aspect, the present application provides a chip, which is used to execute the above-mentioned quantization method.
[0014] In a fifth aspect, the present application provides a computer program product, including a computer program, which implements the above-mentioned quantization method when executed by a processor.
[0015] The quantization method, quantization device, chip, computer-readable storage medium and computer program product provided in the embodiments of the present application determine the model loss by generating initialization data and simulation data, and determining the model loss by the mean and variance of the output data of each batch normalization layer when the initialization data and simulation data are inferred by the model to be quantized, thereby iterating the simulation data until the loss corresponding to the simulation data satisfies the preset condition (such as less than the preset threshold value). It is considered that the simulation data at this time should be able to cover various situations encountered by the model in practical applications, so as to accurately reflect the behavior of the model, and thus serve as the target data for quantizing the model. Based on the target data as calibration data, the model to be quantized can be quantized to obtain a quantized model. Compared with selecting a calibration set in a training set to quantize the model, it is possible to generate target data for the quantization model in the absence of data, and ensure the validity of the target data for quantization, which is conducive to improving the quantization effect, thereby improving the model performance.
[0016] In addition, by calculating the relaxed distribution coefficient to calculate the model loss, the statistical constraints on each batch normalization layer can be relaxed, which is beneficial to improving the generalization ability of the model to be quantized and accelerating the training efficiency of the model, and more efficiently obtaining the target data for iteration.
[0017] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS
[0018] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:
[0019] Figure 1 It is a schematic diagram of an application scenario of the quantification method provided in an embodiment of the present application;
[0020] Figure 2 It is a flowchart of the quantification method provided in the embodiment of the present application;
[0021] Figure 3 is a flow chart of a quantification method provided in an embodiment of the present application; and
[0022] Figure 4 It is a module schematic diagram of the quantization device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0023] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.
[0024] For ease of understanding, the technical background and application scenarios of this application are first introduced below:
[0025] Quantization is the process of approximating the continuous value of a signal to a finite number of discrete values. It can be understood as a method of information compression. When considering this concept in a computer system, it is generally represented by "low bits". Some people also call quantization "fixed-point", but strictly speaking, the range of representation is reduced. Fixed-point refers specifically to linear quantization with a scale of 2, which is a more practical quantization method.
[0026] Convolutional neural networks have very good accuracy, and even surpass human accuracy in some tasks such as face recognition and image classification. However, their shortcomings are also obvious, with a large number of parameters, large amount of calculation, and memory usage. Model quantization can alleviate the problems of large number of parameters, large amount of calculation, and high memory usage of existing convolutional neural networks, and has "potential" advantages such as compressing parameters for neural networks, improving speed, and reducing memory usage.
[0027] DataFree quantization refers to quantization without calibration data. In practical applications, the general PTQ (Post Training Quantization) approach is to use a part of the calibration data to obtain the activation data range, which generally comes from the training set.
[0028] The process of quantizing a model generally includes the following steps:
[0029] 1. Data preparation
[0030] Collect or select a representative subset of data from the training set as the calibration set. This dataset should be able to cover various situations encountered by the model in actual applications in order to accurately reflect the behavior of the model.
[0031] 2. Set up quantitative configuration
[0032] Determine the target bit width for quantization, such as quantizing to 8 bits or less. Choose whether to perform symmetric or asymmetric quantization, and whether to use different quantization schemes for weights and activation functions.
[0033] 3. Calculate the quantification range
[0034] Quantization range of activation function: By traversing the calibration set, let the model run on floating point precision, collect the activation values output by each layer, and calculate the minimum and maximum values of these activation values to determine the quantization range. For asymmetric quantization, the minimum and maximum values directly determine the quantization range; symmetric quantization usually uses the mean and range to determine the quantization zero point and scale factor.
[0035] Quantization range of weights: Weight quantization usually directly analyzes the weight distribution in the model to determine the appropriate quantization parameters. Sometimes, the use of global minimum and maximum values, or independent minimum and maximum values for each weight tensor is also considered.
[0036] 4. Quantitative Operations
[0037] Using the calculated quantization parameters (such as quantization scale and offset), convert the model's weights and activation functions to low-precision representation. This step may involve direct numerical conversion or may require inserting quantization and dequantization operations into the model's computational graph.
[0038] 5. Fine-tuning (Optional)
[0039] After quantization, the accuracy of the model may decrease. In order to restore the accuracy, the quantized model can be fine-tuned. This step usually uses a calibration set or a part of the training data and performs a limited number of iterations at a low learning rate while keeping the model structure unchanged.
[0040] 6. Evaluation
[0041] After quantization and (if any) fine-tuning, use the validation set to evaluate the performance of the model, including key metrics such as accuracy, latency, and model size, to ensure that the quantized model meets the application requirements.
[0042] 7. Deployment
[0043] Finally, the optimized quantization model is deployed to the target hardware platform and a final integration test is performed to ensure that the model performs as expected in the actual environment.
[0044] See also Figure 1 , Figure 1 1 is an application scenario diagram of a quantization method provided in an embodiment of the present application. The application scenario provided in the present application includes a terminal device 101 and a server 102, and the quantization method provided in the present application can be executed by at least one of the terminal device 101 and the server 102.
[0045] Among them, the terminal devices may include but are not limited to: smart phones (such as Android phones, IOS phones, etc.), tablet computers, laptops, desktop computers, smart speakers, smart watches, portable personal computers, mobile Internet devices (Mobile Internet Devices, referred to as MID), intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, wearable devices, etc., and the embodiments of the present application do not limit this.
[0046] The server can be an independent physical server or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms, but the embodiments of the present application do not limit this.
[0047] It should be noted that Figure 1 The number of terminal devices and servers is only used as an example, and the number of terminal devices and servers can be more or less, which is not limited here. The terminal devices and servers can be directly or indirectly connected via wired or wireless communication, which is not limited in this application.
[0048] The quantitative method involved in this application can be implemented based on cloud technology.
[0049] Among them, cloud technology refers to a hosting technology that unifies hardware, software, network and other resources within a wide area network or local area network to achieve data calculation, storage, processing and sharing.
[0050] Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool, which can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The backend services of the technical network system require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the rapid development and application of the Internet industry, in the future, each item may have its own identification mark, which needs to be transmitted to the backend system for logical processing. Data of different levels will be processed separately. All kinds of industry data need strong system backing support, which can only be achieved through cloud computing.
[0051] The quantitative method of the present application can be implemented based on cloud computing. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, so that various application systems can obtain computing power, storage space and information services as needed. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use.
[0052] As a provider of basic cloud computing capabilities, a cloud computing resource pool (referred to as a cloud platform, generally referred to as an IaaS (Infrastructure as a Service) platform) will be established, and various types of virtual resources will be deployed in the resource pool for external customers to choose to use. The cloud computing resource pool mainly includes: computing devices (virtualized machines, including operating systems), storage devices, and network devices.
[0053] The quantization method in the embodiment of the present application can be executed by a chip, and the chip can be installed in at least one of the server and the terminal device, that is, the method can be executed by the server or the terminal device alone, or by the server and the terminal device together. Therefore, the execution subject of each step will not be repeatedly described below.
[0054] The neural network-based quantization method of the present application can generate target data for a quantization model as calibration data in the absence of data, thereby achieving quantization of the model.
[0055] Based on the above technical background and the introduction of related scenarios, the embodiment of the present application provides a quantization method, which is described in detail below:
[0056] See also Figure 2 A quantization method provided in an embodiment of the present application is implemented by steps 011 to 015, which are described in detail below.
[0057] Step 011: Generate initialization data and simulation data.
[0058] The initialization data and simulation data have the same size and both match the input of the model to be quantized.
[0059] Since there is no need to use a training set, in order to generate target data for quantization, it is necessary to actively generate initialization data and simulation data.
[0060] Optionally, based on Gaussian distribution, sampling is performed within a preset range (such as a preset range [0, 1]) to generate the initialization data and simulation data.
[0061] That is, random sampling is performed within a preset range to obtain a set of random data that conforms to the Gaussian distribution (i.e., a set of random numbers with mean μ = 0 and standard deviation σ = 1). The number of random data must match the size of the input tensor of the model to be quantized, such as generating initialization data and simulation data with a size of m*n = 2048.
[0062] Step 012: Processing the initialization data through the model to be quantized to determine the relaxation distribution coefficient of each batch normalization layer, the model to be quantized includes multiple batch normalization layers, and the relaxation distribution coefficient is determined based on the first mean and first variance of the output data of each batch normalization layer.
[0063] The model to be quantified is a model that needs to be quantified.
[0064] Among them, the batch normalization layer (BN layer for short) is a regularization technology widely used in deep learning networks. Its main purpose is to accelerate the network training process and improve the stability and generalization ability of the model.
[0065] Specifically, the initialized data can be inferred through the model to be quantized. During the inference process, each batch normalization layer of the model to be quantized will output data, so that the first mean and first variance of each output data of each batch normalization layer can be calculated.
[0066] The relaxed distribution coefficient of any target batch normalization layer in each batch normalization layer can be calculated according to its corresponding first mean and first variance.
[0067] Optionally, the relaxed distribution coefficient includes an offset coefficient and a scaling coefficient, the offset coefficient of the target batch normalization layer is determined based on a first mean of output data of the target batch normalization layer for the initialization data, and the scaling coefficient of the target batch normalization layer is determined based on a first variance of output data of the target batch normalization layer for the initialization data.
[0068] In order to preserve the learning ability of the network and avoid over-normalization of the data, the batch normalization layer also introduces two learnable parameters γ (scaling factor) and δ (offset). The data after the above standardization will be multiplied by γ and added to δ, so that the network can learn the scale and position that best suits its task.
[0069] Optionally, the offset coefficient δ is determined based on the following formula: Among them, δ i is the offset coefficient of the i-th batch normalization layer, is the first mean corresponding to the i-th batch normalization layer, μ i is the preset mean of the i-th batch normalization layer, and ∈ represents the quantile.
[0070] For example, the model to be quantized includes N batch normalization layers, then i=1, 2, ...N.
[0071] Optionally, the quantile ∈ can be 0.9. and μ i The difference between is no more than 90% of the data points in the data set. In other words, this is saying and μ i The difference is relatively small, so small that only the top 10% of the data set have a larger difference than this.
[0072] Optionally, the scaling factor is determined based on the following formula: Among them, γ i is the scaling factor of the i-th batch normalization layer, is the first variance corresponding to the i-th batch normalization layer, σ i is the preset variance of the i-th batch normalization layer, and ∈ represents the quantile.
[0073] In this way, by initializing the first mean and the first variance of the data, the relaxation distribution coefficient of each batch normalization layer can be quickly calculated.
[0074] Step 013: The simulated data are processed respectively through the model to be quantized to determine the second mean and second variance of the output data of each batch normalization layer.
[0075] Specifically, the simulated data can be inferred through the model to be quantized. During the inference process, each batch normalization layer of the model to be quantized will output data, so that the second mean and second variance of each output data of each batch normalization layer can be calculated.
[0076] Step 014: Determine the total loss of the batch normalization layer, where the total loss of the batch normalization layer is determined based on the relaxation distribution coefficient, and the second mean and second variance corresponding to each batch normalization layer.
[0077] Specifically, after determining the relaxation distribution coefficient, and the second mean and second variance corresponding to each batch normalization layer, the total loss of the batch normalization layer can be calculated.
[0078] Optionally, the total loss of the batch normalization layer can be a relaxed distribution alignment loss, which relaxes the statistical constraints of each BN layer.
[0079] Optionally, a relaxed distribution alignment loss is determined based on the losses of individual batch normalization layers.
[0080] For example, the relaxed distribution alignment loss is the sum of the losses of each batch normalization layer.
[0081] Optionally, the total loss of the batch normalization layer can be determined by a relaxed distribution alignment loss and a preset diversification coefficient to perform stratified sample augmentation with relaxed statistical constraints of each BN layer and provide a specific loss term for each sample.
[0082] Optionally, the total loss of the batch normalization layer is determined based on the following formula, Among them, L DSG is the total loss of the batch normalization layer, X LSE is the preset diversification coefficient, L SDA is the sum of the losses of each batch normalization layer.
[0083] Optionally, the diversification coefficient is determined based on the following formula: LSE =I+11 T; Where I is the n-dimensional identity matrix, 11 T is an n-dimensional matrix of all ones.
[0084] In this way, by setting the diversification coefficient and transforming the simulated data after each iteration, different losses can be generated for the simulated data of each iteration, thereby achieving enhancement of the simulated data.
[0085] Optionally, the loss of the i-th batch normalization layer is determined based on the following formula:
[0086] in, is the loss of the i-th batch normalization layer, is the second mean corresponding to the i-th batch normalization layer, μ i is the preset mean of the i-th batch normalization layer, δ i is the offset coefficient of the i-th batch normalization layer, is the second variance corresponding to the i-th batch normalization layer, σ i is the preset variance of the i-th batch normalization layer, γ i is the scaling factor of the i-th batch normalization layer.
[0087] Specifically, the difference between the absolute value of the difference between the second mean corresponding to the ith batch normalization layer and the preset mean of the ith batch normalization layer and the offset coefficient of the ith batch normalization layer can be subtracted, and the obtained difference can be compared with 0, so as to output the larger square of the second norm; then, the difference between the absolute value of the difference between the second variance corresponding to the ith batch normalization layer and the preset variance of the ith batch normalization layer and the scaling coefficient of the ith batch normalization layer can be subtracted, and the obtained difference can be compared with 0, so as to output the larger square of the second norm. Finally, the square of the second norm of the larger of the two parts is added to obtain the loss of the ith batch normalization layer, and the loss of each batch normalization layer is added to obtain the relaxed distribution alignment loss.
[0088] Step 015: Iterate the simulated data based on the total loss of the batch normalization layer until the target data that makes the total loss of the batch normalization layer meet the preset conditions is determined, and the target data is used to quantize the model to be quantized to generate a quantized model.
[0089] The preset condition refers to the condition under which the total loss of the batch normalization layer makes the model to be quantized converge. For example, the preset condition is that the total loss of the batch normalization layer is less than a preset threshold, or the preset condition is that the total loss corresponding to the simulated data of multiple consecutive iterations remains basically unchanged.
[0090] Specifically, after calculating the total loss of the batch normalization layer, the simulation data can be iterated based on the total loss, such as continuously adjusting the values of each data in the simulation data to achieve gradient descent, so that the total loss continues to decrease until the total loss of the batch normalization layer meets the preset conditions. At this time, the updated iterative simulation data input when the total loss meets the preset conditions can be obtained as the target data. The target data at this time should be able to cover various situations encountered by the model in actual applications, so as to accurately reflect the behavior of the model, and thus serve as the target data for quantifying the model.
[0091] Finally, the target data can be used as a calibration set to quantize the model to be quantized, thereby generating quantization and realizing data-free model quantization.
[0092] The quantization method of the embodiment of the present application determines the model loss by generating initialization data and simulation data, and determining the model loss by the mean and variance of the output data of each batch normalization layer when the initialization data and simulation data are inferred by the model to be quantized, thereby iterating the simulation data until the loss corresponding to the simulation data satisfies the preset conditions (such as less than the preset threshold value). It is considered that the simulation data at this time should be able to cover various situations encountered by the model in practical applications, so as to accurately reflect the behavior of the model, and thus serve as the target data for quantizing the model. Based on the target data as calibration data, the model to be quantized can be quantized to obtain a quantized model. Compared with selecting a calibration set in the training set to quantize the model, it is possible to generate target data for the quantization model in the absence of data, and ensure the validity of the target data for quantization, which is conducive to improving the quantization effect, thereby improving the model performance.
[0093] And by calculating the relaxed distribution coefficient to calculate the model loss, the statistical constraints on each batch normalization layer can be relaxed, which is beneficial to improve the generalization ability of the model to be quantized and accelerate the training efficiency of the model, and can more efficiently obtain the target data for iteration.
[0094] In addition, it can be understood that compared with all samples of the target data, which are iteratively optimized through the same loss function, that is, directly accumulating the loss of each layer of the network to optimize all samples, the characteristic distribution statistics of the samples will tend to be centralized, and there will be a homogenization phenomenon. By using the diversification coefficient, each iteration of the simulated data has a specific loss, which will make the target data completed by the iteration closer to the scattered real data, thereby solving the homogenization problem of the simulated data at two levels, which is conducive to improving the performance of the quantitative model obtained after quantizing the model based on the target data.
[0095] See also Figure 3In some embodiments, step 015: iterating the simulation data based on the total loss of the batch normalization layer until determining the target data that makes the total loss of the batch normalization layer meet the preset conditions, including:
[0096] Step 0151: Iterate the model to be quantized and the simulated data based on the total loss of the batch normalization layer until the target data that makes the total loss of the batch normalization layer meet the preset conditions is determined. The target data is used to quantize the model to be quantized after the iteration to generate a quantized model.
[0097] Specifically, when the simulated data is updated and iterated based on the total loss of the batch normalization layer, the model parameters of the model to be quantized can also be updated and iterated, so that the gradient descent is faster, the model convergence is accelerated, and the efficiency of generating target data is improved.
[0098] For example, model parameters (such as a preset mean, a preset variance, etc.) of each batch normalization layer may be iterated based on the total loss of the batch normalization layer.
[0099] For another example, the model to be quantized further includes a convolution layer and an activation function layer, and at least one of the batch normalization layer, the convolution layer, and the activation layer may be iterated based on the total loss of the batch normalization layer.
[0100] Among them, the convolution layer is used to perform convolution operations. The activation function layer is a key component in the neural network. It is located at the output end of the neuron. Its main purpose is to introduce nonlinearity so that the neural network can learn and express complex nonlinear relationships. Without the activation function, the multi-layer neural network will degenerate into a linear model, and it will not be able to learn nonlinear patterns even with more hidden layers. The activation function layer transforms the output value of the neuron, increases the network's expressive power, and enables the network to fit a wider range of data distributions.
[0101] According to the method described in the above embodiment, the present application embodiment also provides a quantization device 300, which is used to perform the steps in the above quantization method. Figure 4 , Figure 4 3 is a schematic diagram of the structure of a quantization device 300 provided in an embodiment of the present application. The quantization device 300 includes a generation module 301, a first processing module 302, a second processing module 303, a loss determination module 304 and an iteration module 305, wherein:
[0102] In a second aspect, the present application provides a quantization device, the quantization device comprising:
[0103] A generating module 301, used for generating initialization data and simulation data;
[0104] A first processing module 302 is used to process the initialization data through the model to be quantized to determine the relaxation distribution coefficient of each batch normalization layer, the model to be quantized includes multiple batch normalization layers, and the relaxation distribution coefficient is determined based on the first mean and the first variance of the output data of each batch normalization layer;
[0105] A second processing module 303, used to process the simulated data respectively through the model to be quantized to determine the second mean and the second variance of the output data of each batch normalization layer;
[0106] A loss determination module 304 is used to determine a total loss of a batch normalization layer, where the total loss of the batch normalization layer is based on a relaxation distribution coefficient and a second mean and a second variance corresponding to each batch normalization layer;
[0107] The iteration module 305 is used to iterate the simulation data based on the total loss of the batch normalization layer until the target data that makes the total loss of the batch normalization layer meet the preset conditions is determined, and the target data is used to quantize the model to be quantized to generate a quantized model.
[0108] It should be noted that the specific details of each module unit in the above-mentioned quantization device 300 have been described in detail in the embodiment of the above-mentioned quantization method, and will not be repeated here.
[0109] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.
[0110] In some embodiments, the quantization device in the embodiments of the present application can be implemented in hardware, such as a chip, or a component in a chip, such as an integrated circuit or a chip; the quantization device can also be implemented in software, such as as an application installed in a chip.
[0111] In some embodiments, the chip of the embodiment of the present application can execute each process of the embodiment of the above-mentioned quantization method and can achieve the same technical effect. To avoid repetition, it will not be described here.
[0112] Among them, chips, also known as integrated circuits (ICs), microcircuits or microchips, are the core components of modern electronic devices. They are made of semiconductor materials (usually silicon) and integrate millions or even billions of microelectronic components (such as transistors, resistors, capacitors, etc.) through a series of complex manufacturing processes to form an electronic circuit with specific functions. These components and their interconnections are manufactured on the same small silicon wafer. They are small in size, light in weight, but powerful in function. They are the basis of various electronic products such as computers, smart phones, home appliances, automobiles, medical equipment, etc.
[0113] The embodiment of the present application also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the embodiment of the above-mentioned quantization method are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0114] The processor may be the chip in the above embodiment. The computer readable storage medium may be a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.
[0115] Computer readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other solid-state storage technologies, CD-ROM, Digital Versatile Disc (DVD) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above. The above-mentioned system memory 504 and mass storage device 507 can be collectively referred to as memory.
[0116] The embodiment of the present application also provides a computer program product, including a computer program, which implements the above-mentioned quantization method when executed by a processor. The processor may be a chip in the above-mentioned embodiment. When the computer program is executed by the processor, each process of the embodiment of the above-mentioned quantization method is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0117] It is understandable that in the specific implementation of this application, data related to user identity or characteristics is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.
[0118] Although the embodiments of the present application have been shown and described, those skilled in the art will appreciate that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present application, and that the scope of the present application is defined by the claims and their equivalents.
Claims
1. A quantization method based on a neural network, characterized in that: include: Generate initialization data and simulation data; Processing the initialization data through the model to be quantized to determine a relaxation distribution coefficient of each batch normalization layer, wherein the model to be quantized includes multiple batch normalization layers, and the relaxation distribution coefficient is determined based on a first mean and a first variance of output data of each batch normalization layer; Processing the simulated data respectively by the model to be quantized to determine a second mean and a second variance of the output data of each batch normalization layer; Determine a total loss of the batch normalization layer, where the total loss of the batch normalization layer is determined based on the relaxation distribution coefficient and the second mean and the second variance corresponding to each of the batch normalization layers; The simulation data is iterated based on the total loss of the batch normalization layer until target data is determined so that the total loss of the batch normalization layer satisfies a preset condition, and the target data is used to quantize the model to be quantized to generate a quantized model.
2. The quantization method according to claim 1, characterized in that: The generating of initialization data and simulation data comprises: Based on Gaussian distribution, sampling is performed within a preset range to generate the initialization data and simulation data.
3. The quantization method according to claim 1, characterized in that: The initialization data and the simulation data have the same size and match the input of the model to be quantized.
4. The quantization method according to claim 1, characterized in that: The relaxed distribution coefficient includes an offset coefficient and a scaling coefficient, the offset coefficient of the target batch normalization layer is determined based on a first mean of output data of the target batch normalization layer for the initialization data, the scaling coefficient of the target batch normalization layer is determined based on the first variance of output data of the target batch normalization layer for the initialization data, and the target batch normalization layer is any of the batch normalization layers.
5. The quantization method according to claim 4, characterized in that: The offset coefficient is determined based on the following formula: Among them, δ i is the offset coefficient of the i-th batch normalization layer, is the first mean corresponding to the i-th batch normalization layer, μ i is the preset mean of the i-th batch normalization layer, and ∈ represents the quantile.
6. The quantization method according to claim 4 or 5, characterized in that: The scaling factor is determined based on the following formula: Among them, γ i is the scaling factor of the i-th batch normalization layer, is the first variance corresponding to the i-th batch normalization layer, σ i is the preset variance of the i-th batch normalization layer, and ∈ represents the quantile.
7. The quantification method according to any one of claims 4 to 6, characterized in that: The total loss of the batch normalization layer is determined based on the losses of the individual batch normalization layers.
8. The quantification method according to any one of claims 4 to 6, characterized in that: The total loss of the batch normalization layer is determined based on the losses of each batch normalization layer and a preset diversity coefficient.
9. The quantization method according to claim 8, characterized in that: The total loss of the batch normalization layer is determined based on the following formula, Among them, L DSG is the total loss of the batch normalization layer, X LSE is the preset diversification coefficient, L SDA is the sum of the losses of each batch normalization layer.
10. The quantization method according to claim 9, characterized in that: The diversification coefficient is determined based on the following formula: LSE =I+11 T ; Where I is the n-dimensional identity matrix, 11 T is an n-dimensional matrix of all ones.
11. The quantification method according to any one of claims 7 to 10, characterized in that: The loss of the i-th batch normalization layer is determined based on the following formula: Among them, L SDAi is the loss of the i-th batch normalization layer, is the second mean corresponding to the i-th batch normalization layer, μ i is the preset mean of the i-th batch normalization layer, δ i is the offset coefficient of the i-th batch normalization layer, is the second variance corresponding to the i-th batch normalization layer, σ i is the preset variance of the i-th batch normalization layer, γ i is the scaling factor of the i-th batch normalization layer.
12. The quantization method according to claim 1, characterized in that: The iterating the simulation data based on the total loss of the batch normalization layer until determining the target data that makes the total loss of the batch normalization layer meet the preset condition includes: The model to be quantized and the simulation data are iterated based on the total loss of the batch normalization layer until target data is determined so that the total loss of the batch normalization layer meets a preset condition, and the target data is used to quantize the iterated model to be quantized to generate the quantized model.
13. A quantization device, characterized in that: include: A generation module, used to generate initialization data and simulation data; A first processing module, configured to process the initialization data through a model to be quantized to determine a relaxation distribution coefficient of each batch normalization layer, wherein the model to be quantized includes a plurality of batch normalization layers, and the relaxation distribution coefficient is determined based on a first mean and a first variance of output data of each batch normalization layer; A second processing module, used for processing the simulated data respectively through the model to be quantized to determine a second mean and a second variance of the output data of each batch normalization layer; a loss determination module, configured to determine a total loss of the batch normalization layer, wherein the total loss of the batch normalization layer is determined based on the relaxation distribution coefficient, and the second mean and the second variance corresponding to each of the batch normalization layers; An iteration module is used to iterate the simulation data based on the total loss of the batch normalization layer until the target data that makes the total loss of the batch normalization layer meet the preset conditions is determined, and the target data is used to quantize the model to be quantized to generate a quantized model.
14. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the quantization method according to any one of claims 1 to 12 is implemented.
15. A chip, characterized in that: The chip is used to implement the quantification method according to any one of claims 1 to 12.
16. A computer program product, characterized in that The invention comprises a computer program, which implements the quantization method according to any one of claims 1 to 12 when being executed by a processor.