Quantization method and system for generating standard image data and improving precision of neural network model by using standard image data, and application
By scaling, standardizing and interpolation processing of image data, standard image data is generated, and combined with quantization coefficient calculation, the problem of inference performance degradation caused by image preprocessing on FPGA is solved, and the accuracy and efficiency of neural network models are improved.
Patent Information
- Application Number
- CN202411228468.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-03
AI Technical Summary
The prior art has the problem of degradation of inference performance when preprocessing images on FPGAs, especially for higher resolution images, image preprocessing is required to meet input size requirements, but this processing method leads to degradation of inference performance of quantized models. , cannot effectively adapt to the memory and input size limits of FPGA.
By scaling, standardizing and interpolation processing of the input image data, standard image data is generated, and quantization coefficients are calculated using the calibration image, batch standardization layers and convolutional layers in the model are combined, and quantization coefficients are calculated according to the quantization formula to ensure that the model is quantized after the quantization is achieved. The output is consistent with the original floating point number distribution.
Improve the inference performance of neural network models on FPGA, reduce quantization error and accuracy loss, achieve higher accuracy and efficiency, and the detection box of inference results is more accurate, and the average cross-over ratio is increased by about 10 points, while accelerating FPGA inference calculation.
Smart Images

Figure BDA0005025378290000021 
Figure BDA0005025378290000076 
Figure BDA0005025378290000081
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence technology and relates to a quantization method, system and application for generating standard image data and improving the accuracy of a neural network model using the standard image data. Background Art
[0002] The existing symmetrical range-based linear quantization coding technology cannot be well adapted to FPGA. In addition to the need to recalculate the quantization coefficients for each input image data online and the calculated quantization coefficients are floating point numbers, the different preprocessing methods for image data will also affect the performance of the final model inference. For example, the Chinese invention patent application "A quantization method, system and process for optimizing int8" (patent application number: CN202010863091.7). The method mainly provides a method and system for optimizing the quantization of int8, using the floating-point model saved after obtaining the training neural network; calculating the weight quantization scaling factor in each channel in each layer of the neural network; using the KL divergence algorithm to calculate the activation value quantization scaling factor in each layer of the neural network; determining the optimal weight quantization scaling factor and the optimal activation value quantization scaling factor in each layer of the neural network according to the cosine distance; based on the optimal weight quantization scaling factor and the optimal activation value quantization scaling factor, obtaining the integer result of int8. According to the method of the present invention, according to the optimal weight quantization scaling factor and the optimal activation value quantization scaling factor, automatic fine-tuning is performed to obtain the integer result of int8, which can avoid the influence of extreme values; at the same time, the activation value quantization scaling factor is calculated by KL divergence, and the cosine distance is used to automatically perform fine-tuning, which can reduce the quantization error and avoid the influence of extreme values. Chinese invention patent "A method for improving the accuracy of low-bit neural network model by weight preprocessing" (patent application number: CN202010497769.4). The invention provides a method for improving the accuracy of a low-bit neural network model by weight preprocessing. First, the full-precision data of the neural network is obtained, and then the full-precision weights are preprocessed, that is, the weights are preprocessed before quantizing the weights; the data to be quantized is quantized to obtain low-bit data; this method takes the bit width limitation into account before training the low-bit model, so when training the full-precision model, it will be considered that the values represented by the low bits are limited. In order to make full use of each value, the weights will be preprocessed so that each value can be fully utilized, thereby improving the accuracy of the model.
[0003] FPGA requires the input size of the image to be a power of 2, and the quantization coefficient must also be an integer of a power of 2. At the same time, due to the limitation of the memory size of FPGA, the image input size cannot be too large. Therefore, for higher-resolution images, image preprocessing is required to meet the input size requirements of FPGA. Before sending it to the network model for training, in order to reduce the size of the original image, the general image preprocessing process is as follows: first, data normalization is performed, and then the image size is scaled proportionally. In order to ensure that the input size meets the square size requirements, the missing parts are filled with gray borders. However, this image preprocessing method will cause the quantized model to have a significant decrease in inference performance in FPGA, resulting in unusable model inference results. Summary of the invention
[0004] In order to solve the shortcomings of the prior art, the purpose of the present invention is to provide an image preprocessing method for improving image reasoning effect, which can generate standard image data and improve the accuracy of low-bit neural network models through image preprocessing. It is an image processing normalization technology used in FPGA reasoning.
[0005] The present invention provides a method for generating standard image data, wherein the method scales and standardizes input image data, and then obtains the standard image data after interpolation processing, comprising the following steps:
[0006] Step 1: Input the image and scale the image data to the interval [0, 1];
[0007] Step 2: Standardize and process the scaled data obtained in step 1 to obtain standardized data y t ;
[0008]
[0009] Wherein, y is the value after the image data is standardized, ∈ is the standard normal distribution, t is the uniform distribution in the interval [0,1], and π(t) is a monotonically increasing function from 0 to 1; in the specific implementation process, the monotonically increasing function can be a linear function, a logarithmic function, a Sigmoid function, or other high-order functions, etc., and different monotonically increasing functions can be selected according to actual needs;
[0010] Step 3: interpolate the standardized data obtained in step 2 to obtain image data z of the target size.
[0011] In step 1, the pixel values in the input image are divided by 256 one by one to scale the image data;
[0012] and / or,
[0013] In step 2, the scaled data obtained in step 1 is obtained by Standardized to a distribution with a mean of 0 and a variance of 1; where x is the pixel value of the input image, x mean is the average value of the input image pixels, x σ is the variance of the input image pixels;
[0014] and / or,
[0015] In step three, the interpolation processing includes at least one of bilinear interpolation, trilinear interpolation, and nearest neighbor interpolation, so that the image size matches the model input.
[0016] The present invention also proposes a quantization method for improving the accuracy of a neural network model using standard image data, the method comprising the following steps:
[0017] Step I. Acquire standard image data according to the above method for generating standard image data;
[0018] Step II. Inputting the standard image data into a deep learning model for training to obtain an inference network model;
[0019] Step III. Convert the inference network model into an ONNX model, and merge the batch normalization (BN) layer and the convolution layer in the ONNX model;
[0020] Step IV. Use the calibration image to perform KL divergence to calculate the quantization coefficient and obtain the threshold T of the activation function;
[0021] Step V. Calculate the quantization coefficient based on the quantization formula using the weight, bias, variance, and mean information of each network layer, and verify the accuracy of the quantization coefficient.
[0022] The standard image data is adapted to the input size of the deep learning model; and / or,
[0023] In step II, the deep learning model is a neural network model suitable for image processing (such as ResNet, etc.), which is used for reasoning on FPGA; group convolution or separable convolution is used instead of the original convolution operator during training; after the training is completed, cross-validation and testing steps of the model are also included; and / or,
[0024] In step III, the inference network model is exported from its original framework to an ONNX format file and adjusted for compatibility; the trained parameters of the inference network model are extracted from the BN layer, and the weights and biases of the convolutional layer are recalculated so that the convolution operation can directly include the effect of BN, and the model originally including the BN layer and the convolutional layer is replaced with a model including only the merged convolutional layer; and / or,
[0025] In step IV, the calibration image is used to generate an output distribution through the network layer, the output distribution is quantized, the KL divergence between the quantized distribution and the original floating point number distribution is calculated so that the KL divergence reaches a minimum value, and the threshold T of the activation function is determined; and / or,
[0026] In step V, according to the quantization formula x f is the input data, n is the bit width of the quantized value, To quantize the coefficients, the weight information, bias information, variance information, and mean information of each network layer are used to obtain the quantized coefficients.
[0027] When exporting an inference network model from its original framework to an ONNX format file, convert and save the model's architecture and weight information; adapt the model structure to the ONNX format requirements through compatibility adjustments; and / or,
[0028] Perform network parsing on the ONNX model, read the input layer and output layer of the network in sequence and mark them in sequence, to ensure that subsequent processing of the model (such as quantization, optimization, etc.) can be performed based on a clear hierarchical structure without hierarchical confusion or processing errors.
[0029] Step V may also include a quantization self-check step: compare the quantized output of each network layer with the output of the previous layer to ensure the consistency of the output and the accuracy of the quantization coefficient; if the output is found to be inconsistent, automatically adjust the quantization coefficient until the output meets the consistency requirements, ensuring that the inference accuracy of the quantized model is close to that of the unquantized model.
[0030] The present invention also provides a system for implementing the above method, the system comprising: a data preprocessing module, a model training module, a model conversion module, a network analysis module, a calibration and quantization module, and a model verification module;
[0031] Data preprocessing module: This module is responsible for scaling, standardizing, and interpolating the raw image data, converting the image data into a distribution with a mean of 0 and a standard deviation of 1, and adjusting it to the target size. The output of this module is the standardized and adjusted image data, ready to enter the model training.
[0032] Model training module: Input the preprocessed data into the deep learning model for training. This module may include the configuration and training of convolutional neural networks, the optimization of model structure (such as using grouped convolution or separable convolution), and the generation of a preliminary inference network model.
[0033] Model conversion module: This module converts the trained model into ONNX format for cross-platform deployment. During the conversion process, the module also merges the batch normalization (BN) layer with the convolution layer to simplify the model structure and improve the inference efficiency.
[0034] Network parsing module: responsible for reading and analyzing the structure of the ONNX model, extracting the input-output relationship and weight parameters of the network, marking the connection order between layers, and preparing for the subsequent optimization and quantization process.
[0035] Calibration and quantization module: Use the calibration image set to calculate the KL divergence and minimize the error to determine the optimal threshold of the activation function. Then, the weights, biases, variances, mean and other parameters of each layer are quantized according to the quantization formula to generate a low-bit width model suitable for FPGA inference.
[0036] Model Verification Module: Finally, this module verifies the quantized model to ensure that its inference output is consistent with the unquantized model and that the model runs efficiently on the hardware.
[0037] The present invention also provides the above-mentioned method for generating standard image data, the above-mentioned quantization method for improving the accuracy of neural network models, or the application of the above-mentioned system in generating standardized images of matching models, improving model reasoning effects, accelerating model training, reducing model quantization losses, etc.
[0038] The present invention also provides a hardware system for implementing the above method, the hardware system comprising: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the above-mentioned quantization method of generating standard image data and using the standard image data to improve the accuracy of the neural network model is implemented.
[0039] The present invention also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the quantization method for generating standard image data and improving the accuracy of a neural network model using the standard image data is implemented.
[0040] The beneficial effects of the present invention include: the existing symmetrical range-based linear quantization coding technology such as distiller needs to recalculate the quantization coefficient online for each input image data, and its quantization coefficient is a floating point number. The former will lead to a relatively large loss of precision, and the latter will lead to an increase in the demand for storage and computing resources of the FPGA. By performing targeted normalization processing on the original image data and then combining multiple images to calculate its quantization coefficient offline, the detection performance of the quantized model is compared with the detection performance of directly quantizing without performing these processing. The average intersection and union ratio on the test set is about 10 points different. At the same time, the floating point number is replaced by the nearest power of 2 integer, which greatly accelerates the reasoning calculation of the FPGA. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.
[0042] Figure 1 It is a flow chart of a method for improving the accuracy of a neural network model by using standard image data in the present invention.
[0043] Figure 2 It is a schematic diagram of the detection results in the prior art (Scheme A).
[0044] Figure 3 It is a schematic diagram of the detection results after the standard image data is generated according to the present invention (Scheme B). DETAILED DESCRIPTION
[0045] The present invention is further described in detail with reference to the following specific examples and drawings. The process, conditions, experimental methods, etc. for implementing the present invention, except for the contents specifically mentioned below, are all common knowledge and common common sense in the art and are not particularly limited by the present invention.
[0046] In view of the problems in the prior art, the present invention first standardizes the original data, transforms the data into a distribution with a mean of 0 and a standard deviation of 1, and then standardizes the data and performs interpolation methods such as bilinear interpolation to a specified size, and then sends it to a deep learning network for training. The trained model is then converted into an onnx model, and then the onnx model is read and the Conv layer and BN layer in the network are merged. The network connection order and parameter information such as weights of the onnx model are parsed, and then the calibration image is used to calculate the KL divergence to obtain the activation function threshold. Finally, the quantization coefficient is calculated according to the quantization formula from the weights, biases and other information of each layer of the network.
[0047] Specifically,
[0048] The present invention provides a new image data preprocessing process. Inappropriate image preprocessing for deep learning models will lead to precision loss in the model quantization process, resulting in a significant decrease in the model's recall rate. A new image data preprocessing technology is proposed to reduce the precision loss of model quantization caused by adapting to the needs of end-side model reasoning, so that the recall rate during model reasoning remains stable and does not decrease. The fact that the recall rate after model quantization remains basically unchanged is the key to using the model for downstream tasks to achieve practical application results. For example, in the autonomous driving perception module, if the detection box of the obstacle detection model is too large or too small, it will cause a large error in the estimation of the actual boundary area of the obstacle, resulting in an error in the division of the boundary of the drivable area, which will in turn affect the error output of the final control and decision-making, and increase the risk of collision.
[0049] In a specific embodiment, the entire method flow of the present invention is as follows: first, the original image data is standardized, the data is transformed into a data distribution with a mean of 0 and a standard deviation of 1, and then the data is standardized and bilinearly interpolated to the required size, and then sent to the deep learning network for model training, the trained model is converted into an onnx model, the onnx model is read and the BN network layer in the model is merged with the Conv network layer of the previous layer, the input layer and output layer of the network are read in sequence and labeled in sequence, the KL divergence is calculated using the calibration image, the threshold of the activation function is obtained by minimizing the KL divergence, and then the weight information, bias information, variance information, mean information, etc. of each network layer are used according to the quantization formula to obtain the quantization coefficient. Finally, it is verified whether the quantization coefficient meets the requirement that the output of the previous layer is equal to the input of the current layer.
[0050] The difference between this quantization technique and the existing techniques includes the preprocessing process of image data, which is as follows:
[0051] The image data preprocessing process first scales the input image data to the interval [0,1] to improve the convergence speed of the model, and then Standardized to a distribution with a mean of 0 and a variance of 1, where x is the pixel value of the input image, x mean is the average value of the input image pixels, x σ is the variance of the input image pixels, Where ∈ is a standard normal distribution, t is a uniform distribution in the interval [0,1], π(t) is a monotonically increasing function from 0 to 1, and y t Perform bilinear interpolation to obtain image data z of the target size.
[0052] After the standard image data in the method of the present invention is generated, the model training method is performed according to the existing relevant technology, the image data z is sent to the deep learning model for training, the network model structure is modified, and the group convolution and / or separable convolution are used instead of the original convolution operator for model training.
[0053] In a specific implementation, taking the yolov5 model as an example, the convolution operators therein are replaced by grouped convolution and / or separable convolution. Note that only the convolution operators are replaced with convolution operators with a channel number ≥ 256, and the 7*7 pooling layer operators are replaced by multiple 3*3 pooling operators. After the replacement is completed, the model is trained.
[0054] like Figure 1 As shown in the flowchart of the quantization method of the present invention for improving the accuracy of the neural network model using standard image data, the specific steps of implementation are:
[0055] Step 1: Input the training image, divide the image data value by 256, and scale it to the interval [0,1]. The purpose of data normalization is to accelerate the training convergence of the model, and it is also convenient for FPGA to use shift instead of division. However, normalization will affect the original distribution form of the data. Therefore, the present invention performs step 2 on the basis of step 1 to obtain the standardized data y t The operation accelerates the convergence of model training while trying not to destroy the distribution of the original data. Step 1 belongs to the known technology;
[0056] Step 2: The data obtained in step 1 is converted into Standardized to a distribution with a mean of 0 and a variance of 1, where x is the pixel value of the input image, x mean is the average value of the input image pixels, x σ is the variance of the input image pixels. According to the formula Get standardized data y t , step 2 is a non-publicly known technology;
[0057] Among them, y is the standardized value of the image data, ∈ is the standard normal distribution, t is the uniform distribution in the interval [0,1], and π(t) is a monotonically increasing function from 0 to 1;
[0058] Step 3: Substitute the y obtained in step 2 t Perform interpolation methods such as bilinear interpolation and / or trilinear interpolation and / or nearest neighbor interpolation to obtain image data z of target size. Step 3 belongs to the known technology;
[0059] Step 4, sending the preprocessed data into the deep learning model for model training to obtain the required inference network model, step 4 belongs to the known technology; the overall operation of inputting data into the image training model generated by steps 1 to 3 belongs to the unknown technology; sending the data generated by steps 1 to 3 into the deep learning model for training also belongs to the unknown technology;
[0060] Step 5, converting the model obtained in step 4 into an onnx model, and step 5 belongs to a known technology;
[0061] Step 6, read the onnx model of step 5, and merge the BN network layer in the onnx model with the Conv network layer of the previous layer of the network layer. Step 6 belongs to a known technology;
[0062] Step 7, performing network parsing on the merged model, reading the input layer and output layer of the network in sequence and marking them in sequence, step 7 belongs to the known technology;
[0063] Step 8: Use the calibration image to perform KL divergence to calculate the quantization coefficient, and obtain the threshold T of the activation function by minimizing the KL divergence. For example, for the output y of a certain layer of the network, its q layer The calculation formula of the quantization coefficient is: n is the bit width of the quantized value, and the quantization coefficient of the weight Biased quantization coefficient Where T is obtained by minimizing KL divergence. Step 8 belongs to the known technology;
[0064] Step 9: According to the quantitative formula Calculate the quantization results of each layer of the network to obtain the quantized model output results, x f For input data, is the quantization coefficient. Step 9 belongs to the known technology;
[0065] The key point of the present invention is the preprocessing process of image data. There are two points to be protected. The first point is: the first point is to obtain the preprocessed data y t The second point is the entire quantization model process from image preprocessing data to model training quantization.
[0066] Example 1
[0067] This embodiment mainly needs to detect the car in the original image. The existing image preprocessing technology is used to scale the original image (1920×1080) to 640*360, and then fill the black border to a size of 640*512 as the input of the quantization model, which is recorded as solution A. The results are as follows Figure 2 As shown in Figure 2, and using the image preprocessing technology proposed in this paper as input, here recorded as solution B, the results are as follows Figure 3 As shown in the figure, after the same model is quantized, the model reasoning in the FPGA obtains the following detection results respectively. It can be seen that the preprocessing technology proposed in the present invention makes the detection frame of the reasoning result more accurate. mAP is the mean average precision. In the method of the present invention, this value is significantly higher than the image processed by the existing image preprocessing technology method.
[0068]
[0069]
[0070] Example 2
[0071] This embodiment mainly detects multiple targets in the original image, including cars, buses, pedestrians, tricycles, bicycles, etc. Compared with the existing direct quantization scheme, the scheme of the present invention has obvious advantages in terms of average accuracy mean and average time consumption data.
[0072] Direct quantification The quantification method of the present invention quantifies category floating point(%) Fixed point (%) Fixed point (%) Car 81.0 80.1 80.6 Public transportation 83.2 82 90 pedestrian 84.1 82.1 83.3 Three-wheeler 72.3 69.1 70.5 bike 45.5 44.4 45.0 mAP value 53.20 51.7 53.9 Average time comparison 300ms 30ms 21ms
[0073] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of complete hardware embodiments, complete software embodiments, or embodiments in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The scheme in the embodiments of the present application can be implemented in various computer languages, for example, object-oriented programming language Java and literal scripting language JavaScript, etc.
[0074] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0075] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0076] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0077] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.
[0078] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.
[0079] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the attached claims.
Claims
1. A method for generating standard image data, characterized in that: The method scales and normalizes the input image data, and then obtains standard image data after interpolation processing, including the following steps: Step 1: Input the image and scale the image data to the interval [0, 1]; Step 2: Standardize and process the scaled data obtained in step 1 to obtain standardized data y t ; Among them, y is the standardized value of the image data, ∈ is the standard normal distribution, t is the uniform distribution in the interval [0,1], and π(t) is a monotonically increasing function from 0 to 1; Step 3: interpolate the standardized data obtained in step 2 to obtain image data z of the target size.
2. The method according to claim 1, characterized in that In step 1, the pixel values in the input image are divided by 256 one by one to scale the image data; and / or, In step 2, the scaled data obtained in step 1 is obtained by Standardized to a distribution with a mean of 0 and a variance of 1; where x is the pixel value of the input image, x mean is the average value of the input image pixels, x σ is the variance of the input image pixels; and / or, In step three, the interpolation processing includes at least one of bilinear interpolation, trilinear interpolation, and nearest neighbor interpolation, so that the image size matches the model input.
3. A quantitative method for improving the accuracy of a neural network model using standard image data, characterized in that: The method comprises the following steps: Step I. Acquire standard image data according to the method according to any one of claims 1-2; Step II. Inputting the standard image data into a deep learning model for training to obtain an inference network model; Step III. Convert the inference network model into an ONNX model, and merge the batch normalization layer and the convolution layer in the ONNX model; Step IV. Use the calibration image to perform KL divergence to calculate the quantization coefficient and obtain the threshold T of the activation function; Step V. Calculate the quantization coefficient based on the quantization formula using the weight, bias, variance, and mean information of each network layer, and verify the accuracy of the quantization coefficient.
4. The quantization method according to claim 3, characterized in that: The standard image data is adapted to the input size of the deep learning model; and / or, In step II, the deep learning model is a neural network model suitable for image processing, which is used for reasoning on FPGA; group convolution or separable convolution is used instead of the original convolution operator during training; after the training is completed, cross-validation and testing steps of the model are also included; and / or, In step III, the inference network model is exported from its original framework to an ONNX format file and adjusted for compatibility; the trained parameters of the inference network model are extracted from the batch normalization layer, and the weights and biases of the convolutional layer are recalculated so that the convolution operation directly includes the effect of batch normalization, and the model originally including the batch normalization layer and the convolutional layer is replaced with a model including only the merged convolutional layer; and / or, In step IV, the calibration image is used to generate an output distribution through the network layer, the output distribution is quantized, the KL divergence between the quantized distribution and the original floating point number distribution is calculated so that the KL divergence reaches a minimum value, and the threshold T of the activation function is determined; and / or, In step V, according to the quantization formula x f For input data, the quantization coefficients are obtained using the weight information, bias information, variance information, and mean information of each network layer.
5. The quantization method according to claim 4, characterized in that: When exporting an inference network model from its original framework to an ONNX format file, convert and save the model's architecture and weight information; adapt the model structure to the ONNX format requirements through compatibility adjustments; and / or, Perform network parsing on the ONNX model, read the input layer and output layer of the network in sequence and mark them in sequence.
6. The quantization method according to claim 3, characterized in that: Step V also includes a quantization self-checking step: comparing the quantized output of each layer of the network with the output of the previous layer to ensure the consistency of the output and the accuracy of the quantization coefficient; If the output is found to be inconsistent, the quantization coefficient is automatically adjusted until the output meets the consistency requirements, ensuring that the inference accuracy of the quantized model is close to that of the unquantized model.
7. A system for implementing the method according to any one of claims 1 to 2, or the quantization method according to any one of claims 3 to 6, characterized in that: The system includes: a data preprocessing module, a model training module, a model conversion module, a network analysis module, a calibration and quantification module, and a model verification module; The data preprocessing module is responsible for scaling, standardizing and linear interpolation processing of the original image data; The model training module inputs the preprocessed data into the deep learning model for training to generate a preliminary inference network model; The model conversion module converts the trained model into ONNX format and merges the batch normalization layer and the convolution layer; The network parsing module is responsible for reading and analyzing the structure of the ONNX model, extracting the input-output relationship and weight parameters of the network, and marking the connection order between layers; The calibration and quantization module calculates the KL divergence using the calibration image set, minimizes the error and determines the optimal threshold of the activation function; quantizes the weights, biases, variances and means of each layer according to the quantization formula to generate a low bit width model suitable for FPGA reasoning; The model verification module verifies the quantized model.
8. Application of the method for generating standard image data as described in claim 1 or 2, the quantization method as described in any one of claims 3 to 6, or the system as described in claim 7 in generating standardized images of matching models, improving model reasoning effects, accelerating model training, and reducing model quantization losses.
9. A hardware system for implementing the method according to any one of claims 1 to 6, characterized in that: The hardware system comprises: a memory and a processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the method according to any one of claims 1 to 6 is implemented.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
Citation Information
Patent Citations
Radar target HRRP robust identification method in noise environment
CN110824450A
Neural network model establishing method for image key point detection
CN111127495A
Real-time garbage monitoring method and system
CN113592002A
Threshold sensing low-order quantization method based on medical hyperspectral detection network
CN115700806A
Convolutional neural network hybrid computing post-training quantization algorithm for embedded system
CN116341639A
