Model optimization method, image processing method, system, device, and storage medium

Abstract

The application provides a model optimization method, an image processing method, a system, a device and a storage medium. The method comprises the following steps: quantizing a pre-quantization image processing model to obtain a first post-quantization image processing model; inputting an optimization image into the pre-quantization image processing model and the first post-quantization image processing model respectively to obtain first image features and second image features; calculating the quantization accuracy of each level; determining a level to be optimized in the pre-quantization image processing model; performing amplification processing on the parameters of the level to be optimized to obtain a standby post-reconstruction image processing model; adding an output feature adjustment layer at the output position of the level to be optimized in the standby post-reconstruction image processing model to obtain a post-reconstruction image processing model; and quantizing the post-reconstruction image processing model to obtain a second post-quantization image processing model. Through the application, the efficiency of the model in processing images can be improved through quantization, and the image processing accuracy of the post-quantization model is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and in particular to a model optimization method, an image processing method, a system, a device and a storage medium. BACKGROUND

[0002] With the continuous development of deep learning, artificial intelligence technology plays an increasingly important role in various fields, and the demand for related products is increasing. In the field of image processing, artificial intelligence technology has also been widely applied. For example, in the monitoring technology, a face recognition model based on deep learning can be used to analyze and process the images captured by the monitoring device, detect the face position and face attributes therein, and a license plate recognition model based on deep learning can also be used to analyze and process the vehicle images captured by the monitoring device, detect the license plate number, etc. A series of image processing models such as face recognition model, license plate recognition model, target detection model, etc. are generally deployed on chips for running when applied. Through deep learning, we can obtain high-precision image processing models for processing various tasks, but the huge amount of calculation makes it difficult to be widely promoted in practical scenarios due to the limitations of the performance, power consumption, etc. of the end-side intelligent processor. When a high-precision image recognition model is deployed on a chip, the requirement for the chip is very high, and it may run slowly due to insufficient processing capacity of the chip. In order to solve the limitations of end-side intelligent application, improve the running speed of the chip and device deployed with image processing model, and improve the efficiency of image analysis and processing, image processing model compression has become one of the hotspots in the industry in recent years, and image processing model quantization is a very practical model compression technology.

[0003] Image processing model quantization is actually a mapping relationship between the internal floating point data and fixed point data of an image processing model. The operation process of converting the weights, activation values, etc. of the trained deep neural network from high precision to low precision makes it possible to obtain greater benefits at the cost of smaller precision loss, for example, converting 32-bit floating point numbers (FLOAT) to 8-bit integers (INT8), while expecting the accuracy of the converted model to be similar to that before conversion. Compared with the original floating point model, the quantized model occupies less memory and storage, consumes less power, and runs faster. The basic principle of image processing model quantization is as follows: first, find the zero point, i.e. the value corresponding to 0 in the original floating point domain after quantization, and then find an approximate mapping relationship to map the original floating point value to an integer value according to a certain rule. It is worth noting that before the above operation, the range of the quantization object needs to be set, because after quantization to 8-bit integers, the number of values that can be used to represent the quantized values is fixed 2 8 32 ​The floating point number can cause large error in the case of extremely uneven distribution of model parameter distribution, and therefore appropriate clipping can remove maximum and minimum values to achieve high precision after quantization. In the currently common quantization method, the basic principle is to find a proper data truncation range and a suitable mapping relationship to achieve small precision loss. However, this method theoretically removes a part of useful information, especially when the data distribution is extremely uneven and the useful feature information is mostly concentrated near zero. The existing quantization method cannot avoid the quantization error caused by such distribution. When the quantized image processing model is used to analyze and process images, the precision loss is large, which can cause incomplete extraction of image features, reduce the precision of image feature extraction, and further affect the accuracy of image prediction results. After the quantized image processing model is deployed on a chip, the reliability of the analysis and processing results of real-time images is greatly reduced. SUMMARY

[0004] To solve the problems in the prior art, the purpose of the present application is to provide a model optimization method, an image processing method, a system, a device and a storage medium. The efficiency of image analysis and processing of the image processing model can be improved by quantization. The image processing precision of the quantized image processing model is improved by optimizing and reconstructing the image processing model before quantization. The image processing model has high image feature extraction precision on the basis of faster image feature extraction, and the optimization of the image processing model is realized.

[0005] The embodiment of the present application provides a model optimization method, which comprises the following steps:

[0006] quantizing the image processing model before quantization to obtain a first quantized image processing model;

[0007] inputting the optimization image into the image processing model before quantization and the first quantized image processing model respectively to obtain first image features output by each level of the image processing model before quantization and second image features output by each level of the first quantized image processing model respectively;

[0008] calculating the quantization precision of each level of the first quantized image processing model based on the first image features and the corresponding second image features;

[0009] determining the level to be optimized in the image processing model before quantization according to the quantization precision of each level of the first quantized image processing model and the weight parameter distribution of the image processing model before quantization;

[0010] amplifying the parameters of the level to be optimized in the image processing model before quantization to obtain a standby reconstructed image processing model;

[0011] adding an output feature adjustment layer at the output position of the level to be optimized in the backup reconstructed image processing model, the output feature adjustment layer being used to downsize the output feature of the corresponding level to be optimized to ensure that the adjusted output feature of the level is consistent with the output feature of the level in the pre-quantization image processing model, obtaining a reconstructed image processing model;

[0012] quantizing the reconstructed image processing model to obtain a second quantized image processing model.

[0013] The present application first quantizes the pre-quantization image processing model, and extracts features from each level of the pre-quantization image processing model and the first quantized image processing model using the optimization image, and calculates the quantization precision of each level according to the feature extraction results, thereby determining the levels to be optimized with lower quantization precision, amplifying the model parameters of the levels to be optimized, avoiding the truncation and elimination of small parameters in the levels to be optimized during quantization, which affects the image processing precision, adding an output feature adjustment layer at the output position, maintaining the consistency of the output features of the optimized levels and the output features of the pre-optimized levels, then quantizing the reconstructed image processing model to obtain a second quantized image processing model. Through quantization, the efficiency of the model processing image can be improved, the inference speed of the image processing model after deployment on the chip can be improved, and the power consumption of the chip can be reduced. Through the parameter amplification reconstruction of the pre-quantization image processing model, the precision loss caused by the quantization process of the image processing model is reduced, the image processing precision of the quantized image processing model is improved, the image processing model has higher image feature extraction precision on the basis of faster image feature extraction, the time loss caused by repeated model training due to model loss is avoided, and the optimization of the image processing model is realized.

[0014] In some embodiments, based on the first image features and the corresponding second image features, the quantization precision of each level of the first quantized image processing model is calculated, including the following steps:

[0015] The similarity of the first image features of each level of the pre-quantization image processing model and the second image features of the corresponding level of the first quantized image processing model is calculated as the quantization precision of the corresponding level of the first quantized image processing model.

[0016] In some embodiments, calculating the similarity of the first image features of each level of the pre-quantization image processing model and the second image features of the corresponding level of the first quantized image processing model includes calculating at least one of the dot product similarity, Euclidean similarity and cosine similarity of the first image features of each level of the pre-quantization image processing model and the second image features of the corresponding level of the first quantized image processing model.

[0017] In some embodiments, the layers to be optimized in the pre-quantization image processing model are determined according to the quantization precision of each layer of the first post-quantization image processing model and the weight parameter distribution of the pre-quantization image processing model, including the following steps:

[0018] The weight standard deviation of each layer of the pre-quantization image processing model is obtained based on the weight parameter distribution of the pre-quantization image processing model.

[0019] A mapping curve between the weight standard deviation of each layer of the pre-quantization image processing model and the quantization precision is established.

[0020] According to the set quantization precision threshold and the mapping curve, a weight standard deviation reference value corresponding to the quantization precision threshold is determined.

[0021] The layers of the pre-quantization image processing model with a weight standard deviation less than the weight standard deviation reference value are determined as the layers to be optimized.

[0022] In some embodiments, the establishment of the mapping curve between the weight standard deviation of each layer of the pre-quantization image processing model and the precision includes the following steps:

[0023] According to the weight standard deviation of each layer of the pre-quantization image processing model and the quantization precision of the corresponding layer of the first post-quantization image processing model, a mapping curve is fitted, and the first direction coordinate of the mapping curve is the weight standard deviation, and the second direction coordinate is the quantization precision.

[0024] In some embodiments, the parameters of the layers to be optimized in the pre-quantization processing model are amplified to obtain a standby reconstructed post-quantization image processing model, including the following steps:

[0025] The ratio of the weight standard deviation of the layers to be optimized in the pre-quantization processing model to the weight standard deviation reference value is calculated, and an amplification coefficient is calculated according to the ratio.

[0026] The parameters of the layers to be optimized in the pre-quantization processing model are multiplied by the amplification coefficient for amplification.

[0027] The corresponding parameters of the layers to be optimized in the pre-quantization processing model are updated using the amplified parameters to obtain a standby reconstructed post-quantization image processing model.

[0028] The embodiments of the present application also provide an image processing method, including the following steps:

[0029] Obtaining a to-be-processed image;

[0030] The to-be-processed image is input into the second post-quantization image processing model obtained by using the above-mentioned model optimization method.

[0031] obtaining an image processing result output by the second quantized image processing model.

[0032] In the image processing method, after obtaining the image to be processed, the image to be processed is input into the second quantized image processing model obtained by the model optimization method, and the image is processed based on the second quantized image processing model. After the second quantized image is processed, the processing efficiency is higher, and through the amplification processing of the smaller parameters in the lower precision level, the second quantized image processing model has higher precision in extracting features of the image and predicting results, thereby improving the efficiency and precision of image processing.

[0033] The application also provides an image processing system applied to the image processing method, and the system comprises:

[0034] an image acquisition module configured to acquire an image to be processed;

[0035] an image input module configured to input the image to be processed into a second quantized image processing model obtained by the model optimization method;

[0036] an image output module configured to obtain an image processing result output by the second quantized image processing model.

[0037] In the image processing system, after the image acquisition module acquires the image to be processed, the image input module inputs the image to be processed into the second quantized image processing model obtained by the model optimization method, the image is processed based on the second quantized image processing model, and the image output module obtains the image processing result. After the second quantized image is processed, the processing efficiency is higher, and through the amplification processing of the smaller parameters in the lower precision level, the second quantized image processing model has higher precision in extracting features of the image and predicting results, thereby improving the efficiency and precision of image processing.

[0038] The application also provides an image processing device comprising:

[0039] a processor;

[0040] a memory having executable instructions of the processor stored therein;

[0041] The processor is configured to execute the steps of the image processing method by executing the executable instructions.

[0042] By using the image processing device provided by the application, the processor executes the image processing method when executing the executable instructions, thereby obtaining the beneficial effects of the image processing method.

[0043] An embodiment of the present application also provides a computer-readable storage medium for storing a program, which implements the steps of the image processing method when executed by a processor.

[0044] By adopting the computer-readable storage medium provided in the present application, the program stored therein implements the steps of the image processing method when being executed, thereby achieving the beneficial effects of the above-mentioned image processing method. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Other features, objects and advantages of the present application will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings.

[0046] Figure 1 is a flow chart of a model optimization method according to an embodiment of the present application;

[0047] Figure 2 is a schematic diagram of an output feature adjustment layer according to an embodiment of the present application;

[0048] Figure 3 is a flowchart of an image processing method according to an embodiment of the present application;

[0049] Figure 4 is a schematic diagram of an image processing system according to an embodiment of the present application;

[0050] Figure 5 is a structural diagram of an image processing device according to an embodiment of the present application;

[0051] Figure 6 It is a structural diagram of a computer storage medium according to an embodiment of the present application. DETAILED DESCRIPTION

[0052] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this application will be comprehensive and complete and will fully convey the concepts of the example embodiments to those skilled in the art. In the figures, the same reference numerals represent the same or similar structures, and thus their repeated description will be omitted. Although "first" or "second" or the like is used in this specification to indicate certain features, this is only to indicate the function and is not intended to limit the number or importance of specific features.

[0053] like Figure 1 As shown, in one embodiment, the present application provides a model optimization method, comprising the following steps:

[0054] S110: quantizing the pre-quantization image processing model to obtain a first post-quantization image processing model;

[0055] The pre-quantization image processing model is a model to be optimized, which is a floating-point model, for example, where the parameters are 32-bit floating-point numbers (FLOAT). The pre-quantization processing model can be quantized using existing integer quantization methods to obtain 8-bit integers (INT8) or 16-bit integers (INT16). For example, first set the range of the quantization object, and eliminate parameters that are outside the range of the quantization object (including eliminating parameters greater than the maximum value of the range of the quantization object and eliminating parameters less than the minimum value of the range of the quantization object). For parameters that are not eliminated, map the floating-point values of the parameters in the pre-quantization processing model to integer values according to a certain rule through a pre-set mapping relationship. For some parameters that are less than the minimum value of the range of the quantization object, eliminating them will cause a large loss of precision, resulting in very low quantization accuracy for the corresponding level.

[0056] The image processing model is used to extract features and recognize images. For example, the image processing model can be a face recognition model that can recognize the position and / or attributes (gender, age, identity, etc.) of a face in an image, or the image processing model can be a target detection model that detects the position of a specified type of target in an image and / or classifies the target.

[0057] S120: input the pre-quantization image processing model and the first quantized image processing model to the optimization image respectively, to obtain first image features output by each level of the pre-quantization image processing model and second image features output by each level of the first quantized image processing model respectively.

[0058] The optimization image can be one or more, and is preferably an image corresponding to the purpose of the image processing model. For example, for a face recognition model, an image including a face is selected as the optimization image.

[0059] Taking the pre-quantization image processing model as a convolutional neural network as an example, each level includes a convolution layer (convolution) and a fully connected layer (inner product), each level extracts features from the input image of the level and outputs corresponding image features.

[0060] S130: calculate the quantization accuracy of each level of the first quantized image processing model based on the first image features and the corresponding second image features.

[0061] When an optimized image is selected, the quantization precision is calculated directly based on the first image feature and the second image feature, and the calculation of the quantization precision here is a comparison process based on the first image feature and the second image feature to determine the precision loss of the second image feature compared with the first image feature;

[0062] S140: According to the quantization precision of each level of the first quantized image processing model and the weight parameter distribution of the pre-quantization image processing model, determine the level to be optimized in the pre-quantization image processing model;

[0063] The level to be optimized determined here is the level with greater precision loss after quantization;

[0064] S150: Perform amplification processing on the parameters of the level to be optimized in the pre-quantization processing model to obtain a standby reconstructed image processing model;

[0065] Here, only the parameters of the level to be optimized are amplified, and the data type of the parameters is not changed, so the reconstructed image processing model obtained by processing is still a floating-point model;

[0066] Since the removal of some small parameters in some levels based on the range of the quantization object has a greater impact on the precision of these levels during the quantization process, the parameters are amplified here to avoid being removed during quantization, thereby avoiding the precision decline caused by removing these parameters;

[0067] S160: Add an output feature adjustment layer at the output position of the level to be optimized in the standby reconstructed image processing model, and the output feature adjustment layer is used to reduce the output feature of the corresponding level to be optimized to ensure that the adjusted output feature of the level is consistent with the output feature of the level in the pre-quantization image processing model, thereby obtaining a reconstructed image processing model;

[0068] Here, the adjusted output feature of the level is consistent with the output feature of the level in the pre-quantization image processing model, which means that the value of the output feature map of the level after adjustment is consistent with the value of the output feature map of the level in the pre-quantization image processing model;

[0069] S170: Quantize the reconstructed image processing model to obtain a second quantized image processing model;

[0070] The method of quantizing the reconstructed image processing model can be the same as the quantization method of step S110, and the difference lies in that the object of quantization is different, that is, not the original image processing model before quantization, but the reconstructed image processing model after the magnification of the parameters. The second quantized image processing model obtained by this step is the optimized image processing model, which can be deployed on the chip as a model that can execute image processing tasks.

[0071] The present application first quantizes the image processing model before quantization in step S110, and then through steps S120 and S130, the features of the optimization image are extracted by using the layers of the image processing model before quantization and the first quantized image processing model, respectively, and the quantization accuracy of each layer is calculated according to the feature extraction result, so as to determine the layer to be optimized with lower quantization accuracy. The model parameters of the layer to be optimized are magnified in step S150 to avoid the small parameters in the layer to be optimized being truncated and excluded from the influence on the accuracy during quantization. The output feature adjustment layer is added in step S160 to maintain the output feature consistency of the optimized layer, and then the reconstructed image processing model after the magnification of the parameters is quantized in step S170 to obtain the second quantized image processing model. Through quantization, the efficiency of the model processing image can be improved, the inference speed of the image processing model deployed on the chip can be improved, and the power consumption of the chip can be reduced. Through the magnification and reconstruction of the parameters of the image processing model before quantization, the accuracy loss caused by the quantization process of the image processing model is reduced, the image processing accuracy of the quantized image processing model is improved, the image processing model has higher image feature extraction accuracy on the basis of faster image feature extraction, the time loss caused by repeated model training due to model loss is avoided, and the optimization of the image processing model is realized.

[0072] In this embodiment, the step S130 of calculating the quantization accuracy of each layer of the first quantized image processing model based on the first image features and the corresponding second image features comprises the following steps:

[0073] The similarity between the first image features of each layer of the image processing model before quantization and the second image features of the corresponding layer of the first quantized image processing model is calculated as the quantization accuracy of the corresponding layer of the first quantized image processing model. Here, for the first image features of a layer of the image processing model before quantization, the output of the same layer in the first quantized image processing model is the second image features corresponding to the first image features.

[0074] Before calculating the similarity between the first image features of each layer of the image processing model before quantization and the corresponding second image features, the first image features and the second image features can be normalized by using the following formula:

[0075]

[0076]

[0077] in, is the normalized first image feature, is the normalized second image feature, f1 is the first image feature obtained in step S120, f2 is the second image feature obtained in step S120, and ‖‖2 represents the second norm. In step S130, the similarity between the first image feature of each level of the image processing model before quantization and the second image feature of the level corresponding to the first image processing model after quantization is calculated, which can be calculated by calculating at least one of the cosine similarity, dot product similarity, and Euclidean similarity between the two. Here, dot product similarity is taken as an example, and dot product similarity includes calculating the normalized first image feature And the normalized second image feature of the corresponding level The dot product between Prec(f1, f2) is used as the quantization accuracy of the corresponding level of the first quantized image processing model.

[0078] In other alternative embodiments, other similarity calculation methods may also be used to calculate the quantization accuracy of each level of the first quantized image processing model. For example, Euclidean similarity, cosine similarity, etc. may be used to calculate the similarity between the first image feature and the second image feature of the corresponding level as the quantization accuracy of the corresponding level, etc., all of which fall within the scope of protection of this application.

[0079] In this embodiment, step S140 of determining the level to be optimized in the image processing model before quantization based on the quantization accuracy of each level of the first quantized image processing model and the weight parameter distribution of the image processing model before quantization includes the following steps:

[0080] Obtaining a weight standard deviation of each level in the image processing model before quantization based on a weight parameter distribution of the image processing model before quantization;

[0081] Establishing a mapping curve between the weight standard deviation of the layers in the pre-quantization image processing model and the quantization accuracy;

[0082] Determining a weight standard deviation reference value corresponding to the quantization precision threshold according to the set quantization precision threshold and the mapping curve; the quantization precision threshold set here may be a pre-set threshold, or a quantization precision threshold re-determined based on the quantization precision of all levels after obtaining the quantization precision;

[0083] The layer whose weight standard deviation is less than the weight standard deviation reference value in the image processing model before quantization is taken as a layer to be optimized.

[0084] Most of the weight parameters in the layer whose weight standard deviation is less than the weight standard deviation reference value are small in value, and are generally truncated and removed during quantization, which affects the accuracy of the model after quantization. Therefore, the weight standard deviation reference value is used to determine the layer to be optimized, which can achieve more accurate selection of the layer to be optimized. However, the present application is not limited thereto, and in another alternative embodiment, the layer whose quantization accuracy is less than the quantization accuracy threshold can be directly selected as the layer to be optimized.

[0085] In this embodiment, the step of establishing the mapping curve between the weight standard deviation of the layer in the image processing model before quantization and the accuracy includes the following steps:

[0086] According to the weight standard deviation of each layer in the image processing model before quantization and the quantization accuracy of the corresponding layer in the first image processing model after quantization, a mapping curve is fitted, the first direction coordinate of the mapping curve is the weight standard deviation, and the second direction coordinate is the quantization accuracy. Thus, when the weight standard deviation reference value is determined subsequently, the weight standard deviation reference value can be found in the mapping curve based on the quantization accuracy threshold. Here, the mapping curve can be fitted by using a numerical analysis fitting method, taking the quantization accuracy and the corresponding weight standard deviation of each layer as known points, constructing a simple function S(d i , Prec i ), i = 0, 1, 2,..., n (n is the number of model layers), constructing a simple function S(d) so that the error of ‖S(d)-Prec(d)‖ is as small as possible, and performing approximate linear fitting according to the distribution of the point list to obtain the mapping curve corresponding to S(d). Then, for example, the quantization accuracy threshold is set to 0.95, and the weight standard deviation reference value θ can be obtained according to the quantization accuracy threshold and the mapping curve between the quantization accuracy and the standard deviation.

[0087] Here, only one method of establishing the mapping curve between the weight standard deviation of the layer in the image processing model before quantization and the quantization accuracy of the first image processing model after quantization is given. In another alternative embodiment, the mapping function between the weight standard deviation of the layer in the image processing model before quantization and the quantization accuracy of the first image processing model after quantization can be set according to empirical values in advance to obtain the corresponding mapping curve.

[0088] In this embodiment, the step S150 of performing amplification processing on the parameters of the layer to be optimized in the processing model before quantization to obtain a standby reconstructed image processing model includes the following steps:

[0089] calculating a ratio of a weight standard deviation of the level to be optimized in the pre-quantization processing model and the weight standard deviation reference value, calculating an amplification coefficient according to the ratio;

[0090] amplifying parameters of the level to be optimized in the pre-quantization processing model by the amplification coefficient;

[0091] Taking a full connection layer as an example, a forward inference formula of the layer is as follows:

[0092] y = xw + b

[0093] wherein y is an output feature of a current level, x is input data of the current level, w is a weight parameter, and b is a bias parameter;

[0094] Suppose that a weight standard deviation calculated by the current level is γ, then in order to optimize precision loss, the weight parameter and the bias parameter of the layer need to be amplified by δ times respectively, θ is the weight standard deviation reference value, k is a preset coefficient, and is a value greater than 1, for example, k is selected as 10, but the application is not limited thereto, and ceil(θ / γ) represents returning the smallest integer greater than or equal to θ / γ;

[0095] updating corresponding parameters of the level to be optimized in the pre-quantization processing model by using the amplified parameters to obtain a standby reconstructed image processing model.

[0096] Herein, only an example way of calculating the amplification coefficient is given, but the application is not limited thereto, for example, in another alternative embodiment, the amplification coefficient can also be a fixed setting value according to experience.

[0097] In this embodiment, since the parameters of the level to be optimized of the model are amplified, the output feature of the corresponding level is also amplified by δ times, in order to maintain the consistency of the output feature, a feature adjustment layer, for example, a scale layer, is needed to reduce the output feature of the level by δ times. Specifically, the step S160: an output feature adjustment layer is added at an output position of the level to be optimized in the standby reconstructed image processing model, the output feature adjustment layer is used to reduce the output feature of the corresponding level to be optimized by δ times to ensure that the output feature of the level after adjustment is consistent with the output feature of the level in the pre-quantization image processing model, and a reconstructed image processing model is obtained.

[0098] Taking a full connection layer as an example, a forward inference formula of the layer is as follows: Figure 2 ​The local network structure after processing of one full connection layer (InnerProduct) is shown in the figure, wherein BatchNorm+Scale is an existing standardization network structure, Flatten is used to "flatten" the input feature map, Convolution is a convolution layer connected after the full connection layer, a Scale layer is connected after the InnerProduct layer, i.e., a feature adjustment layer, which is used to reduce the output feature value of the InnerProduct layer after amplifying the parameter by δ times.

[0099] Taking the image processing model as a face recognition model as an example, the accuracy of the first quantized image processing model and the second quantized image processing model is tested and compared. For the same image, a first processing result is obtained by using the image processing model before quantization, a second processing result is obtained by using the first quantized image processing model, and a third processing result is obtained by using the second quantized image processing model. The first cosine similarity between the first processing result and the second processing result is only 38.79%, and the second cosine similarity between the first processing result and the third processing result is as high as 96.45%. It can be seen that by using the method of the present application, not only the size of the model is reduced and the processing efficiency of the model is improved, but also the accuracy loss caused by quantization is reduced. Compared with the traditional quantization method, the image processing accuracy is higher.

[0100] As shown in Figure 3 The embodiment of the present application also provides an image processing method, comprising the following steps:

[0101] S210: acquiring an image to be processed;

[0102] S220: inputting the image to be processed into the second quantized image processing model obtained by using the model optimization method described above;

[0103] Before inputting the image to be processed into the second quantized image processing model, the image to be processed can be processed first so that the size meets the input size requirement of the second quantized image processing model;

[0104] S230: acquiring an image processing result output by the second quantized image processing model;

[0105] For example, when the second quantized image processing model is a face recognition model, the image processing result here is the position recognition result and / or attribute recognition result of the face in the image to be processed. When the second quantized image processing model is a target detection model, the image processing result here is the position recognition result and / or target classification result of the target in the image to be processed.

[0106] In the image processing method of the present application, after the to-be-processed image is obtained, the to-be-processed image is input into the second quantized image processing model obtained through the model optimization method, and the image is processed based on the second quantized image processing model. After the second quantized image is processed, the processing efficiency is higher, and through the amplification processing of the smaller parameters in the lower precision level, the second quantized image processing model has higher precision in extracting features of the image and predicting results, thereby improving the efficiency and precision of image processing.

[0107] As shown in Figure 4 The present application also provides an image processing system applied to the image processing method, and the system comprises:

[0108] An image acquisition module M100 is configured to acquire a to-be-processed image.

[0109] An image input module M200 is configured to input the to-be-processed image into the second quantized image processing model obtained through the model optimization method.

[0110] An image output module M300 is configured to acquire an image processing result output by the second quantized image processing model.

[0111] In the image processing system of the present application, after the to-be-processed image is acquired by the image acquisition module M100, the to-be-processed image is input into the second quantized image processing model obtained through the model optimization method by the image input module M200, the image is processed based on the second quantized image processing model, and the image processing result is acquired by the image output module M300. After the second quantized image is processed, the processing efficiency is higher, and through the amplification processing of the smaller parameters in the lower precision level, the second quantized image processing model has higher precision in extracting features of the image and predicting results, thereby improving the efficiency and precision of image processing.

[0112] The embodiments of the present application also provide an image processing device, which comprises a processor, a memory having executable instructions of the processor stored therein, and the processor is configured to execute the steps of the image processing method by executing the executable instructions.

[0113] Those skilled in the art can understand that each aspect of the present application can be implemented as a system, a method or a program product. Therefore, each aspect of the present application can be embodied as a complete hardware embodiment, a complete software embodiment (including firmware, microcode, etc.), or an embodiment combining hardware and software, which can be collectively referred to as "circuitry", "module" or "system".

[0114] The following will be described with reference to Figure 5The electronic device 600 according to this embodiment of the present application will be described. Figure 5 The electronic device 600 shown is merely one example and should not be taken as limiting the scope of the present application.

[0115] As shown in Figure 5 The electronic device 600 is in the form of a general computing device. Components of the electronic device 600 can include, but are not limited to, at least one processing unit 610, at least one memory unit 620, a bus 630 that connects different system components including the memory unit 620 and the processing unit 610, a display unit 640, etc.

[0116] The memory unit stores program codes which can be executed by the processing unit 610, so that the processing unit 610 performs the steps described in the above electronic prescription flow processing method section according to various exemplary embodiments of the present application. For example, the processing unit 610 can perform the steps as shown in Figure 1

[0117] The memory unit 620 can include a readable medium in the form of volatile memory units, such as a random access memory (RAM) 6201 and / or a cache memory unit 6202, and can further include a read-only memory (ROM) 6203.

[0118] The memory unit 620 can further include program / utility 6204 having a set of the program modules 6205, such as an operating system, one or more application programs, other program modules, and program data, and can include implementation of a network environment in each of these or some combination thereof.

[0119] The bus 630 can be representative of one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration bus, a processor or local bus using any of a variety of bus structures, and the like.

[0120] ​The electronic device 600 can also communicate with one or more external devices 700 such as a keyboard or pointing devices, display devices, etc.; one or more devices that enable a user to interact with the electronic device 600; and / or one or more devices (e.g. routers, modems, etc.) that enable the electronic device 600 to communicate with one or more other computing devices. Such communication can be facilitated by an Input / Output (I / O) interface 650. Still yet, the electronic device 600 can communicate with one or more networks, such as a local area network (LAN), a wide area network (WAN), and / or the Internet, through a network adapter 660. The network adapter 660 can be communicatively coupled to the other components of the electronic device 600 through a bus 630. It should be appreciated that the bus 630 can be any bus or interconnect that allows bus master capable hardware devices to communicate with one another. The bus 630 allows data to be transferred between the electronic device 600 and other peripheral devices. Although the electronic device 600 is not shown with both a bus and a network adapter, such an electronic device typically would be implemented with both. Of course, the electronic device 600 can communicate with an external device 700 in a variety of other ways as well.

[0121] By employing the image processing device provided in the present application, the processor executes the image processing method when executing the executable instructions, thereby obtaining the beneficial effects of the image processing method.

[0122] The embodiments of the present application further provide a computer readable storage medium for storing a program, which, when executed by a processor, implements the steps of the image processing method. In some possible implementation manners, each aspect of the present application can also be implemented in the form of a program product, which includes program codes for causing an end device to perform the steps described in the image processing method part of the present specification according to various exemplary embodiments of the present application when the program product runs on the end device.

[0123] Reference Figure 6 As shown, the program product 800 for implementing the above method according to the embodiments of the present application is described, which can adopt a portable compact disc read-only memory (CD-ROM) and include program codes, and can run on an end device, such as a personal computer. However, the program product of the present application is not limited thereto, and in the present document, the readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with an instruction execution system, device or apparatus.

[0124] The program product can employ any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0125] The computer-readable storage medium can include a data signal traveling over a carrier wave in a baseband or propagated as part of a carrier wave. Such a propagated signal can take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer-readable storage medium can also be any computer-readable medium other than a storage medium that can be a source or destination of computer-readable instructions for execution by a computer. The instructions on the computer-readable storage medium can be implemented by any suitable high-level, low-level, or proprietary programming language, tooling, or combination thereof.

[0126] The program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider. The application program code can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device.

[0127] By employing the computer-readable storage medium provided by the present application, the program stored therein, when executed, implements the steps of the image processing method, and thus the beneficial effects of the image processing method can be obtained.

[0128] The above description is further detailed in combination with specific preferred embodiments of the present application, and cannot be deemed to limit the specific implementation of the present application to these descriptions. For those of ordinary skill in the art to which the present application belongs, several simple deductions or replacements can be made without departing from the concept of the present application, and all of them shall be deemed to fall within the protection scope of the present application.

Claims

1. A model optimization method, characterized in that: The steps include: quantizing the pre-quantization image processing model to obtain a first post-quantization image processing model; Inputting the optimization image into the pre-quantization image processing model and the first post-quantization image processing model, respectively, to obtain first image features output by each layer of the pre-quantization image processing model and second image features output by each layer of the first post-quantization image processing model; Calculating the quantization accuracy of each level of the first quantized image processing model based on the first image feature and the corresponding second image feature; According to the quantization accuracy of each level of the first quantized image processing model and the weight parameter distribution of the image processing model before quantization, the level to be optimized in the image processing model before quantization is determined, including: obtaining the weight standard deviation of each level in the image processing model before quantization based on the weight parameter distribution of the image processing model before quantization; establishing a mapping curve between the weight standard deviation and quantization accuracy of the level in the image processing model before quantization; determining the weight standard deviation reference value corresponding to the quantization accuracy threshold according to the set quantization accuracy threshold and the mapping curve; and taking the level in the image processing model before quantization whose weight standard deviation is less than the weight standard deviation reference value as the level to be optimized; Amplifying the parameters of the level to be optimized in the quantization pre-processing model to obtain a spare reconstructed image processing model; adding an output feature adjustment layer at the output position of the level to be optimized in the spare reconstructed image processing model, wherein the output feature adjustment layer is used to reduce the output features of the corresponding level to be optimized to ensure that the adjusted output features of the level are consistent with the output features of the level in the pre-quantization image processing model, thereby obtaining a reconstructed image processing model; The reconstructed image processing model is quantized to obtain a second quantized image processing model.

2. The model optimization method according to claim 1, characterized in that Calculating the quantization accuracy of each level of the first quantized image processing model based on the first image feature and the corresponding second image feature includes the following steps: The similarity between the first image features of each level of the image processing model before quantization and the second image features of the corresponding level of the first image processing model after quantization is calculated as the quantization accuracy of the corresponding level of the first image processing model after quantization.

3. The model optimization method according to claim 2, characterized in that: Calculate the similarity between the first image features of each level of the image processing model before quantization and the second image features of the level corresponding to the first image processing model after quantization, including calculating at least one of the dot product similarity, Euclidean similarity and cosine similarity of the first image features of each level of the image processing model before quantization and the second image features of the level corresponding to the first image processing model after quantization.

4. The model optimization method according to claim 1, characterized in that The step of establishing a mapping curve between the weight standard deviation and the accuracy of the layers in the pre-quantization image processing model comprises the following steps: According to the weight standard deviation of each level of the image processing model before quantization and the quantization accuracy of the corresponding level of the first image processing model after quantization, a mapping curve is fitted, wherein the first direction coordinate of the mapping curve is the weight standard deviation, and the second direction coordinate is the quantization accuracy.

5. The model optimization method according to claim 1, characterized in that: Amplifying the parameters of the level to be optimized in the quantization pre-processing model to obtain a spare reconstructed image processing model includes the following steps: Calculating a ratio of a weight standard deviation of a level to be optimized in the quantization pre-processing model to a weight standard deviation reference value, and calculating a magnification factor based on the ratio; amplifying the parameters of the level to be optimized in the quantization pre-processing model by the amplification coefficient; The amplified parameters are used to update the corresponding parameters of the level to be optimized in the quantization pre-processing model to obtain a spare reconstructed image processing model.

6. An image processing method, characterized in that: The steps include: Get the image to be processed; Inputting the image to be processed into a second quantized image processing model obtained by the model optimization method according to any one of claims 1 to 5; Obtain an image processing result output by the second quantized image processing model.

7. An image processing system, characterized in that: Applied to the image processing method according to claim 6, the system comprises: An image acquisition module, used for acquiring an image to be processed; An image input module, configured to input the image to be processed into a second quantized image processing model obtained by the model optimization method; An image output module is used to obtain the image processing result output by the second quantized image processing model.

8. An image processing device, characterized in that include: processor; a memory storing executable instructions for the processor; The processor is configured to perform the steps of the image processing method according to claim 6 by executing the executable instructions.

9. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the image processing method according to claim 6 are implemented.

Images