Model screening method for image processing model quantification and image processing model quantification method

Abstract

The invention provides a model screening method for image processing model quantification and an image processing model quantification method, and relates to the technical field of deep learning, and the method comprises the steps: calculating a comprehensive quantification friendliness score of each image processing model, and screening each image processing model, and an image processing model with high comprehensive quantization friendliness can be screened out for subsequent quantization operation. Therefore, the number of models needing to be quantized can be greatly reduced, the quantization cost is further reduced, the quantization efficiency is improved, and hardware resources can be saved.

Description

Technical Field

[0001] This invention relates to the field of deep learning technology, and in particular to a model selection method and an image processing model quantization method for image processing model quantization. Background Technology

[0002] In modern computer vision and deep learning applications, convolutional neural networks are commonly used in image processing scenarios to form image processing models.

[0003] In practical applications, quantization of image processing models is a key technology. Currently, the most common quantization method for image processing models is Post-Training Quantization (PTQ), which jointly quantizes the weight matrix and activation values ​​of a pre-trained model. Here, the activation values ​​are the outputs of each neuron in the pre-trained model, containing the model's output.

[0004] PTQ technology can compress the weight matrix and activation values ​​of image processing models from floating-point data to fixed-point data, thereby significantly reducing the model's storage space and computational requirements, making the model more suitable for deployment and inference on resource-constrained devices. However, model quantization introduces a loss of accuracy, the degree of which is related to the distribution of the model's weight and activation values. Therefore, how to efficiently obtain image processing models with minimal quantization accuracy loss for deployment is a crucial issue. Summary of the Invention

[0005] This invention provides a model selection method and an image processing model quantization method for image processing model quantization, in order to solve the defects existing in related technologies.

[0006] This invention provides a model selection method for image processing model quantization, comprising: Acquire the target sample images for model training and the multiple image processing models generated during training; By traversing the multiple image processing models, for the current image processing model, the weight tensors of each layer structure in the current image processing model are determined, and the target sample image is input into the current image processing model to obtain the activation tensors output by each layer structure; Based on the first ratio of the maximum absolute value of the weights of all output channels in the weight tensor of each layer structure to the maximum absolute value of the weights of each output channel, the weight quantization friendliness score of the current image processing model is calculated, and based on the second ratio of the maximum absolute value of the activation of all channels in the activation tensor output by each layer structure to the maximum absolute value of the activation of each channel, the activation quantization friendliness score of the current image processing model is calculated. Based on the weight quantization friendliness score and the activation quantization friendliness score, the comprehensive quantization friendliness score of the current image processing model is calculated, and the multiple image processing models are screened based on the comprehensive quantization friendliness scores of the multiple image processing models.

[0007] According to a model selection method for image processing model quantization provided by the present invention, the method for calculating the weight quantization friendliness score of the current image processing model based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of each layer structure to the maximum absolute value of the weight of each output channel includes: Traverse each layer structure. For the current layer structure, determine the weight quantification evaluation index for each output channel based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of the current layer structure to the maximum absolute value of the weight of each output channel. Based on the weighted quantification evaluation index of each output channel, calculate the first proportion of the number of output channels with the weighted quantification evaluation index within the first specified range in the total number of output channels; Based on the average first proportion corresponding to each layer structure, the weight quantization friendliness score of the current image processing model is calculated.

[0008] According to a model selection method for image processing model quantization provided by the present invention, the method for determining the weight quantization evaluation index of each output channel based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of the current layer structure to the maximum absolute value of the weight of each output channel includes: For any output channel in the weight tensor of the current layer structure, calculate the logarithm of the first ratio corresponding to the output channel, and use the logarithm as the weight quantification evaluation index of the output channel.

[0009] According to a model selection method for image processing model quantization provided by the present invention, the first specified range is a weight quantization evaluation index that is greater than or equal to a first threshold and less than a second threshold; correspondingly, the step of calculating the weight quantization friendliness score of the current image processing model based on the first average proportion corresponding to each layer structure includes: The difference between 1 and the first average percentage is calculated as the weighted quantification friendliness score.

[0010] According to a model selection method for image processing model quantization provided by the present invention, the method for calculating the activation quantization friendliness score of the current image processing model based on the second ratio of the maximum absolute activation value of all channels in the activation tensor output by each layer structure to the maximum absolute activation value of each channel includes: Traverse each layer structure. For the current layer structure, determine the activation quantification evaluation index for each channel based on the current second ratio of the maximum absolute value of activation of all channels in the activation tensor output by the current layer structure to the maximum absolute value of activation of each channel. Based on the activation quantification evaluation index of each channel, calculate the second proportion of the number of channels with the activation quantification evaluation index within a second specified range in the total number of channels of the activation tensor output by the current layer structure. Based on the average second proportion corresponding to each layer structure, the activation value quantization friendliness score of the current image processing model is calculated.

[0011] According to a model selection method for image processing model quantization provided by the present invention, the step of determining the activation quantization evaluation index for each channel based on the current second ratio of the maximum absolute activation value of all channels in the activation tensor output by the current layer structure to the maximum absolute activation value of each channel includes: For any channel in the activation tensor output by the current layer structure, calculate the logarithm of the current second ratio corresponding to the channel, and use the logarithm as the activation quantification evaluation index for the channel.

[0012] According to the model selection method for image processing model quantization provided by the present invention, the second specified range is a weight quantization evaluation index that is greater than or equal to a third threshold and less than a fourth threshold; correspondingly, the step of calculating the activation quantization friendliness score of the current image processing model based on the second average proportion corresponding to each layer structure includes: The difference between 1 and the second average percentage is calculated as the activation quantification friendliness score.

[0013] The present invention also provides an image processing model quantization method, comprising: Obtain a preset number of models to be quantized, wherein the models to be quantized are image processing models obtained by screening based on the above-mentioned model screening method for image processing model quantization; Each of the aforementioned models to be quantized is quantized to obtain a quantized model. Acquire quantization test images, analyze the quantization test images using the quantization models described above, and obtain the test results of each quantization model. Based on the test results of each of the quantization models, the target quantization model is determined.

[0014] The present invention also provides a model selection device for image processing model quantization, comprising: The first acquisition module is used to acquire the target sample image for model training and multiple image processing models generated during training. The tensor determination module is used to traverse the multiple image processing models, determine the weight tensors of each layer structure in the current image processing model for the current image processing model, and input the target sample image into the current image processing model to obtain the activation tensors output by each layer structure; The first score determination module is used to calculate the weight quantization friendliness score of the current image processing model based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of each layer structure to the maximum absolute value of the weight of each output channel, and to calculate the activation quantization friendliness score of the current image processing model based on the second ratio of the maximum absolute value of the activation of all channels in the activation tensor output by each layer structure to the maximum absolute value of the activation of each channel. The model filtering module is used to calculate the comprehensive quantization friendliness score of the current image processing model based on the weight quantization friendliness score and the activation quantization friendliness score, and to filter the multiple image processing models based on the comprehensive quantization friendliness score of the multiple image processing models.

[0015] The present invention also provides an image processing model quantization device, comprising: The second acquisition module is used to acquire a preset number of models to be quantized, wherein the models to be quantized are image processing models obtained by screening based on the above-mentioned model screening method for image processing model quantization. The model quantization module is used to quantize each of the models to be quantized to obtain each quantized model. The model testing module is used to acquire quantization test images, analyze the quantization test images using each of the quantization models, and obtain the test results of each of the quantization models. The model determination module is used to determine the target quantization model based on the test results of each of the quantization models.

[0016] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a model selection method for image processing model quantization as described above, or an image processing model quantization method.

[0017] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the model selection method or the image processing model quantization method as described above.

[0018] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements a model selection method or an image processing model quantization method as described above.

[0019] This invention provides a model selection method and an image processing model quantization method for image processing models. This method calculates the comprehensive quantization friendliness score for each image processing model, and selects models with strong comprehensive quantization friendliness for subsequent quantization operations. This significantly reduces the number of models requiring quantization, thereby lowering quantization costs, improving quantization efficiency, and saving hardware resources. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in this invention or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic flowchart of the model selection method for image processing model quantization provided by the present invention.

[0022] Figure 2 This is one of the flowcharts illustrating the image processing model quantization method provided by the present invention.

[0023] Figure 3 This is a partial structural diagram of the image processing model in the model selection method for image processing model quantization provided by the present invention.

[0024] Figure 4 This is the second flowchart of the image processing model quantization method provided by the present invention.

[0025] Figure 5 This is a schematic diagram of the model selection device for image processing model quantization provided by the present invention.

[0026] Figure 6 This is a schematic diagram of the image processing model quantization device provided by the present invention.

[0027] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] Because existing PTQ techniques introduce model accuracy loss, the degree of which is related to the distribution of model weights and activation values, reducing quantization accuracy loss is a crucial issue that urgently needs to be addressed. Current techniques directly perform batch quantization on the original image processing models at multiple storage points during a single training iteration to obtain various quantized models. Then, each quantized model is tested against performance metrics to determine the model with the highest metrics. This increases the success rate of model quantization and reduces the problem of significant accuracy loss and low accuracy in quantized models obtained by randomly assigning model parameters and selecting a single model for quantization.

[0030] However, this approach requires quantizing, testing, and statistically analyzing the original image processing models at multiple storage points during training, which results in wasted hardware resources and low efficiency.

[0031] Based on this, this embodiment of the invention provides a model selection method for image processing model quantization. By quantizing the selected image processing models, the number of models that need to be quantized can be greatly reduced, thereby reducing quantization costs, improving quantization efficiency, and saving hardware resources.

[0032] Figure 1 This is a flowchart illustrating a model selection method for image processing model quantization provided in an embodiment of the present invention, as shown below. Figure 1 As shown, the method includes: S11, Obtain the target sample image for model training and multiple image processing models generated during training; S12, traverse the multiple image processing models, for the current image processing model, determine the weight tensors of each layer structure in the current image processing model, and input the target sample image into the current image processing model to obtain the activation tensors output by each layer structure; S13, based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of each layer structure to the maximum absolute value of the weight of each output channel, calculate the weight quantization friendliness score of the current image processing model, and based on the second ratio of the maximum absolute value of the activation of all channels in the activation tensor output by each layer structure to the maximum absolute value of the activation of each channel, calculate the activation quantization friendliness score of the current image processing model. S14. Based on the weight quantization friendliness score and the activation quantization friendliness score, calculate the comprehensive quantization friendliness score of the current image processing model, and filter the multiple image processing models based on the comprehensive quantization friendliness scores of the multiple image processing models.

[0033] Specifically, the model selection method for image processing model quantization provided in this embodiment of the invention is executed by a model selection device for image processing model quantization. This device can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0034] First, step S11 is executed to obtain the target sample images for model training and multiple image processing models generated during training. Here, the sample images used for model training can include multiple images, which can constitute a sample image set. The target sample images can be one or more sample images randomly selected from various scenes within this sample image set. For example, the target sample images can include two sample images from each scene. To improve the accuracy of each quantization friendliness score, the selection criteria for target sample images can be to avoid overexposed, remote scenes, and pure black / pure white sample images, and instead select typical scene images with high recognition rates and uniform colors.

[0035] The initial processing model is iteratively trained using a set of sample images. After reaching the training termination condition, multiple candidate processing models are obtained. From these models, the one with preset precision and preset recall is selected as the final image processing model. The training termination condition can be reaching a preset number of iterations or the loss function used during training converging. The preset precision and preset recall can be set as needed and are not specifically limited here. Multiple image processing models can be obtained.

[0036] Understandably, image processing models can be used to perform operations such as target recognition, segmentation, and detection on images.

[0037] Then, step S12 is executed, traversing multiple image processing models. The same operation is performed on each traversed image processing model: for the current traversed image processing model, the weight tensors of each layer in the current image processing model are determined. Each image processing model has a multi-layered structure, such as... Figure 2 As shown, each layer structure can include an input layer, convolutional layer, activation layer, output layer, etc. Figure 2 As can be seen, not every layer has a corresponding weight tensor. Convolutional layers have weight tensors, while input layers, activation layers, and output layers do not have weight tensors, meaning their weight tensors are null.

[0038] The weight tensor can include weight values ​​for each dimension, and the dimensions of the weight tensor can be represented as OIHW. Here, O is the number of output channels of the layer, I is the number of input channels of the layer, and H and W are the height and width of the convolution kernel of the layer, respectively.

[0039] By inputting the target sample image into the current image processing model, the activation tensors output by each layer of the current image processing model can be obtained. For example... Figure 2 As shown, each layer has an corresponding output activation tensor, which can be the output tensor of that layer, i.e., the feature map output by that layer. The activation tensor can include activation values ​​in each dimension, and the dimensions of the activation tensor can be represented as NCHW. Here, N represents the batch size, i.e., the number of sample images selected from the sample image set for each training iteration; C is the number of channels in the activation tensor output by that layer; and H and W are the height and width of the activation tensor output by that layer, respectively.

[0040] Next, step S13 is executed to calculate the first ratio of the maximum absolute weight of all output channels in the weight tensor of each layer to the first ratio of the maximum absolute weight of each output channel. For each layer where the weight tensor is not empty, there is a first ratio equal to the number of output channels, and the first ratio corresponds one-to-one with the output channels in each layer. By using the maximum absolute weight of all output channels in the weight tensor of that layer to the first ratio of each output channel, the first ratio corresponding to each output channel can be determined.

[0041] For the weight tensor of the j-th layer structure, the maximum absolute value of the weight of the i-th output channel and the maximum absolute value of the weights of all output channels They can be represented as follows:

[0042] in, This represents the maximum absolute value of the weight of the i-th channel, that is, the maximum absolute value of the weight values ​​of the i-th channel. This represents the minimum weight value of the i-th channel. This represents the maximum weight value of the i-th channel. It is an absolute value function. It is a function that takes the maximum value. This represents the maximum absolute value of the weights across all output channels, i.e., the maximum absolute value of the weights across all output channels.

[0043] The first ratio corresponding to the i-th output channel in the weight tensor of the j-th layer structure can be expressed as: .

[0044] By using the first ratio corresponding to each output channel in the weight tensor of each layer, the weight quantization friendliness score of the current image processing model can be calculated. The weight quantization friendliness score is used to assess the risk of accuracy loss in the current image processing model after weight tensor quantization.

[0045] For example, the first ratio corresponding to each output channel can be used as the weight quantization evaluation index for that output channel. The first proportion of the number of output channels with weight quantization evaluation in each layer within a first specified range is calculated in the total number of output channels. Then, the average of the first proportions corresponding to each layer is used as the weight quantization friendliness score of the current image processing model. That is: ; in, The weights are used to quantize the friendliness score, where M is the number of layers in the current image processing model. This represents the first proportion corresponding to the k-th layer structure.

[0046] Here, the first specified range can be less than the first threshold A1, and the weight quantization friendliness score can be used. The smaller the value, the better the image processing model is at handling weight quantization. This means the model is more suitable for weight tensor quantization, and the lower the risk of accuracy loss after weight tensor quantization. In other words:

[0047] in, This represents the weight quantification evaluation index for the i-th output channel.

[0048] It can also calculate the second ratio of the maximum absolute value of activation of all channels in the activation tensor output by each layer structure to the maximum absolute value of activation of each channel.

[0049] Since the dimension of the activation tensor can be represented as NCHW, in order to improve computational efficiency and reduce computational difficulty, the dimension of the activation tensor can be transformed, for example, from NCHW to CNHW, so that the maximum absolute value of activation can be calculated by channel later.

[0050] Each layer has a second ratio that corresponds to the same number of channels in the output activation tensor. These second ratios are one-to-one with the channels in the output activation tensor of each layer. The second ratio for each channel can be determined by comparing the maximum absolute activation value of all channels in the output activation tensor of that layer with the maximum absolute activation value of each individual channel.

[0051] For the activation tensor of the j-th layer structure, the maximum absolute activation value of the i-th channel and the maximum absolute value of activation of all channels They can be represented as follows:

[0052] in, This represents the maximum absolute activation value of the i-th channel, that is, the maximum absolute value of the activation value of the i-th channel. This represents the maximum absolute activation value across all channels, i.e., the maximum absolute value of the activation values ​​across all channels. This represents the maximum activation value of the i-th channel. This represents the minimum activation value of the i-th channel.

[0053] The second ratio corresponding to the i-th channel in the activation tensor output by the j-th layer structure can be expressed as: .

[0054] By using the second ratio corresponding to each channel in the activation tensor output by each layer, the activation quantization friendliness score of the current image processing model can be calculated. The activation quantization friendliness score is used to assess the risk of accuracy loss in the current image processing model after activation tensor quantization.

[0055] For example, the second ratio corresponding to each channel can be used as the activation quantization evaluation index for that channel. The second proportion of the number of channels with activation quantization evaluation indices within a second specified range in each layer is calculated relative to the total number of channels. Then, the average of the second proportions corresponding to each layer is used as the activation quantization friendliness score of the current image processing model. That is:

[0056] in, To activate the quantization friendliness score, M represents the number of structural layers in the current image processing model. This represents the second proportion corresponding to the k-th layer structure.

[0057] Here, the second specified range can be less than the third threshold A2, activating the quantification of friendliness score. The smaller the value, the more favorable the current image processing model is to activation quantization, the more suitable the model is for activation tensor quantization, and the lower the risk of accuracy loss after activation tensor quantization. This third threshold A2 can be set as needed; for example, it can be the same as the first threshold A1, both set to 3. That is:

[0058] in, This represents the activation quantification evaluation index of the i-th output channel.

[0059] Finally, step S14 is executed to calculate the overall quantization friendliness score of the current image processing model using the weight quantization friendliness score and the activation quantization friendliness score. For example, the average of the weight quantization friendliness score and the activation quantization friendliness score can be used as the overall quantization friendliness score of the current image processing model. That is: .

[0060] Furthermore, by iterating through each image processing model and performing the above process, the overall quantization friendliness score of each image processing model is determined. This overall quantization friendliness score can then be used to filter multiple image processing models. For example, a predetermined number of image processing models with strong overall quantization friendliness can be selected from among the various image processing models as models to be quantized for subsequent quantization operations.

[0061] Lower scores for both weight quantization friendliness and activation quantization friendliness indicate stronger quantization friendliness. Therefore, a lower overall quantization friendliness score indicates stronger overall quantization friendliness for the image processing model, making it more suitable for quantizing weight and activation tensors, and reducing the risk of accuracy loss after quantization. A predetermined number of image processing models with low overall quantization friendliness scores can be selected as models to be quantized. This predetermined number can be set as needed, and can be one or more, for example, 3, or other values.

[0062] It should be noted that the operations on the weight tensor and the operations on the activation tensor in steps S12 and S13 are independent of each other. The weight tensor can be operated on first and then the activation tensor in the order of steps S12 and S13, or the activation tensor can be operated on first and then the weight tensor in the order of steps S12 and S13, or the weight tensor and the activation tensor can be operated on simultaneously in each step in the order of steps S12 and S13.

[0063] The model selection method for image processing model quantization provided in this embodiment of the invention first obtains the target sample image for model training and multiple image processing models generated by training. Then, it iterates through the multiple image processing models. For the current image processing model, it determines the weight tensors of each layer structure and inputs the target sample image into the current image processing model to obtain the activation tensors output by each layer structure. Subsequently, based on the first ratio of the maximum absolute weight value of all output channels in the weight tensors of each layer structure to the maximum absolute weight value of each output channel, it calculates the weight quantization friendliness score of the current image processing model. Based on the second ratio of the maximum absolute activation value of all channels in the activation tensors output by each layer structure to the maximum absolute activation value of each channel, it calculates the activation quantization friendliness score of the current image processing model. Finally, using the weight quantization friendliness score and the activation quantization friendliness score, it calculates the comprehensive quantization friendliness score of the current image processing model, and based on the comprehensive quantization friendliness score of multiple image processing models, it selects multiple image processing models. This method calculates the comprehensive quantization friendliness score of each image processing model separately and selects the image processing models with strong comprehensive quantization friendliness for subsequent quantization operations. This can greatly reduce the number of models that need to be quantized, thereby reducing quantization costs, improving quantization efficiency, and saving hardware resources.

[0064] Based on the above embodiments, the first ratio of the maximum absolute value of the weights of all output channels in the weight tensor of each layer structure to the maximum absolute value of the weights of each output channel is used to calculate the weight quantization friendliness score of the current image processing model, including: Traverse each layer structure. For the current layer structure, determine the weight quantification evaluation index for each output channel based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of the current layer structure to the maximum absolute value of the weight of each output channel. Based on the weighted quantification evaluation index of each output channel, calculate the first proportion of the number of output channels with the weighted quantification evaluation index within the first specified range in the total number of output channels; Based on the average first proportion corresponding to each layer structure, the weight quantization friendliness score of the current image processing model is calculated.

[0065] Specifically, when calculating the weight quantization friendliness score of the current image processing model, the same operation can be performed on each layer of the current image processing model. For the current layer of the current image processing model, the weight quantization evaluation index of each output channel can be determined by using the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of the current layer to the maximum absolute value of the weight of each output channel.

[0066] Here, for any output channel in the weight tensor of the current layer structure, the logarithm of the first ratio corresponding to any output channel can be calculated, and this logarithm can be used as the weight quantification evaluation index for any output channel. This logarithm can be the logarithm with a specified base, which can be set as needed, such as 2, e, 10, etc., without specific limitations here.

[0067] Taking a specified value of 2 as an example, the weight quantization evaluation index of the i-th output channel is... It can be represented as:

[0068] Subsequently, using the weight quantification evaluation index of each output channel, the first proportion of the number of output channels within the first specified range of the weight quantification evaluation index in the total number of output channels is calculated.

[0069] The first specified range can also be greater than or equal to the first threshold A1 and less than the second threshold B1. That is, for the current image processing model with an output channel whose weighted quantization evaluation index is greater than or equal to the second threshold B1, it can be considered unsuitable for quantization. In this case, we have:

[0070] The first threshold A1 can be set to 3, the second threshold A2 can be set to 5, and both can be set to other values.

[0071] Furthermore, by utilizing the average first proportion corresponding to each layer, the weight quantization friendliness score of the current image processing model can be calculated. For example, the difference between 1 and the average first proportion can be used as the weight quantization friendliness score. That is:

[0072] At this point, the weighted quantification friendliness score The larger the value, the more favorable the current image processing model is for weight quantization.

[0073] In this embodiment of the invention, by calculating the first proportion of the number of output channels within a first specified range in the total number of output channels of the weight quantization evaluation index, the average first proportion corresponding to each layer structure is obtained, and then the weight quantization friendliness score of the current image processing model is calculated. This quantifies the weight quantization friendliness and improves the accuracy of the weight quantization friendliness score. Furthermore, the weight quantization friendliness score... The larger the value, the more favorable the current image processing model is for weight quantization.

[0074] Based on the above embodiments, the calculation of the activation quantization friendliness score of the current image processing model by the second ratio of the maximum absolute activation value of all channels in the activation tensor output by each layer structure to the maximum absolute activation value of each channel includes: Traverse each layer structure. For the current layer structure, determine the activation quantification evaluation index for each channel based on the current second ratio of the maximum absolute value of activation of all channels in the activation tensor output by the current layer structure to the maximum absolute value of activation of each channel. Based on the activation quantification evaluation index of each channel, calculate the second proportion of the number of channels with the activation quantification evaluation index within a second specified range in the total number of channels of the activation tensor output by the current layer structure. Based on the average second proportion corresponding to each layer structure, the activation value quantization friendliness score of the current image processing model is calculated.

[0075] Specifically, when calculating the activation quantization friendliness score of the current image processing model, the same operation can be performed on each layer of the current image processing model. For the current layer of the current image processing model, the weight quantization evaluation index of each channel can be determined by using the second ratio of the maximum absolute activation value of all channels in the activation tensor output by the current layer to the maximum absolute activation value of each channel.

[0076] Here, for any channel in the activation tensor output by the current layer structure, the logarithm of the second ratio corresponding to any channel can be calculated, and this logarithm can be used as the activation quantification evaluation index for any channel. This logarithm can be the logarithm with a specified base, which can be set as needed, such as 2, e, 10, etc., without specific limitations here.

[0077] Taking a specified value of 2 as an example, the activation quantification evaluation index of the i-th channel is... It can be represented as:

[0078] Subsequently, using the activation quantification evaluation index of each channel, the second proportion of the number of channels with the activation quantification evaluation index within the second specified range in the total number of channels of the activation tensor output by the current layer structure is calculated.

[0079] The second specified range can also be greater than or equal to the third threshold A2 and less than the fourth threshold B2. That is, for a current image processing model with channels whose activation quantization evaluation index is greater than or equal to the fourth threshold B2, it can be considered unsuitable for quantization. In this case, we have:

[0080] The third threshold A2 can be set to 3, the fourth threshold B2 can be set to 5, and both can be set to other values.

[0081] Furthermore, by utilizing the average of the second proportions corresponding to each layer, the activation quantization friendliness score of the current image processing model can be calculated. For example, the difference between 1 and the average of the second proportions can be calculated as the activation quantization friendliness score. That is:

[0082] At this point, the quantitative friendliness score is activated. The larger the value, the more responsive the current image processing model is to activation quantization.

[0083] In this embodiment of the invention, by calculating the second proportion of the number of channels within a second specified range of the activation quantization evaluation index in the total number of channels of the activation tensor output by the current layer structure, the average of the second proportions corresponding to each layer structure is obtained. This allows for the calculation of the activation quantization friendliness score of the current image processing model, quantifying the activation quantization friendliness and improving the accuracy of the activation quantization friendliness score. Furthermore, the activation quantization friendliness score... The larger the value, the more favorable the activation quantization of the current image processing model.

[0084] like Figure 3 As shown, based on the above embodiments, this embodiment of the invention also provides an image processing model quantization method, which includes: S21, Obtain a preset number of models to be quantized, wherein the models to be quantized are image processing models obtained by screening based on the model screening method for image processing model quantization provided in the above embodiments; S22, perform model quantization on each of the models to be quantized to obtain each quantized model; S23, acquire quantization test images, analyze the quantization test images using each of the quantization models, and obtain the test results of each of the quantization models respectively; S24. Based on the test results of each of the quantization models, determine the target quantization model.

[0085] Specifically, the image processing model quantization method provided in this embodiment of the invention is executed by an image processing model quantization device, which can be configured in a computer. The computer can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0086] First, step S21 is executed to obtain a preset number of models to be quantized. These models are image processing models obtained by screening based on the model screening method for image processing model quantization provided in the above embodiments.

[0087] Then, step S22 is executed to quantize each model to be quantized, resulting in quantized models. Here, conventional model quantization methods can be used to quantize each model; no specific limitations are imposed in this embodiment. In actual quantization, model quantization can only effectively quantize parameters within a certain range. If the proportion of abnormal channels in the model parameters to be quantized is large, it will cause significant quantization errors, leading to a substantial decrease in the accuracy of the final quantized model. It is understood that the quantized models obtained here are all successfully quantized models.

[0088] Next, step S23 is executed to obtain quantization test images. These images are then analyzed using various quantization models to obtain the test results for each model. The quantization test images can be randomly selected from the test image set of the image processing model.

[0089] For each quantization model, the quantization test images are analyzed using that model to obtain the test results. The test results of the quantization model can be parameters such as accuracy, error rate, detection rate, or false positive rate, which can be customized according to the actual situation.

[0090] Finally, step S24 is executed to determine the target quantization model for actual deployment using the test results of each quantization model.

[0091] For example, if the test results include accuracy, the quantization model with the highest accuracy can be selected as the target quantization model; if the test results include detection rate, the quantization model with the highest detection rate can be selected as the target quantization model; if the test results include false detection rate, the quantization model with the lowest false detection rate can be selected as the target quantization model, and so on.

[0092] The target quantization model inherits the functionality of the image processing model and can be used for image processing or target recognition. For example, if the image processing model is used for vehicle recognition, the target quantization model can also be used for vehicle recognition; if the image processing model is used for image scene classification, the target quantization model can also be used for image scene classification. This target quantization model has higher accuracy in image processing, reducing the problem of low accuracy in quantized models obtained by randomly assigning model parameters and selecting a single model for quantization, and increasing the success rate of model quantization.

[0093] The image processing model quantization method provided in this embodiment of the invention uses the model screening method for image processing model quantization provided in the above embodiments to screen the obtained image processing model as the model to be quantized, and selects the target quantization model from the obtained quantization models. This can reduce the cost of model quantization, improve quantization efficiency, save hardware resources, and increase the quantization success rate.

[0094] Based on the above embodiments, such as Figure 4 The diagram shown illustrates the complete process of the image processing model quantization method provided in this embodiment of the invention. The method includes: Multiple image processing models generated during training are obtained and formed into a model pool; Iterate through all image processing models in the model pool. For the current image processing model, calculate the weight quantization friendliness score and activation quantization friendliness score of the current image processing model using the model screening method for image processing model quantization provided in the above embodiments. The comprehensive quantization friendliness score of the current image processing model is calculated by using the weighted quantization friendliness score and the activation quantization friendliness score. The comprehensive quantization friendliness scores of multiple image processing models are then used to filter the multiple image processing models and obtain a target number of image processing models as models to be quantized. The quantitative test images are analyzed using the various quantitative models described above, and the test results of each quantitative model are obtained. The target quantitative model is then determined using the test results of each quantitative model.

[0095] like Figure 5 As shown, based on the above embodiments, this embodiment of the invention provides a model selection device for image processing model quantization, comprising: The first acquisition module 51 is used to acquire the target sample image for model training and multiple image processing models generated during training. Tensor determination module 52 is used to traverse the multiple image processing models, determine the weight tensors of each layer structure in the current image processing model for the current image processing model, and input the target sample image into the current image processing model to obtain the activation tensors output by each layer structure; The scoring determination module 53 is used to calculate the weight quantization friendliness score of the current image processing model based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of each layer structure to the maximum absolute value of the weight of each output channel, and to calculate the activation quantization friendliness score of the current image processing model based on the second ratio of the maximum absolute value of the activation of all channels in the activation tensor output by each layer structure to the maximum absolute value of the activation of each channel. The model filtering module 54 is used to calculate the comprehensive quantization friendliness score of the current image processing model based on the weight quantization friendliness score and the activation quantization friendliness score, and to filter the multiple image processing models based on the comprehensive quantization friendliness score of the multiple image processing models.

[0096] Based on the above embodiments, the model screening device for image processing model quantization provided in this embodiment of the invention, wherein the score determination module is specifically used for: Traverse each layer structure. For the current layer structure, determine the weight quantification evaluation index for each output channel based on the first ratio of the maximum absolute value of the weight of all output channels in the weight tensor of the current layer structure to the maximum absolute value of the weight of each output channel. Based on the weighted quantification evaluation index of each output channel, calculate the first proportion of the number of output channels with the weighted quantification evaluation index within the first specified range in the total number of output channels; Based on the average first proportion corresponding to each layer structure, the weight quantization friendliness score of the current image processing model is calculated.

[0097] Based on the above embodiments, the model screening device for image processing model quantization provided in this embodiment of the invention, wherein the score determination module is specifically used for: For any output channel in the weight tensor of the current layer structure, calculate the logarithm of the first ratio corresponding to the output channel, and use the logarithm as the weight quantification evaluation index of the output channel.

[0098] Based on the above embodiments, the model screening device for image processing model quantization provided in this embodiment of the invention, wherein the score determination module is specifically used for: The difference between 1 and the first average percentage is calculated as the weighted quantification friendliness score.

[0099] Based on the above embodiments, the model screening device for image processing model quantization provided in this embodiment of the invention, wherein the score determination module is specifically used for: Traverse each layer structure. For the current layer structure, determine the activation quantification evaluation index for each channel based on the current second ratio of the maximum absolute value of activation of all channels in the activation tensor output by the current layer structure to the maximum absolute value of activation of each channel. Based on the activation quantification evaluation index of each channel, calculate the second proportion of the number of channels with the activation quantification evaluation index within a second specified range in the total number of channels of the activation tensor output by the current layer structure. Based on the average second proportion corresponding to each layer structure, the activation value quantization friendliness score of the current image processing model is calculated.

[0100] Based on the above embodiments, the model screening device for image processing model quantization provided in this embodiment of the invention, wherein the score determination module is specifically used for: For any channel in the activation tensor output by the current layer structure, calculate the logarithm of the current second ratio corresponding to the channel, and use the logarithm as the activation quantification evaluation index for the channel.

[0101] Based on the above embodiments, the model screening device for image processing model quantization provided in this embodiment of the invention specifies that the second designated range is a weight quantization evaluation index that is greater than or equal to a third threshold and less than a fourth threshold; correspondingly, the score determination module is specifically used for: The difference between 1 and the second average percentage is calculated as the activation quantification friendliness score.

[0102] Specifically, the functions of each module in the model screening device for image processing model quantization provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0103] like Figure 6 As shown, based on the above embodiments, this embodiment of the invention provides an image processing model quantization device, comprising: The second acquisition module 61 is used to acquire a preset number of models to be quantized, wherein the models to be quantized are image processing models obtained by screening based on the model screening method for image processing model quantization provided in the above embodiments. The model quantization module 62 is used to perform model quantization on each of the models to be quantized to obtain each quantized model. The model testing module 63 is used to acquire quantization test images, analyze the quantization test images using each of the quantization models, and obtain the test results of each of the quantization models. The model determination module 64 is used to determine the target quantization model based on the test results of each of the quantization models.

[0104] Specifically, the functions of each module in the image processing model quantization device provided in this embodiment correspond one-to-one with the operation flow of each step in the above method embodiment, and the achieved effects are also the same. Please refer to the above embodiments for details, and this will not be repeated in this embodiment.

[0105] Figure 7 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 7As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, the communications interface 720, and the memory 730 communicate with each other via the communication bus 740. The processor 710 may call logical instructions in the memory 730 to execute the model selection method or the image processing model quantization method provided in the above embodiments.

[0106] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0107] On the other hand, the present invention also provides a computer program product, the computer program product including a computer program, the computer program being stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer is able to execute the model selection method for image processing model quantization or the image processing model quantization method provided in the above embodiments.

[0108] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the model selection method or image processing model quantization method provided in the above embodiments.

[0109] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0110] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A model screening method for image processing model quantization, characterized in that, The method comprises the following steps: obtaining a target sample image for model training and a plurality of image processing models generated by training; traversing the plurality of image processing models, for a current image processing model, determining weight tensors of each layer structure in the current image processing model, and inputting the target sample image into the current image processing model to obtain activation tensors output by the each layer structure; based on a first ratio of a maximum weight absolute value of all output channels in the weight tensors of the each layer structure to a maximum weight absolute value of each output channel, calculating a weight quantization friendliness score of the current image processing model, and based on a second ratio of a maximum activation absolute value of all channels in the activation tensors output by the each layer structure to a maximum activation absolute value of each channel, calculating an activation quantization friendliness score of the current image processing model; based on the weight quantization friendliness score and the activation quantization friendliness score, calculating a comprehensive quantization friendliness score of the current image processing model, and based on the comprehensive quantization friendliness scores of the plurality of image processing models, screening the plurality of image processing models.

2. The model screening method for image processing model quantization according to claim 1, characterized in that, The method comprises the following steps: traversing the each layer structure, for a current layer structure, based on a first ratio of a maximum weight absolute value of all output channels in the weight tensors of the current layer structure to a maximum weight absolute value of each output channel, determining a weight quantization evaluation index of each output channel; based on the weight quantization evaluation index of each output channel, calculating a first proportion of the number of output channels with the weight quantization evaluation index in a first specified range in the total number of output channels; based on a mean value of the first proportions corresponding to the each layer structure, calculating the weight quantization friendliness score of the current image processing model.

3. The model screening method for image processing model quantization according to claim 2, characterized in that, The method comprises the following steps: for any output channel in the weight tensors of the current layer structure, calculating a logarithm of the first ratio corresponding to the any output channel, and taking the logarithm as the weight quantization evaluation index of the any output channel.

4. The model screening method for image processing model quantization of claim 2, wherein, The first specified range is that the weight quantization evaluation index is greater than or equal to a first threshold value and less than a second threshold value; accordingly, the method comprises the following steps: calculating a difference between 1 and the mean value of the first proportions as the weight quantization friendliness score.

5. The model screening method for image processing model quantization according to any one of claims 1-4, characterized in that, The method comprises the following steps: The activation quantization evaluation index of each channel is determined based on a current second ratio of a maximum activation absolute value of all channels in the activation tensor output by the current layer structure to a maximum activation absolute value of each channel; A second proportion of the number of channels whose activation quantization evaluation index is within a second specified range in the total number of channels of the activation tensor output by the current layer structure is calculated based on the activation quantization evaluation index of each channel; The activation value quantization friendliness score of the current image processing model is calculated based on the average of the second proportions corresponding to the layer structures.

6. The model screening method for image processing model quantization according to claim 5, characterized in that, The activation quantization evaluation index of each channel is determined based on a current second ratio of a maximum activation absolute value of all channels in the activation tensor output by the current layer structure to a maximum activation absolute value of each channel; For any channel in the activation tensor output by the current layer structure, the logarithm of the current second ratio corresponding to the channel is calculated, and the logarithm is taken as the activation quantization evaluation index of the channel.

7. The model screening method for image processing model quantization of claim 5, wherein, The second specified range is that the weight quantization evaluation index is greater than or equal to a third threshold value and less than a fourth threshold value; accordingly, the activation quantization friendliness score of the current image processing model is calculated based on the average of the second proportions corresponding to the layer structures, comprising: The difference between 1 and the average of the second proportions is taken as the activation quantization friendliness score.

8. An image processing model quantization method, characterized in that, Comprising: Obtaining a preset number of to-be-quantized models, the to-be-quantized models being image processing models obtained by screening based on the model screening method for image processing model quantization in any one of claims 1-6; Respectively, each of the to-be-quantized models is subjected to model quantization to obtain each quantized model; Obtaining a quantization test picture, and analyzing the quantization test picture using each of the quantized models to obtain a test result of each of the quantized models; Based on the test results of each of the quantized models, a target quantized model is determined.

9. A model screening device for image processing model quantization, characterized in that, Comprising: A first obtaining module is configured to obtain a target sample image used for model training and a plurality of image processing models generated by training; A tensor determination module is configured to traverse the plurality of image processing models, determine a weight tensor of each layer structure in a current image processing model, and input the target sample image into the current image processing model to obtain an activation tensor output by the layer structure; A first score determination module is configured to calculate a weight quantization friendliness score of the current image processing model based on a first ratio of a maximum weight absolute value of all output channels in the weight tensor of the layer structure to a maximum weight absolute value of each output channel, and calculate an activation quantization friendliness score of the current image processing model based on a second ratio of a maximum activation absolute value of all channels in the activation tensor output by the layer structure to a maximum activation absolute value of each channel. The model screening module is configured to calculate a comprehensive quantization-friendliness score of the current image processing model based on the weight quantization-friendliness score and the activation quantization-friendliness score, and screen the plurality of image processing models based on the comprehensive quantization-friendliness scores of the plurality of image processing models.

10. An image processing model quantization apparatus, characterized by comprising: The method comprises the following steps: The second obtaining module is configured to obtain a preset number of to-be-quantized models, the to-be-quantized models being image processing models screened based on the model screening method for image processing model quantization according to any one of claims 1 to 6; The model quantization module is configured to perform model quantization on each of the to-be-quantized models to obtain a quantized model; The model testing module is configured to obtain a quantization test picture, analyze the quantization test picture using each of the quantized models, and obtain a test result of each of the quantized models; The model determination module is configured to determine a target quantized model based on the test result of each of the quantized models.

11. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the model screening method for image processing model quantization according to any one of claims 1 to 7, or the image processing model quantization method according to claim 8.

12. A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the model screening method for image processing model quantization according to any one of claims 1 to 7, or the image processing model quantization method according to claim 8.

Images