A model compression method and system for lightweight industrial image processing network

Abstract

The present application provides a model compression method and system for a lightweight industrial image processing network. Among them, multiple industrial image processing network models are obtained; the multiple industrial image processing network models are compressed according to preset compression rules to obtain a compressed lightweight model; the model parameters of the compressed lightweight model are extracted, including convolution kernel size, number of channels, layer depth, etc.; wherein the compression rules include pruning, quantization, and knowledge distillation technology, so that the compressed lightweight model can reduce the demand for computing resources while maintaining sufficient accuracy. The technical solution provided by the present application can significantly reduce the demand for computing resources while ensuring the accuracy of the model.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of graphics processing technology, and in particular to a model compression method and system for a lightweight industrial image processing network. Background Art

[0002] In industrial production environments, image processing tasks are widely used in quality inspection, automated assembly and other links. With the development of Industry 4.0, the requirements for real-time and accuracy are getting higher and higher. Traditional image processing network models are difficult to meet the needs of rapid response in modern industrial sites due to their high computational complexity and large number of parameters. In order to meet this demand, the industry urgently needs a lightweight image processing network model that can not only run quickly on embedded devices, but also maintain sufficient recognition accuracy to meet the high standards of industrial production.

[0003] Existing technologies often suffer from a loss of accuracy when compressing models. Especially in industrial applications, even a slight loss of accuracy may lead to serious consequences. Many compression technologies are only applicable to specific types of data sets or specific tasks and are difficult to adapt to changing industrial environments. Although the model is compressed, there may still be a waste of computing resources due to the lack of an effective multi-objective optimization mechanism. Summary of the invention

[0004] The embodiments of the present application provide a model compression method and system for a lightweight industrial image processing network, so as to solve the problem of poor model compression efficiency leading to waste of computing resources in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a model compression method for a lightweight industrial image processing network, including:

[0006] Obtain multiple industrial image processing network models;

[0007] Compressing the multiple industrial image processing network models according to preset compression rules to obtain compressed lightweight models, wherein the compression rules at least include pruning, quantization, and knowledge distillation technology;

[0008] Extract model parameters of the compressed lightweight model to process the lightweight model according to the model parameters; wherein, processing the lightweight model according to the model parameters at least includes fine-tuning the lightweight model based on the model parameters or deploying the lightweight model based on the model parameters, wherein the model parameters include at least convolution kernel size, number of channels, and layer depth.

[0009] Optionally include:

[0010] Using pruning technology to remove unimportant weight connections in the multiple industrial image processing network models;

[0011] Using quantization technology to reduce the weight accuracy of the multiple industrial image processing network models;

[0012] Using knowledge distillation technology, the adjusted multiple industrial image processing network models are trained through a teacher-student framework to obtain a compressed lightweight model.

[0013] Optionally include:

[0014] Calculating the L1 norm or L2 norm of each weight in the multiple industrial image processing network models, and removing the weight according to the dynamically adjusted threshold;

[0015] The dynamically adjusted threshold T Calculated by the following formula:

[0016]

[0017] in, yes The adjustment factor is Expressed as the average of the absolute values ​​of all weights, yes The adjustment factor is Expressed as the mean of the L2 norm of all weights, yes The adjustment factor is It is expressed as the absolute value of each weight divided by the sum of the maximum absolute values ​​of all weights, w i It is i weights, N is the total number of weights, max (| w |) is the absolute maximum of all weights.

[0018] Optionally, the quantization technique includes a non-uniform quantization technique;

[0019] The method of using quantization technology to reduce the weight accuracy of the plurality of industrial image processing network models includes:

[0020] The weight accuracy of the multiple industrial image processing network models is reduced by using non-uniform quantization technology, wherein the adopted quantization interval is calculated according to the following formula:

[0021]

[0022] in, It is jThe quantization interval of the group weight is used to indicate the distance between adjacent quantized values ​​after quantization. It is the index of the weight grouping. n j It corresponds to j The number of quantization bits of the group, max j ( w ) is the j The maximum value of the group weight, min j ( w ) is the j The minimum value of the group weight, It is the adjustment factor of the quantization interval, which is used to adjust the influence of the quantization interval. w i It is i weights, N is the total number of weights;

[0023] The quantization error used is calculated by the following formula:

[0024]

[0025] in, E j It is j The quantization error of the group weight is used to measure the error caused by the approximation of the weight value by the quantization process. is the adjustment factor of the quantization error, which is used to adjust the influence of the quantization error. j It corresponds to j The number of quantization bits for the group.

[0026] Optionally, it also includes:

[0027] determining a cumulative effect of the quantization error;

[0028] The cumulative effect of the quantization error is calculated by the following formula:

[0029]

[0030] in, E total represents the cumulative effect of quantization error, J is the number of weight groups, v is the adjustment factor for the cumulative effect of quantization error, E j It is j Quantization error of group weights, w i It is i weights, N is the total number of weights, max j ( w ) is the jThe maximum value of the group weight, min j ( w ) is the j Minimum value for group weight.

[0031] Optionally, the using of knowledge distillation technology to train the adjusted multiple industrial image processing network models through a teacher-student framework to obtain a compressed lightweight model includes:

[0032] Determining a cross entropy loss function, the cross entropy loss function is used to measure the difference between the predicted probability distribution of the compressed lightweight model and the adjusted soft labels produced by the multiple industrial image processing network models;

[0033] The adjusted multiple industrial image processing network models are trained using the cross entropy loss function through a teacher-student framework to obtain a compressed lightweight model;

[0034] Among them, the cross entropy loss function is calculated by the following formula:

[0035]

[0036] in, L CE is the cross entropy loss function, y i It is a soft tag. p i is a compressed lightweight model for the category i The predicted probability distribution of C is the number of categories, T is the temperature factor, is the adjustment factor of the cross entropy loss function, is the sum of absolute errors, representing the average deviation between the predicted probability distribution of the compressed lightweight model and the adjusted soft labels produced by the multiple industrial image processing network models, max(y) is the maximum value of the soft label, indicating the maximum probability value output by the multiple industrial image processing network models after adjustment in all categories. min(y) is the minimum value of the soft label, indicating the minimum probability value of the outputs of the adjusted multiple industrial image processing network models in all categories.

[0037] Optionally, the method of training the adjusted multiple industrial image processing network models by using the cross entropy loss function through a teacher-student framework to obtain a compressed lightweight model further includes:

[0038] Determine a multi-objective optimization mechanism, wherein the optimization formula of the multi-objective optimization mechanism is calculated by the following formula:

[0039] in, Represents the minimization optimization objective function , are the parameters of the compressed lightweight model, is the cross entropy loss function, is the zero norm of the model parameters, It is The number of floating point operations of the layer, is the weight coefficient for balancing different objectives, Represents a set of constraints.

[0040] In a second aspect, an embodiment of the present application provides a lightweight industrial image processing network model compression system, including:

[0041] An input module for obtaining multiple industrial image processing network models;

[0042] An acquisition module compresses the multiple industrial image processing network models according to a preset compression rule to obtain a compressed lightweight model, wherein the compression rule at least includes pruning, quantization, and knowledge distillation technology;

[0043] A processing module is used to extract model parameters of the compressed lightweight model to process the lightweight model according to the model parameters.

[0044] In a third aspect, an embodiment of the present application provides a computing device, comprising a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a model compression method for a lightweight industrial image processing network as described in any one of the first aspects.

[0045] In a fourth aspect, an embodiment of the present application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a model compression method for a lightweight industrial image processing network as described in any one of the first aspects.

[0046] In an embodiment of the present application, multiple industrial image processing network models are obtained; the multiple industrial image processing network models are compressed according to preset compression rules to obtain a compressed lightweight model; the model parameters of the compressed lightweight model are extracted, including convolution kernel size, number of channels, layer depth, etc.; wherein the compression rules include pruning, quantization, and knowledge distillation technology, so that the compressed lightweight model can reduce the demand for computing resources while maintaining sufficient accuracy. The technical solution provided by the present application can significantly reduce the demand for computing resources while ensuring the accuracy of the model.

[0047] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0049] Figure 1 A flowchart of a model compression method for a lightweight industrial image processing network provided in an embodiment of the present application;

[0050] Figure 2 A structural schematic diagram of a model compression system for a lightweight industrial image processing network is provided for an embodiment of the present application;

[0051] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0052] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0053] In some of the processes described in the specification and claims of this application and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0054] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0055] Figure 1A flowchart of a model compression method for a lightweight industrial image processing network is provided for an embodiment of the present application, such as Figure 1 As shown, the method includes:

[0056] 101. Obtain multiple industrial image processing network models;

[0057] In this step, "obtaining multiple industrial image processing network models" means selecting multiple neural network models for industrial image processing tasks from an existing model library or from trained models. These models are usually trained and optimized for specific industrial application scenarios. The purpose of obtaining these models is to prepare for subsequent model compression operations.

[0058] Industrial image processing network model: refers to a deep learning model designed for industrial applications, such as defect detection on production lines, part classification, and other tasks. These models usually have high accuracy and robustness to adapt to complex lighting, occlusion, and other factors in industrial environments.

[0059] In the embodiment of this application, we will select several commonly used industrial image processing network models from the existing model library, including Residual Network (ResNet), Mobile Network (MobileNet) and Efficient Network (EfficientNet). These models have been widely used in the field of industrial image processing and have different characteristics and advantages.

[0060] Implementation steps:

[0061] Model selection: First, we need to select multiple industrial image processing network models from the model library. Assume that our model library contains a variety of pre-trained models, such as ResNet-50, MobileNetV2, and EfficientNet-B0. These models have been trained and optimized for specific industrial image processing tasks.

[0062] Model download: Next, we will download the pre-trained weight files of these models. These files are usually stored in formats such as .pth or .h5, and contain all the parameter information of the model. Assuming that we choose ResNet-50 as one of the models, its parameter file name is resnet50_pretrained.pth.

[0063] Model loading: Use appropriate tools or software to load these models. For example, for ResNet-50, we can use the tools or command line interface provided by the deep learning framework to load the model. Assuming that we are using a software platform that supports deep learning model loading, we can load the model in the following ways:

[0064] Open the software platform and enter the model management interface.

[0065] Select the "Import Model" function.

[0066] Browse and select the resnet50_pretrained.pth file you downloaded earlier.

[0067] Click the "Import" button and wait for the model to load.

[0068] Model configuration: For the loaded model, we need to configure it for subsequent model compression operations. The configuration content may include setting the input and output size of the model, specifying the working environment of the model (such as CPU or GPU), selecting the compression target, etc. For example, for ResNet-50, we need to configure its input image size to 224x224 pixels and specify that the model runs on the CPU.

[0069] Model check: After the model configuration is complete, you need to perform a preliminary check on the model to ensure that the model is loaded correctly and can work properly. You can check this by inputting a set of test images and viewing the model output results. For example, input a 224x224 pixel industrial image to see if the model can correctly classify or detect it.

[0070] Assume that the model we choose is ResNet-50, which is pre-trained on the ImageNet dataset and has achieved good performance in industrial image processing tasks. We download the resnet50_pretrained.pth file from the model library and load it in using the deep learning software platform.

[0071] Basic information of the model:

[0072] Model name: Residual Network-50 (ResNet-50)

[0073] Input size: 224x224 pixels

[0074] Output categories: 1000 (ImageNet classification task)

[0075] Number of parameters: about 23.5 million

[0076] Pre-trained weight file: resnet50_pretrained.pth

[0077] Model loading process:

[0078] Open the deep learning software platform and enter the model management interface.

[0079] Select the "Import Model" function.

[0080] Browse and select the resnet50_pretrained.pth file.

[0081] Click the "Import" button and wait for the model to load.

[0082] Model configuration:

[0083] Set the input image size to 224x224 pixels.

[0084] Specifies that the model runs on the CPU.

[0085] The compression goal is to reduce the computational resource requirements while maintaining accuracy.

[0086] Model Checking:

[0087] Input is a 224x224 pixel industrial image.

[0088] Observe the model output to ensure that the model can correctly classify or detect.

[0089] Through the above steps, we have successfully obtained an industrial image processing network model, ResNet-50, and prepared for its subsequent compression operations. The same process can be applied to other models, such as MobileNetV2 and EfficientNet-B0.

[0090] 102. Compress the multiple industrial image processing network models according to a preset compression rule to obtain a compressed lightweight model, wherein the compression rule at least includes pruning, quantization, and knowledge distillation technology;

[0091] Optionally, the step 102 of compressing the multiple industrial image processing network models according to preset compression rules to obtain compressed lightweight models includes: using pruning technology to remove unimportant weight connections in the multiple industrial image processing network models; using quantization technology to reduce the weight accuracy of the multiple industrial image processing network models; using knowledge distillation technology to train the adjusted multiple industrial image processing network models through a teacher-student framework to obtain a compressed lightweight model.

[0092] Optionally, the use of pruning technology in step 102 to remove unimportant weight connections in the multiple industrial image processing network models includes: calculating the L1 norm or L2 norm of each weight in the multiple industrial image processing network models, and removing the weight according to a dynamically adjusted threshold; wherein the dynamically adjusted threshold T Calculated by the following formula:

[0093]

[0094] in, yes The adjustment factor is Expressed as the average of the absolute values ​​of all weights, yes The adjustment factor is Expressed as the mean of the L2 norm of all weights, yes The adjustment factor is It is expressed as the absolute value of each weight divided by the sum of the maximum absolute values ​​of all weights, w i It is i weights, N is the total number of weights, max (| w |) is the absolute maximum of all weights.

[0095] Optionally, the quantization technology in step 102 includes a non-uniform quantization technology; the using of the quantization technology to reduce the weight accuracy of the multiple industrial image processing network models includes: using the non-uniform quantization technology to reduce the weight accuracy of the multiple industrial image processing network models, wherein the adopted quantization interval is calculated according to the following formula:

[0096]

[0097] in, It is j The quantization interval of the group weight is used to indicate the distance between adjacent quantized values ​​after quantization. j is the index of the weight grouping, n j It corresponds to j The number of quantization bits of the group, max j ( w ) is the j The maximum value of the group weight, min j ( w ) is the j The minimum value of the group weight, yes The adjustment factor is used to adjust the influence of the quantization interval. w i It is i weights, N is the total number of weights;

[0098] The quantization error used is calculated by the following formula:

[0099]

[0100] in, E j It is j The quantization error of the group weight is used to measure the error caused by the approximation of the weight value by the quantization process. yes E j The adjustment factor is used to adjust the influence of quantization error, h j It corresponds to j The number of quantization bits for the group.

[0101] Optionally, the step 102 uses the knowledge distillation technology to train the adjusted multiple industrial image processing network models through a teacher-student framework to obtain a compressed lightweight model, including: determining a cross entropy loss function, the cross entropy loss function is used to measure the difference between the predicted probability distribution of the compressed lightweight model and the soft labels produced by the adjusted multiple industrial image processing network models; through the teacher-student framework, using the cross entropy loss function to train the adjusted multiple industrial image processing network models to obtain a compressed lightweight model; wherein the cross entropy loss function is calculated by the following formula:

[0102]

[0103] in, L CE is the cross entropy loss function, y i It is a soft tag. p i is a compressed lightweight model for the category i The predicted probability distribution of C is the number of categories, T is the temperature factor, yes L CE The adjustment factor, is the sum of absolute errors, representing the average deviation between the predicted probability distribution of the compressed lightweight model and the adjusted soft labels produced by the multiple industrial image processing network models, max( y) is the maximum value of the soft label, indicating the maximum probability value of the output of the multiple industrial image processing network models adjusted in all categories, min( y ) is the minimum value of the soft label, which represents the minimum probability value output by the adjusted multiple industrial image processing network models in all categories.

[0104] In this step, "compressing the multiple industrial image processing network models according to preset compression rules to obtain compressed lightweight models" means applying a series of compression techniques, such as pruning, quantization, knowledge distillation, etc., to the selected industrial image processing network models to reduce the computing resource requirements of the models while keeping the accuracy of the models as unchanged as possible. Specifically, it includes the following sub-steps:

[0105] Pruning: Reducing the number of model parameters by removing unimportant weight connections in the model.

[0106] Quantization: Further compresses the model size by reducing the precision of the weights.

[0107] Knowledge distillation: A smaller student model is trained through a teacher-student framework, thereby reducing the model size while maintaining accuracy.

[0108] Pruning: It is a model compression technique that reduces the number of model parameters by removing unimportant weights or neurons in the model, thereby reducing computational costs.

[0109] Quantization: Convert the model’s floating-point weights into a lower-bit representation, thereby reducing the model’s storage space and computing resource requirements.

[0110] Knowledge Distillation (KD): is a technique that guides the student model to learn through a teacher model. Usually the teacher model is larger and more complex than the student model, but the student model can achieve similar or even better performance with the help of the teacher model.

[0111] In the embodiment of this application, we will take a specific industrial image processing network model as an example to show how to use pruning, quantization and knowledge distillation techniques to compress it to obtain a lightweight model. Assume that the model we choose is ResNet-50 (Residual Network, ResNet).

[0112] Pruning techniques remove unimportant weight connections:

[0113] Calculate the L1 norm or L2 norm of each weight in the model and remove weights based on a dynamically adjusted threshold.

[0114] Dynamically adjusted thresholds TCalculated by the following formula:

[0115] in, yes The adjustment factor is Expressed as the average of the absolute values ​​of all weights, yes The adjustment factor is Expressed as the mean of the L2 norm of all weights, yes The adjustment factor is It is expressed as the absolute value of each weight divided by the sum of the maximum absolute values ​​of all weights, w i It is i weights, N is the total number of weights, max (| w |) is the absolute maximum of all weights.

[0116] Assume that there are 10 million weights in the model, and the absolute value of the largest weight is 10. We set , , , then the calculated threshold T as follows:

[0117]

[0118] Quantization techniques reduce weight precision:

[0119] Non-uniform quantization technology is used to reduce the weight accuracy of the model.

[0120] Quantization interval Calculated by the following formula:

[0121]

[0122] in, j is the index of the weight grouping, n j It corresponds to j The number of quantization bits of the group, max j ( w ) is the j The maximum value of the group weight, min j ( w ) is the j The minimum value of the group weight, yes The adjustment factor, w i It is i weights,i is the total number of weights.

[0123] Assumptions j represents the weight of the first group, , , , , then the quantization interval as follows:

[0124]

[0125] Knowledge distillation technology trains lightweight models:

[0126] Determine the cross entropy loss function that measures the difference between the predicted probability distribution of the compressed lightweight model and the soft labels produced by the teacher model.

[0127] The cross entropy loss function is calculated as follows:

[0128] in, y i It is a soft tag. p i is a compressed lightweight model for the category i The predicted probability distribution of C is the number of categories, T is the temperature factor, yes L CE The adjustment factor.

[0129] Assumptions , , , then the cross entropy loss function L CE as follows:

[0130]

[0131] Through the above steps, we successfully compressed the selected industrial image processing network model ResNet-50, obtained a lightweight model, and reduced the demand for computing resources while maintaining accuracy. These steps can adjust parameters and methods according to actual conditions to adapt to different application scenarios and technical requirements.

[0132] 103. Extracting model parameters of the compressed lightweight model to process the lightweight model according to the model parameters;

[0133] Among them, processing the lightweight model according to the model parameters at least includes fine-tuning the lightweight model based on the model parameters or deploying the lightweight model based on the model parameters, wherein the model parameters at least include convolution kernel size, number of channels, and layer depth.

[0134] In this step, "extracting model parameters of the compressed lightweight model to process the lightweight model according to the model parameters" refers to extracting key parameters from the compressed lightweight model and further processing the model according to these parameters. Specifically, this includes fine-tuning or deploying the lightweight model based on the model parameters. Model parameters include at least convolution kernel size, number of channels, layer depth, etc.

[0135] Model parameters: refers to the basic elements that constitute the deep learning model, including but not limited to convolutional kernel size, number of channels, layer depth, etc.

[0136] Kernel size: The size of the filter in the convolutional layer, which determines how the model slides over the input data to extract features.

[0137] Number of channels: The number of filters in each layer, reflecting the width of the model.

[0138] Layer depth: The number of model layers reflects the depth of the model.

[0139] Fine-Tuning: Based on the pre-trained model, continue to train the model using data from a specific field to adapt to new tasks or data sets.

[0140] Deployment: Deploy the trained model to the actual application environment so that it can run on the target platform and perform specific tasks.

[0141] In the embodiments of this application, we will take a specific industrial image processing network model as an example to show how to extract the model parameters of the compressed lightweight model and fine-tune or deploy the model based on these parameters. Assume that the model we choose is ResNet-50 (Residual Network, ResNet), and after pruning, quantization and knowledge distillation technology compression, a lightweight model is obtained.

[0142] Extract model parameters:

[0143] Extract key parameters from the compressed lightweight model, including convolution kernel size, number of channels, layer depth, etc.

[0144] Assume we have the following parameters:

[0145] Convolution kernel size: 3x3;

[0146] Number of channels: 64;

[0147] Layer depth: 18 layers;

[0148] Fine-tuning the model:

[0149] Use domain-specific datasets to fine-tune the compressed lightweight model to adapt to new tasks or datasets.

[0150] The fine-tuning process includes:

[0151] Prepare dataset: Collect and preprocess industrial image data.

[0152] Set fine-tuning parameters: choose a suitable optimizer, learning rate, etc.

[0153] Training model: Use the dataset to iteratively train the model until the model converges.

[0154] Evaluate the model: Evaluate the performance of the model on the validation set.

[0155] Deploy the model:

[0156] Deploy the trained lightweight model to the actual application environment so that it can run on the target platform and perform specific tasks.

[0157] The deployment process includes:

[0158] Import model: Load the compressed lightweight model onto the target platform.

[0159] Configure the environment: Set the necessary operating environment and parameters.

[0160] Application interface: Provides API or other interfaces for external programs to call.

[0161] Performance monitoring: Monitor the performance of the model in actual applications to ensure its stable operation.

[0162] Assume that we have compressed the ResNet-50 model through pruning, quantization, and knowledge distillation techniques and obtained the following parameters:

[0163] Convolution kernel size: 3x3;

[0164] Number of channels: 64;

[0165] Layer depth: 18 layers;

[0166] Fine-tuning the model:

[0167] Prepare the dataset: 1,000 industrial image data were collected and divided into training and validation sets.

[0168] Set fine-tuning parameters: select Adam optimizer, learning rate 0.001, batch size 32.

[0169] Training model: Use the training set to iteratively train the model until the accuracy of the model on the validation set reaches the expected standard.

[0170] Evaluate the model: Evaluate the performance of the model on the validation set to ensure that its accuracy is above 95%.

[0171] Deploy the model:

[0172] Import model: Load the compressed lightweight model into the industrial computer vision system.

[0173] Configure the environment: Set up the necessary operating environment, such as memory allocation, processor usage, etc.

[0174] Application interface: Provides RESTful API for external programs to call to realize image uploading and processing functions.

[0175] Performance monitoring: Monitor the performance of the model in actual applications to ensure its stable operation in industrial environments.

[0176] Through the above steps, we successfully extracted the key parameters of the compressed lightweight model and fine-tuned and deployed the model based on these parameters. These steps can be adjusted according to actual application scenarios and technical requirements to ensure the optimal performance of the model in specific tasks.

[0177] Optionally, it also includes:

[0178] determining a cumulative effect of the quantization error;

[0179] The cumulative effect of the quantization error is calculated by the following formula:

[0180]

[0181] in, represents the cumulative effect of quantization error, J is the number of weight groups, and v is adjustment factor, Ej is the quantization error of the jth group of weights, wi is the ith weight, N is the total number of weights, maxj(w) is the maximum value of the jth group of weights, and minj(w) is the minimum value of the jth group of weights.

[0182] In this step, "determining the cumulative effect of the quantization error" means calculating the cumulative effect of the error introduced during the quantization process in the entire model. The cumulative effect of the quantization error can help us evaluate the impact of the quantization process on the overall performance of the model, so as to better adjust the quantization strategy to achieve higher accuracy and better performance.

[0183] Quantization error: The error caused by the loss of precision when converting floating-point weights to a low-bit representation during the quantization process.

[0184] Cumulative effect: The cumulative effect of quantization error among the layers of the model reflects the impact of the quantization operation on the performance of the entire model.

[0185] Adjustment Factor (v): It is used to adjust the impact of quantization error, helping us better balance the relationship between accuracy loss and model compression.

[0186] In the embodiments of this application, we will take a specific industrial image processing network model as an example to show how to calculate the cumulative effect of quantization error and adjust the quantization strategy based on these results. Assume that the model we choose is ResNet-50 (Residual Network, ResNet), and quantization technology is used in the model compression process.

[0187] Calculate the quantization error for a single group:

[0188] Use non-uniform quantization techniques to reduce the precision of model weights.

[0189] Quantization interval Calculated by the following formula:

[0190] in, j is the index of the weight grouping, n j It corresponds to j The number of quantization bits of the group, max j ( w ) is the j The maximum value of the group weight, min j ( w ) is the j The minimum value of the group weight, yes The adjustment factor, w i It is i weights, N is the total number of weights.

[0191] Quantization Error E j Calculated by the following formula:

[0192] in, yes E j The adjustment factor.

[0193] Calculate the cumulative effect of quantization error:

[0194] The cumulative effect of quantization error can be calculated by the following formula:

[0195] in, represents the cumulative effect of quantization error, J is the number of weight groups, v yes The adjustment factor, E j It is j Quantization error of group weights, w i It is i weights, N is the total number of weights, max j ( w ) is the j The maximum value of the group weight, min j ( w ) is the j Minimum value for group weight.

[0196] Suppose we have an industrial image processing network model that contains multiple weight groups. Suppose we choose the ResNet-50 model and quantize it. The specific steps are as follows:

[0197] Calculate the quantization error for a single group:

[0198] Suppose we have two sets of weights, each with 10,000 weights.

[0199] The maximum value of the first group of weights is 0.5, the minimum value is -0.5, the number of quantization bits is 8, and the adjustment factor , adjustment factor .

[0200] The maximum value of the second group of weights is 0.7, the minimum value is -0.7, the number of quantization bits is 8, and the adjustment factor , adjustment factor .

[0201] The first set of quantization intervals and quantization error as follows:

[0202]

[0203]

[0204] The second set of quantization intervals and quantization error as follows:

[0205]

[0206] Calculate the cumulative effect of quantization error:

[0207] Suppose we have two sets of weights, the number of weight groups , adjustment factor .

[0208] Cumulative Effects of Quantization Error as follows:

[0209]

[0210] Through the above steps, we have successfully calculated the cumulative effect of quantization error and adjusted the quantization strategy based on these results to ensure that the model has minimal performance loss after quantization. These steps can be adjusted according to specific application scenarios and technical requirements to ensure the best performance of the model.

[0211] Optionally, it also includes:

[0212] Determine a multi-objective optimization mechanism, wherein the optimization formula of the multi-objective optimization mechanism is calculated by the following formula:

[0213] in, Represents the minimization optimization objective function , are the parameters of the compressed lightweight model, is the cross entropy loss function, is the zero norm of the model parameters, It is The number of floating point operations of the layer, is the weight coefficient for balancing different objectives, Represents a set of constraints.

[0214] In this step, “determining a multi-objective optimization mechanism” means minimizing the optimization objective function of the compressed lightweight model through a multi-objective optimization formula. , to achieve a balance between model accuracy, model size, and computational efficiency. Specifically, this multi-objective optimization mechanism takes into account the cross entropy loss function, the zero norm of model parameters, and the number of floating-point operations per layer, and ensures the effectiveness of the optimization through a series of constraints.

[0215] Multi-objective Optimization: Multiple objectives are considered simultaneously during the optimization process to find an optimal or suboptimal solution.

[0216] Optimization objective function: The function that needs to be minimized or maximized, used to measure the effect of optimization.

[0217] Cross-Entropy Loss Function ): Measures the difference between the model's predicted probability distribution and the true label, used to supervise model training.

[0218] Zero Norm ): The zero norm of the model parameters, which indicates the number of non-zero parameters in the model and is used to measure the model size.

[0219] Floating Point Operations ): The number of floating-point operations per layer, used to measure the computational complexity of the model.

[0220] Weight Coefficients ): Used to balance the importance of different goals.

[0221] Constraints ): Limit the range or specific properties of model parameters during optimization.

[0222] In this embodiment of the present application, we will take a specific industrial image processing network model as an example to demonstrate how to minimize the optimization objective function through a multi-objective optimization mechanism. , to achieve a balance between model accuracy, model size and computational efficiency. Assume that the model we choose is ResNet-50 (Residual Network, ResNet), and pruning, quantization and knowledge distillation techniques are used in the model compression process.

[0223] Define the optimization objective function:

[0224] Define the optimization objective function for:

[0225]

[0226] in, is the cross entropy loss function The weight coefficient of is zero norm The weight coefficient of is the number of floating point operations The weight coefficient of Represents a set of constraints.

[0227] Set the weight factor:

[0228] Choose the appropriate weight factor To balance model accuracy, model size and computational efficiency.

[0229] Suppose we set .

[0230] Calculate the optimization objective function:

[0231] Calculate the cross entropy loss function

[0232]

[0233] in, y i It is a soft tag. p i is a compressed lightweight model for the category i The predicted probability distribution of C is the number of categories, T is the temperature factor, yes The adjustment factor.

[0234] Assumptions , , , then the cross entropy loss function as follows:

[0235]

[0236] Calculate zero norm :

[0237] Assume that there are 10 million parameters in the model, of which 100,000 are non-zero parameters, then the zero norm 100,000.

[0238] Calculate the number of floating point operations :

[0239] Assume that there are 18 layers in the model and the number of floating point operations per layer is , ,…, .

[0240] Assuming that the number of floating-point operations for each layer is 10 million times, the total number of floating-point operations is 18×10 million times.

[0241] Calculate the value of the optimization objective function:

[0242] Substitute the above numerical calculation to optimize the objective function :

[0243]

[0244] Set up constraints:

[0245] Set a series of constraints to constrain the range of model parameters or specific properties during the optimization process.

[0246] Assume that the constraint condition is that the zero norm of the model parameters does not exceed 150,000, that is, .

[0247] Assume that we have compressed the ResNet-50 model through pruning, quantization, and knowledge distillation techniques and obtained the following parameters:

[0248] Cross Entropy Loss Function : Assume the calculated result is 0.5.

[0249] Zero norm :100 thousand.

[0250] Number of floating point operations : 18 layers, 10 million times per layer.

[0251] Define the optimization objective function

[0252] Define the optimization objective function for:

[0253]

[0254] in, , , .

[0255] Calculate the value of the optimization objective function:

[0256] Substitute the above numerical calculation to optimize the objective function :

[0257]

[0258] Set up constraints:

[0259] Set the constraint that the zero norm of the model parameters does not exceed 150,000. .

[0260] Through the above steps, we have successfully defined the multi-objective optimization mechanism and calculated the optimization objective function through specific numerical calculations. These steps can be adjusted according to specific application scenarios and technical requirements to ensure the best performance of the model in practical applications.

[0261] Figure 2 A structural diagram of a model compression system for a lightweight industrial image processing network is provided for an embodiment of the present application. Figure 2 As shown, the device comprises:

[0262] An input module 21, used to obtain multiple industrial image processing network models;

[0263] An acquisition module 22 compresses the multiple industrial image processing network models according to a preset compression rule to obtain a compressed lightweight model, wherein the compression rule at least includes pruning, quantization, and knowledge distillation technology;

[0264] The processing module 23 is used to extract model parameters of the compressed lightweight model to process the lightweight model according to the model parameters.

[0265] Figure 2 The model compression system of a lightweight industrial image processing network can perform Figure 1 The implementation principle and technical effect of the model compression method of a lightweight industrial image processing network described in the illustrated embodiment will not be repeated. The specific manner in which each module and unit performs operations in the model compression system of a lightweight industrial image processing network in the above embodiment has been described in detail in the embodiment of the method, and will not be elaborated here.

[0266] In one possible design, Figure 2 The model compression system of a lightweight industrial image processing network of the illustrated embodiment can be implemented as a computing device, such as Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0267] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are called and executed by the processing component 32 .

[0268] The processing component 32 is used to: obtain multiple industrial image processing network models; compress the multiple industrial image processing network models according to preset compression rules to obtain compressed lightweight models; extract model parameters of the compressed lightweight models, including convolution kernel size, number of channels, layer depth, etc.; wherein the compression rules include pruning, quantization, and knowledge distillation techniques, so that the compressed lightweight model can reduce the demand for computing resources while maintaining sufficient accuracy.

[0269] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above method. Of course, the processing component may also be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0270] The storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0271] Of course, the computing device may also include other components, such as input / output interfaces, display components, communication components, etc.

[0272] The input / output interface provides an interface between the processing component and the peripheral interface module, which may be an output device, an input device, etc.

[0273] The communication component is configured to facilitate, among other things, wired or wireless communications between the computing device and other devices.

[0274] Among them, the computing device can be a physical device or an elastic computing host provided by a cloud computing platform, etc. In this case, the computing device can refer to a cloud server, and the above-mentioned processing components, storage components, etc. can be basic server resources rented or purchased from the cloud computing platform.

[0275] The present application also provides a computer storage medium storing a computer program, wherein the computer program can achieve the above-mentioned Figure 1 A model compression method for a lightweight industrial image processing network in the illustrated embodiment.

[0276] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0277] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0278] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0279] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A model compression method for a lightweight industrial image processing network, characterized in that: include: Obtain multiple industrial image processing network models; Compressing the multiple industrial image processing network models according to preset compression rules to obtain compressed lightweight models, wherein the compression rules at least include pruning, quantization, and knowledge distillation technology; Extracting model parameters of the compressed lightweight model to process the lightweight model according to the model parameters; wherein the processing of the lightweight model according to the model parameters at least includes fine-tuning the lightweight model based on the model parameters or deploying the lightweight model based on the model parameters, wherein the model parameters at least include convolution kernel size, number of channels, and layer depth; The step of compressing the plurality of industrial image processing network models according to a preset compression rule to obtain a compressed lightweight model includes: Using pruning technology to remove unimportant weight connections in the multiple industrial image processing network models; Using quantization technology to reduce the weight accuracy of the multiple industrial image processing network models; Using knowledge distillation technology, training the adjusted multiple industrial image processing network models through a teacher-student framework to obtain a compressed lightweight model; The method of removing unimportant weight connections in the plurality of industrial image processing network models using pruning technology includes: Calculating the L1 norm or L2 norm of each weight in the multiple industrial image processing network models, and removing the weight according to the dynamically adjusted threshold; The dynamically adjusted threshold T Calculated by the following formula: ； in, yes The adjustment factor is Expressed as the average of the absolute values ​​of all weights, yes The adjustment factor is Expressed as the mean of the L2 norm of all weights, yes The adjustment factor is It is expressed as the absolute value of each weight divided by the sum of the maximum absolute values ​​of all weights, It is weights, N is the total number of weights, is the absolute maximum value of all weights; The quantization technique includes a non-uniform quantization technique; The method of using quantization technology to reduce the weight accuracy of the plurality of industrial image processing network models includes: The weight accuracy of the plurality of industrial image processing network models is reduced by using non-uniform quantization technology, wherein the adopted quantization interval is calculated according to the following formula: ； in, It is The quantization interval of the group weight is used to indicate the distance between adjacent quantized values ​​after quantization. is the index of the weight grouping, It corresponds to The number of quantization bits for the group, It is The maximum value of the group weight, It is The minimum value of the group weight, It is the adjustment factor of the quantization interval, which is used to adjust the influence of the quantization interval. It is weights, is the total number of weights; The quantization error used is calculated by the following formula: ； in, It is The quantization error of the group weight is used to measure the error caused by the approximation of the weight value by the quantization process. It is the adjustment factor of the quantization error, which is used to adjust the influence of the quantization error. It corresponds to The number of quantization bits for the group; The method uses the knowledge distillation technology to train the adjusted multiple industrial image processing network models through a teacher-student framework to obtain a compressed lightweight model, including: Determining a cross entropy loss function, the cross entropy loss function is used to measure the difference between the predicted probability distribution of the compressed lightweight model and the adjusted soft labels produced by the multiple industrial image processing network models; The adjusted multiple industrial image processing network models are trained using the cross entropy loss function through a teacher-student framework to obtain a compressed lightweight model; Among them, the cross entropy loss function is calculated by the following formula: ； in, is the cross entropy loss function, It is a soft tag. is a compressed lightweight model for the category The predicted probability distribution of is the number of categories, is the temperature factor, is the adjustment factor of the cross entropy loss function, is the sum of absolute errors, representing the average deviation between the predicted probability distribution of the compressed lightweight model and the adjusted soft labels produced by the multiple industrial image processing network models, is the maximum value of the soft label, indicating the maximum probability value output by the multiple industrial image processing network models after adjustment in all categories. is the minimum value of the soft label, indicating the minimum probability value of the output of the adjusted multiple industrial image processing network models in all categories.

2. The method according to claim 1, characterized in that Also includes: determining a cumulative effect of the quantization error; The cumulative effect of the quantization error is calculated by the following formula: ； in, represents the cumulative effect of quantization error, is the number of weight groups, is the adjustment factor for the cumulative effect of quantization error, It is Quantization error of group weights, yes weights, is the total number of weights, It is The maximum value of the group weight, It is Minimum value for group weight.

3. The method according to claim 1, characterized in that The method further comprises: training the adjusted multiple industrial image processing network models by using the cross entropy loss function through the teacher-student framework to obtain a compressed lightweight model; Determine a multi-objective optimization mechanism, wherein the optimization formula of the multi-objective optimization mechanism is calculated by the following formula: ； in Represents the minimization optimization objective function , are the parameters of the compressed lightweight model, is the cross entropy loss function, is the zero norm of the model parameters, It is The number of floating point operations of the layer, is the weight coefficient for balancing different objectives, Represents a set of constraints.

4. A lightweight industrial image processing network model compression system, characterized in that: include: An input module for obtaining multiple industrial image processing network models; An acquisition module compresses the multiple industrial image processing network models according to a preset compression rule to obtain a compressed lightweight model, wherein the compression rule at least includes pruning, quantization, and knowledge distillation technology; A processing module, used for extracting model parameters of the compressed lightweight model to process the lightweight model according to the model parameters; The step of compressing the plurality of industrial image processing network models according to a preset compression rule to obtain a compressed lightweight model includes: Using pruning technology to remove unimportant weight connections in the multiple industrial image processing network models; Using quantization technology to reduce the weight accuracy of the multiple industrial image processing network models; Using knowledge distillation technology, training the adjusted multiple industrial image processing network models through a teacher-student framework to obtain a compressed lightweight model; The method of removing unimportant weight connections in the plurality of industrial image processing network models using pruning technology includes: Calculating the L1 norm or L2 norm of each weight in the multiple industrial image processing network models, and removing the weight according to the dynamically adjusted threshold; The dynamically adjusted threshold T Calculated by the following formula: ； in, yes The adjustment factor is Expressed as the average of the absolute values ​​of all weights, yes The adjustment factor is Expressed as the mean of the L2 norm of all weights, yes The adjustment factor is It is expressed as the absolute value of each weight divided by the sum of the maximum absolute values ​​of all weights, It is weights, N is the total number of weights, is the absolute maximum value of all weights; The quantization technique includes a non-uniform quantization technique; The method of using quantization technology to reduce the weight accuracy of the plurality of industrial image processing network models includes: The weight accuracy of the plurality of industrial image processing network models is reduced by using non-uniform quantization technology, wherein the adopted quantization interval is calculated according to the following formula: ； in, It is The quantization interval of the group weight is used to indicate the distance between adjacent quantized values ​​after quantization. is the index of the weight grouping, It corresponds to The number of quantization bits for the group, It is The maximum value of the group weight, It is The minimum value of the group weight, It is the adjustment factor of the quantization interval, which is used to adjust the influence of the quantization interval. It is weights, is the total number of weights; The quantization error used is calculated by the following formula: ； in, It is The quantization error of the group weight is used to measure the error caused by the approximation of the weight value by the quantization process. It is the adjustment factor of the quantization error, which is used to adjust the influence of the quantization error. It corresponds to The number of quantization bits for the group; The method uses the knowledge distillation technology to train the adjusted multiple industrial image processing network models through a teacher-student framework to obtain a compressed lightweight model, including: Determining a cross entropy loss function, the cross entropy loss function is used to measure the difference between the predicted probability distribution of the compressed lightweight model and the adjusted soft labels produced by the multiple industrial image processing network models; The adjusted multiple industrial image processing network models are trained using the cross entropy loss function through a teacher-student framework to obtain a compressed lightweight model; Among them, the cross entropy loss function is calculated by the following formula: ； in, is the cross entropy loss function, It is a soft tag. is a compressed lightweight model for the category The predicted probability distribution of is the number of categories, is the temperature factor, is the adjustment factor of the cross entropy loss function, is the sum of absolute errors, representing the average deviation between the predicted probability distribution of the compressed lightweight model and the adjusted soft labels produced by the multiple industrial image processing network models, is the maximum value of the soft label, indicating the maximum probability value output by the multiple industrial image processing network models after adjustment in all categories. is the minimum value of the soft label, indicating the minimum probability value of the output of the adjusted multiple industrial image processing network models in all categories.

5. A computing device, characterized in that: It comprises a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are used to be called and executed by the processing component to implement a model compression method for a lightweight industrial image processing network as described in any one of claims 1 to 3.

6. A computer storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a computer, a model compression method for a lightweight industrial image processing network as described in any one of claims 1 to 3 is implemented.

Images