A robust model compression method for distributed outdoor scene image recognition

Abstract

The application discloses a robust model compression method for distributed external scene image recognition. On one hand, the model parameters are fine-tuned using training data, and the distributed internal sample representation capability is inherited. On the other hand, a method for avoiding overfitting is combined, and sparse optimization is used to screen parameters to ensure the generalization capability of the distributed external samples. The technology is easy to implement and has strong universality. The main steps include: in the pre-pruning stage, a Dropout is added before the feature processing part and the final classification layer, and an L1 regularization term is introduced to adjust the original model structure, so that the sparsity is enhanced, and overfitting caused by subsequent pruning is avoided; in the pruning stage, the Dropout layer is shielded, and the Taylor expansion formula is used to evaluate the influence of all parameters on the model loss before and after being set to zero as an importance index; based on the evaluation result and the compression ratio, the parameters with the lowest importance are removed; finally, the model performance is recovered through fine-tuning. The evaluation parameters and fine-tuning are carried out at the same time, and the importance change brought by the parameter adjustment is adapted according to the current parameters.

Description

Technical Field

[0001] This invention relates to a robust model compression method for out-of-distribution scene image recognition, belonging to the fields of machine learning technology and image classification application technology. Background Technology

[0002] Deep learning-based image recognition models have made significant progress in numerous application tasks, such as image classification, object detection, and image segmentation. These achievements are attributed to both the widespread availability of training data and the ability of model architectures to handle large numbers of parameters. However, the sheer volume of model parameters leads to a significant increase in computational and storage costs. Reducing costs, efficiently storing data, and rapidly completing model inference remain challenging problems.

[0003] To address these challenges, neural network compression is a mainstream technique for image recognition models, focusing on efficient model storage and inference. A representative technique is model pruning algorithms, which aim to leverage the inherent sparsity of neural networks to remove "less important" weight parameters, thereby reducing the number of parameters and computational load. Specifically, model pruning algorithms accelerate model inference and deployment by minimizing hardware storage and computational requirements, while simultaneously maintaining the accuracy of the model in recognizing images as much as possible.

[0004] However, numerous experiments have shown that as the compression ratio (pruning ratio) increases, while existing pruning algorithms can maintain the accuracy of image recognition models on independent and identically distributed data, their performance is often poor when dealing with out-of-distribution samples in open scenes. They suffer from transfer difficulties and insufficient generalization, which seriously affects the versatility of the model in downstream scenarios. Summary of the Invention

[0005] Purpose of the Invention: Analysis of the shortcomings of existing technologies revealed a key factor: limited observation data. This causes the model pruning process to retain only the feature recognition ability of training image data, making it difficult to consider the feature recognition ability of test samples. This results in performance degradation and overfitting, hindering robust generalization to tasks outside the training data. To address the problem of limited application scenarios for image recognition models after pruning in existing technologies, a two-stage model pruning framework, TSP, is proposed. This framework aims to optimize the model structure and enhance sparsity through pre-pruning, while preserving the original model's generalization ability, thus making it suitable for other downstream tasks.

[0006] Technical solution: A robust model compression method for out-of-distribution scene image recognition, comprising:

[0007] 1) Obtain the original image recognition model and dataset to be pruned. The training method of the original image recognition model includes known loss calculation formulas, and the dataset can be a regular dataset or a multi-domain dataset.

[0008] 2) Adjust the structure of the original image recognition model by adding Dropout before the feature processing and final classification layers of the original image recognition model and introducing L1 as the target loss for fine-tuning in the pre-pruning stage;

[0009] 3) Disable the Dropout layer and use Taylor expansion to evaluate the impact of all parameters on the image recognition model loss before and after setting them to zero as an importance indicator;

[0010] 4) Based on the evaluation results, gradually screen the parameters with the lowest importance until the pruning rate requirement is met;

[0011] 5) The final image recognition model is obtained by fine-tuning the recovery performance.

[0012] The image recognition model can be any other recognition model.

[0013] The training method and necessary training details of the original image recognition model are known, so that they can be used to restore the performance of the image recognition model later.

[0014] The training dataset can be a regular dataset such as ImageNet, or a multi-domain dataset such as PACS.

[0015] The Dropout layer should be placed after the batch normalization layer and before the linear classification layer to ensure the correct parameter normalization relationship.

[0016] The steps for using Taylor expansion to evaluate the impact of all parameters on the change in model loss before and after setting them to zero as an importance indicator are as follows:

[0017] 31) Use the classification loss plus the Coral regularization term as the loss in the evaluation criterion;

[0018] 32) Use Taylor expansion to approximate the change in loss before and after setting the parameters to zero;

[0019] 33) Select the parameter with the least change, i.e. the "least important parameter", as the parameter to be pruned in the current step.

[0020] A robust model compression device for out-of-distribution scene image recognition, the device comprising:

[0021] The acquisition unit is used to acquire the original image recognition model and dataset to be pruned. The training method of the original image recognition model, including the loss calculation formula, is known. The dataset can be a regular dataset or a multi-domain dataset.

[0022] The first reconstruction unit is used to modify the original image recognition model structure and add a Dropout layer to adjust the relationship between different parameters.

[0023] The first fine-tuning unit is used to fine-tune the image recognition model. It introduces the L1 norm on the basis of the original training loss. On the one hand, as a typical overfitting mitigation method, it works together with the Dropout layer to help the image recognition model learn general category features, thereby enhancing reliability and generalization ability. On the other hand, the sparse optimization of the L1 norm facilitates subsequent parameter selection and pruning.

[0024] The first pruning unit is used to perform pruning compression according to the given pruning rate. It first evaluates the impact of each parameter on the loss before and after setting it to zero as an importance index, and gradually eliminates the parameters with the least impact until the given pruning rate requirement is met.

[0025] The second fine-tuning unit fine-tunes the pruned image recognition model to restore performance. For the domain generalization task and corresponding dataset, the fine-tuned loss is set to the original training loss plus a Coral constraint term to balance the loss gradients across different domains. The final model's generalization ability is close to that of the original model.

[0026] The implementation process and method of the device are the same and will not be described again.

[0027] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the robust model compression method for out-of-distribution scene image recognition as described above.

[0028] A computer-readable storage medium storing a computer program that performs the robust model compression method for out-of-distribution scene image recognition as described above.

[0029] Beneficial Effects: Addressing the issue of reduced generalization ability of image recognition models to out-of-distribution samples after pruning, this invention provides a robust model compression method and apparatus for out-of-distribution scene image recognition, consisting of two stages: a fine-tuning (pre-pruning) stage and a pruning stage including parameter evaluation. Compared to existing technologies, this invention addresses overfitting caused by changes in model structure when relying solely on observation data. It introduces dropout layers in the fine-tuning stage to reduce inter-parameter dependencies, while using sparse optimization to facilitate the subsequent pruning process. This effectively improves the classification ability and generality of the compressed image recognition model for samples of other styles or forms outside the training data. Furthermore, this invention supports the original model training method, is applicable to initial models with different architectures and image recognition tasks, and possesses a certain degree of robustness and versatility. Attached Figure Description

[0030] Figure 1 This is a flowchart of a method according to an embodiment of the present invention;

[0031] Figure 2 This is an overall framework diagram of an embodiment of the present invention;

[0032] Figure 3 This is a flowchart of step (2) of the present invention, which involves modifying and fine-tuning the model structure.

[0033] Figure 4 This is a flowchart of the pruning stage in steps (3), (4), and (5) of the embodiment of the present invention;

[0034] Figure 5 This is a block diagram of the robust model compression device for image recognition in distributed out-of-scene scenarios according to an embodiment of the present invention. Detailed Implementation

[0035] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading the present invention, any modifications of the present invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.

[0036] like Figure 1 As shown, robust model compression methods for out-of-distribution scene image recognition mainly include:

[0037] (1) Obtain the original image recognition model to be pruned and the corresponding dataset. The training method of the original image recognition model, including the loss calculation formula, is known so that it can be used to restore the performance of the image recognition model. The dataset can be a regular dataset or a multi-domain dataset. In this embodiment of the invention, a multi-domain dataset is used as an example. It contains several image samples of different styles. It is necessary to ensure the performance of the pruned model on the style images that appear in the training data.

[0038] In this embodiment of the invention, the process of acquiring the target image dataset can be carried out according to existing acquisition methods, such as setting up an interface dedicated to target image data to acquire the target dataset.

[0039] (2) Adjusting and optimizing the model structure. Adding the Dropout layer before the batch normalization layer will cause a shift in the variance between training and testing, i.e., the variance statistics with and without Dropout are inconsistent. To avoid this problem, Dropout is added after the batch normalization layer of the original image recognition model. Taking the ResNet50 network as an example, in existing technologies, Dropout is usually only used before the classification layer. In this invention, Dropout with different dropout ratios is introduced before both the convolutional layer and the classification layer to decouple the parameters of each neuron and facilitate subsequent pruning.

[0040] The formula for calculating the loss during the subsequent fine-tuning phase is as follows:

[0041]

[0042] in, The first term represents the training loss of the original model, the second term represents the sum of the absolute values ​​of the model parameters (i.e., the L1 norm), and λ is the regularization coefficient. More complex models have larger L1 norm values. The addition of L1 helps the model minimize training error while maintaining its simplicity, mitigating to some extent the problem of excessively complex remaining parameters after pruning, leading to decreased generalization ability due to fitting the individual characteristics of the samples. The unique sparsity of the L1 regularization term can be considered as "pre-pruning"; even if the lowered parameters are subsequently set to zero, the impact on the classification surface of the sample space will be less disruptive.

[0043] (3) Parameter importance assessment. The calculation formula is as follows:

[0044]

[0045] Among them l I This indicates that when the model parameter is ω, it is in the training set D. tr The loss on the parameter ω to be evaluated i The change in model loss before and after setting the value to zero is the parameter importance assessment result I(ω). i For multi-domain datasets, Adding a Coral term to the training loss of the original model aligns the second-order statistics between different domains to find domain-invariant relationships and filter parameters that are more beneficial to out-of-distribution samples:

[0046]

[0047] in This refers to the Coral regularization term, where d represents the output dimension of the fully connected layer of the model, and C... S C R Let be the feature covariance matrix for different sample domains.

[0048] To avoid the large amount of computation caused by multiple predictions, the Taylor formula can be used to expand the evaluation formula, retaining only the first derivative, i.e., the gradient g. i An approximate evaluation is obtained:

[0049] I(ω i )=(g i ·ω i ) 2

[0050] The parameters that are "least important" based on the evaluation results are the parameters to be pruned.

[0051] (4) Prune to the specified compression ratio. This process is carried out simultaneously with fine-tuning the recovery performance. After several iterations, the evaluation results of all parameters are obtained through the formula in step (3). The n parameters with the least impact are set to zero, and then the fine-tuning iteration continues until the total number of pruned parameters N reaches the pruning rate requirement:

[0052] P = N / M

[0053] Where M represents the total number of parameters in the original image recognition model.

[0054] (5) The final image recognition model is obtained through fine-tuning. After pruning, fine-tuning continues for several epochs to restore performance. The final model is obtained by inputting the original model f and the original dataset D. tr When the pruning ratio requirement P is met, the algorithm of this invention... The resulting image recognition model:

[0055]

[0056] This invention addresses the overfitting problem caused by focusing solely on the original training set during model pruning, which severely limits the model's general applicability. This invention introduces a pre-pruning stage, guiding the enhancement of model sparsity while preserving its ability to recognize common features in the dataset, thus mitigating overfitting; while the process of selecting parameters based on the original dataset ensures model performance within the distribution. Compared to existing pruning and compression algorithms, the image recognition model obtained by this invention has stronger applicability in relatively complex downstream tasks and application scenarios.

[0057] like Figure 5 As shown, the robust model compression device for image recognition in out-of-distribution scenes according to the present invention specifically includes:

[0058] The acquisition unit is used to acquire the original image recognition model and dataset to be pruned. The training method of the original image recognition model, including the loss calculation formula, is known. The dataset can be a regular dataset or a multi-domain dataset.

[0059] The first reconstruction unit is used to modify the original model structure and add a Dropout layer to adjust the relationship between different parameters.

[0060] The first fine-tuning unit is used to fine-tune the model. It introduces the L1 norm on the basis of the original training loss. On the one hand, as a typical overfitting mitigation method, it works together with the Dropout layer to encourage the model to learn common category features, thereby enhancing reliability and generalization ability. On the other hand, the sparse optimization of the L1 norm facilitates subsequent parameter selection and pruning.

[0061] The first pruning unit is used to perform pruning compression according to the given pruning rate. It first evaluates the impact of each parameter on the loss before and after setting it to zero as an importance index, and gradually eliminates the parameters with the least impact until the given pruning rate requirement is met.

[0062] The second fine-tuning unit fine-tunes the pruned model to restore performance. For the domain generalization task and the corresponding dataset, the fine-tuned loss is set to the original training loss plus a Coral constraint term to balance the loss gradients across different domains. The final model's generalization ability is close to that of the original model.

[0063] Furthermore, such as Figure 5 As shown, the first pruning unit includes:

[0064] The acquisition module is used to evaluate and determine the parameters to be pruned;

[0065] Build modules for updating model parameters during the iteration process.

[0066] This example addresses the issue of reduced generalization ability and robustness of deep learning-based image recognition models on out-of-distribution (OOD) data after pruning. It achieves reduced inference costs while maintaining performance in applicable scenarios consistent with the original model. The method and apparatus of this invention are primarily based on the following observations: Existing research indicates that neural network models inherently possess a certain degree of sparsity, a crucial foundation for feasible network pruning; however, existing pruning methods mainly focus on evaluating the performance of pruned models on in-distribution data, neglecting generalization on out-of-distribution (OOD) data. This invention addresses the overfitting problem caused by the disruption and subsequent repair of the classification surface during pruning. It enhances parameter sparsity and the robustness of subsequent pruning by introducing pre-pruning, and then optimizes the parameter selection process using a new importance evaluation score, ensuring the final model is applicable to downstream tasks with different data distributions.

[0067] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0068] Obviously, those skilled in the art should understand that the steps of the robust model compression technology for distributed scene image recognition or the units of the robust model compression device for distributed scene image recognition described in the above embodiments of the present invention can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. Optionally, they can be implemented using computer-executable program code, thereby storing them in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the embodiments of the present invention are not limited to any particular hardware and software combination.

[0069] This example validates the effectiveness on the widely used out-of-distribution generalization datasets PACS and Office Home, containing 7 and 65 classes of samples, respectively. The PACS dataset includes four different image styles: artwork, cartoons, photos, and line drawings. The Office Home dataset includes four styles: artwork, clip art, products, and photos. In the experiment, one style is selected as the final test environment. The original model is trained using the other three styles and then pruned. The accuracy of the final model and the original model in the test environment is compared under different pruning methods. Comparison methods include baseline pruning (no pre-pruning stage, using only the original training loss evaluation parameters), DG pruning algorithm (adding loss variance to evaluate model parameters under different environments), FPGM algorithm, and SWAD algorithm (using parameter-weighted averaging in the final fine-tuning stage). The experimental results show the average accuracy of different methods using the PACS and Office Home datasets with a 50% pruning rate for the ResNet50 model.

[0070] PACS Experiment Results Office Home Experiment Results Standard pruning method 79.47 62.31 DG pruning algorithm 79.70 62.44 FPGM algorithm 63.90 50.73 SWAD algorithm 79.49 62.37 Method of the present invention 80.10 63.34

Claims

1. A robust model compression method for out-of-distribution scene image recognition, characterized in that, include: 1) Obtain the original image recognition model and dataset to be pruned, where the training method of the original image recognition model includes known loss calculation formulas; 2) Adjust the structure of the original image recognition model, add Dropout before the feature processing and final classification layers of the original image recognition model, and introduce an L1 regularization term as the target loss for fine-tuning in the pre-pruning stage. Use the original training data to fine-tune the modified image recognition model. 3) Disable the Dropout layer and use Taylor expansion to evaluate the impact of all parameters on the image recognition model loss before and after setting them to zero as an importance indicator; 4) Based on the evaluation results, gradually screen the parameters with the lowest importance until the pruning rate requirement is met; 5) The final image recognition model is obtained by fine-tuning the recovery performance; The Dropout layer is placed after the batch normalization layer and before the linear classification layer. Dropout with different dropout ratios is introduced before both the convolutional and classification layers to decouple the parameters of each neuron. The Taylor expansion is used to evaluate the impact of all parameters on the change in model loss before and after setting them to zero as an importance indicator. The steps are as follows: 31) Use the classification loss plus the Coral regularization term as the loss in the evaluation criterion; 32) Use Taylor expansion to approximate the change in loss before and after setting the parameters to zero; 33) Select the parameter with the least change, i.e., the "least important parameter", as the parameter to be pruned in the current step; The process of screening the least important parameters is carried out simultaneously with fine-tuning the recovery performance to obtain the adjusted importance index for the current parameters. During the operation, an evaluation is performed every few iterations, removing some parameters until the pruning ratio requirement is met, and then fine-tuning is performed for several epochs.

2. The robust model compression method for out-of-distribution scene image recognition according to claim 1, characterized in that, The training method of the original image recognition model is known.

3. The robust model compression method for out-of-distribution scene image recognition according to claim 1, characterized in that, The training dataset is either the regular ImageNet dataset or the multi-domain PACS dataset.

4. A robust model compression apparatus for implementing the method of any one of claims 1-3 for distributed scene image recognition, characterized in that, The device includes: The acquisition unit is used to acquire the original image recognition model and dataset to be pruned, wherein the training method of the original image recognition model includes known loss calculation formulas; The first reconstruction unit is used to modify the original image recognition model structure and add a Dropout layer to adjust the relationship between different parameters. The first fine-tuning unit is used to fine-tune the image recognition model by introducing the L1 norm on top of the original training loss. The first pruning unit is used to perform pruning compression according to the given pruning rate. It first evaluates the impact of each parameter on the loss before and after setting it to zero as an importance index, and gradually eliminates the parameters with the least impact until the given pruning rate requirement is met. The second fine-tuning unit fine-tunes the pruned image recognition model to restore performance. For the domain generalization task and the corresponding dataset, the fine-tuning loss is set to the original training loss plus a Coral constraint term to balance the loss gradients of different domains.

5. A computer device, characterized in that: The computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the robust model compression method for out-of-distribution scene image recognition as described in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program that performs a robust model compression method for image recognition of out-of-distribution scenes as described in any one of claims 1-3.

Images