Continuous learning image recognition method and device based on model parameters and pruning strategy
The continuous learning image recognition model built through knowledge distillation and pruning strategies solves the problems of catastrophic forgetting and parameter growth, and improves image recognition accuracy and efficiency.
Patent Information
- Application Number
- CN202210063640.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-20
- Publication Date
- 2025-09-30
- Estimated Expiration
- 2042-01-20
AI Technical Summary
Existing continuous learning methods are prone to catastrophic forgetting when faced with new categories of image data, resulting in poor image recognition accuracy and a rapid growth rate of parameters.
A continuous learning method based on model parameters and pruning strategy is adopted. The model is trained through knowledge distillation and pruning strategy is used to remove unimportant convolution kernels. The rehearsal strategy is combined to save some old category data to build a continuous learning image recognition model.
It effectively alleviates the catastrophic forgetting problem, slows down the growth rate of parameters, and improves image recognition accuracy and efficiency.
Smart Images
Figure CN114418094B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method and device for continuous learning image recognition based on model parameters and pruning strategies. Background Art
[0002] Image recognition is an image processing method that distinguishes objects of different categories based on their distinct features. In this field, models are typically trained using historical image data. However, when new image data emerges, this data may no longer be available due to various reasons, such as data loss. Consequently, the model must be retrained with the new data and parameters adjusted to recognize the new data. However, this can cause the model to forget or even completely forget its past tasks, a phenomenon known as catastrophic forgetting. This problem is typically addressed through continuous learning. When new image data emerges, a model is trained using a small amount of, or even no, historical data to achieve performance close to that of a model trained using the full data set. Currently, two main approaches in the field of continuous learning are regularization and rehearsal. Regularization adds a regularization term to the loss function to prevent excessive changes in model parameters, thereby preventing the model from forgetting the characteristics of historical categories. Rehearsal retains a portion of data from previous categories for continuous review. This method, while the model learns new data, mixes historical data with the new data, allowing it to learn new data while also taking into account old data. Although the above methods have alleviated catastrophic forgetting to a certain extent, the accuracy of image recognition is still poor. Summary of the Invention
[0003] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the existing technology and provide a continuous learning image recognition method and device based on model parameters and pruning strategy. This method can effectively alleviate the problem of catastrophic forgetting in the continuous learning process, while slowing down the growth rate of the parameter amount, effectively improving the continuous learning ability and image recognition accuracy.
[0004] In order to achieve the above object, the present invention adopts the following technical solutions:
[0005] In one aspect, the present invention provides a method for continuous learning image recognition based on model parameters and pruning strategy, comprising the following steps:
[0006] Collect newly added category data and historical category data;
[0007] Constructing a continuous learning image recognition model; the continuous learning image recognition model includes multiple feature extractors, multiple FC layers and a NME classifier; the feature extractor is used to extract features of category data; the FC layer is used to filter the features; and the NME classifier is used to classify the features;
[0008] Merge the newly added category data with some of the old category data saved by the Rehearsal strategy to form the training data;
[0009] Input the training data into the continuous learning image recognition model and add feature extractors and FC layers;
[0010] Use knowledge distillation training to continuously learn image recognition models and use pruning strategies to remove unimportant convolution kernels in the newly added feature extractor;
[0011] Use the Rehearsal strategy for newly added category data and historical category data to save some category data;
[0012] The test data is input into the continuous learning image recognition model for testing to obtain the image recognition results.
[0013] As a preferred technical solution, the feature extractors in the continuous learning image recognition model correspond one-to-one to the FC layers, and the number is consistent.
[0014] As a preferred technical solution, the newly added feature extractor and FC layer specifically refer to:
[0015] When new category data is collected in the Tth round, the Tth feature extractor and the corresponding Tth FC layer are added to the continuous learning image recognition model.
[0016] As a preferred technical solution, the continuous learning image recognition model using knowledge distillation training is specifically as follows:
[0017] Fix the model parameters of the first T-1 rounds of feature extractors in the continuous learning image recognition model, and use the weights of the T-1 round feature extractors to initialize the T round feature extractors;
[0018] The T-th round feature extractor is trained on the newly added category data using the cross entropy function, and the knowledge is distilled from the T-1-th round feature extractor to the T-th round feature extractor through the knowledge distillation loss function.
[0019] As a preferred technical solution, the pruning strategy is used to remove unimportant convolution kernels in the newly added feature extractor, specifically:
[0020] Each layer in the new feature extractor will use the BN layer, and its expression is:
[0021] Y=Ax+B
[0022] Where Y represents the output of the BN layer, A and B are both learnable parameters, the dimension is consistent with the number of convolution kernels, and x is the output of the convolution layer in the newly added feature extractor;
[0023] Apply an L1 regularization term to A to make its value approach 0, indicating that the features extracted by the convolutional layer in the newly added feature extractor are not important;
[0024] If the value of A approaches 0, the convolution kernel corresponding to A is removed from the newly added feature extractor.
[0025] As a preferred technical solution, the storage of partial category data is specifically as follows:
[0026] Use the Rehearsal strategy to retain some historical category data;
[0027] Assume that there are K images and N categories in the historical category data, then save K / N images for each category, discard images with more than K / N images in each category, and obtain partial historical category data;
[0028] The newly added category data is sampled using the herding strategy; the newly added category data of each category is input into the continuous learning image recognition model to extract the feature vector;
[0029] Calculate the average value of each category’s feature vector as the category center of the category;
[0030] Select K / N images from the newly added category image data and merge them with part of the historical category data to obtain and save part of the category data for the next training.
[0031] As a preferred technical solution, the image recognition result is obtained by:
[0032] Perform preprocessing operations on the test data;
[0033] The pre-processed test data is input into T feature extractors for feature extraction and output feature vectors;
[0034] The output T feature vectors are input into T FC layers for feature screening and filtering;
[0035] Average the outputs of T FC layers to obtain the final feature vector;
[0036] The final feature vector is input into the NME classifier for classification to obtain the image recognition result.
[0037] As a preferred technical solution, the NME classifier is a classifier based on feature similarity. In the feature space, according to the category center of each category, the similarity between the final feature vector of the test data and the category center is calculated, and finally the test data is classified into the category with the highest similarity.
[0038] On the other hand, the present invention provides a continuous learning image recognition system based on model parameters and pruning strategies, which is applied to the above-mentioned continuous learning image recognition method based on model parameters and pruning strategies, including a data collection module, a model construction module, a data composition module, a model addition module, a training and pruning module, a data storage module and an image recognition module;
[0039] The data collection module is used to collect newly added category data and historical category data;
[0040] The model building module is used to build a continuous learning image recognition model; the continuous learning image recognition model includes multiple feature extractors, multiple FC layers and NME classifiers;
[0041] The data composition module is used to merge the newly added category data with part of the old category data saved by the Rehearsal strategy to form training data;
[0042] The model adds a new module that inputs training data into the continuous learning image recognition model, adding a feature extractor and FC layer;
[0043] The training and pruning module uses knowledge distillation training to continuously learn the image recognition model and uses a pruning strategy to remove unimportant convolution kernels in the newly added feature extractor;
[0044] The data storage module uses the Rehearsal strategy for the newly added category data and the historical category data, saving some category data for next use;
[0045] The image recognition module is used to input the test data into the continuous learning image recognition model for testing to obtain image recognition results.
[0046] In another aspect, the present invention provides a computer-readable storage medium storing a program, which, when executed by a processor, implements the above-mentioned continuous learning image recognition method based on model parameters and pruning strategy.
[0047] Compared with the prior art, the present invention has the following advantages and beneficial effects:
[0048] 1. This invention solves the problem of continuous learning from the perspective of model parameters. It uses knowledge distillation to store knowledge in parameters, eliminating the need to focus on knowledge transfer. This effectively alleviates the problem of catastrophic forgetting during continuous learning and improves continuous learning capabilities and image recognition performance.
[0049] 2. The present invention adopts a pruning strategy to remove unimportant convolution kernels, which slows down the growth rate of the parameter quantity, saves computing costs, and improves image recognition efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0051] Figure 1 Flowchart of a method for continuous learning image recognition based on model parameters and pruning strategy in an embodiment of the present invention;
[0052] Figure 2 This is a schematic diagram of the structure of the newly added feature extractor and FC layer in an embodiment of the present invention;
[0053] Figure 3 4 is a structural diagram of a continuous learning image recognition system based on model parameters and pruning strategy in an embodiment of the present invention;
[0054] Figure 4 A schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention;
[0055] Figure 5 This figure shows the performance test results of an embodiment of the present invention on the CIFAR100 dataset. DETAILED DESCRIPTION
[0056] In order to enable those skilled in the art to better understand the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0057] References to "embodiments" in this application mean that a particular feature, structure, or characteristic described in connection with the embodiment may be included in at least one embodiment of the application. The appearance of this phrase in various places in the specification does not necessarily refer to the same embodiment, nor does it constitute an independent or alternative embodiment that is mutually exclusive of other embodiments. It is understood, both explicitly and implicitly, by those skilled in the art that the embodiments described in this application may be combined with other embodiments.
[0058] Example
[0059] like Figure 1 As shown, this embodiment provides a continuous learning image recognition method based on model parameters and pruning strategy, including the following steps:
[0060] S1. Collect new category data and historical category data;
[0061] S2. Build a continuous learning image recognition model, including multiple feature extractors, multiple FC layers, and NME classifiers. The feature extractors are used to extract features of categorical image data; the FC layers are used to filter features; and the NME classifier is used to classify features.
[0062] S3, merge the newly added category data with some old category data saved by the Rehearsal strategy to form training data;
[0063] S4. Input the training data into the continuous learning image recognition model and add a feature extractor and FC layer;
[0064] S5. Use knowledge distillation training to continuously learn the image recognition model and use pruning strategies to remove unimportant convolution kernels in the newly added feature extractor;
[0065] S6. Use the Rehearsal strategy for newly added category data and historical category data, saving some category data for next use;
[0066] S7. Input the test data into the continuous learning image recognition model for testing to obtain image recognition results.
[0067] More specifically, Figure 2 As shown, the continuous learning image recognition model contains T-1 feature extractors and corresponding FC layers. In step S4, when the T-th round of new category data is collected, the T-th feature extractor and the corresponding T-th FC layer are added to the continuous learning image recognition model;
[0068] More specifically, in step S5, the image recognition model is continuously learned using knowledge distillation training, specifically:
[0069] Fix the model parameters of the first T-1 rounds of feature extractors in the continuous learning image recognition model, and use the weights of the T-1 round feature extractors to initialize the T round feature extractors;
[0070] The T-th round feature extractor is trained on the newly added category data using the cross entropy function, and the knowledge is distilled from the T-1-th round feature extractor to the T-th round feature extractor through the knowledge distillation loss function.
[0071] Existing knowledge distillation distills the knowledge in the teacher model into the student model, so that the performance of the student model is similar to that of the teacher network. Knowledge distillation is usually used for model compression to distill the knowledge in a large model into a small model. In continuous learning, knowledge distillation distills the old knowledge in the old model into the new model. That is, when new types of data appear, the new model is initialized with the weights of the old model. While the new model learns new categories through cross entropy, knowledge distillation is used to distill the knowledge related to the old categories into the new model.
[0072] The present invention creatively applies the knowledge distillation method to model parameters, fixes the model parameters of the first T-1 round feature extractors in the continuous learning image recognition model, uses the weights of the T-1 round feature extractor to initialize the T round feature extractor, performs cross-entropy training on it, and uses the knowledge distillation loss function to distill the knowledge from the old feature extractor to the T round feature extractor, so that the T round feature extractor can distinguish the features of the newly added category data that are more distinctive than the historical category data as much as possible; based on the model parameters, the present invention stores the knowledge in the model parameters, and does not need to pay attention to how the knowledge is transferred from the old feature extractor to the new feature extractor, thereby enhancing the ability of continuous learning.
[0073] More specifically, since the present invention is based on model parameters, the number of model parameters is bound to increase. In order to reduce the growth rate of the number of model parameters, the present invention introduces a pruning strategy in step S5 to remove unimportant convolution kernels in the newly added feature extractor, specifically:
[0074] Each layer in the new feature extractor will use the BN layer, and its expression is:
[0075] Y=Ax+B
[0076] Where Y represents the output of the BN layer, A and B are both learnable parameters, the dimension is consistent with the number of convolution kernels, and x is the output of the convolution layer in the newly added feature extractor;
[0077] Apply an L1 regularization term to A to make its value approach 0, indicating that the features extracted by the convolutional layer in the newly added feature extractor are not important;
[0078] If the value of A approaches 0, the convolution kernel corresponding to A is removed from the newly added feature extractor.
[0079] There is a hyperparameter in the pruning strategy, namely the number of pruned convolution kernels. The number of convolution kernels pruned by the present invention is 50% of the convolution kernels of a single feature extractor, and the number of parameters of a single feature extractor can be reduced by 75%; it is worth mentioning that the present invention is not sensitive to the pruning strategy, and users can completely replace the pruning strategy of the present invention according to personal needs.
[0080] More specifically, in step S6, some category data is saved, specifically:
[0081] Use the Rehearsal strategy to retain some historical category data;
[0082] Assume that there are K images and N categories in the historical category data, then save K / N images for each category, discard images with more than K / N images in each category, and obtain partial historical category data;
[0083] The newly added category data is sampled using the herding strategy; the newly added category data of each category is input into the continuous learning image recognition model to extract the feature vector;
[0084] Calculate the average value of each category’s feature vector as the category center of the category;
[0085] Select K / N images from the newly added category image data and merge them with some historical category data to obtain and save some category data for the next training;
[0086] In this way, the next time new category data arrives, there is no need to use all the historical category data. You only need to combine the partial category data saved by the Reshearsal strategy with the new category data to form the training data to train the model, thus avoiding repeated operations.
[0087] In this embodiment, there are 2000 images in the historical category data, and K / N images are selected from the newly added category data. The average value of the feature vectors of these K / N images is close to the category center of the new category.
[0088] More specifically, in step S7, the image recognition result is obtained, specifically:
[0089] Perform preprocessing operations on the test data;
[0090] The pre-processed test data is input into T feature extractors for feature extraction and output feature vectors;
[0091] The output T feature vectors are input into T FC layers for feature screening and filtering;
[0092] Average the outputs of T FC layers to obtain the final feature vector;
[0093] The final feature vector is input into the NME classifier for classification to obtain the image recognition result.
[0094] The NME classifier in this embodiment is a classifier based on feature similarity. In the feature space, according to the category center of each category, the similarity between the final feature vector of the test data and the category center is calculated, and finally the test data is classified into the category with the highest similarity to obtain the image recognition result.
[0095] Figure 5 This is the implementation result of the performance test of the embodiment of the present invention on the CIFAR100 dataset. "Ours" represents the image recognition performance of this method on the three network structures of ResNet34, WRN, and SENet18. ACC represents the image classification accuracy and variance. Mean represents the average accuracy of 10 categories in 10 rounds of continuous learning. It can be seen that the image recognition performance of this method is very excellent and is not sensitive to the network structure.
[0096] It should be noted that, for the sake of convenience, the aforementioned method embodiments are all expressed as a series of action combinations, but those skilled in the art should know that the present invention is not limited to the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously.
[0097] Based on the same concept as the continuous learning image recognition method based on model parameters and pruning strategies in the above-mentioned embodiment, the present invention also provides a continuous learning image recognition system based on model parameters and pruning strategies, which can be used to execute the above-mentioned continuous learning image recognition method based on model parameters and pruning strategies. For ease of explanation, the structural diagram of the embodiment of the continuous learning image recognition system based on model parameters and pruning strategies only shows the parts related to the embodiment of the present invention. Those skilled in the art will understand that the illustrated structure does not constitute a limitation of the device, and it can include more or fewer components than shown, or combine certain components, or arrange the components differently.
[0098] like Figure 3 As shown, another embodiment of the present invention provides a continuous learning image recognition system based on model parameters and pruning strategy, which includes at least the following modules:
[0099] The data collection module is used to collect newly added category data and historical category data;
[0100] The model building module is used to build a continuous learning image recognition model, including multiple feature extractors, multiple FC layers and NME classifiers;
[0101] The data composition module is used to merge the newly added category data with some old category data saved by the Rehearsal strategy to form training data;
[0102] The new module of the model inputs training data into the continuous learning image recognition model, adding feature extractors and FC layers;
[0103] The training and pruning module uses knowledge distillation training to continuously learn the image recognition model and uses a pruning strategy to remove unimportant convolution kernels in the newly added feature extractor;
[0104] The data preservation module uses the Rehearsal strategy for newly added category data and historical category data, saving some category data for next use;
[0105] The image recognition module is used to input the test data into the continuous learning image recognition model for testing and obtain image recognition results.
[0106] It should be noted that the continuous learning image recognition system based on model parameters and pruning strategies of the present invention corresponds one-to-one to the continuous learning image recognition method based on model parameters and pruning strategies of the present invention. The technical features and beneficial effects described in the above-mentioned embodiment of the continuous learning image recognition method based on model parameters and pruning strategies are applicable to the embodiment of the continuous learning image recognition system based on model parameters and pruning strategies. For specific contents, please refer to the description in the embodiment of the method of the present invention. No further details will be given here. This is hereby declared.
[0107] In addition, in the implementation of the continuous learning image recognition system based on model parameters and pruning strategies in the above-mentioned embodiment, the logical division of each program module is only an example. In actual applications, the above-mentioned functions can be assigned to different program modules as needed, for example, for the configuration requirements of the corresponding hardware or the convenience of software implementation. That is, the internal structure of the continuous learning image recognition system based on model parameters and pruning strategies is divided into different program modules to complete all or part of the functions described above.
[0108] like Figure 4 As shown, in one embodiment, a computer-readable storage medium is provided, which stores a program in a memory. When the program is executed by a processor, the continuous learning image recognition method based on model parameters and pruning strategy is implemented, specifically:
[0109] Collect newly added category data and historical category data;
[0110] Build a continuous learning image recognition model, including multiple feature extractors, multiple FC layers and NME classifiers;
[0111] Merge the newly added category data with some of the old category data saved by the Rehearsal strategy to form the training data;
[0112] Input the training data into the continuous learning image recognition model and add feature extractors and FC layers;
[0113] Use knowledge distillation training to continuously learn image recognition models and use pruning strategies to remove unimportant convolution kernels in the newly added feature extractor;
[0114] Use the Rehearsal strategy for newly added category data and historical category data to save some category data;
[0115] The test data is input into the continuous learning image recognition model for testing to obtain the image recognition results.
[0116] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-described methods. In particular, any reference to memory, storage, database, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory.
[0117] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.
[0118] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.
Claims
1. A continuous learning image recognition method based on model parameters and pruning strategy, characterized by: The steps include: Collect newly added category data and historical category data; Constructing a continuous learning image recognition model; the continuous learning image recognition model includes multiple feature extractors, multiple FC layers and a NME classifier; the feature extractor is used to extract features of category data; the FC layer is used to filter the features; and the NME classifier is used to classify the features; Merge the newly added category data with some of the old category data saved by the Rehearsal strategy to form the training data; Input the training data into the continuous learning image recognition model and add feature extractors and FC layers; The newly added feature extractor and FC layer specifically refer to: When new category data is collected in the Tth round, the Tth feature extractor and the corresponding Tth FC layer are added to the continuous learning image recognition model; Use knowledge distillation training to continuously learn image recognition models and use pruning strategies to remove unimportant convolution kernels in the newly added feature extractor; The continuous learning image recognition model using knowledge distillation training is specifically as follows: Fix the model parameters of the first T-1 rounds of feature extractors in the continuous learning image recognition model, and use the weights of the T-1 round feature extractors to initialize the T round feature extractors; The feature extractor in round T is trained on the newly added category data using the cross entropy function, and the knowledge is distilled from the feature extractor in round T-1 to the feature extractor in round T using the knowledge distillation loss function. The pruning strategy is used to remove unimportant convolution kernels in the newly added feature extractor, specifically: Each layer in the new feature extractor will use the BN layer, and its expression is: Y=Ax+B Where Y represents the output of the BN layer, A and B are both learnable parameters, the dimension is consistent with the number of convolution kernels, and x is the output of the convolution layer in the newly added feature extractor; Apply an L1 regularization term to A to make its value approach 0, indicating that the features extracted by the convolutional layer in the newly added feature extractor are not important; If the value of A approaches 0, the convolution kernel corresponding to A is removed from the newly added feature extractor; Use the Rehearsal strategy for newly added category data and historical category data to save some category data; The test data is input into the continuous learning image recognition model for testing to obtain the image recognition results.
2. The continuous learning image recognition method based on model parameters and pruning strategy according to claim 1 is characterized in that The feature extractors in the continuous learning image recognition model correspond one-to-one to the FC layers, and the number is the same.
3. The continuous learning image recognition method based on model parameters and pruning strategy according to claim 1 is characterized in that The said part of category data is stored as follows: Use the Rehearsal strategy to retain some historical category data; Assume that there are K images and N categories in the historical category data, then save K / N images for each category, discard images with more than K / N images in each category, and obtain partial historical category data; Sample the newly added category data through the herding strategy; Input the newly added category data of each category into the continuous learning image recognition model to extract the feature vector; Calculate the average value of each category’s feature vector as the category center of the category; Select K / N images from the newly added category image data and merge them with part of the historical category data to obtain and save part of the category data for the next training.
4. The continuous learning image recognition method based on model parameters and pruning strategy according to claim 3 is characterized in that The image recognition result is obtained as follows: Perform preprocessing operations on the test data; The pre-processed test data is input into T feature extractors for feature extraction and output feature vectors; The output T feature vectors are input into T FC layers for feature screening and filtering; Average the outputs of T FC layers to obtain the final feature vector; The final feature vector is input into the NME classifier for classification to obtain the image recognition result.
5. The image recognition method based on model parameters and pruning strategy according to claim 4, characterized in that: The NME classifier is a classifier based on feature similarity. In the feature space, according to the category center of each category, the similarity between the final feature vector of the test data and the category center is calculated, and the test data is finally classified into the category with the highest similarity.
6. A continuous learning image recognition system based on model parameters and pruning strategies, characterized by: The continuous learning image recognition method based on model parameters and pruning strategy applied to any one of claims 1-5 comprises a data collection module, a model construction module, a data composition module, a model addition module, a training and pruning module, a data storage module and an image recognition module; The data collection module is used to collect newly added category data and historical category data; The model building module is used to build a continuous learning image recognition model; the continuous learning image recognition model includes multiple feature extractors, multiple FC layers and NME classifiers; The data composition module is used to merge the newly added category data with part of the old category data saved by the Rehearsal strategy to form training data; The model adds a new module that inputs training data into the continuous learning image recognition model, adding a feature extractor and FC layer; The newly added feature extractor and FC layer specifically refer to: When new category data is collected in the Tth round, the Tth feature extractor and the corresponding Tth FC layer are added to the continuous learning image recognition model; The training and pruning module uses knowledge distillation training to continuously learn the image recognition model and uses a pruning strategy to remove unimportant convolution kernels in the newly added feature extractor; The continuous learning image recognition model using knowledge distillation training is specifically as follows: Fix the model parameters of the first T-1 rounds of feature extractors in the continuous learning image recognition model, and use the weights of the T-1 round feature extractors to initialize the T round feature extractors; The feature extractor in round T is trained on the newly added category data using the cross entropy function, and the knowledge is distilled from the feature extractor in round T-1 to the feature extractor in round T using the knowledge distillation loss function. The pruning strategy is used to remove unimportant convolution kernels in the newly added feature extractor, specifically: Each layer in the new feature extractor will use the BN layer, and its expression is: Y=Ax+B Where Y represents the output of the BN layer, A and B are both learnable parameters, the dimension is consistent with the number of convolution kernels, and x is the output of the convolution layer in the newly added feature extractor; Apply an L1 regularization term to A to make its value approach 0, indicating that the features extracted by the convolutional layer in the newly added feature extractor are not important; If the value of A approaches 0, the convolution kernel corresponding to A is removed from the newly added feature extractor; The data storage module uses the Rehearsal strategy for the newly added category data and the historical category data, saving some category data for next use; The image recognition module is used to input the test data into the continuous learning image recognition model for testing to obtain image recognition results.
7. A computer-readable storage medium storing a program, characterized in that: When the program is executed by a processor, the continuous learning image recognition method based on model parameters and pruning strategy described in any one of claims 1 to 5 is implemented.