Image classification method, and training method and device of image classification model

Abstract

The application relates to an image classification method and a training method and device of an image classification model. The method comprises the following steps: acquiring an image to be classified; inputting the image to be classified into an image classification model and acquiring an image classification result output by the image classification model. The image classification model is generated after a pre-training model is continuously executed for multiple training tasks; each training task is used for performing separate training on a classification head unit in the pre-training model according to a training sample set and a playback sample set of the training task, so as to update weight parameters of the classification head unit for the training task. Since separate training is adopted, only the weight parameters of the classification head unit for the training task are updated, so that the old training task and the new training task can be separated, the influence on the old training task is avoided, overfitting of the playback sample is avoided, and the recognition accuracy of the image classification model generated by training is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to an image classification method and a training method and device of an image classification model. BACKGROUND

[0002] With the development of artificial intelligence (AI) technology, deep learning is increasingly applied in the field of image classification. Continuous learning is a special way of training a deep learning model. Continuous learning allows a deep learning model to sequentially learn multiple training tasks, and cannot see the data of historical training tasks when learning subsequent training tasks.

[0003] Class-incremental learning is an important way of continuous learning. In class-incremental learning, each sample set is considered an independent training task, and accordingly, when predicting, class-incremental learning needs to distinguish all classes that have appeared in all training tasks. Since the model does not see the data of all classes at the same time, class-incremental learning cannot learn features for distinguishing different task classes.

[0004] In related technologies, replay samples are usually used to learn general features. However, since replay samples can have overfitting problems, the recognition ability for pictures of the same class outside the replay samples is weak, thereby leading to poor recognition accuracy of the image classification model generated by class-incremental learning. SUMMARY

[0005] Therefore, it is necessary to provide an image classification method and a training method and device of an image classification model capable of improving the recognition accuracy of the image classification model.

[0006] In a first aspect, the present application provides an image classification method. The method comprises:

[0007] obtaining an image to be classified;

[0008] inputting the image to be classified into an image classification model and obtaining an image classification result output by the image classification model;

[0009] The image classification model is generated by continuously performing multiple training tasks on a pre-trained model; each training task is used to separately train a classification head unit in the pre-trained model according to a training sample set and a replay sample set of the training task to update the weight parameters of the classification head unit for the training task; and the replay sample set is generated according to the trained training sample set.

[0010] In one of the embodiments, before the inputting the image to be classified into the image classification model and obtaining the image classification result output by the image classification model, the method further comprises:

[0011] obtaining a training sample set and a replay sample set of a target training task to be executed currently;

[0012] updating a total number of categories of total sample images according to a number of categories of sample images in the training sample set of the target training task;

[0013] determining a replay loss balance coefficient of the target training task according to the number of categories of sample images in the training sample set of the target training task and the total number of categories of total sample images;

[0014] performing separate training on the classification head unit according to the replay loss balance coefficient of the target training task, the training sample set and the replay sample set, to update weight parameters of the classification head unit for the target training task.

[0015] In one of the embodiments, the determining the replay loss balance coefficient of the target training task according to the number of categories of sample images in the training sample set of the target training task and the total number of categories of total sample images comprises:

[0016] determining a ratio of the number of categories of sample images in the training sample set of the target training task to the total number of categories of total sample images as the replay loss balance coefficient of the target training task.

[0017] In one of the embodiments, the performing separate training on the classification head unit according to the replay loss balance coefficient of the target training task, the training sample set and the replay sample set, to update weight parameters of the classification head unit for the target training task comprises:

[0018] performing a loop process until the number of times of updating the weight parameters of the classification head unit for the target training task is equal to a threshold value;

[0019] wherein, the loop process comprises:

[0020] sampling a preset number of sample images from the training sample set of the target training task to form a training sample subset, and sampling a preset number of sample images from the replay sample set of the target training task to form a replay sample subset;

[0021] determining loss function values of classification prediction of the pre-training model on the training sample subset and the replay sample subset respectively;

[0022] determine a comprehensive loss function value of the pre-training model according to the loss function values of the classification prediction of the pre-training model on the training sample subset and the replay sample subset respectively and the replay loss balancing coefficient of the target training task;

[0023] update the weight parameters of the classification head unit for the target training task according to the comprehensive loss function value of the pre-training model and determine the number of times of updating the weight parameters of the classification head unit for the target training task.

[0024] In one of the embodiments, after the total number of categories of the total sample images is updated according to the number of categories of the sample images in the training sample set of the target training task, the method further comprises:

[0025] determine the extraction ratio of the sample images in the training sample set and the replay sample set of the target training task in the next training task according to the number of categories of the sample images in the training sample set of the target training task and the total number of categories of the total sample images;

[0026] randomly extract the sample images in the training sample set and the replay sample set of the target training task respectively according to the extraction ratio;

[0027] generate the replay sample set of the next training task according to the extracted sample images.

[0028] In one of the embodiments, the pre-training model comprises a preset feature extractor, and the preset feature extractor is used to extract general features for any training task.

[0029] In a second aspect, the present application provides a training method of an image classification model. The method comprises:

[0030] obtain a training sample set and a replay sample set of a target training task to be executed currently, the target training task being any training task corresponding to a pre-training model, and the pre-training model being used to generate the image classification model after continuous training for multiple times;

[0031] update the total number of categories of the total sample images according to the number of categories of the sample images in the training sample set of the target training task;

[0032] determine a replay loss balancing coefficient of the target training task according to the number of categories of the sample images in the training sample set of the target training task and the total number of categories of the total sample images;

[0033] According to the replay loss balance coefficient of the target training task, the training sample set and the replay sample set, the classification head unit in the pre-training model is separately trained to update the weight parameter of the classification head unit for the target training task.

[0034] In a third aspect, the present application provides an image classification device. The device comprises:

[0035] A first obtaining module is configured to obtain an image to be classified.

[0036] An identifying module is configured to input the image to be classified into an image classification model and obtain an image classification result output by the image classification model.

[0037] The image classification model is generated after the pre-training model is continuously trained for multiple training tasks; each training task is used to separately train a classification head unit in the pre-training model according to a training sample set and a replay sample set of the training task, so as to update a weight parameter of the classification head unit for the training task; and the replay sample set is generated according to the trained training sample set.

[0038] In one of the embodiments, the device further comprises:

[0039] A training module is configured to obtain a training sample set and a replay sample set of a target training task to be currently executed, the target training task being any training task corresponding to the pre-training model; update a total number of categories of total sample images according to a number of categories of sample images in the training sample set of the target training task; determine a replay loss balance coefficient of the target training task according to the number of categories of sample images in the training sample set of the target training task and the total number of categories of total sample images; and separately train the classification head unit according to the replay loss balance coefficient of the target training task, the training sample set and the replay sample set, so as to update the weight parameter of the classification head unit for the target training task.

[0040] In one of the embodiments, the training module is specifically configured to determine a ratio of the number of categories of sample images in the training sample set of the target training task to the total number of categories of total sample images as the replay loss balance coefficient of the target training task.

[0041] In one of the embodiments, the training module is specifically configured to perform a loop process until the number of times of updating the weight parameter of the classification head unit for the target training task is equal to a threshold value.

[0042] The loop process comprises:

[0043] sample a preset number of sample images from the training sample set of the target training task to form a training sample subset, and sample a preset number of sample images from the replay sample set of the target training task to form a replay sample subset;

[0044] determine loss function values of classification predictions of the pre-training model on the training sample subset and the replay sample subset respectively;

[0045] determine a comprehensive loss function value of the pre-training model according to the loss function values of the classification predictions of the pre-training model on the training sample subset and the replay sample subset respectively, and a replay loss balancing coefficient of the target training task;

[0046] update the weight parameters of the classification head unit for the target training task according to the comprehensive loss function value of the pre-training model, and determine the number of times of updating the weight parameters of the classification head unit for the target training task.

[0047] In one of the embodiments, the training module is further configured to determine extraction ratios of sample images in a replay sample set of a next training task in the training sample set and the replay sample set of the target training task according to a number of categories of the sample images in the training sample set of the target training task and a total number of categories of the total sample images, extract sample images in the training sample set and the replay sample set of the target training task randomly according to the extraction ratios, and generate the replay sample set of the next training task according to the extracted sample images.

[0048] In one of the embodiments, the pre-training model comprises a preset feature extractor, and the preset feature extractor is configured to extract general features for any training task.

[0049] In a fourth aspect, the present application provides a device for training an image classification model. The device comprises:

[0050] a second obtaining module configured to obtain a training sample set and a replay sample set of a target training task to be executed currently, the target training task being any training task corresponding to a pre-training model, and the pre-training model being configured to generate the image classification model after being trained for a plurality of times successively;

[0051] an updating module configured to update a total number of categories of total sample images of the image classification model according to a number of categories of sample images in the training sample set of the target training task;

[0052] The processing module is configured to determine a replay loss balancing coefficient of the target training task according to a number of categories of sample images in a training sample set of the target training task and a total number of categories of the total sample images; and perform separate training on a classification head unit in the pre-training model according to the replay loss balancing coefficient of the target training task, the training sample set and a replay sample set, so as to update weight parameters of the classification head unit for the target training task.

[0053] In a fifth aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the image classification method in the first aspect or the training method of the image classification model in the second aspect when executing the computer program.

[0054] In a sixth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the image classification method in the first aspect or the training method of the image classification model in the second aspect.

[0055] In a seventh aspect, the present application provides a computer program product. The computer program product comprises a computer program. The computer program is executed by a processor to implement the image classification method in the first aspect or the training method of the image classification model in the second aspect.

[0056] The image classification method, the training method and the device of the image classification model are configured to first acquire a to-be-classified image, then input the to-be-classified image into an image classification model, and acquire an image classification result output by the image classification model. The image classification model is generated by continuously performing a plurality of training tasks on a pre-training model. Each training task is configured to perform separate training on a classification head unit in the pre-training model according to a training sample set and a replay sample set of the training task, so as to update weight parameters of the classification head unit for the training task. The replay sample set is generated according to the trained training sample set. Since the pre-training model is trained according to the training sample set and the replay sample set by using separate training, only the weight parameters of the classification head unit for the training task are updated, so that the old training task and the new training task are separated, the influence on the old training task is avoided, the overfitting of the replay sample is avoided, and the recognition accuracy of the image classification model generated by training is improved. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 An application environment of an image classification method provided by an embodiment of the present application is shown in the following figure;

[0058] Figure 2A flowchart of an image classification method provided by an embodiment of the present application is shown in FIG. 1.

[0059] Figure 3 A flowchart of a training method of an image classification model provided by an embodiment of the present application is shown in FIG. 2.

[0060] Figure 4 A flowchart of a separate training provided by an embodiment of the present application is shown in FIG. 3.

[0061] Figure 5 A flowchart of another image classification method provided by an embodiment of the present application is shown in FIG. 4.

[0062] Figure 6 A structural block diagram of an image classification device provided by an embodiment of the present application is shown in FIG. 5.

[0063] Figure 7 A structural block diagram of a training device of an image classification model provided by an embodiment of the present application is shown in FIG. 6.

[0064] Figure 8 An internal structural diagram of a computer device provided by an embodiment of the present application is shown in FIG. 7. DETAILED DESCRIPTION

[0065] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0066] First, the technical names involved in the present application are explained below.

[0067] The training sample set can be a set composed of a large number of sample images to be trained and class labels of the sample images.

[0068] The playback sample set can be a set composed of a small number of sample images and labels of the sample images reserved from the learned training task.

[0069] The loss function value is used to quantify the gap between the current model and the expected effect. The smaller the loss function, the smaller the gap, and the better the model effect.

[0070] The feature extractor is a part close to the input of the pre-training model except the last layer, which is used to convert the original input image into high-level features for classification by the classification head.

[0071] The related art is described below.

[0072] Generally, training an image classification model with deep learning requires collecting a large number of instances of each class of images to form a training set in advance, and then using a deep model to optimize the entire training set by stochastic gradient descent. This training method will bring the following two problems:

[0073] The first problem is that the training set must be collected before training, but in many online application scenarios, training data is gradually generated during application deployment, which makes the model unable to adapt to changing new requirements, and periodically retraining the model on the expanded training set will bring high cost;

[0074] The second problem is that in edge devices such as Internet of Things devices, it is usually impossible to store all the data generated during deployment for training.

[0075] The above two problems can be solved by continuous learning. Continuous learning, also known as incremental learning, is a special method of training deep learning models. Continuous learning allows a deep model to sequentially learn multiple training tasks and cannot see the data of historical training tasks when learning subsequent training tasks. Therefore, using the continuous learning method, a large-scale training set can be divided into several smaller training sets, and training can be performed on each smaller training set in turn. When all training is complete, the effect of training on the entire training set can be achieved.

[0076] The continuous learning method is divided into task incremental learning and class incremental learning. Both methods can make the model sequentially learn multiple disjoint training sets of classes. The difference is that in task incremental learning, each training set is considered an independent task training, and the model only needs to distinguish between classes in the task when predicting, while class incremental learning requires the model to predict on all classes. Since class incremental learning needs to distinguish between class training in different tasks, it is more difficult than task incremental learning.

[0077] Since the model does not see all class data at the same time, class incremental learning cannot learn features for distinguishing different task classes, and catastrophic forgetting will occur during sequential learning. Catastrophic forgetting refers to when the model is trained on two training sets in sequence, the model can only guarantee the expected performance on the second training set, and will forget the knowledge learned on the first data set, resulting in a significant drop in performance compared to when the first task is just learned. This is because the gradient-based optimization algorithm modifies the already learned weight parameters, causing interference.

[0078] Currently, the methods of class-incremental continual learning mainly include: incremental classifier and representation learning (iCaRL), dark experience replay++ (DER++), and separated-softmax for incremental learning (SS-IL). These class-incremental continual learning methods all use replay samples for learning general features and avoiding forgetting. Replay samples refer to a small number of samples retained from the learned tasks. Because the number is very small compared with the original data set, the space overhead can be ignored.

[0079] Among them, iCaRL mixes the replay samples and the samples of the new training task for training, and uses the nearest center classifier to adapt to the changing features. DER++ stores the prediction probabilities of the model on all classes in addition to the replay samples, which are used as dark knowledge in addition to the labels of the image samples to resist catastrophic forgetting in subsequent training; SS-IL separates the normalization operation of the prediction probability of the classifier into two parts of the new and old training tasks, trying to avoid the prediction bias caused by too few replay samples.

[0080] However, the above-mentioned class-incremental continual learning methods do not solve the problem of overfitting caused by using replay samples, and the recognition ability of the same class pictures other than the replay samples is weak, thereby causing the recognition accuracy of the image classification model generated by class-incremental learning to be poor.

[0081] To solve the above problems, the present application provides an image classification method, a training method and device of an image classification model. The pre-training model is trained by using separate training according to the training sample set and the replay sample set, and only the weight parameters of the classification head unit for the training task are updated, so that the old training task and the new training task are separated, the influence on the old training task is avoided, the overfitting of the replay sample is avoided, and the recognition accuracy of the image classification model generated by training is improved.

[0082] The image classification method provided by the embodiments of the present application can be applied to the application environment as shown in the figure. Figure 1 As shown in the figure, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data required to be processed by the server 104. The data storage system can be integrated on the server 104, or placed on the cloud or other network servers.

[0083] The server 104 can perform a plurality of continuous training tasks to train the pre-trained model, thereby generating the image classification model. In each training task, the server 104 can perform separate training when training the pre-trained model according to the training sample set and the replay sample set, and only update the weight parameters of the classification head unit for the training task. After the server 104 completes the plurality of continuous training tasks and generates the image classification model, the terminal 102 can send an image classification request to the server 104, the image classification request including a to-be-classified image. The server 104 then inputs the to-be-classified image into the image classification model and obtains an image classification result output by the image classification model. Finally, the server 104 can send the image classification result to the terminal 102.

[0084] The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle-mounted device, and the like. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, and the like. The server 104 can be implemented by a stand-alone server or a server cluster composed of multiple servers.

[0085] In an embodiment, as shown in FIG. 1, an image classification method is provided. The method is applied to the server in FIG. 1 for example, and includes S201-S202. Figure 2 Figure 1 S201, obtaining a to-be-classified image.

[0086] S201, obtaining a to-be-classified image.

[0087] In the present application, when an image needs to be identified to determine the category of the image, the server can obtain the to-be-classified image.

[0088] It should be understood that the embodiments of the present application do not limit how to obtain the to-be-classified image. In some embodiments, when a user needs to perform image classification identification, the terminal device can send an image classification request to the server, the image classification request can include the to-be-classified image. The server can obtain the to-be-classified image from the image classification request to perform subsequent image classification identification.

[0089] In other embodiments, the server can receive a to-be-classified image collected in real time and store the to-be-classified image in a database. The user can send an image classification identification instruction to the server through the terminal device to instruct the server to perform image classification identification on the to-be-classified image in a certain time period and / or under a certain scene. Subsequently, the server obtains the to-be-classified image from the database to perform subsequent image classification identification.

[0090] ​It should be understood that the embodiments of the present application do not limit the categories of the image to be classified, and can include two categories or multiple categories.

[0091] For example, the image to be classified can include multiple categories such as traffic signs of different categories. For example, the image to be classified can include multiple categories such as animals of different categories. For example, the image to be classified can also include two categories of head portraits of a certain person and head portraits of a person other than the certain person.

[0092] S202, input the image to be classified into the image classification model, and obtain the image classification result output by the image classification model.

[0093] The image classification model is generated after the pre-training model is continuously executed multiple times. Each training task is used to separately train the classification head unit in the pre-training model according to the training sample set and the replay sample set of the training task, so as to update the weight parameters of the classification head unit for the training task.

[0094] In this step, when the server obtains the image to be classified, the image to be classified can be input into the image classification model, and the image classification result output by the image classification model can be obtained.

[0095] It should be understood that the embodiments of the present application do not limit the type of the above-mentioned pre-training model. For example, the above-mentioned pre-training model can include a region-based convolutional neural network (MaskR-CNN) model, a mobile network (MobileNet), a vision transformer (Vision Transformer) model, etc.

[0096] It should be noted that the pre-training model provided by the embodiments of the present application includes a preset feature extractor, which is used to extract general features for any training task. For example, the server can set the initialization feature extractor of the pre-training model, and extract general features based on fixed parameters θ at each training task.

[0097] The image classification model provided by the embodiments of the present application uses a pre-training model based on large-scale data and sets a fixed feature extractor, so as to provide general features for all training tasks and avoid catastrophic forgetting caused by relearning features.

[0098] It should be understood that the multiple training tasks performed by the above-mentioned pre-training model are independent of each other, and each training task has an independent training sample set. Each training sample set includes multiple sample images and a category label corresponding to each sample image.

[0099] It should be understood that the above playback sample set is generated according to the trained training sample set. After each training task of the pre-trained model, a small number of sample images and their type labels in the training sample set corresponding to the training task can be saved in the playback sample set.

[0100] It should be noted that the embodiments of the present application do not limit the generation of the image classification model based on the pre-trained model training. In some embodiments, the server can first obtain the training sample set and the playback sample set of the current target training task, and the target training task is any training task corresponding to the pre-trained model. Secondly, the server can update the total number of categories of the total sample images according to the number of categories of the sample images in the training sample set of the target training task. Thirdly, the server can determine the playback loss balance coefficient of the target training task according to the number of categories of the sample images in the training sample set of the target training task and the total number of categories of the total sample images. Finally, the server can perform separate training on the classification head unit according to the playback loss balance coefficient of the target training task, the training sample set and the playback sample set, to update the weight parameters of the classification head unit for the target training task.

[0101] Secondly, the training sample set of the target training task can be input to the server by the user, or can be pre-set and stored in the server, and the embodiments of the present application do not limit this. The playback sample set of the target training task can be generated based on the training sample set of the last training task of the target training task.

[0102] In some embodiments, the server can first count the category labels in the training sample set of the target training task, and determine the number of categories of the sample images in the training sample set of the target training task. Subsequently, the server can obtain the total number of categories of the trained sample images before performing the target training task, and add the total number of categories of the trained sample images to the number of categories of the sample images in the training sample set of the target training task, thereby updating the total number of categories of the total sample images.

[0103] For example, the total number of categories of the total sample images is C, the initial value of C is 0, and the number of categories of the sample images in the training sample set of the target training task T is C T Therefore, when performing the target training task, the total number of categories of the total sample images can be updated by C←C T +.

[0104] In some embodiments, after updating the total number of categories of the total sample images according to the number of categories of the sample images in the training sample set of the target training task, the server can further determine the extraction ratio of the sample images in the replay sample set of the next training task in the training sample set and the replay sample set of the target training task according to the number of categories of the sample images in the training sample set of the target training task and the total number of categories of the total sample images. Subsequently, the server can randomly extract the sample images in the training sample set and the replay sample set of the target training task according to the extraction ratio. Finally, the server can generate the replay sample set of the next training task according to the extracted sample images.

[0105] It should be noted that the above extraction ratio is not limited by the embodiments of the present application, and can be determined by the ratio of the difference between the number of categories of the sample images in the training sample set of the target training task and the total number of categories of the total sample images to the total number of categories of the total sample images.

[0106] For example, the server can generate the replay sample set of the next training task by formula (1).

[0107]

[0108] wherein M is the replay sample set, C T is the number of categories of the sample images in the training sample set of the target training task T, C is the total number of categories of the total sample images, N is the upper limit of the number of samples of the replay sample set, D T is the training sample set of the target training task T, and Sample is a random sampling function.

[0109] It should be understood that the above replay loss balancing coefficient is used to give the pre-training model the same learning weight for new and old training tasks, and the embodiments of the present application do not limit how to determine the replay loss balancing coefficient. In some embodiments, the server can determine the ratio of the number of categories of the sample images in the training sample set of the target training task to the total number of categories of the total sample images as the replay loss balancing coefficient of the target training task.

[0110] For example, the server can determine the replay loss balancing coefficient by formula (2).

[0111]

[0112] wherein λ is the replay loss balancing coefficient, C T is the number of categories of the sample images in the training sample set of the target training task T, and C is the total number of categories of the total sample images.

[0113] It should be understood that the embodiments of the present application do not limit how to balance the coefficients according to the replay loss of the target training task, the training sample set and the replay sample set, and separate training of the classification head unit. In some embodiments, the server can continuously extract sample subsets from the training sample set and the replay sample set, respectively, and determine the loss function value of the prediction of the pre-trained model corresponding to the sample subsets. After balancing the loss function values corresponding to the training sample set and the replay sample set by the replay loss balancing coefficient, the comprehensive loss function value is obtained. Subsequently, the weight parameters of the classification head unit for the target training task are updated multiple times by the comprehensive loss function value until the number of updates reaches the upper limit.

[0114] It should be noted that the weight parameters of the target training task can be randomly initialized before separate training.

[0115] For example, the server can perform a loop process until the number of updates of the weight parameters of the classification head unit for the target training task is equal to the number threshold.

[0116] The loop process includes: sampling a preset number of sample images from the training sample set of the target training task to form a training sample subset, and sampling a preset number of sample images from the replay sample set of the target training task to form a replay sample subset; determining the loss function value of the classification prediction of the pre-trained model on the training sample subset and the replay sample subset, respectively; determining the comprehensive loss function value of the pre-trained model according to the loss function value of the classification prediction of the pre-trained model on the training sample subset and the replay sample subset, respectively, and the replay loss balancing coefficient of the target training task; updating the weight parameters of the classification head unit for the target training task according to the comprehensive loss function value of the pre-trained model, and determining the number of updates of the weight parameters of the classification head unit for the target training task.

[0117] The preset number can be set according to actual conditions, for example, 1000, 5000, etc. The number threshold can also be set according to actual conditions, for example, 100 times, 1000 times, etc.

[0118] The image classification method provided by the embodiments of the present application comprises the following steps: first, obtaining an image to be classified; then, inputting the image to be classified into an image classification model and obtaining an image classification result output by the image classification model. The image classification model is generated after a pre-trained model is continuously executed for multiple training tasks. Each training task is used to separately train a classification head unit in the pre-trained model according to a training sample set and a playback sample set of the training task to update weight parameters of the classification head unit for the training task. The playback sample set is generated according to a trained training sample set. Since the pre-trained model is trained according to the training sample set and the playback sample set by using separate training, only the weight parameters of the classification head unit for the training task are updated, so that the old training task and the new training task can be separated, the influence on the old training task is avoided, the overfitting of the playback sample is avoided, and the recognition accuracy of the image classification model generated by training is improved.

[0119] The training of the image classification model is described below. Figure 3 The flowchart of the training method of the image classification model provided by the embodiments of the present application is shown in FIG. 1. Figure 3 The training method of the image classification model comprises the following steps.

[0120] S301, obtaining a training sample set and a playback sample set of a target training task to be executed, the target training task being any training task corresponding to a pre-trained model, the pre-trained model being used to generate an image classification model after being trained for multiple times.

[0121] The multiple training tasks performed by the pre-trained model are independent of each other, and the target training task is any training task corresponding to the pre-trained model. Each training task has an independent training sample set, and each training sample set contains multiple sample images and a class label corresponding to each sample image.

[0122] The playback sample set is generated according to a trained training sample set. After the pre-trained model is trained for one time, a small number of sample images and their type labels in the training sample set corresponding to the training task are saved in the playback sample set.

[0123] S302, updating the total number of classes of the total sample images according to the number of classes of the sample images in the training sample set of the target training task.

[0124] In some embodiments, the server can first count the class labels in the training sample set of the target training task, and determine the number of classes of the sample images in the training sample set of the target training task. Subsequently, the server can obtain the total number of classes of the trained sample images before performing the target training task, and add the total number of classes of the trained sample images to the number of classes of the sample images in the training sample set of the target training task, thereby updating the total number of classes of the total sample images.

[0125] S303, determining the replay loss balancing coefficient of the target training task according to the number of classes of the sample images in the training sample set of the target training task and the total number of classes of the total sample images.

[0126] In some embodiments, the server can determine the ratio of the number of classes of the sample images in the training sample set of the target training task to the total number of classes of the total sample images as the replay loss balancing coefficient of the target training task.

[0127] S304, performing separate training on the classification head unit in the pre-training model according to the replay loss balancing coefficient of the target training task, the training sample set and the replay sample set, to update the weight parameters of the classification head unit for the target training task.

[0128] In some embodiments, the server can perform a loop process until the number of times of updating the weight parameters of the classification head unit for the target training task is equal to the number threshold.

[0129] In some embodiments, the server can perform a loop process until the number of times of updating the weight parameters of the classification head unit for the target training task is equal to the number threshold. Figure 4 As shown in the above loop, S3041-S3046 are included:

[0130] S3041, sampling a preset number of sample images from the training sample set of the target training task to form a training sample subset, and sampling a preset number of sample images from the replay sample set of the target training task to form a replay sample subset.

[0131] In some embodiments, the server can determine the number of classes of the sample images in the training sample set of the target training task, and determine the number of classes of the sample images in the replay sample set of the target training task.

[0132] For example, if the preset number is B, the server can sample B sample images from the training sample set D T of the target training task to form a training sample subset {(x1, y1), (x2, y2), …, (xB, yB)}, and sample B sample images from the replay sample set M to form a replay sample subset B . B

[0133] In some embodiments, the server can determine the number of classes of the sample images in the training sample set of the target training task, and determine the number of classes of the sample images in the replay sample set of the target training task. B B ​​a class label of the Bth sample image in the training sample subset, a class label of the Bth sample image in the playback sample subset, a class label of the Bth sample image in the playback sample subset.

[0134] S3042, determine the loss function value of the classification prediction of the pre-training model on the training sample subset and the playback sample subset respectively.

[0135] In some embodiments, the server can first calculate the classification prediction probability of the pre-training model on the training sample subset and the classification prediction probability on the playback sample subset respectively. Then, the server calculates the loss function value of the classification prediction of the pre-training model on the training sample subset and the playback sample subset respectively based on the classification prediction probability of both.

[0136] For example, the server can determine the classification prediction probability of the pre-training model on the training sample subset by formula (3).

[0137]

[0138] wherein p i is the classification prediction probability of the ith sample image in the training sample subset, is the transpose matrix of W 1：T , W 1：T is the weight parameter of the classification head unit in the first T training tasks; h θ (x i ) is the feature vector output by the pre-training model with input parameter θ for the ith sample image x i in the training sample subset; π(z) is a vector normalization function, defined as

[0139] Correspondingly, the server can determine the classification prediction probability of the pre-training model on the playback sample subset by formula (4).

[0140]

[0141] wherein, is the classification prediction probability of the ith sample image in the playback sample subset; is the feature vector output by the pre-training model with input parameter θ for the ith sample image in the playback sample subset.

[0142] For example, the server can determine the loss function value of the classification prediction of the pre-training model on the training sample subset by formula (5), and determine the loss function value of the classification prediction of the pre-training model on the playback sample subset by formula (6).

[0143]

[0144]

[0145] wherein, is a loss function value of the classification prediction of the pre-trained model on the training sample subset, is a loss function value of the classification prediction of the pre-trained model on the replay sample subset.

[0146] S3043, determining a comprehensive loss function value of the pre-trained model according to the loss function values of the classification predictions of the pre-trained model on the training sample subset and the replay sample subset respectively, and the replay loss balancing coefficient of the target training task.

[0147] In some embodiments, the server can balance the loss function values of the classification predictions of the pre-trained model on the training sample subset and the replay sample subset respectively by the replay loss balancing coefficient of the target training task, so as to determine the comprehensive loss function value of the pre-trained model.

[0148] For example, the server can determine the comprehensive loss function value of the pre-trained model by formula (7).

[0149]

[0150] wherein, is a comprehensive loss function value of the pre-trained model, is a loss function value of the classification prediction of the pre-trained model on the training sample subset, is a loss function value of the classification prediction of the pre-trained model on the replay sample subset.

[0151] S3044, updating the weight parameters of the classification head unit for the target training task according to the comprehensive loss function value of the pre-trained model, and determining the number of times of updating the weight parameters of the classification head unit for the target training task.

[0152] It should be understood that the embodiments of the present application do not limit how to update the weight parameters of the classification head unit for the target training task, and in some embodiments, the weight parameters of the target training task can be updated using gradient descent.

[0153] For example, the server can update the weight parameters of the classification head unit for the target training task by formula (8).

[0154]

[0155] wherein, W T is a weight parameter of the classification head unit for the target training task, and η is a learning rate, loss function with respect to the matrix W T partial derivative.

[0156] S3045, determine whether the number of times of updating the weight parameter of the classification head unit for the target training task is equal to the number threshold.

[0157] If yes, end the loop, and if no, perform S3041.

[0158] The number threshold can also be set according to actual conditions, for example, 100 times, 1000 times, etc.

[0159] S3046, complete the separate training of the classification head unit in the pre-training model by the target training task.

[0160] In the embodiments of the present application, the pre-training model using large-scale data and the fixed feature extractor can provide general features for all tasks, avoiding the catastrophic forgetting caused by relearning features. Moreover, the learning of the new training task is separated from the old training task, and only the classification head of the new task is updated to avoid affecting the old task. In addition, the replay loss balancing coefficient is introduced to give the pre-training model the same learning weight for new and old training tasks, avoiding the pre-training model from being biased towards outputting the class of the new training task.

[0161] In one embodiment, as shown in Figure 5 Another image classification method is provided, including the following steps S501-S509:

[0162] S501, obtaining a training sample set and a replay sample set of a current target training task to be executed, the target training task being any training task corresponding to the pre-training model.

[0163] S502, updating the total number of categories of total sample images according to the number of categories of sample images in the training sample set of the target training task.

[0164] S503, determining the replay loss balancing coefficient of the target training task according to the number of categories of sample images in the training sample set of the target training task and the total number of categories of total sample images.

[0165] S504, performing separate training of the classification head unit according to the replay loss balancing coefficient of the target training task, the training sample set and the replay sample set, to update the weight parameter of the classification head unit for the target training task.

[0166] S505, determining whether the pre-training model is trained.

[0167] If yes, perform S507, and if no, perform S506.

[0168] S506. Generate the replay sample set for the next training session.

[0169] After step S506, step S501 is executed.

[0170] S507. Determine the generated image classification model.

[0171] S508. Obtain the image to be classified.

[0172] S509. Input the image to be classified into the image classification model and obtain the image classification result output by the image classification model.

[0173] The image classification method provided in this application first acquires an image to be classified, then inputs the image to be classified into an image classification model, and obtains the image classification result output by the image classification model. The image classification model is generated by continuously executing multiple training tasks on a pre-trained model. Each training task is used to perform separate training on the classification head unit in the pre-trained model based on the training sample set and the playback sample set of that training task, thereby updating the weight parameters of the classification head unit for that training task. The playback sample set is generated based on the already trained training sample set. Because separate training is used when training the pre-trained model based on the training sample set and the playback sample set, only the weight parameters of the classification head unit for that training task are updated. This separates the old training tasks from the new training tasks, avoiding the influence of the old training tasks, thus avoiding overfitting of the playback samples and improving the recognition accuracy of the trained image classification model.

[0174] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0175] Based on the same inventive concept, this application also provides an image classification apparatus for implementing the image classification method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image classification apparatus embodiments provided below can be found in the limitations of the image classification method described above, and will not be repeated here.

[0176] In one embodiment, such as Figure 6 As shown, an image classification device 600 is provided, including: a first acquisition module 601, a recognition module 602, and a training module 603, wherein:

[0177] The first acquisition module 601 is used to acquire the image to be classified;

[0178] The recognition module 602 is used to input the image to be classified into the image classification model and obtain the image classification result output by the image classification model;

[0179] The image classification model is generated by continuously executing multiple training tasks on the pre-trained model. Each training task is used to perform separate training on the classification head unit in the pre-trained model based on the training sample set and the replay sample set of that training task, so as to update the weight parameters of the classification head unit for that training task. The replay sample set is generated based on the already trained training sample set.

[0180] In one embodiment, the image classification device 600 further includes:

[0181] The training module 603 is used to obtain the training sample set and replay sample set of the target training task to be executed. The target training task is any training task corresponding to the pre-trained model. Based on the number of categories of the sample images in the training sample set of the target training task, the total number of categories of the total sample images is updated. Based on the number of categories of the sample images in the training sample set of the target training task and the total number of categories of the total sample images, the replay loss balance coefficient of the target training task is determined. Based on the replay loss balance coefficient of the target training task, the training sample set and the replay sample set, the classification head unit is separately trained to update the weight parameters of the classification head unit for the target training task.

[0182] In one embodiment, the training module 603 is specifically used to determine the replay loss balance coefficient of the target training task as the ratio of the number of categories of sample images in the training sample set of the target training task to the total number of categories of the total sample images.

[0183] In one embodiment, the training module 603 is specifically configured to perform a loop process until the number of times the weight parameters of the classification head unit are updated for the target training task is equal to the number of updates threshold.

[0184] The cyclic process includes:

[0185] A training sample subset is formed by sampling a predetermined number of sample images from the training sample set of the target training task, and a replay sample subset is formed by sampling a predetermined number of sample images from the replay sample set of the target training task.

[0186] Determine the loss function values ​​for the classification predictions of the pre-trained model on the training sample subset and the playback sample subset, respectively;

[0187] The comprehensive loss function value of the pre-trained model is determined based on the loss function values ​​of the classification predictions of the pre-trained model on the training sample subset and the replay sample subset, respectively, and the replay loss balance coefficient of the target training task.

[0188] Based on the comprehensive loss function value of the pre-trained model, update the weight parameters of the classification head unit for the target training task, and determine the number of times the weight parameters of the classification head unit for the target training task are updated.

[0189] In one embodiment, the training module 603 is further configured to determine the extraction ratio of sample images in the replay sample set of the next training task in the training sample set and the replay sample set of the target training task based on the number of categories of sample images in the training sample set of the target training task and the total number of categories of the total sample images; randomly extract sample images from the training sample set and the replay sample set of the target training task according to the extraction ratio; and generate the replay sample set of the next training task based on the extracted sample images.

[0190] In one embodiment, the pre-trained model includes a preset feature extractor, which is used to extract general features for any training task.

[0191] Based on the same inventive concept, this application also provides a training apparatus for an image classification model to implement the training method for the image classification model described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image classification apparatus embodiments provided below can be found in the limitations of the image classification model training method described above, and will not be repeated here.

[0192] In one embodiment, such as Figure 7 As shown, a training device 700 for an image classification model is provided, comprising: a second acquisition module 701, an update module 702, and a processing module 703, wherein:

[0193] The second acquisition module 701 is used to acquire the training sample set and replay sample set of the target training task to be executed. The target training task is any training task corresponding to the pre-trained model. The pre-trained model is used to generate an image classification model after multiple consecutive training sessions.

[0194] The update module 702 is used to update the total number of categories of the total sample images of the image classification model based on the number of categories of the sample images in the training sample set of the target training task.

[0195] The processing module 703 is used to determine the replay loss balance coefficient of the target training task based on the number of categories of sample images in the training sample set of the target training task and the total number of categories of the total sample images; and to perform separate training on the classification head unit in the pre-trained model based on the replay loss balance coefficient of the target training task, the training sample set and the replay sample set, so as to update the weight parameters of the classification head unit for the target training task.

[0196] Each module in the aforementioned image classification device and image classification model training device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0197] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this computer device includes a processor, memory, input / output interfaces (I / O), and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The database stores data. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network. When the computer program is executed by the processor, it implements an image classification method or a training method for an image classification model.

[0198] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0199] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described training method for the image classification method or image classification model.

[0200] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described training method for the image classification method or image classification model.

[0201] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described training method for implementing the image classification method or image classification model.

[0202] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0203] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0204] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image classification method, characterized in that, The method includes: Obtain the training sample set and replay sample set of the target training task to be executed, wherein the target training task is any training task corresponding to the pre-trained model; Update the total number of categories of the total sample images based on the number of categories of the sample images in the training sample set of the target training task; The replay loss balance coefficient for the target training task is determined based on the number of categories of sample images in the training sample set of the target training task and the total number of categories of the total sample images. Based on the replay loss balance coefficient of the target training task, the training sample set, and the replay sample set, the classification head unit is subjected to separate training to update the weight parameters of the classification head unit for the target training task. Obtain the image to be classified; The image to be classified is input into the image classification model, and the image classification result output by the image classification model is obtained; The image classification model is generated by continuously executing multiple training tasks on a pre-trained model. Each training task is used to perform separate training on the classification head unit in the pre-trained model based on the training sample set and the replay sample set of that training task, so as to update the weight parameters of the classification head unit for that training task. The replay sample set is generated based on the already trained training sample set. The step of performing separate training on the classification head unit based on the replay loss balance coefficient of the target training task, the training sample set, and the replay sample set to update the weight parameters of the classification head unit for the target training task includes: The loop process continues until the number of times the weight parameters of the classification head unit are updated for the target training task equals the number of updates threshold. The cyclic process includes: A training sample subset is formed by sampling a predetermined number of sample images from the training sample set of the target training task, and a replay sample subset is formed by sampling a predetermined number of sample images from the replay sample set of the target training task. Determine the loss function values ​​for classification predictions of the pre-trained model on the training sample subset and the replay sample subset, respectively; The comprehensive loss function value of the pre-trained model is determined based on the loss function values ​​of the classification predictions of the pre-trained model on the training sample subset and the replay sample subset, respectively, and the replay loss balance coefficient of the target training task. Based on the comprehensive loss function value of the pre-trained model, update the weight parameters of the classification head unit for the target training task, and determine the number of times the weight parameters of the classification head unit for the target training task are updated.

2. The method according to claim 1, characterized in that, The step of determining the playback loss balance coefficient for the target training task based on the number of categories of sample images in the training sample set of the target training task and the total number of categories of the total sample images includes: The ratio of the number of categories of sample images in the training sample set of the target training task to the total number of categories of the total sample images is determined as the playback loss balance coefficient of the target training task.

3. The method according to claim 1, characterized in that, After updating the total number of categories of the total sample images based on the number of categories of the sample images in the training sample set of the target training task, the method further includes: Based on the number of categories of sample images in the training sample set of the target training task and the total number of categories of the total sample images, determine the extraction ratio of sample images in the replay sample set of the next training task in the training sample set and the replay sample set of the target training task. According to the extraction ratio, sample images are randomly extracted from the training sample set and the replay sample set of the target training task, respectively. Based on the extracted sample images, a replay sample set for the next training task is generated.

4. The method according to claim 1, characterized in that, The pre-trained model includes a preset feature extractor, which is used to extract general features for any training task.

5. A training method for an image classification model, characterized in that, The method includes: Obtain the training sample set and replay sample set of the target training task to be executed. The target training task is any training task corresponding to the pre-trained model. The pre-trained model is used to generate the image classification model after multiple consecutive training sessions. Update the total number of categories of the total sample images based on the number of categories of the sample images in the training sample set of the target training task; The replay loss balance coefficient for the target training task is determined based on the number of categories of sample images in the training sample set of the target training task and the total number of categories of the total sample images. Based on the replay loss balance coefficient of the target training task, the training sample set, and the replay sample set, the classification head unit in the pre-trained model is subjected to separate training to update the weight parameters of the classification head unit for the target training task. The step of performing separate training on the classification head unit based on the replay loss balance coefficient of the target training task, the training sample set, and the replay sample set to update the weight parameters of the classification head unit for the target training task includes: The loop process continues until the number of times the weight parameters of the classification head unit are updated for the target training task equals the number of updates threshold. The cyclic process includes: A training sample subset is formed by sampling a predetermined number of sample images from the training sample set of the target training task, and a replay sample subset is formed by sampling a predetermined number of sample images from the replay sample set of the target training task. Determine the loss function values ​​for classification predictions of the pre-trained model on the training sample subset and the replay sample subset, respectively; The comprehensive loss function value of the pre-trained model is determined based on the loss function values ​​of the classification predictions of the pre-trained model on the training sample subset and the replay sample subset, respectively, and the replay loss balance coefficient of the target training task. Based on the comprehensive loss function value of the pre-trained model, update the weight parameters of the classification head unit for the target training task, and determine the number of times the weight parameters of the classification head unit for the target training task are updated.

6. An image classification device, characterized in that, The device includes: The training module is used to obtain the training sample set and replay sample set of the target training task to be executed. The target training task is any training task corresponding to the pre-trained model. Based on the number of categories of the sample images in the training sample set of the target training task, the total number of categories of the total sample images is updated. Based on the number of categories of the sample images in the training sample set of the target training task and the total number of categories of the total sample images, the replay loss balance coefficient of the target training task is determined. Based on the replay loss balance coefficient of the target training task, the training sample set, and the replay sample set, the classification head unit is trained separately to update the weight parameters of the classification head unit for the target training task. The first acquisition module is used to acquire the image to be classified. The recognition module is used to input the image to be classified into the image classification model and obtain the image classification result output by the image classification model; The image classification model is generated by continuously executing multiple training tasks on a pre-trained model. Each training task is used to perform separate training on the classification head unit in the pre-trained model based on the training sample set and the replay sample set of that training task, so as to update the weight parameters of the classification head unit for that training task. The replay sample set is generated based on the already trained training sample set. The training module is specifically used to execute a loop process until the number of times the weight parameters of the classification head unit are updated for the target training task is equal to the number of updates threshold. The cyclic process includes: A training sample subset is formed by sampling a predetermined number of sample images from the training sample set of the target training task, and a replay sample subset is formed by sampling a predetermined number of sample images from the replay sample set of the target training task. The loss function values ​​of the classification prediction of the pre-trained model on the training sample subset and the replay sample subset are determined respectively. Based on the loss function values ​​of the classification prediction of the pre-trained model on the training sample subset and the replay sample subset, and the replay loss balance coefficient of the target training task, the comprehensive loss function value of the pre-trained model is determined. Based on the comprehensive loss function value of the pre-trained model, the weight parameters of the classification head unit for the target training task are updated, and the number of times the weight parameters of the classification head unit for the target training task are updated is determined.

7. A training device for an image classification model, characterized in that, The device includes: The second acquisition module is used to acquire the training sample set and replay sample set of the target training task to be executed. The target training task is any training task corresponding to the pre-trained model. The pre-trained model is used to generate the image classification model after multiple consecutive training sessions. The update module is used to update the total number of categories of the total sample images of the image classification model based on the number of categories of the sample images in the training sample set of the target training task. The processing module is used to determine the replay loss balance coefficient of the target training task based on the number of categories of sample images in the training sample set of the target training task and the total number of categories of the total sample images; and to perform separate training on the classification head unit in the pre-trained model based on the replay loss balance coefficient of the target training task, the training sample set, and the replay sample set, so as to update the weight parameters of the classification head unit for the target training task. The processing module is specifically used to execute a loop process until the number of times the weight parameters of the classification head unit are updated for the target training task is equal to the number of updates threshold. The cyclic process includes: A training sample subset is formed by sampling a predetermined number of sample images from the training sample set of the target training task, and a replay sample subset is formed by sampling a predetermined number of sample images from the replay sample set of the target training task. The loss function values ​​of the classification prediction of the pre-trained model on the training sample subset and the replay sample subset are determined respectively. Based on the loss function values ​​of the classification prediction of the pre-trained model on the training sample subset and the replay sample subset, and the replay loss balance coefficient of the target training task, the comprehensive loss function value of the pre-trained model is determined. Based on the comprehensive loss function value of the pre-trained model, the weight parameters of the classification head unit for the target training task are updated, and the number of times the weight parameters of the classification head unit for the target training task are updated is determined.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 5.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 5.

Images