Image dual classification method and device in online continuous learning scene

Abstract

The invention discloses an image dual classification method and device in an online continuous learning scene, and the method comprises the steps: enabling a memory cache to be mixed with the data of a current new task during the learning of the new task, enabling the mixed data to serve as training data, inputting the training data into a classification model, and outputting the classification prediction probability distribution; inputting the classification prediction probability distribution into a proxy classifier, and outputting proxy prediction probability distribution; calculating classification loss according to proxy prediction probability distribution and a real category label, fixing parameters of a proxy classifier in a training process, and only updating parameters of a classification model so as to minimize a prediction error; training a proxy classifier by using the data in the memory cache to assist the classification model in correcting classification prediction probability distribution; and evaluating the accuracy of the training result by using the average classification precision and the forgetting index. The problem of how to find an optimal balance point between historical task reservation and new task learning in an online continuous learning scene so as to correct the tendency facing the new task in the classification process is solved.

Description

Technical Field

[0001] The present application relates to the field of continuous learning technology, and in particular to an image dual classification method and device in an online continuous learning scenario. Background Art

[0002] Continuous learning (CL) is an important research direction in the field of artificial intelligence. It aims to build intelligent systems that can continuously learn from the continuous influx of new data and tasks while retaining and integrating existing knowledge. This technology breaks through the limitation of traditional machine learning models that stop learning after training, allowing the system to continuously optimize itself in practical applications and adapt to complex and changing environments. As a subset of continuous learning, online continuous learning requires that the model can update parameters in a timely manner when the data stream arrives in real time, so as to quickly adapt to changes in data distribution and ensure the efficiency and real-time performance of the system.

[0003] In continuous learning, especially in online continuous learning scenarios, machine learning models are prone to bias towards new tasks. This problem is mainly manifested in that when the model encounters a new task, it will pay too much attention to the characteristics of the new task, resulting in forgetting or ignoring the knowledge of the old task, and then over-predicting the new task samples in tasks such as classification, seriously damaging the overall classification accuracy. Taking image classification as an example, during the continuous learning process, deep neural networks will classify all newly input image samples into the category of the new task, causing a sharp drop in classification performance. Existing technologies ignore the complexity of the dynamic changes in data distribution in continuous learning scenarios, and it is difficult to find the best balance between retaining historical tasks and learning new tasks.

[0004] Therefore, in the online continuous learning scenario, how to find the best balance between retaining historical tasks and learning new tasks in order to correct the tendency towards new tasks in the classification process has become a problem to be solved. Summary of the invention

[0005] In an embodiment of the present application, by providing an image dual classification method in an online continuous learning scenario, the problem of how to find the best balance between retaining historical tasks and learning new tasks in an online continuous learning scenario to correct the tendency towards new tasks in the classification process is solved.

[0006] In a first aspect, an embodiment of the present application provides an image dual classification method in an online continuous learning scenario, the method comprising: constructing a continuous learning image classification dataset; wherein the continuous learning image classification dataset comprises multiple image classification tasks, each classification task comprises multiple images and their corresponding true category labels; saving a small amount of historical task data as a memory cache, and when learning a new task, mixing the memory cache with the current new task data, and inputting them into the classification model as training data, the classification model outputs a classification prediction probability distribution; iteratively executing an iterative training step until a stopping condition is met; wherein the stopping condition is: reaching a preset number of iterations or the value of the classification loss is reduced to less than a preset threshold; the iterative training step comprises: inputting the classification prediction probability distribution into a proxy classifier, and the proxy classifier outputs a proxy prediction probability distribution; calculating the classification loss according to the proxy prediction probability distribution and the true category label, and fixing the parameters of the proxy classifier during the training process, and only updating the parameters of the classification model to minimize the prediction error; training the proxy classifier with the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; after the iterative training step is completed, using the average classification accuracy and forgetting index to evaluate the accuracy of the training results.

[0007] In one possible implementation, the data of a small amount of historical tasks are saved as a memory cache. When learning a new task, the memory cache is mixed with the current new task data and input into the classification model together as training data. The classification model outputs a classification prediction probability distribution, including: when learning a new task, a batch of data is randomly sampled from the memory cache, and the batch of data is mixed with the data of the current new task to serve as training data; the training data is input into the classification model, and the classification prediction probability distribution corresponding to each category label is output through forward propagation.

[0008] In a possible implementation, the classification prediction probability distribution is input into the proxy classifier, and the proxy classifier outputs the proxy prediction probability distribution, including: the proxy classifier includes a category-specific module and a category-independent module; the category-specific module is used to extract unique information of each category from the classification prediction probability distribution vector of the classification model, and generate a vector of the proxy prediction probability distribution of the corresponding category to model the characteristics of each category; the category-independent module inputs a two-element vector and outputs a corresponding vector of proxy new and old task prediction probabilities; wherein the two-element vector reflects the classification model's predicted probability that a given sample belongs to a new task or a historical task; the proxy classifier combines the vector of the proxy prediction probability distribution with the vector of the proxy new and old task prediction probabilities to output the proxy prediction probability distribution.

[0009] In a possible implementation, updating the parameters of the classification model includes: updating the parameters of the classification model using a stochastic gradient descent algorithm.

[0010] In one possible implementation, the method of training a proxy classifier with data in a memory cache to assist a classification model in correcting a classification prediction probability distribution includes: using the classification model after updating parameters to calculate the classification loss value of another batch of samples randomly selected from the memory cache, and updating the parameters of the proxy classifier during the training of the proxy classifier through the Adam algorithm.

[0011] In a possible implementation, the training process of the classification model and the training process of the proxy classifier use different objective functions respectively; the expression of the objective function used in the classification model training process is: Among them, θ * (ω) is the optimal parameter of the classification model when the parameter of the fixed proxy classifier is ω, argmin is the operator of the parameter that can minimize the function, is the classification loss function of the classification model, is a random sampling of a batch of data and sample data of the new task from the memory cache, B is the number of samples currently trained by the classification model, and x i is the i-th training sample, y i is the i-th true category label, is the classification model f θ Then cascade the proxy classifier g v The augmented classification model of θ is the parameter of the classification model, and ° is the cascade operation; the expression of the objective function used in the training process of the proxy classifier is: Among them, ω * is the optimal parameter of the proxy classifier, is the training loss function of the proxy classifier, is a random sample of a batch of data from the memory cache, B′ is the number of samples currently trained by the proxy classifier, and f θ*(ω) is a classification model in which the parameter θ is set to the optimal value based on the current ω; l(f θ*(ω) (x i ),y i ) is the loss function of a single sample, which is used to calculate the classification model for the training sample x i The predicted result f θ*(ω) (x i ) and the true category label y i The difference between.

[0012] In a possible implementation, the method for obtaining the average classification accuracy and the forgetting index includes: for each task, inputting the samples in its test set into the classification model, and the classification model generates a prediction result for each sample; wherein the test set is divided from the continuous learning image classification data set; if the prediction result of the classification model matches the true category label of the sample, it is regarded as a successful prediction; the prediction accuracy of each task is defined as the number of successful predictions divided by the total number of samples in the test set of the task; the maximum prediction accuracy of each task is the highest prediction accuracy achieved by the task during the entire training process; the average of the maximum prediction accuracy of all tasks is calculated as the average classification accuracy of the classification model; for each task, the difference between its prediction accuracy and the maximum prediction accuracy is calculated, and the difference is used as the forgetting index of the task.

[0013] In a possible implementation, the use of average classification accuracy and forgetting index to evaluate the accuracy of the training result includes: the expression of average classification accuracy is: Among them, ACC is the average classification accuracy, T is the number of all tasks in the training process, and a t,T is the prediction accuracy of the tth task calculated after the training process is completed; the expression of the forgetting index is: Among them, FM is the forgetting index, T is the number of all tasks in the training process, and a * is the maximum prediction accuracy of the t-th task during the training process, a t,T is the prediction accuracy of the tth task calculated after the training process is completed, and T is the number of all tasks in the training process.

[0014] In the second aspect, an embodiment of the present application provides an image dual classification device in an online continuous learning scenario, the device comprising: a construction module, used to construct a continuous learning image classification dataset; wherein the continuous learning image classification dataset comprises multiple image classification tasks, each classification task comprises multiple images and their corresponding true category labels; a mixing module, used to save the data of a small number of historical tasks as a memory cache, and when learning a new task, the memory cache is mixed with the current new task data, and the two are input into the classification model as training data, and the classification model outputs a classification prediction probability distribution; an iteration module, used to iteratively execute iterative training steps until the stopping condition is met ; Wherein, the stopping condition is: reaching a preset number of iterations or the value of the classification loss is reduced to less than a preset threshold; the iterative training step includes: inputting the classification prediction probability distribution into the proxy classifier, and the proxy classifier outputs the proxy prediction probability distribution; calculating the classification loss according to the proxy prediction probability distribution and the true category label, and fixing the parameters of the proxy classifier during the training process, and only updating the parameters of the classification model to minimize the prediction error; training the proxy classifier with data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; an evaluation module is used to evaluate the accuracy of the training results using the average classification accuracy and forgetting indicators after the iterative training step is completed.

[0015] One or more technical solutions provided in the embodiments of the present application have at least the following technical effects:

[0016] The embodiment of the present application provides an image dual classification method in an online continuous learning scenario. By saving a small amount of historical task data as a memory cache and mixing it with the current task data for training when learning a new task, this method can solve the problem of the model forgetting the ability of historical tasks when learning a new task. This hybrid training strategy enhances the generalization ability of the model, enabling it to handle multiple tasks at the same time and keep the memory of historical task knowledge. By introducing a proxy classifier, fixing its parameters during the training process, and only updating the parameters of the classification model, the output of the proxy classifier can be used to assist in optimizing the classification prediction without directly interfering with the main structure of the classification model. This dual supervision mechanism helps to refine the classification boundaries and improve the classification accuracy, especially when facing complex or blurred images. During the iterative training process, the calculation of the classification loss is directly based on the proxy prediction probability distribution and the true category label, which makes the calculation of the classification loss function more direct and efficient. At the same time, since the proxy classifier is only fine-tuned on the new task data instead of completely retraining, this significantly reduces the computational cost and time consumption and improves the training efficiency. The introduction of the proxy classifier enhances the interpretability of the classification results of the classification model. In addition, with the assistance of the proxy classifier, the classification model can be more robust when facing noisy data or outliers, improving the robustness of the model. The average classification accuracy and forgetting index are used as the evaluation criteria for the training results, which can fully reflect the comprehensive performance of the classification model on different tasks. In the online continuous learning scenario, it solves the problem of how to find the best balance between retaining historical tasks and learning new tasks to correct the tendency to face new tasks in the classification process. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments of the present application or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A flowchart of an image dual classification method in an online continuous learning scenario provided by an embodiment of the present application;

[0019] Figure 2 A specific flow chart of the iterative training steps provided in the embodiment of the present application;

[0020] Figure 3 A specific flow chart of the method for obtaining the average classification accuracy and forgetting index provided in the embodiment of the present application;

[0021] Figure 4 A schematic diagram of the structure of the proxy classifier provided in the embodiment of the present application;

[0022] Figure 5 A comparison chart of the average classification accuracy of the CIFAR-100 data set monitored in real time during the online continuous learning process provided in the embodiment of the present application;

[0023] Figure 6 A comparative schematic diagram of the probability distribution of each category predicted by the classification model of the CIFAR-100 dataset provided in the embodiment of the present application;

[0024] Figure 7 A schematic diagram of an image dual classification device in an online continuous learning scenario provided by an embodiment of the present application;

[0025] Figure 8 A schematic diagram of an image dual classification server in an online continuous learning scenario provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0027] The following describes some of the techniques involved in the embodiments of the present application to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, some descriptions of well-known functions and structures are omitted in the following description.

[0028] The present application embodiment provides an image dual classification method in an online continuous learning scenario, such as Figure 1 As shown, the method includes steps S101 to S104. Among them, Figure 1 This is only an execution order shown in the embodiment of the present application, and does not represent the only execution order of the image dual classification method in an online continuous learning scenario. If the final result can be achieved, Figure 1 The steps shown may be performed in parallel or reversed.

[0029] S101: Construct a continuous learning image classification dataset, wherein the continuous learning image classification dataset includes multiple image classification tasks, and each classification task includes multiple images and their corresponding true category labels.

[0030] Specifically, the CIFAR-10, CIFAR-100, and Tiny-ImageNet datasets can be downloaded from the official website. This application uses the CIFAR-100 dataset as a continuous learning image classification dataset because it provides rich category diversity and is suitable for evaluating the model's ability to continuously learn new tasks. The dataset contains images of 100 categories, each category has 6,000 32×32 color images, divided into 5,000 training images and 1,000 test images. In order to construct continuous learning tasks, the CIFAR-100 dataset is evenly divided into 10 non-overlapping image classification tasks, each task containing 10 categories of data. This division ensures that each task is independent in category, thereby simulating the situation where tasks arrive continuously in actual scenarios. Of course, the number of divided classification tasks can also be other, and this application is not limited to the above values. For each task t (t=1,2,...,10), define its training dataset And the test dataset Training Dataset Among them, N t is the number of training samples for this task, is the i-th training sample in the t-th task And its corresponding true category label The sample pairs are composed of. Test data set in, is the number of test samples for this task, is the jth test sample of the tth task and its corresponding true category label In the continuous learning scenario, the classification model learns these tasks in sequence. After each task is learned, the classification model needs to be able to accurately predict all the learned task categories at the same time. When learning a new task, the classification model can only access the training data of the current task, and the data of historical tasks are invisible. This requires the classification model to adopt an effective strategy to avoid forgetting old tasks.

[0031] S102: Save a small amount of historical task data as a memory cache. When learning a new task, mix the memory cache with the current new task data and input them into the classification model as training data. The classification model outputs a classification prediction probability distribution. Save a small amount of historical task data as a memory cache. When learning a new task, mix the memory cache with the current new task data and input them into the classification model as training data. The classification model outputs a classification prediction probability distribution, including: when learning a new task, randomly sample a batch of data from the memory cache, and mix the batch of data with the current new task data to serve as training data. Input the training data into the classification model, and output the classification prediction probability distribution corresponding to each category label through forward propagation.

[0032] It should be noted that the memory cache only stores a small portion of the data of historical tasks, and there is no need to store all the data. In the continuous learning image classification dataset, classification tasks are divided into historical tasks and new tasks. Specifically, according to the learning order, classification tasks that have been learned are considered historical tasks, while classification tasks that have not yet been learned are defined as new tasks. This process reflects the characteristics of continuous learning, that is, the model is constantly learning new classification tasks, and it also needs to retain knowledge of previously learned historical tasks.

[0033] Specifically, any commonly used deep neural network can be used as the classification model. This application selects ResNet-18 as the classification model, denoted by f θ (x). Where x is the data input into the classification model, and θ is the parameter of the classification model, which will be updated according to the training data during the training process. The training process in this application can be understood as an online continuous learning process. The ResNet-18 model can effectively handle complex image classification tasks, and alleviate the gradient vanishing or gradient exploding problems in deep network training through residual connections. In order to avoid catastrophic forgetting of the classification model in continuous learning tasks, a memory cache mechanism is introduced. Memory cache It is defined as a set containing a small number of old task data samples, in the form of: Where M represents the total number of samples that can be accommodated in the memory cache. Represents the i-th image in the memory cache and its corresponding true category label M<<N t , which means that the size of the memory cache is much smaller than the number of training samples for any task.

[0034] Table 1 is a schematic diagram of the network structure of the classification model provided by the embodiment of the present application. As shown in Table 1, the present application selects ResNet-18 as the classification model, Conv1 is the first convolution layer of the classification model network, and the input image is convolved through a 7×7 convolution kernel, the number of output channels is 64, and the stride is 2. Since the stride is 2, this layer will reduce the size of the feature map. The size of the output feature map is 112×112. The Conv2_x stage includes two residual blocks (ResidualBlocks), each of which contains two 3×3 convolution layers, and the number of output channels of the convolution layers is 64. Before the two residual blocks, there will be a maximum pooling layer, which uses a window size of 3×3 and a stride of 2 to further reduce the size of the feature map. Therefore, after Conv2_x, the size of the feature map becomes 56×56. The Conv3_x stage also includes two residual blocks, each of which uses a 3×3 convolution layer, but the number of output channels is increased to 128 at this time. Similarly, by stacking residual blocks, richer feature representations can be learned. The size of the output feature map is 28×28. In the Conv4_x stage, the number of residual blocks remains unchanged, but the number of convolutional layer output channels of each residual block increases to 256. After this stage, the size of the feature map is reduced to 14×14. Conv5_x is the last convolution stage of the network, which contains two residual blocks, and the number of convolutional layer output channels of each block is 512. After this stage, the size of the feature map is reduced to 7×7. After all convolutional layers, there is a fully connected layer MLP. The 1×1 convolution kernel in the fully connected layer is used for cross-channel feature integration, which does not change the spatial size of the feature map, but the number of output channels becomes the final number of categories.

[0035] Table 1

[0036]

[0037] S103: Iteratively perform the iterative training steps until a stop condition is met, wherein the stop condition is: reaching a preset number of iterations or the classification loss value is reduced to less than a preset threshold.

[0038] Figure 2 A specific flow chart of the iterative training steps provided in the embodiment of the present application, such as Figure 2 As shown, it includes steps S201 to S203.

[0039] S201: Inputting the classification prediction probability distribution into the proxy classifier, and the proxy classifier outputs the proxy prediction probability distribution.

[0040] The classification prediction probability distribution is input into the proxy classifier, and the proxy classifier outputs the proxy prediction probability distribution, including: the proxy classifier includes a category-specific module and a category-independent module. The category-specific module is used to extract the unique information of each category from the vector of the classification prediction probability distribution of the classification model, and generate a vector of the proxy prediction probability distribution of the corresponding category to model the characteristics of each category. The category-independent module inputs a two-element vector and outputs the corresponding vector of proxy new and old task prediction probabilities. Among them, the two-element vector reflects the classification model's predicted probability that a given sample belongs to a new task or a historical task. The proxy classifier combines the vector of the proxy prediction probability distribution with the vector of the proxy new and old task prediction probabilities to output the proxy prediction probability distribution.

[0041] Figure 4 A schematic diagram of the structure of the proxy classifier provided in the embodiment of the present application, such as Figure 4 As shown, the proxy classifier is a lightweight neural network that aims to perform fine correction and enhancement on the prediction of the classification model through two modules, namely the category-specific module and the category-independent module. These two modules work together to not only extract and model the unique information of each category, but also capture and characterize the relationship between new and old tasks, thereby improving the performance of the classification model in continuous learning scenarios. This application uses a two-layer multilayer perceptron model with a ReLU activation function as the network structure of these two modules. This network structure is only one option, and other more complex network structures can also be used as proxy classifiers.

[0042] The expression for the class-specific module is Its input is That is, the classification model predicts the probability distribution vector of the input sample x. The category-specific module outputs the proxy prediction probability distribution vector of the corresponding category, which is used to model the characteristics of each category, that is, the specific information of each category. The category-specific module helps the proxy classifier maintain high discrimination in complex multi-class classification tasks by strengthening the unique representation of each category. The expression of the category-independent module is Its input is a two-element vector,

[0043] include and in, The sum of the predicted probabilities that the classification model predicts that the sample belongs to the new task category, The sum of the predicted probabilities that the classification model predicts that the sample belongs to the historical task category, is the category set of the new task, is the category set of old tasks, Represents the classification prediction probability vector The predicted probability corresponding to the cth category of the class-independent module is the vector of predicted probabilities of the proxy's new and old tasks, which is used to characterize the dynamic relationship between the new and old tasks. The class-independent module provides a robust foundation for cross-task learning for the proxy classifier by stably modeling the interaction between new and old tasks, which helps to maintain stable performance in a continuous learning environment. The proxy prediction probability distribution vector is combined with the proxy prediction probability vector of the new and old tasks, and the proxy classifier finally outputs the proxy prediction probability distribution, which combines the fine representation of each category and the stable relationship between the new and old tasks, providing more comprehensive and robust information support for classification decisions.

[0044] The training process of the classification model and the proxy classifier adopts different objective functions.

[0045] The expression of the objective function used in the classification model training process is: Among them, θ * (ω) is the optimal parameter of the classification model when the parameter of the fixed proxy classifier is ω, argmin is the operator of the parameter that can minimize the function, is the classification loss function of the classification model, is a random sampling of a batch of data and sample data of the new task from the memory cache, B is the number of samples currently trained by the classification model, and x i is the i-th training sample, y i is the i-th true category label, is the classification model f θ Then cascade the proxy classifier g ω The augmented classification model is θ, θ is the parameter of the classification model, and ° is the cascade operation.

[0046] S202: Calculate the classification loss based on the proxy prediction probability distribution and the true category label, and fix the parameters of the proxy classifier during the training process, and only update the parameters of the classification model to minimize the prediction error.

[0047] Updating parameters of the classification model, including: updating the parameters of the classification model using a stochastic gradient descent algorithm.

[0048] Specifically, during the training process, the proxy classifier is first used to predict the input data, and the output is the probability value corresponding to each category, which constitutes the proxy prediction probability distribution. Each training sample has a true category label, which indicates the true category of the sample. This label is the supervision information in the training process and is used to evaluate the prediction accuracy of the classification model.

[0049] Use the stochastic gradient descent algorithm (SGD) to update the parameters of the classification model. The stochastic gradient descent algorithm first calculates the gradient of the classification loss function with respect to the parameters of the classification model. This gradient indicates the direction in the parameter space where the classification loss function value increases fastest. Update the parameters in the opposite direction of the gradient, that is, in the direction where the classification loss function value decreases. Specifically, subtract the product of a step size (learning rate) proportional to the gradient of the parameter from each parameter. After such a step of gradient descent, the parameters of the classification model are updated so that under the new parameters, the model's prediction results are closer to the true label, thereby reducing the classification loss.

[0050] Specifically, the objective function used in the classification model training process aims to find the optimal value of the parameters of the classification model when the proxy classifier parameters ω are fixed. The classification loss function is used to measure the difference between the classification model prediction and the true category label. By minimizing the classification loss function, it is hoped that the classification model can more accurately predict the category of the sample. The proxy classifier provides a knowledge transfer mechanism for the classification model through its stable parameter ω. In the scenario of continuous learning, the classification model is prone to bias towards the latest learned task, resulting in forgetting of historical tasks. The existence of the proxy classifier helps to alleviate this tendency towards new tasks because it can provide comprehensive guidance for all tasks seen. The tendency towards new tasks is a common problem in continuous learning, which manifests itself as the classification model gradually forgetting the knowledge of historical tasks during the continuous learning process and over-focusing on new tasks. By introducing the proxy classifier and cascading it to the classification model, an augmented classification model is constructed. During the training process, this augmented model minimizes the difference between the predicted probability distribution and the true category label, while being guided by the proxy classifier, thereby avoiding the problem of excessive bias towards new tasks. In the process of continuous learning, by randomly sampling data from the memory cache and mixing it with sample data of the new task, a training batch containing information of both new and old tasks can be constructed. This way of utilizing data helps the classification model keep its memory of historical tasks while learning new tasks.

[0051] In summary, the classification model training process aims to solve the problem of new task orientation in continuous learning by designing an augmented classification model that includes a proxy classifier and minimizing the classification loss under its guidance. By optimizing the parameters of the classification model and fixing the parameters of the proxy classifier during the training process, a classification model that can learn new task knowledge and keep the memory of historical tasks can be trained. This process is crucial to achieve efficient continuous learning.

[0052] The objective function expression used in the training process of the proxy classifier is: Among them, ω * is the optimal parameter of the proxy classifier, is the training loss function of the proxy classifier, is to randomly sample a batch of data from the memory cache. The samples in this batch are relatively balanced for all new tasks. B′ is the number of samples currently trained by the proxy classifier. θ*(ω) is a classification model where the parameter θ is set to the optimal value based on the current ω. (ω) (x i ),y i ) is the loss function of a single sample, which is used to calculate the classification model for the training sample x i The predicted result f θ*(ω) (x i ) and the true category label y i The difference between.

[0053] It should be noted that a batch of data is relatively balanced for all new tasks. In fact, it is an ideal expectation that this batch can contain enough and evenly distributed samples of different old tasks so that when training new tasks, the model can review and consolidate the knowledge of these historical tasks at the same time. However, in actual operation, due to the limited size of the memory cache, the diversity of historical tasks and the uncertainty of random sampling, it is difficult to ensure that each batch is strictly balanced, so it is relatively balanced.

[0054] Specifically, in order to ensure that the proxy classifier can effectively absorb and correct the training deviations of the classification model during the continuous learning process, a joint optimization framework is adopted, in which the parameters of the proxy classifier are regarded as hyperparameters of the classification model parameters, and the two are trained in an alternating or simultaneous optimization manner.

[0055] First, during the classification model training process, only the parameters of the classification model are optimized, while the parameters ω of the proxy classifier remain fixed. The purpose of this step is to allow the classification model to reach a certain performance level on the task or dataset. Next, the data in the memory cache is used as an approximation of the balanced dataset. A batch of data is randomly sampled from the memory cache for training the proxy classifier. The training goal of the proxy classifier is to minimize its training loss function, which calculates the difference between the prediction results of the classification model on the memory cache data and the true category label. The optimal parameters of the proxy classifier are found by optimizing the training objective function of the proxy classifier. The proxy classifier adapts to the changes of the classification model by optimizing its parameters. This mutual adaptation mechanism helps the proxy classifier maintain the effectiveness of its guidance during the continuous learning process. By minimizing the training loss function, the proxy classifier learns how to help the classification model correct its classification prediction probability distribution, thereby avoiding the problem of bias towards new tasks. This makes the classification model more stable on the balanced dataset or the approximation of the memory cache data, reducing the risk of forgetting old task knowledge. The cross entropy loss function is used as a criterion for measuring the difference between the prediction results and the true label. During the training process of the proxy classifier, this loss function prompts the proxy classifier to learn how to better guide the classification model so that it can fit the data distribution of all tasks. Through this joint optimization method, the proxy classifier and the classification model evolve together, achieving effective consolidation and transfer of knowledge. While avoiding forgetting historical tasks, the classification model can also efficiently learn the knowledge of new tasks.

[0056] S203: Use the data in the memory cache to train the proxy classifier to assist the classification model in correcting the classification prediction probability distribution.

[0057] The proxy classifier is trained with the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution, including: using the classification model after updating the parameters to calculate the classification loss value of another batch of samples randomly selected from the memory cache, and updating the parameters of the proxy classifier in the process of training the proxy classifier through the Adam algorithm. Specifically, through the Adam algorithm, the gradient information of the loss function is passed from the classification model to the proxy classifier to guide the update of the proxy classifier parameters. In this way, the classification model can continuously consolidate the historical task knowledge that has been learned with the assistance of the proxy classifier, while quickly adapting to and learning the knowledge of new tasks. This mechanism provides an effective way to achieve continuous learning, enabling the classification model to maintain the stability and generalization ability of its performance in a constantly changing environment.

[0058] S104: When the iterative training step is completed, the accuracy of the training results is evaluated using the average classification accuracy and the forgetting index.

[0059] Figure 3A specific flow chart of the method for obtaining the average classification accuracy and forgetting index provided in the embodiment of the present application, such as Figure 3 As shown, it includes steps S301 to S306.

[0060] S301: For each task, input samples in its test set into the classification model, and the classification model generates a prediction result for each sample. The test set is divided from the continuous learning image classification dataset.

[0061] Specifically, the test set is pre-split from the entire continuous learning image classification dataset to evaluate the performance of the model on unseen data. For each test sample, it is input into the trained classification model. The classification model receives the sample as input and generates a prediction result for each sample based on its internally learned feature representation and classification rules. This prediction result is a probability distribution that indicates the likelihood that the sample belongs to each category, but when evaluating, the category with the highest probability is selected as the final prediction category.

[0062] S302: If the prediction result of the classification model matches the true category label of the sample, it is regarded as a successful prediction.

[0063] Specifically, for the prediction results generated by the classification model for each test sample, it is necessary to compare them with the true class label of the sample. If the highest probability class in the prediction result matches the true class label of the sample, it is considered a successful prediction. The number of successful predictions will be used to calculate the prediction accuracy of the task later.

[0064] S303: The prediction accuracy of each task is defined as the number of successful predictions divided by the total number of samples in the test set of the task.

[0065] Specifically, after completing the prediction of all test samples and determining the successful predictions, the prediction accuracy of each task can be calculated. The prediction accuracy is obtained by dividing the number of successful predictions by the total number of samples in the test set of the task. This ratio reflects the accuracy of the classification model on the task, that is, the ability of the classification model to correctly classify the samples in the test set of the task.

[0066] S304: The maximum prediction accuracy of each task is the highest prediction accuracy achieved by the task during the entire training process.

[0067] Specifically, during the continuous learning training process, the classification model will go through multiple iterations or training cycles. At the end of each cycle, the prediction accuracy of the current model is calculated. In order to evaluate the best performance of the model throughout the training process, it is necessary to record the highest prediction accuracy achieved by each task during the entire training process, that is, the maximum prediction accuracy of the task. This indicator reflects the best classification effect that the classification model can achieve on the task.

[0068] S305: Calculate the average of the maximum prediction accuracies of all tasks as the average classification accuracy of the classification model.

[0069] Specifically, after the maximum prediction accuracy of all tasks is recorded, the average of these maximum prediction accuracies can be calculated as the average classification accuracy of the classification model. The average classification accuracy is a global indicator that measures the overall classification accuracy of the classification model on all tasks it has seen. This indicator is important for evaluating the generalization ability and stability of the classification model on multiple tasks.

[0070] S306: For each task, calculate the difference between its prediction accuracy and the maximum prediction accuracy, and use the difference as the forgetting index of the task.

[0071] Specifically, the forgetting index is used to measure the degree to which the classification model forgets the knowledge of historical tasks during the continuous learning training process. For each task, the final prediction accuracy is calculated, that is, the difference between the prediction accuracy obtained after all training cycles and the maximum prediction accuracy. This difference is the forgetting index of the task. The smaller the forgetting index, the less the classification model forgets the knowledge of historical tasks during the continuous learning process, and the better the stability of the model. On the contrary, the larger the forgetting index, the easier it is for the classification model to forget the knowledge of old tasks when learning new tasks, which may lead to a decline in the performance of the classification model.

[0072] The accuracy of the training results was evaluated using average classification accuracy and forgetting indicators, including:

[0073] The expression of average classification accuracy is: Among them, ACC is the average classification accuracy, T is the number of all tasks in the training process, and a t,T It is the prediction accuracy of the t-th task calculated after the training process is completed, that is, after all tasks have completed training. Specifically, the prediction accuracy of the t-th task is obtained by dividing the number of successfully predicted samples in the test set of the t-th task by the total number of samples in the test set of the task.

[0074] Specifically, the higher the average classification accuracy, the better the overall classification performance of the classification model on all tasks, that is, the classification model can more accurately identify samples of different tasks.

[0075] The expression of the forgetting indicator is: Among them, FM is the forgetting index, T is the number of all tasks in the training process, and a * is the maximum prediction accuracy of the t-th task during the training process, a t,T is the prediction accuracy of the tth task calculated after the training process is completed, and T is the number of all tasks in the training process.

[0076] Specifically, the smaller the forgetting index is, the less the classification model forgets the knowledge of historical tasks during the continuous learning process, and the better the stability of the classification model. On the contrary, the larger the forgetting index is, the easier it is for the classification model to forget the knowledge of historical tasks when learning new tasks, which will lead to a decrease in the performance of the classification model on historical tasks.

[0077] In the scenario of continuous learning, the classification model does not learn all tasks at once, but learns them one by one in sequence. Whenever the model completes training for a new task, it may forget the previously learned tasks because of learning the new task. In order to evaluate the performance of the classification model in the process of continuous learning, it is necessary to test the model's predictive ability for all learned tasks after the training of each task is completed. Specifically, when the model completes training for the qth task, we will use the test set of the vth task (where v can be any value less than or equal to q, because the classification model has learned the first v tasks at the end of the qth task) to evaluate the model's prediction accuracy for the i-th task. This prediction accuracy is recorded as a q,v . Where q represents the index of the task being evaluated, and v represents how many tasks have been trained so far. q,v represents the prediction accuracy of the model for the qth task after training the vth task. In this way, we can track how the prediction performance of the classification model for each task changes during the continuous learning process. In particular, we can observe whether the model has forgotten the historical tasks because of learning new tasks, which can be observed by comparing a under different v values. q,v If for a certain historical task v, as q increases (that is, as the classification model learns more new tasks), a q,v A significant decrease indicates that the classification model has forgotten.

[0078] Table 2 is a performance comparison of the three classification tasks of CIFAR-10, CIFAR-100 and Tiny-ImageNet provided by the embodiment of the present application. Three data sets are listed at the top of Table 2, namely CIFAR-10, CIFAR-100 and Tiny-ImageNet. These data sets are commonly used classification task benchmark data sets in the field of computer vision. The Method in Table 2 is the method part. Table 2 shows different model configurations, including M=0.2k, M=0.5k, M=2k, M=5k, etc., representing different parameter settings of the model. In addition, methods with the suffix "proxy classifier" are also listed, which means that a proxy classifier is introduced on the basis of these methods. ACC represents the average classification accuracy of each method on each data set. FM represents the forgetting index. For the CIFAR-10 data set, the accuracy of the method with M=0.2k is 35.21%, and the forgetting index is 50.28%; while the accuracy of the method with M=0.5k is increased to 42.32%, and the forgetting index is reduced to 40.80%. This shows that the performance improves with the increase of model capacity. Methods that introduce proxy classifiers (such as ER-proxy classifier, DER-proxy classifier, etc.) show higher accuracy and lower forgetting metrics than original methods (such as ER, DER++, etc.) in most cases, which shows that proxy classifiers are effective in improving model performance and alleviating forgetting problems. Similar trends are observed for CIFAR-100 and Tiny-ImageNet datasets, that is, with the increase of model capacity and the introduction of proxy classifiers, the average classification accuracy of the model is improved and the forgetting index is reduced.

[0079] It should be noted that ER, DER++, RAR and CLSER are four different continuous learning methods, each of which uses different strategies to optimize the classification model to reduce or avoid forgetting in the continuous learning task. The present application adds a proxy classifier on the basis of the classification model, which can be directly applied to these four continuous learning methods to further improve the performance on their basis. Specifically, ER (Experience Replay) is an experience replay strategy. DER++ (DarkExperience Replay++) is an enhanced version of ER, that is, a distillation loss is added to ER. RAR (Repeated Augmented Rehearsal) uses a random augmentation strategy based on DER++, that is, randomly transforms the data to increase the diversity of samples. CLSER (Complementary Learning System ExperienceReplay) constructs a stability model and a plasticity model to better transfer knowledge to the classification model.

[0080] Table 2

[0081]

[0082] Table 3 is a performance comparison of different methods provided in the embodiments of the present application when processing the three datasets of CIFAR10, CIFAR100 and Tiny-ImageNet. For the CIFAR10 dataset, Table 3 lists different parameter combinations (such as a 1,5 ,a 2,5etc.), which represent different configurations of the model. For each parameter combination, Table 3 gives the values ​​of the average classification accuracy (ACC) and the forgetting measure (FM). After the introduction of the proxy classifier, the ER value decreases, indicating that the proxy classifier helps to reduce the error rate and improve the model performance. For the CIFAR100 dataset, Table 3 lists another set of parameter combinations, which are different from the parameters used in the CIFAR10 dataset, reflecting the configuration adjustment of the model on a more complex dataset. Similar to CIFAR10, Table 3 gives the ACC and FM values ​​of each parameter combination. Under the setting of M=5k, the ACC and FM values ​​of different parameter combinations are different, but overall, the error rate is generally high, reflecting the higher difficulty of the CIFAR100 dataset relative to CIFAR10. Similarly, after the introduction of the proxy classifier, the FM value decreases, indicating that the proxy classifier also has the effect of improving the model performance on the CIFAR100 dataset. For the Tiny-ImageNet dataset, the table lists another set of parameter combinations, which are optimized for the characteristics of the Tiny-ImageNet dataset. Table 3 gives the ACC and FM values ​​of each parameter combination. Under the setting of M = 5k, the ACC and FM values ​​of different parameter combinations reflect the performance of the model on the Tiny-ImageNet dataset. After the introduction of the proxy classifier, the FM value also decreases, indicating that the proxy classifier can also effectively reduce the error rate and improve the model performance on the Tiny-ImageNet dataset.

[0083] Table 3

[0084]

[0085] Table 4 shows the classification performance at different scales for the CIFAR-10 dataset in the online Blurry-10 setting under different methods provided by the embodiments of the present application. Among them, the online Blurry-10 setting means that the training data between tasks are partially overlapped during the online continuous learning and training process, that is, about 10% of the training data for each task will randomly appear in the training data of other tasks, and at the same time, the task will also contain about 10% of the training data from other tasks. This setting requires that the classification model is not restricted by the task boundaries, and is a more practical and flexible experimental setting. CIFAR-10 is a continuous learning image classification dataset. Table 4 compares the performance in the two cases of M=0.2k and M=0.5k. It can be seen from Table 4 that after the introduction of the proxy classifier, the classification accuracy and negative sample minimization of all methods have been significantly improved, especially the RAR-proxy classifier and CLSER-proxy classifier have performed well in multiple indicators. The results show that the technical solution can effectively improve the classification accuracy of the corresponding methods (ER, DER, RAR and CLSER) under this setting and greatly reduce the degree of forgetting, indicating that the proxy classifier proposed in this application is not affected by the task boundary and is a very flexible method that can be applied to a variety of more practical continuous learning experimental settings.

[0086] Table 4

[0087]

[0088] Figure 5 A comparison chart of the average classification accuracy of the CIFAR-100 dataset monitored in real time during the online continuous learning process provided by the embodiment of this application. The proxy classifier can help the classification model correct the classification prediction probability distribution during the entire continuous learning process, avoid the tendency of the classification model to face new tasks, thereby helping the classification model achieve better convergence results and effectively improving the classification accuracy of online continuous learning.

[0089] Figure 6 A comparative schematic diagram of the probability distribution of each category predicted by the classification model of the CIFAR-100 dataset provided in an embodiment of the present application. Figure 6 The Task Index in the figure is the task index. The X-axis of the chart marks the task index values ​​from 0 to 9, and the Y-axis represents the task distribution of the ER and ER-proxy classifier methods. The proxy classifier can significantly help the classification model correct the probability distribution of each predicted category by suppressing the prediction probability corresponding to the new task and improving the prediction probability of the previous historical tasks, effectively helping the original classification model avoid the tendency to face new tasks in the continuous learning process.

[0090] Table 5 is a comparative result of offline continuous learning setting for CIFAR-10 data set under different methods provided by the embodiment of the present application. Wherein, offline continuous learning means that each task can be iteratively trained multiple times until convergence during the continuous learning process. Compared with the setting that can only process streaming data at the same time in online continuous learning, the data distribution change of offline continuous learning is relatively stable. With the support of more computing resources, offline continuous learning can usually achieve better convergence results of classification models, thereby obtaining higher prediction accuracy. The results show that the technical solution can effectively improve the classification accuracy of corresponding methods (ER, DER, RAR and CLSER) under offline continuous learning settings and greatly reduce the degree of forgetting. It is explained that the present application can be applied not only to online continuous learning with drastic changes in data distribution, but also to conventional offline continuous learning, and effectively improve the performance of its corresponding method.

[0091] Table 5

[0092]

[0093] The present application embodiment also provides an image pair classification device 700 in an online continuous learning scenario, such as Figure 7 As shown, the device includes: a construction module 701, a mixing module 702, an iteration module 703 and an evaluation module 704.

[0094] The construction module 701 is used to construct a continuous learning image classification dataset, wherein the continuous learning image classification dataset includes multiple image classification tasks, and each classification task includes multiple images and their corresponding true category labels.

[0095] The mixing module 702 is used to save a small amount of historical task data as a memory cache. When learning a new task, the memory cache is mixed with the current new task data and input into the classification model as training data. The classification model outputs the classification prediction probability distribution.

[0096] The iteration module 703 is used to iteratively execute the iterative training step until the stopping condition is met. The stopping condition is: reaching a preset number of iterations or the value of the classification loss is reduced to less than a preset threshold. The iterative training step includes: inputting the classification prediction probability distribution into the proxy classifier, and the proxy classifier outputs the proxy prediction probability distribution. The classification loss is calculated based on the proxy prediction probability distribution and the true category label, and the parameters of the proxy classifier are fixed during the training process, and only the parameters of the classification model are updated to minimize the prediction error. The proxy classifier is trained with data in the memory cache to assist the classification model in correcting the classification prediction probability distribution.

[0097] The evaluation module 704 is used to evaluate the accuracy of the training results using the average classification accuracy and the forgetting index after the iterative training step is completed.

[0098] Some modules in the apparatus described in the present application can be described in the general context of computer executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments, in which tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media including storage devices.

[0099] The devices or modules described in the above application embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above devices are described in various modules according to their functions. When implementing the embodiments of the present application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, the module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.

[0100] The methods, devices or modules described in this application can be implemented in a computer-readable program code mode. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro) processor, a logic gate, a switch, an application-specific integrated circuit (English: Application Specific Integrated Circuit, referred to as: ASIC), a programmable logic controller and an embedded microcontroller. Examples of controllers include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a pure computer-readable program code mode, the controller can be completely implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included in it for implementing various functions can also be regarded as structures within the hardware component. Or even, the means for realizing various functions may be regarded as both a software module for realizing the method and a structure within a hardware component.

[0101] like Figure 8As shown, an embodiment of the present application also provides an image dual classification server in an online continuous learning scenario, including a memory 801 and a processor 802; the memory 801 is used to store computer executable instructions; the processor 802 is used to execute computer executable instructions to implement an image dual classification method in an online continuous learning scenario described above in an embodiment of the present application.

[0102] An embodiment of the present application also provides a computer-readable storage medium, which stores executable instructions. When a computer executes the executable instructions, it can implement the image dual classification method in an online continuous learning scenario described above in the embodiment of the present application.

[0103] It can be known from the description of the above implementation mode that a person skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, or it can be reflected in the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the method described in the embodiment of the present application.

[0104] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations.

[0105] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some or all of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.

Claims

1. An image dual classification method in an online continuous learning scenario, characterized in that: include: Constructing a continuous learning image classification dataset; wherein the continuous learning image classification dataset includes multiple image classification tasks, each classification task includes multiple images and their corresponding real category labels; A small amount of historical task data is saved as a memory cache. When learning a new task, the memory cache is mixed with the current new task data and input into the classification model as training data. The classification model outputs the classification prediction probability distribution. The iterative training steps are iteratively performed until the stopping condition is met. The stopping condition is: reaching a preset number of iterations or the classification loss value is reduced to less than a preset threshold. The iterative training step comprises: The classification prediction probability distribution is input into the proxy classifier, and the proxy classifier outputs the proxy prediction probability distribution; The classification loss is calculated based on the proxy prediction probability distribution and the true category label. During the training process, the parameters of the proxy classifier are fixed and only the parameters of the classification model are updated to minimize the prediction error. Use the data in the memory cache to train the proxy classifier to assist the classification model in correcting the classification prediction probability distribution; When the iterative training steps are completed, the accuracy of the training results is evaluated using the average classification accuracy and the forgetting metric.

2. The image dual classification method in the online continuous learning scenario according to claim 1 is characterized in that: The data of a small amount of historical tasks is saved as a memory cache. When learning a new task, the memory cache is mixed with the current new task data and input into the classification model as training data. The classification model outputs the classification prediction probability distribution, including: When learning a new task, randomly sample a batch of data from the memory cache and mix the batch of data with the data of the current new task to serve as training data together; The training data is input into the classification model, and the classification prediction probability distribution corresponding to each category label is output through forward propagation.

3. The image dual classification method in the online continuous learning scenario according to claim 1 is characterized in that: The step of inputting the classification prediction probability distribution into the proxy classifier, and the proxy classifier outputting the proxy prediction probability distribution, comprises: The proxy classifier includes a category-specific module and a category-independent module; The category-specific module is used to extract unique information of each category from the classification prediction probability distribution vector of the classification model, and generate a vector of proxy prediction probability distribution of the corresponding category to model the characteristics of each category; The category-independent module inputs a two-element vector and outputs a vector of the corresponding predicted probabilities of the agent's new and old tasks; the two-element vector reflects the classification model's predicted probability that a given sample belongs to a new task or a historical task; The agent classifier outputs the agent prediction probability distribution by combining the agent's prediction probability vector with the agent's prediction probability vectors for the new and old tasks.

4. The image dual classification method in the online continuous learning scenario according to claim 1 is characterized in that: The updating of the parameters of the classification model includes: updating the parameters of the classification model using a stochastic gradient descent algorithm.

5. The image dual classification method in the online continuous learning scenario according to claim 1 is characterized in that: The method of using the data in the memory cache to train the proxy classifier to assist the classification model in correcting the classification prediction probability distribution includes: using the classification model after updating the parameters to calculate the classification loss value of another batch of samples randomly selected from the memory cache, and updating the parameters of the proxy classifier during the training of the proxy classifier through the Adam algorithm.

6. The image dual classification method in the online continuous learning scenario according to claim 1, characterized in that: The training process of the classification model and the surrogate classifier adopts different objective functions; The expression of the objective function used in the classification model training process is: Among them, θ * (ω) is the optimal parameter of the classification model when the parameter of the fixed proxy classifier is ω, argmin is the operator of the parameter that can minimize the function, is the classification loss function of the classification model, is a random sampling of a batch of data and sample data of the new task from the memory cache, B is the number of samples currently trained by the classification model, and x i is the i-th training sample, y i is the i-th true category label, is the classification model f θ Then cascade the proxy classifier g ω The augmented classification model, θ is the parameter of the classification model, For cascade operation; The objective function expression used in the training process of the proxy classifier is: Among them, ω * is the optimal parameter of the proxy classifier, is the training loss function of the proxy classifier, is a random sample of a batch of data from the memory cache, B′ is the number of samples currently trained by the proxy classifier, and f θ*(ω) is a classification model in which the parameter θ is set to the optimal value based on the current ω; l(f θ*(ω) (x ′ ),y i ) is the loss function of a single sample, which is used to calculate the classification model for the training sample x i The predicted result f θ*(ω) (x i ) and the true category label y i The difference between.

7. The image dual classification method in the online continuous learning scenario according to claim 1 is characterized in that: The methods for obtaining the average classification accuracy and forgetting index include: For each task, the samples in its test set are input into the classification model, and the classification model generates a prediction result for each sample; wherein the test set is divided from the continuous learning image classification dataset; If the prediction result of the classification model matches the true category label of the sample, it is considered a successful prediction; The prediction accuracy of each task is defined as the number of successful predictions divided by the total number of samples in the test set for that task; The maximum prediction accuracy of each task is the highest prediction accuracy achieved by the task during the entire training process; Calculate the average of the maximum prediction accuracy of all tasks as the average classification accuracy of the classification model; For each task, the difference between its prediction accuracy and the maximum prediction accuracy is calculated, and the difference is used as the forgetting indicator of the task.

8. The image dual classification method in the online continuous learning scenario according to claim 1, characterized in that: The accuracy of the training results is evaluated using the average classification accuracy and forgetting index, including: The expression of average classification accuracy is: Among them, ACC is the average classification accuracy, T is the number of all tasks in the training process, and a t,T The prediction accuracy of the tth task calculated after the training process is completed; The expression of the forgetting indicator is: Among them, FM is the forgetting index, T is the number of all tasks in the training process, and a * is the maximum prediction accuracy of the t-th task during the training process, a t,T is the prediction accuracy of the tth task calculated after the training process is completed, and T is the number of all tasks in the training process.

9. An image dual classification device in an online continuous learning scenario, characterized in that: include: A construction module is used to construct a continuous learning image classification dataset; wherein the continuous learning image classification dataset includes multiple image classification tasks, each classification task includes multiple images and their corresponding real category labels; The hybrid module is used to save a small amount of historical task data as a memory cache. When learning a new task, the memory cache is mixed with the current new task data and input into the classification model as training data. The classification model outputs the classification prediction probability distribution. An iteration module is used to iteratively execute the iterative training step until a stop condition is met; wherein the stop condition is: reaching a preset number of iterations or the value of the classification loss is reduced to less than a preset threshold; the iterative training step includes: inputting the classification prediction probability distribution into the proxy classifier, and the proxy classifier outputs the proxy prediction probability distribution; calculating the classification loss according to the proxy prediction probability distribution and the true category label, and fixing the parameters of the proxy classifier during the training process, and only updating the parameters of the classification model to minimize the prediction error; training the proxy classifier with the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; The evaluation module is used to evaluate the accuracy of the training results using the average classification accuracy and forgetting indicators after the iterative training steps are completed.

Images