Image pair classification method and device in online continual learning scenario

Abstract

The application discloses an image pair classification method and device in an online continuous learning scene, comprising the following steps: when learning a new task, mixing a memory cache with current new task data, inputting the mixed data into a classification model as training data, and outputting a classification prediction probability distribution; inputting the classification prediction probability distribution into an agent classifier, and outputting an agent prediction probability distribution; calculating a classification loss according to the agent prediction probability distribution and a real class label, fixing parameters of the agent classifier in a training process, and only updating parameters of the classification model to minimize a prediction error; training the agent classifier with data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; and evaluating the accuracy of a training result by using average classification accuracy and a forgetting index. The application solves the problem of how to find a best balance point between retaining historical tasks and learning new tasks to correct a new-task-oriented tendency in a classification process in the online continuous learning scene.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of continual learning, and particularly relates to an image pair classification method and device in an online continual learning scenario. BACKGROUND

[0002] Continual Learning (CL) is an important research direction in the field of artificial intelligence, aiming to build an intelligent system that can continuously learn from new data and tasks that are constantly emerging, while retaining and integrating existing knowledge. This technology breaks through the limitation of traditional machine learning models that stop learning after training, enabling the system to continuously optimize itself in practical applications and adapt to complex and changing environments. Online continual learning, as a subset of continual learning, further requires the model to update parameters in real time as data streams arrive, to quickly adapt to changes in data distribution and ensure the efficiency and real-time performance of the system.

[0003] In the scenario of continual learning, especially online continual learning, machine learning models are prone to have a tendency to face new tasks. This problem mainly manifests that when the model encounters a new task, it will pay excessive attention to the features of the new task, leading to forgetting or ignoring the knowledge of old tasks, and thus showing excessive prediction of new task samples in classification tasks, which seriously damages the overall classification accuracy. Taking image classification as an example, in the process of continual learning, a deep neural network will classify all newly input image samples into the class of the new task, causing a sharp decline in classification performance. Existing technologies ignore the complexity of dynamic changes in data distribution in the scenario of continual learning, and are difficult to find the best balance point between retaining historical tasks and learning new tasks.

[0004] Therefore, in the online continual learning scenario, how to find the best balance point between retaining historical tasks and learning new tasks to correct the tendency to face new tasks in the classification process has become a problem to be solved. SUMMARY

[0005] In the embodiments of the present application, by providing an image pair classification method in an online continual learning scenario, the problem of how to find the best balance point between retaining historical tasks and learning new tasks to correct the tendency to face new tasks in the classification process in the online continual learning scenario is solved.

[0006] In a first aspect, the embodiments of the present application provide an image pair classification method in an online continual learning scenario, which comprises: constructing a continual learning image classification dataset; wherein the continual learning image classification dataset comprises a plurality of image classification tasks, each classification task comprising a plurality of images and corresponding real class labels; saving data of a small number of historical tasks as a memory cache, and mixing the memory cache with current new task data as training data when learning a new task, and inputting the training data into a classification model, and the classification model outputs a classification prediction probability distribution; iteratively performing an iterative training step until a stop condition is met; wherein the stop condition is that a preset iteration number is reached or a classification loss value is reduced to less than a preset threshold; the iterative training step comprises: inputting the classification prediction probability distribution into an agent classifier, and the agent classifier outputs an agent prediction probability distribution; calculating a classification loss according to the agent prediction probability distribution and the real class labels, and fixing the parameters of the agent classifier during the training process, and only updating the parameters of the classification model to minimize the prediction error; training the agent classifier with the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; and after the iterative training step is completed, using average classification accuracy and a forgetting index to evaluate the accuracy of the training result.

[0007] In a possible implementation, the saving of the data of the small number of historical tasks as the memory cache, the mixing of the memory cache with the current new task data as the training data when learning the new task, and the inputting of the training data into the classification model to output the classification prediction probability distribution comprise: when learning the new task, randomly sampling a batch of data from the memory cache, and mixing the batch of data with the data of the current new task as the training data; inputting the training data into the classification model to output the classification prediction probability distribution corresponding to each class label through forward propagation.

[0008] In a possible implementation, the inputting of the classification prediction probability distribution into the agent classifier to output the agent prediction probability distribution comprises: the agent classifier comprises a class-specific module and a class-agnostic module; the class-specific module is used to extract unique information of each class from a vector of the classification prediction probability distribution of the classification model, to generate a vector of the agent prediction probability distribution corresponding to the class, so as to model the characteristics of each class; the class-agnostic module inputs a two-element vector to output a vector of agent new-old task prediction probability corresponding thereto; wherein the two-element vector reflects the prediction probability of the classification model for a given sample belonging to a new task or a historical task; and the agent classifier combines the vector of the agent prediction probability distribution and the vector of the agent new-old task prediction probability to output the agent prediction probability distribution.

[0009] In a possible implementation, the updating of the parameters of the classification model comprises: updating the parameters of the classification model using a stochastic gradient descent algorithm.

[0010] In a possible implementation, the training of the agent classifier with the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution comprises: calculating the value of the classification loss of another batch of samples randomly selected from the memory cache using the classification model with the updated parameters, and updating the parameters of the agent classifier in the process of training the agent classifier by using the Adam algorithm.

[0011] In a possible implementation, the training processes of the classification model and the agent classifier respectively adopt different objective functions; the expression of the objective function adopted in the training process of the classification model is: wherein θ * is the optimal parameter of the classification model when the parameter of the fixed agent classifier is ω, argmin is an operator of the parameter that can make the function reach the minimum value, is the classification loss function of the classification model, is a batch of data randomly sampled from the memory cache and sample data of a new task, B is the number of samples currently trained by the classification model, x i is the i-th training sample, y i is the i-th real class label, is the augmented classification model of the classification model f θ after cascading the agent classifier g v , θ is the parameter of the classification model, ° is the cascading operation; the expression of the objective function adopted in the training process of the agent classifier is: wherein ω * is the optimal parameter of the agent classifier, is the training loss function of the agent classifier, is a batch of data randomly sampled from the memory cache, B' is the number of samples currently trained by the agent classifier, f θ*(ω) is the classification model in which the parameter θ is set to the optimal value based on the current ω; l(f θ*(ω) (x i ), y i ) is the loss function of a single sample, used to calculate the difference between the result f i (x i ) predicted by the classification model for the training sample x θ*(ω) and the real class label y i .

[0012] In a possible implementation, the average classification accuracy and the forgetting index are obtained in the following manner: for each task, input samples in the test set of the task into the classification model, the classification model generates a prediction result for each sample; wherein the test set is divided from the continuous learning image classification dataset; if the prediction result of the classification model matches the true class label of the sample, it is considered as a successful prediction; the prediction accuracy of each task is defined as the number of successful predictions divided by the total number of samples in the test set of the task; the maximum prediction accuracy of each task is the highest prediction accuracy achieved by the task during the entire training process; calculate the average of the maximum prediction accuracies of all tasks as the average classification accuracy of the classification model; for each task, calculate the difference between the prediction accuracy and the maximum prediction accuracy as the forgetting index of the task.

[0013] In a possible implementation, the accuracy of the training result is evaluated using the average classification accuracy and the forgetting index in the following manner: the expression of the average classification accuracy is: wherein ACC is the average classification accuracy, T is the number of all tasks in the training process, a t,T is the prediction accuracy of the tth task calculated after the end of the training process; the expression of the forgetting index is: wherein FM is the forgetting index, T is the number of all tasks in the training process, a * is the maximum prediction accuracy of the tth task in the training process, a t,T is the prediction accuracy of the tth task calculated after the end of the training process, and T is the number of all tasks in the training process.

[0014] In a second aspect, the embodiments of the present application provide an image pair classification device in an online continual learning scenario, the device comprising: a construction module configured to construct a continual learning image classification dataset; wherein the continual learning image classification dataset comprises a plurality of image classification tasks, and each classification task comprises a plurality of images and corresponding real class labels; a mixing module configured to save data of a small number of historical tasks as a memory cache, and mix the memory cache with current new task data when learning a new task, and input the mixed data into a classification model as training data; the classification model outputs a classification prediction probability distribution; an iteration module configured to iteratively perform an iteration training step until a stop condition is met; wherein the stop condition is that a preset iteration number is reached or a classification loss value is reduced to less than a preset threshold; the iteration training step comprises: inputting the classification prediction probability distribution into an agent classifier, and the agent classifier outputs an agent prediction probability distribution; calculating a classification loss according to the agent prediction probability distribution and the real class labels, and fixing parameters of the agent classifier and only updating parameters of the classification model in the training process to minimize a prediction error; training the agent classifier with data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; and an evaluation module configured to evaluate accuracy of a training result using an average classification accuracy and a forgetting index after the iteration training step is completed.

[0015] The one or more technical solutions provided in the embodiments of the present application have at least the following technical effects:

[0016] The embodiment of the application provides a kind of image dual classification method in online continuous learning scene, by saving the data of a small amount of historical tasks as memory cache, and when learning new task, it is mixed training with current task data, this method can solve the ability of model forgets historical task when learning new task.This mixed training strategy enhances the generalization ability of model, so that it can handle multiple tasks at the same time, keep the memory of historical task knowledge.By introducing proxy classifier, fix its parameters in the training process, only update the parameters of classification model, can use the output of proxy classifier to assist optimization classification prediction without directly interfering with the main structure of classification model.This double supervision mechanism helps to refine the classification boundary and improve classification accuracy, especially when facing complex or ambiguous images.In the iterative training process, the calculation of classification loss is directly based on the proxy prediction probability distribution and the real class label, which makes the calculation of classification loss function more direct and efficient.At the same time, since the proxy classifier is only fine-tuned on new task data, not completely retrained, which significantly reduces the computational cost and time consumption, improves the training efficiency.The introduction of proxy classifier enhances the explainability of the classification results of the classification model.In addition, with the assistance of proxy classifier, the classification model can be more robust when facing noisy data or outliers, improving the robustness of the model.The average classification accuracy and forgetting index are used as the evaluation criteria for the training results, which can comprehensively reflect the overall performance of the classification model on different tasks.Solved the problem of how to find the best balance point between preserving historical tasks and learning new tasks in online continuous learning scenario to correct the tendency of new tasks in the classification process. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the application or the prior art, the drawings needed to be used in the description of the embodiments of the application or the prior art will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the application, and those skilled in the art can obtain other drawings without creative labor based on these drawings.

[0018] Figure 1 The flowchart of the image dual classification method in online continuous learning scenario provided by the embodiment of the application is shown in the figure.

[0019] Figure 2 The specific flowchart of the iterative training step provided by the embodiment of the application is shown in the figure.

[0020] Figure 3 The specific flowchart of the acquisition method of average classification accuracy and forgetting index provided by the embodiment of the application is shown in the figure.

[0021] Figure 4 The structure diagram of the proxy classifier provided by the embodiment of the application is shown in the figure.

[0022] Figure 5 A comparison chart of the average classification accuracy of the CIFAR-100 dataset monitored in real time in the online continuous learning process provided by the embodiments of the present application;

[0023] Figure 6 A comparison diagram of predicting the probability distribution of each category by the classification model using the CIFAR-100 dataset provided by the embodiments of the present application;

[0024] Figure 7 A schematic diagram of an image pair classification device in an online continuous learning scenario provided by the embodiments of the present application;

[0025] Figure 8 A schematic diagram of an image pair classification server in an online continuous learning scenario provided by the embodiments of the present application. DETAILED DESCRIPTION

[0026] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0027] The following describes some technologies related to the embodiments of the present application to help understanding, which should be considered only as exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Also, for the sake of clarity and brevity, the description below omits the description of some well-known functions and structures.

[0028] The embodiments of the present application provide an image pair classification method in an online continuous learning scenario, as shown in the figure, the method comprises steps S101 to S104. Wherein, Figure 1 Only one execution order is shown for the embodiments of the present application, which does not represent the only execution order of the image pair classification method in an online continuous learning scenario, and Figure 1 The steps shown can be executed in parallel or in reverse. Figure 1

[0029] S101: Construct a continuous learning image classification dataset. Wherein, the continuous learning image classification dataset comprises a plurality of image classification tasks, and each classification task comprises a plurality of images and their corresponding true category labels.

[0030] ​Specifically, the CIFAR-10, CIFAR-100 and Tiny-ImageNet datasets can be downloaded from the official website, and the CIFAR-100 dataset is used as the continual learning image classification dataset in this application because it provides rich class diversity and is suitable for evaluating the ability of the model to learn new tasks continuously. The dataset contains 6000 color images of 32x32 for each of the 100 classes, divided into 5000 training images and 1000 test images. In order to construct the continual learning task, the CIFAR-100 dataset is evenly divided into 10 mutually disjoint image classification tasks, each containing data of 10 classes. This division ensures that each task is independent in terms of classes, thereby simulating the situation where tasks continue to arrive in actual scenarios. Of course, the number of classification tasks can also be other, and the application is not limited to the above values. For each task t (t = 1, 2,..., 10), define its training dataset and test dataset Training dataset where N t is the number of training samples of the task, is the i-th training sample in the t-th task and its corresponding real class label constitute a sample pair. The test dataset where is the number of test samples of the task, is the j-th test sample of the t-th task and its corresponding real class label constitute a sample pair. In the continual learning scenario, the classification model learns these tasks in order, and after each task is learned, the classification model needs to be able to accurately predict all the learned task classes at the same time. When learning a new task, the classification model can only access the training data of the current task, while the data of the historical tasks is invisible. This requires the classification model to use effective strategies to avoid forgetting the old tasks.

[0031] S102: Save a small amount of data of historical tasks as a memory cache, and mix the memory cache with the current new task data when learning the new task, and input them together as training data into the classification model, and the classification model outputs a classification prediction probability distribution. Save a small amount of data of historical tasks as a memory cache, and mix the memory cache with the current new task data when learning the new task, and input them together as training data into the classification model, and the classification model outputs a classification prediction probability distribution, including: when learning a new task, randomly sampling a batch of data from the memory cache, and mixing the batch of data with the current new task data to serve as training data together. Input the training data into the classification model, and output the classification prediction probability distribution corresponding to each class label through forward propagation.

[0032] It should be noted that the memory cache only saves a small amount of data of historical tasks, and does not need to store all data. In the continuous learning image classification dataset, the classification tasks are divided into historical tasks and new tasks. Specifically, according to the learning order, the classification tasks that have completed learning are regarded as historical tasks, and the classification tasks that have not been learned are defined as new tasks. This process embodies the characteristics of continuous learning, that is, the model is constantly learning new classification tasks, while also retaining the knowledge of previously learned historical tasks.

[0033] Specifically, any commonly used deep neural network can be used as a classification model, and ResNet-18 is chosen as the classification model in this application, denoted as f θ (x). Where x is the data input into the classification model, and θ is the parameter of the classification model, which will be updated according to the training data during the training process. The training process in this application can be understood as an online continuous learning process. The ResNet-18 model can effectively handle complex image classification tasks and alleviate the gradient vanishing or gradient explosion problem in deep network training through residual connection. In order to avoid catastrophic forgetting of the classification model in the continuous learning task, a memory cache mechanism is introduced. The memory cache is defined as a set containing a small amount of old task data samples, and the specific form is: Where M represents the total number of samples that can be accommodated in the memory cache, represents the i-th image in the memory cache and its corresponding real class label M << N t , which means that the size of the memory cache is much smaller than the number of training samples of any task.

[0034] Table 1 is a schematic diagram of a classification model network structure provided by the embodiment of the present application. As shown in Table 1, the present application selects ResNet-18 as the classification model. Conv1 is the first convolutional layer of the classification model network. The input image is subjected to a convolution operation by a 7x7 convolutional kernel, and the output channel number is 64, and the stride is 2. Since the stride is 2, the size of the feature map is reduced. The size of the output feature map is 112x112. The Conv2_x stage includes two residual blocks (Residual Blocks). Each residual block contains two 3x3 convolutional layers, and the output channel number of each convolutional layer is 64. Before the two residual blocks, there is a max-pooling layer, which uses a 3x3 window size and a stride of 2, to further reduce the size of the feature map. Therefore, after Conv2_x, the size of the feature map becomes 56x56. The Conv3_x stage also contains two residual blocks, each of which uses a 3x3 convolutional layer, but the output channel number is increased to 128. Through the stacking of residual blocks, more rich feature representations can be learned. The size of the output feature map is 28x28. In the Conv4_x stage, the number of residual blocks remains unchanged, but the output channel number of the convolutional layer of each residual block is increased to 256. After this stage, the size of the feature map is reduced to 14x14. Conv5_x is the last convolutional stage of the network, which contains two residual blocks, and the output channel number of the convolutional layer of each block is 512. After the feature map passes through this stage, the size is reduced to 7x7. After all the convolutional layers, there is a fully connected layer MLP. The 1x1 convolutional kernel in the fully connected layer is used for cross-channel feature integration, and does not change the spatial size of the feature map, but the output channel number becomes the final class number.

[0035] Table 1

[0036]

[0037] S103: iteratively performing the iterative training step until a stop condition is met. The stop condition is that a preset number of iterations is reached or the value of the classification loss is reduced to less than a preset threshold.

[0038] Figure 2 The specific flowchart of the iterative training step provided by the embodiment of the present application is shown in Figure 2 , which includes steps S201 to S203.

[0039] S201: inputting the classification prediction probability distribution into the agent classifier, and the agent classifier outputs an agent prediction probability distribution.

[0040] The classification prediction probability distribution is input into the proxy classifier, and the proxy classifier outputs a proxy prediction probability distribution, including: the proxy classifier includes a class-specific module and a class-agnostic module. The class-specific module is used to extract unique information of each class from the vector of the classification prediction probability distribution of the classification model, to generate a vector of the proxy prediction probability distribution corresponding to the class, so as to model the characteristics of each class. The class-agnostic module inputs a two-element vector and outputs a vector of proxy new-old task prediction probability corresponding to the two-element vector. The two-element vector reflects the prediction probability of the classification model for the given sample belonging to the new task or the historical task. The proxy classifier combines the vector of the proxy prediction probability distribution and the vector of the proxy new-old task prediction probability to output the proxy prediction probability distribution.

[0041] Figure 4 The structural diagram of the proxy classifier provided by the embodiment of the application is shown in Figure 4 The proxy classifier is a lightweight neural network, which aims to finely correct and enhance the prediction of the classification model through two modules, i.e., the class-specific module and the class-agnostic module. The two modules work together to not only extract and model unique information of each class, but also capture and represent the relationship between the new task and the old task, so as to improve the performance of the classification model in the continuous learning scene. The two-layer multilayer perceptron model with the ReLU activation function is adopted as the network structure of the two modules in the application. The network structure is only a selection, and other more complex network structures can also be used as the proxy classifier.

[0042] The expression of the class-specific module is The input of the class-specific module is i.e., the vector of the classification prediction probability distribution of the input sample x by the classification model, and the class-specific module outputs the vector of the proxy prediction probability distribution corresponding to the class, which is used to model the characteristics of each class, i.e., the specific information of each class. The class-specific module helps the proxy classifier to maintain high discriminability in complex multi-class classification tasks by strengthening the unique representation of each class. The expression of the class-agnostic module is The input of the class-agnostic module is a two-element vector,

[0043] including and wherein, is the sum of the prediction probability of the classification model for predicting that the sample belongs to the new task class, is the sum of the prediction probability of the classification model for predicting that the sample belongs to the historical task class, is the class set of the new task, is the class set of the old task, represents the classification prediction probability vector The prediction probability corresponding to the cth category of the agent, and the category-independent module outputs a vector of prediction probabilities corresponding to the new and old tasks of the agent, which is used to depict the dynamic relationship between the new and old tasks. The category-independent module stably models the interaction between the new and old tasks, provides a robust basis for cross-task learning for the agent classifier, and helps to maintain the stability of performance in the continuous learning environment. The vector of prediction probability distribution of the agent and the vector of prediction probability of the new and old tasks of the agent are combined, and the agent prediction probability distribution output by the final agent classifier is obtained, which integrates the fine representation of each category and the stable relationship between the new and old tasks, and provides more comprehensive and robust information support for classification decision.

[0044] The training processes of the classification model and the agent classifier adopt different objective functions.

[0045] The expression of the objective function adopted in the training process of the classification model is: Where θ * is the optimal parameter of the classification model when the parameter of the fixed agent classifier is ω, argmin is an operator that can make the function reach the minimum value, is the classification loss function of the classification model, is the sample data of the new task and a batch of data randomly sampled from the memory cache, B is the number of sample data currently trained by the classification model, x i is the ith training sample, y i is the ith real category label, is the augmented classification model of the classification model f θ The agent classifier g ω is cascaded after the augmented classification model, θ is the parameter of the classification model, ° is the cascading operation.

[0046] S202: Calculate the classification loss according to the agent prediction probability distribution and the real category label, and fix the parameters of the agent classifier during the training process, and only update the parameters of the classification model to minimize the prediction error.

[0047] Update the parameters of the classification model, including: updating the parameters of the classification model using the stochastic gradient descent algorithm.

[0048] Specifically, during the training process, first, the input data is predicted by the agent classifier, and the output is the probability value corresponding to each category, which constitutes the agent prediction probability distribution. Each training sample has a real category label, which represents the real belonging category of the sample. This label is the supervision information in the training process, which is used to evaluate the prediction accuracy of the classification model.

[0049] The parameters of the classification model are updated using a stochastic gradient descent algorithm (SGD). The stochastic gradient descent algorithm first computes the gradient of the classification loss function with respect to the parameters of the classification model. This gradient indicates the direction in which the value of the classification loss function increases most rapidly in the parameter space. The parameters are updated in the direction opposite to the gradient, i.e., the direction in which the value of the classification loss function decreases. Specifically, each parameter is reduced by a product of a step size (learning rate) proportional to the gradient of that parameter. After such a step of gradient descent, the parameters of the classification model are updated so that, under the new parameters, the prediction results of the model are closer to the true labels, thereby reducing the classification loss.

[0050] Specifically, the objective function used in the classification model training process aims to find the optimal value of the parameters of the classification model when the parameters of the proxy classifier ω are fixed. The classification loss function is used to measure the difference between the prediction of the classification model and the true class label. By minimizing the classification loss function, it is hoped that the classification model can more accurately predict the class of the sample. The proxy classifier provides a mechanism for knowledge transfer to the classification model through its stable parameters ω. In the context of continual learning, the classification model tends to be biased towards the latest learned task, leading to forgetting of historical tasks. The presence of the proxy classifier helps to alleviate this new-task-oriented bias problem, as it can provide comprehensive guidance for all seen tasks. The new-task-oriented bias problem is a common problem in continual learning, which manifests as the classification model gradually forgetting the knowledge of historical tasks while overemphasizing new tasks during the continual learning process. By introducing the proxy classifier and cascading it to the classification model, an augmented classification model is constructed. This augmented model minimizes the difference between the predicted probability distribution and the true class label during the training process, while being guided by the proxy classifier, thereby avoiding the problem of excessive bias towards new tasks. In the continual learning process, by randomly sampling data from the memory buffer and mixing it with the sample data of the new task, a training batch containing information of new and old tasks can be constructed This way of using data helps the classification model to learn new tasks while maintaining the memory of historical tasks.

[0051] In summary, the classification model training process aims to solve the new-task-oriented bias problem in continual learning by designing an augmented classification model containing a proxy classifier and minimizing the classification loss under its guidance. By optimizing the parameters of the classification model and fixing the parameters of the proxy classifier during the training process, a classification model that can learn new task knowledge while maintaining historical task memory can be trained. This process is crucial for efficient continual learning.

[0052] The expression of the objective function used in the training process of the proxy classifier is: where ω * is the optimal parameter of the proxy classifier, a training loss function for the proxy classifier, randomly sample a batch of data from the memory buffer, the batch of samples is relatively balanced for all new tasks, B' is the number of samples currently trained for the proxy classifier, f θ*(ω) is set to the classification model based on the optimal value of the current ω. l(f (ω) (x i ), y i ) is the loss function of a single sample, used to calculate the difference between the result f i (x θ*(ω) ) predicted by the classification model for the training sample x i and the real class label y i .

[0053] It should be noted that the batch of data is relatively balanced for all new tasks, in fact, it is ideally expected that this batch can contain enough and uniformly distributed different old task samples, so that the model can review and consolidate the knowledge of these historical tasks at the same time when training new tasks. However, in actual operation, due to the limited size of the memory buffer, the diversity of historical tasks and the uncertainty of random sampling, it is difficult to guarantee that each batch is strictly balanced, so it is relatively balanced.

[0054] Specifically, in order to ensure that the proxy classifier can effectively absorb and correct the training bias of the classification model during the continuous learning process, a joint optimization framework is adopted. In this framework, the parameters of the proxy classifier are regarded as the hyperparameters of the classification model parameters, and the two are trained by alternating or simultaneous optimization.

[0055] First, during the training process of the classification model, only the parameters of the classification model are optimized, while the parameters ω of the proxy classifier remain fixed. The purpose of this step is to allow the classification model to achieve a certain level of performance on the task or dataset. Next, the data in the memory cache is used as an approximation of the balanced dataset. A batch of data is randomly sampled from the memory cache for training the proxy classifier. The training objective of the proxy classifier is to minimize its training loss function, which calculates the difference between the classification model's prediction results and the true class labels on the memory cache data. The optimal parameters of the proxy classifier are found by optimizing the training objective function. The proxy classifier adapts to changes in the classification model through the optimization of its parameters. This mutual adaptation mechanism helps the proxy classifier maintain its guidance effectiveness during continuous learning. By minimizing the training loss function, the proxy classifier learns how to help the classification model correct its classification prediction probability distribution, thereby avoiding the tendency problem facing new tasks. This makes the classification model perform more stably on the balanced dataset or the approximation of the memory cache data, reducing the risk of forgetting old task knowledge. The cross-entropy loss function is used as a standard to measure the difference between the prediction results and the true labels. During the training process of the proxy classifier, this loss function encourages the proxy classifier to learn how to better guide the classification model to fit the data distribution of all tasks. Through this joint optimization method, the proxy classifier and the classification model co-evolve, achieving effective consolidation and transfer of knowledge. While avoiding forgetting historical tasks, the classification model can also efficiently learn the knowledge of new tasks.

[0056] S203: Train the proxy classifier with data in the memory cache to assist the classification model in correcting the classification prediction probability distribution.

[0057] Training the proxy classifier with data in the memory cache to assist the classification model in correcting the classification prediction probability distribution includes: using the updated classification model to calculate the classification loss value of another batch of randomly selected samples in the memory cache, and updating the parameters of the proxy classifier during the training of the proxy classifier using the Adam algorithm. Specifically, through the Adam algorithm, the gradient information of the loss function is passed from the classification model to the proxy classifier to guide the update of the proxy classifier parameters. In this way, the classification model can continuously consolidate the historical task knowledge it has learned with the assistance of the proxy classifier, while quickly adapting and learning the knowledge of new tasks. This mechanism provides an effective way to achieve continuous learning, enabling the classification model to maintain its performance stability and generalization ability in a changing environment.

[0058] S104: When the iterative training step is completed, the average classification accuracy and the forgetting index are used to evaluate the accuracy of the training results.

[0059] Figure 3The specific flowchart of the acquisition method of the average classification accuracy and the forgetting index provided by the embodiments of the present application is shown in Figure 3 and includes steps S301 to S306.

[0060] S301: For each task, input the samples in its test set into the classification model, and the classification model generates a prediction result for each sample. The test set is divided from the continuous learning image classification data set.

[0061] Specifically, the test set is pre-divided from the entire continuous learning image classification data set and is used to evaluate the performance of the model on unseen data. For each test sample, it is input into the already trained classification model. The classification model receives the sample as input and generates a prediction result for each sample based on the feature representation and classification rules learned internally. This prediction result is a probability distribution indicating the likelihood of the sample belonging to each class, but when evaluating, the class with the highest probability is selected as the final predicted class.

[0062] S302: If the prediction result of the classification model matches the true class label of the sample, it is considered a successful prediction.

[0063] Specifically, for the prediction result generated by the classification model for each test sample, it needs to be compared with the true class label of the sample. If the highest probability class in the prediction result matches the true class label of the sample, it is considered a successful prediction. The number of successful predictions will be used to calculate the prediction accuracy of the task later.

[0064] S303: The prediction accuracy of each task is defined as the number of successful predictions divided by the total number of samples in the test set of the task.

[0065] Specifically, after completing the prediction of all test samples and determining the successful predictions, the prediction accuracy of each task can be calculated. The prediction accuracy is obtained by dividing the number of successful predictions by the total number of samples in the test set of the task. This ratio reflects the accuracy of the classification model on the task, i.e. the ability of the classification model to correctly classify the samples in the test set of the task.

[0066] S304: The maximum prediction accuracy of each task is the highest prediction accuracy achieved by the task during the entire training process.

[0067] Specifically, during the training process of continual learning, the classification model will go through multiple iterations or training cycles. At the end of each cycle, the prediction accuracy of the current model is calculated. In order to evaluate the best performance of the model throughout the training process, the highest prediction accuracy achieved by each task throughout the training process, i.e., the maximum prediction accuracy of the task, needs to be recorded. This indicator reflects the best classification effect that the classification model can achieve on the task.

[0068] S305: Calculate the average of all task maximum prediction accuracies as the average classification accuracy of the classification model.

[0069] Specifically, after the maximum prediction accuracies of all tasks are recorded, the average of these maximum prediction accuracies can be calculated as the average classification accuracy of the classification model. The average classification accuracy is a global indicator that measures the overall classification accuracy of the classification model on all seen tasks. This indicator is of great significance for evaluating the generalization ability and stability of the classification model on multiple tasks.

[0070] S306: For each task, calculate the difference between its prediction accuracy and maximum prediction accuracy, and take the difference as the forgetting indicator of the task.

[0071] Specifically, the forgetting indicator is used to measure the degree of forgetting of historical task knowledge by the classification model during the training process of continual learning. For each task, the difference between its final prediction accuracy, i.e., the prediction accuracy obtained after all training cycles end, and the maximum prediction accuracy is calculated. This difference is the forgetting indicator of the task. The smaller the forgetting indicator, the less the classification model forgets the knowledge of historical tasks during the continual learning process, and the better the stability of the model. Conversely, the larger the forgetting indicator, the more likely the classification model forgets the knowledge of old tasks when learning new tasks, which may lead to a decline in the performance of the classification model.

[0072] The accuracy of the training results is evaluated using the average classification accuracy and the forgetting indicator, including:

[0073] The expression of the average classification accuracy is: where ACC is the average classification accuracy, T is the number of all tasks in the training process, and a t,T is the prediction accuracy of the tth task calculated after the training process ends, i.e., all tasks have completed training. Specifically, the prediction accuracy of the tth task is obtained by dividing the number of successfully predicted samples in the test set of the tth task by the total number of samples in the test set of the tth task.

[0074] Specifically, the higher the average classification accuracy, the better the overall classification performance of the classification model on all tasks, i.e., the classification model can more accurately identify samples of different tasks.

[0075] The expression of the forgetting index is: where FM is the forgetting index, T is the number of all tasks in the training process, a * is the maximum prediction accuracy of the t-th task in the training process, a t,T is the prediction accuracy of the t-th task calculated after the training process, and T is the number of all tasks in the training process.

[0076] Specifically, the smaller the forgetting index, the less the classification model forgets the knowledge of historical tasks in the process of continuous learning, and the better the stability of the classification model. On the contrary, the larger the forgetting index, the more likely the classification model forgets the knowledge of historical tasks when learning new tasks, which will lead to the decline of the performance of the classification model on historical tasks.

[0077] In the context of continuous learning, the classification model does not learn all tasks at once, but learns one after another in sequence. Whenever the model completes the training of a new task, it may forget the previously learned tasks due to learning new tasks. In order to evaluate the performance of the classification model in the process of continuous learning, the prediction ability of the model on all learned tasks needs to be tested after the training of each task is completed. Specifically, when the model completes the training of the q-th task, we use the test set of the v-th task (where v can be any value less than or equal to q, because the classification model has learned the first v tasks by the end of the q-th task) to evaluate the prediction accuracy of the model on the i-th task. This prediction accuracy is denoted as a q,v . Where q represents the index of the task being evaluated, and v represents how many tasks have been trained so far. a q,v represents the prediction accuracy of the model on the q-th task after training the v-th task. In this way, it can be tracked how the prediction performance of the classification model on each task changes in the process of continuous learning. In particular, it can be observed whether the model forgets historical tasks due to learning new tasks, which can be achieved by comparing a q,v for different v values. If for a certain historical task v, a q,v significantly decreases as q increases (i.e., as the classification model learns more new tasks), it indicates that the classification model has forgotten.

[0078] Table 2 provides performance comparison of CIFAR-10, CIFAR-100 and Tiny-ImageNet classification tasks according to embodiments of the present application. The top of Table 2 lists three datasets, which are CIFAR-10, CIFAR-100 and Tiny-ImageNet, respectively. These datasets are commonly used benchmark datasets for classification tasks in the field of computer vision. The Method column in Table 2 shows different model configurations, including M = 0.2k, M = 0.5k, M = 2k, M = 5k, etc., which represent different parameter settings of the model. In addition, methods with the suffix “proxy classifier” are also listed, which means that proxy classifiers are introduced based on these methods. ACC represents the average classification accuracy of each method on each dataset. FM represents the forgetting index. For the CIFAR-10 dataset, the accuracy of the method with M = 0.2k is 35.21%, and the forgetting index is 50.28%. The accuracy of the method with M = 0.5k is improved to 42.32%, and the forgetting index is reduced to 40.80%. This shows that with the increase of model capacity, the performance is improved. The methods with proxy classifiers (such as ER-proxy classifier, DER-proxy classifier, etc.) show higher accuracy and lower forgetting index than the original methods (such as ER, DER++, etc.) in most cases, which shows that the proxy classifier is effective in improving the model performance and reducing the forgetting problem. For CIFAR-100 and Tiny-ImageNet datasets, similar trends are also observed, that is, with the increase of model capacity and the introduction of proxy classifiers, the average classification accuracy of the model is improved, and the forgetting index is reduced.

[0079] It needs to be explained that ER, DER++, RAR and CLSER are four different continuous learning methods, each of which adopts different strategies to optimize the classification model to reduce or avoid the forgetting phenomenon in the continuous learning task. The present application adds an agent classifier on the basis of the classification model, which can be directly applied to the four continuous learning methods and further improve the performance on the basis of them. Specifically, ER (Experience Replay) is an experience replay strategy. DER++ (Dark Experience Replay++) is an enhanced version of ER, that is, an item of distillation loss is added on the basis of ER. RAR (Repeated Augmented Rehearsal) is a random augmentation strategy used on the basis of DER++, that is, random transformation is performed on the data to increase the diversity of samples. CLSER (Complementary Learning System Experience Replay) constructs a stability model and a plasticity model to better transfer knowledge to the classification model.

[0080] Table 2

[0081]

[0082] Table 3 is the performance comparison of different methods provided by the embodiments of the present application in processing CIFAR10, CIFAR100 and Tiny-ImageNet data sets. For the CIFAR10 data set, Table 3 lists different parameter combinations (such as a 1,5 ,a 2,5These parameters represent different configurations of the model. For each parameter combination, Table 3 gives the values of the average classification accuracy (ACC) and the forgetting metric (FM). With the introduction of the proxy classifier, the ER values decrease, indicating that the proxy classifier helps to reduce the error rate and improve the model performance. For the CIFAR100 dataset, Table 3 lists another set of parameter combinations, which are different from the ones used for the CIFAR10 dataset, reflecting the adjustment of the model configurations on the more complex dataset. Similar to CIFAR10, Table 3 gives the ACC and FM values for each parameter combination. With M = 5k, the ACC and FM values for different parameter combinations vary, but overall, the error rates are generally higher, reflecting the higher difficulty of the CIFAR100 dataset relative to CIFAR10. Similarly, with the introduction of the proxy classifier, the FM values decrease, indicating that the proxy classifier also has an effect of improving the model performance on the CIFAR100 dataset. For the Tiny-ImageNet dataset, Table lists another set of parameter combinations, which are optimized for the characteristics of the Tiny-ImageNet dataset. Table 3 gives the ACC and FM values for each parameter combination. With M = 5k, the ACC and FM values for different parameter combinations reflect the performance of the model on the Tiny-ImageNet dataset. With the introduction of the proxy classifier, the FM values also decrease, indicating that the proxy classifier can effectively reduce the error rate and improve the model performance on the Tiny-ImageNet dataset.

[0083] Table 3

[0084]

[0085] Table 4 is the classification performance of different scales provided by the embodiments of the present application under different methods for CIFAR-10 dataset under online Blurry-10 setting. Among them, the online Blurry-10 setting refers to the training data of each task in the online continuous learning training process has a partial overlap, that is, about 10% of the training data of each task randomly appears in the training data of other tasks, and at the same time, the task also contains about 10% of the training data from other tasks. The setting requires the classification model to be not limited by the task boundary, which is a more practical and flexible experimental setting. CIFAR-10 is a continuous learning image classification dataset, and Table 4 compares the performance of M=0.2k and M=0.5k. As can be seen from Table 4, after introducing the proxy classifier, the classification accuracy and negative sample minimization of all methods are significantly improved, especially RAR-proxy classifier and CSLER-proxy classifier perform excellently in multiple indicators. The results show that the technical solution can effectively improve the classification accuracy of the corresponding method (ER, DER, RAR and CSLER) and greatly reduce the degree of forgetting under this setting, which shows that the proxy classifier proposed in the present application is not affected by the task boundary, and is a very flexible way that can be applied to various more practical continuous learning experimental settings.

[0086] Table 4

[0087]

[0088] Figure 5 The average classification accuracy comparison chart of CIFAR-100 dataset monitored in real time in the online continuous learning process provided by the embodiments of the present application. The proxy classifier can help the classification model correct the classification prediction probability distribution throughout the continuous learning process, avoid the problem of the classification model facing the new task, so as to help the classification model to achieve better convergence results, and effectively improve the classification accuracy of online continuous learning.

[0089] Figure 6 The comparison diagram of the probability distribution of each class predicted by the classification model using the CIFAR-100 dataset provided by the embodiments of the present application. Figure 6 Task Index in the figure is the task index, and the X-axis of the chart marks the task index values from 0 to 9, and the Y-axis represents the task distribution of ER and ER-proxy classifier methods. The proxy classifier can obviously help the classification model to correct the predicted probability distribution of each class, suppress the prediction probability of the new task, and at the same time improve the prediction probability of the previous historical task, effectively helping the original classification model to avoid the problem of facing the new task in the continuous learning process.

[0090] Table 5 is a comparison result of the embodiments of the present application under different methods for the CIFAR-10 dataset offline continual learning setting. Offline continual learning means that each task can be iteratively trained multiple times until convergence during the continual learning process. Compared with the online continual learning setting which can only process streaming data at the same time, the data distribution change of offline continual learning is relatively stable. With the support of more computing resources, offline continual learning can usually achieve better convergence results of the classification model, thereby obtaining higher prediction accuracy. The results show that the technical solution can effectively improve the classification accuracy of the corresponding method (ER, DER, RAR and CLSER) and greatly reduce the forgetting degree under the offline continual learning setting. It is illustrated that the present application can not only be applied to online continual learning with dramatic changes in data distribution, but also be applied to conventional offline continual learning, and effectively improve the performance of the corresponding method.

[0091] Table 5

[0092]

[0093] The embodiments of the present application also provide an image pair classification device 700 in an online continual learning scenario, as shown in the figure, the device comprises a construction module 701, a mixing module 702, an iteration module 703 and an evaluation module 704. Figure 7 The construction module 701 is configured to construct a continual learning image classification dataset. The continual learning image classification dataset comprises a plurality of image classification tasks, and each classification task comprises a plurality of images and corresponding true class labels.

[0094] The construction module 701 is configured to construct a continual learning image classification dataset. The continual learning image classification dataset comprises a plurality of image classification tasks, and each classification task comprises a plurality of images and corresponding true class labels.

[0095] The mixing module 702 is configured to save a small amount of historical task data as a memory cache, and mix the memory cache with current new task data when learning a new task, and input the mixed data into a classification model as training data. The classification model outputs a classification prediction probability distribution.

[0096] The iteration module 703 is configured to iteratively perform an iterative training step until a stop condition is met. The stop condition is that a preset iteration number is reached or a classification loss value is reduced to less than a preset threshold. The iterative training step comprises: inputting the classification prediction probability distribution into a proxy classifier, and the proxy classifier outputs a proxy prediction probability distribution. The classification loss is calculated according to the proxy prediction probability distribution and the true class label, and the parameters of the proxy classifier are fixed during the training process, and only the parameters of the classification model are updated to minimize the prediction error. The proxy classifier is trained with the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution.

[0097] The evaluation module 704 is configured to evaluate the accuracy of the training result using the average classification accuracy and the forgetting index after the iterative training step is completed.

[0098] Some of the modules in the apparatus described in this application can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, classes, etc., that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0099] The apparatus or modules described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. For the convenience of description, the above apparatus is described as various modules with functions. In the implementation of the embodiments of the application, the functions of the modules can be implemented in one or more software and / or hardware. Of course, the modules that implement certain functions can also be implemented by a combination of multiple sub-modules or sub-units.

[0100] The methods, apparatuses or modules described in this application can be implemented in a computer-readable program code in any appropriate manner. For example, the controller can take the form of, for example, a microprocessor or a processor, and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, application specific integrated circuits (Application Specific Integrated Circuit, abbreviated as: ASIC), programmable logic controllers and embedded microcontrollers. Examples of the controller include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in a pure computer-readable program code manner, the same function can also be achieved by logically programming the method steps to implement the controller in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, such a controller can be considered as a hardware component, and the means included therein for implementing various functions can also be regarded as structures within the hardware component. Alternatively, the means for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0101] As Figure 8As shown, the embodiment of the present application further provides an image pair classification server in an online continuous learning scenario, comprising a memory 801 and a processor 802; the memory 801 is used to store computer executable instructions; the processor 802 is used to execute the computer executable instructions to realize the image pair classification method in an online continuous learning scenario as described above.

[0102] The embodiment of the present application further provides a computer readable storage medium, which stores executable instructions, and the computer executes the executable instructions to realize the image pair classification method in an online continuous learning scenario as described above.

[0103] From the above description of the embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software and the necessary hardware. Based on such an understanding, the technical solutions of the present application can be embodied in the form of a software product or in the form of data migration in the implementation process. The computer software product can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the method described in the embodiments of the present application.

[0104] The various embodiments in the specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referred to each other, and each embodiment mainly describes the difference from other embodiments. The whole or part of the present application can be used in many general or special computer system environments or configurations.

[0105] The above embodiments are only used to illustrate the technical solutions of the present application, and not to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the present application.

Claims

1. An image pair-wise classification method in an online continual learning scenario, characterized in that, The method comprises the following steps: constructing a continuous learning image classification dataset; wherein the continuous learning image classification dataset comprises a plurality of image classification tasks, and each classification task comprises a plurality of images and corresponding true class labels; saving data of a small number of historical tasks as a memory cache, mixing the memory cache with current new task data when learning a new task, and inputting the mixed data into a classification model as training data, wherein the classification model outputs a classification prediction probability distribution; iteratively performing an iterative training step until a stop condition is met; wherein the stop condition is that a preset number of iterations is reached or a classification loss value is reduced to less than a preset threshold; the iterative training step comprises: inputting the classification prediction probability distribution into an agent classifier, and the agent classifier outputs an agent prediction probability distribution; the inputting of the classification prediction probability distribution into the agent classifier and the outputting of the agent prediction probability distribution by the agent classifier comprise that the agent classifier comprises a class-specific module and a class-agnostic module; the class-specific module is used to extract unique information of each class from a vector of the classification prediction probability distribution of the classification model, generate a vector of the agent prediction probability distribution corresponding to the class, and model the characteristics of each class; the class-agnostic module inputs a two-element vector and outputs a vector of agent new-old task prediction probability; wherein the two-element vector reflects the prediction probability of the classification model for a given sample belonging to a new task or a historical task; the agent classifier combines the vector of the agent prediction probability distribution and the vector of the agent new-old task prediction probability to output the agent prediction probability distribution; a classification loss is calculated according to the agent prediction probability distribution and the true class label, and the parameters of the agent classifier are fixed during the training process, and only the parameters of the classification model are updated to minimize the prediction error; the agent classifier is trained with the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; after the iterative training step is completed, the accuracy of the training result is evaluated using average classification accuracy and forgetting index; The training processes for the classification model and the surrogate classifier employ different objective functions; the expression for the objective function used in the classification model training process is as follows: ;in, The parameters of the fixed agent classifier are The optimal parameters for the time-series classification model. An operator that takes a parameter that minimizes the function. , Let be the classification loss function of the classification model. To randomly sample a batch of data from the memory cache, as well as sample data from new tasks, This represents the number of samples currently being trained on the classification model. For the i-th training sample, For the i-th real category label, , For classification models Then cascaded agent classifier augmented classification model, For the parameters of the classification model, This is a cascaded operation; the objective function used in the training process of the surrogate classifier is expressed as: ;in, The optimal parameters for the surrogate classifier. , Let be the training loss function for the surrogate classifier. To randomly sample a batch of data from the memory cache, This represents the number of samples currently used for training the surrogate classifier. For parameters Set based on the current The optimal value classification model; The loss function for a single sample is used to calculate the classification model's performance on the training samples. The result of the prediction With real category labels The differences between them; the average classification accuracy and the forgetting index are obtained in the following manner: for each task, input the samples in the test set into the classification model, and the classification model generates a prediction result for each sample; wherein the test set is divided from the continuous learning image classification dataset; if the prediction result of the classification model matches the true class label of the sample, it is considered as a successful prediction; the prediction accuracy of each task is defined as the number of successful predictions divided by the total number of samples in the test set of the task; the maximum prediction accuracy of each task is the highest prediction accuracy achieved during the entire training process; the average value of the maximum prediction accuracies of all tasks is calculated as the average classification accuracy of the classification model; for each task, the difference between the prediction accuracy and the maximum prediction accuracy is calculated as the forgetting index of the task.

2. The image pair-wise classification method under online continual learning scenario according to claim 1, characterized in that, the saving of the data of the small number of historical tasks as the memory cache, the mixing of the memory cache with the current new task data when learning the new task, and the inputting of the mixed data into the classification model as the training data to output the classification prediction probability distribution comprise: When learning a new task, a batch of data is randomly sampled from the memory cache and mixed with the data of the current new task as training data; The training data is input into the classification model, and the classification prediction probability distribution corresponding to each class label is output through forward propagation.

3. The image pair-wise classification method under online continual learning scenario according to claim 1, wherein, The parameters of the classification model are updated, including using the stochastic gradient descent algorithm to update the parameters of the classification model.

4. The image pair-wise classification method under online continual learning scenario according to claim 1, wherein, The proxy classifier is trained using the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution, including calculating the value of the classification loss of another batch of samples randomly selected from the memory cache using the updated parameters of the classification model, and updating the parameters of the proxy classifier in the process of training the proxy classifier through the Adam algorithm.

5. The image pair-wise classification method under online continual learning scenario according to claim 1, characterized in that, The accuracy of the training result is evaluated using the average classification accuracy and the forgetting index, including: The expression for average classification accuracy is: ;in, To achieve average classification accuracy, This represents the number of all tasks during the training process. The calculated number of lines after the training process is complete. Prediction accuracy for each task; The expression for the forgetting index is: ;in, Forgetting indicator, This represents the number of all tasks during the training process. Let be the maximum prediction accuracy for the t-th task during training. The calculated number of lines after the training process is complete. Prediction accuracy for each task This represents the number of all tasks during the training process.

6. An image pair-wise classification apparatus in an online continual learning scenario, comprising: including: A construction module is configured to construct a continuous learning image classification dataset; wherein the continuous learning image classification dataset includes a plurality of image classification tasks, and each classification task includes a plurality of images and corresponding real class labels; A mixing module is configured to save a small amount of data of historical tasks as a memory cache, and when learning a new task, the memory cache is mixed with the data of the current new task as training data input into the classification model, and the classification model outputs a classification prediction probability distribution; An iteration module is configured to iteratively execute an iterative training step until a stop condition is met; wherein the stop condition is that a preset iteration number is reached or the value of the classification loss is reduced to less than a preset threshold; the iterative training step includes inputting the classification prediction probability distribution into the proxy classifier, and the proxy classifier outputs a proxy prediction probability distribution; the inputting of the classification prediction probability distribution into the proxy classifier and the outputting of the proxy prediction probability distribution by the proxy classifier includes that the proxy classifier includes a class-specific module and a class-independent module; the class-specific module is configured to extract unique information of each class from a vector of the classification prediction probability distribution of the classification model, generate a vector of the proxy prediction probability distribution corresponding to the class, and model the characteristics of each class; the class-independent module inputs a two-element vector and outputs a vector of corresponding proxy new-old task prediction probability; wherein the two-element vector reflects the prediction probability of the classification model for a given sample belonging to a new task or a historical task; the proxy classifier combines the vector of the proxy prediction probability distribution and the vector of the proxy new-old task prediction probability to output the proxy prediction probability distribution; the classification loss is calculated according to the proxy prediction probability distribution and the real class label, and the parameters of the proxy classifier are fixed in the training process, and only the parameters of the classification model are updated to minimize the prediction error; the proxy classifier is trained using the data in the memory cache to assist the classification model in correcting the classification prediction probability distribution; An evaluation module is configured to evaluate the accuracy of the training result using the average classification accuracy and the forgetting index when the iterative training step is completed. The training processes for the classification model and the surrogate classifier employ different objective functions; the expression for the objective function used in the classification model training process is as follows: ;in, The parameters of the fixed agent classifier are The optimal parameters for the time-series classification model. An operator that takes a parameter that minimizes the function. , Let be the classification loss function of the classification model. To randomly sample a batch of data from the memory cache, as well as sample data from new tasks, This represents the number of samples currently being trained on the classification model. For the i-th training sample, For the i-th real category label, , For classification models Then cascaded agent classifier augmented classification model, For the parameters of the classification model, This is a cascaded operation; the objective function used in the training process of the surrogate classifier is expressed as: ;in, The optimal parameters for the surrogate classifier. , Let be the training loss function for the surrogate classifier. To randomly sample a batch of data from the memory cache, This represents the number of samples currently used for training the surrogate classifier. For parameters Set based on the current The optimal value classification model; The loss function for a single sample is used to calculate the classification model's performance on the training samples. The result of the prediction With real category labels The differences between them; The average classification accuracy and the forgetting index are obtained in the following manner: for each task, inputting samples in the test set of the task into the classification model, the classification model generating a prediction result for each sample; wherein the test set is divided from the continuous learning image classification dataset; if the prediction result of the classification model matches the true class label of the sample, it is considered as a successful prediction; the prediction accuracy of each task is defined as the number of successful predictions divided by the total number of samples in the test set of the task; the maximum prediction accuracy of each task is the highest prediction accuracy reached by the task during the entire training process; calculating the average of the maximum prediction accuracies of all tasks as the average classification accuracy of the classification model; for each task, calculating the difference between the prediction accuracy and the maximum prediction accuracy as the forgetting index of the task.

Images