Image classification method and device based on class incremental learning model, equipment and medium

Abstract

The invention discloses an image classification method and device based on a class incremental learning model, equipment and a medium, and relates to the technical field of image recognition. Determining task general knowledge features corresponding to the current image data and task specific knowledge features corresponding to the target historical image data; determining a first image classification result corresponding to the current image data based on the task general knowledge features by using a duplex classifier of the initial class incremental learning model, and determining a second image classification result corresponding to the target historical image data based on the task specific knowledge features; and determining a target classification loss based on the first image classification result and the second image classification result, and updating model parameters of the initial class incremental learning model based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model. The problem of disastrous forgetting during class incremental learning of the deep neural network can be relieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image recognition, and in particular to an image classification method and device based on a class-incremental learning model, an image classification equipment and a medium. BACKGROUND

[0002] As a core technology of computer vision, deep neural networks have achieved excellent results in image classification tasks. However, in real-world dynamic scenarios, in order to subsequently learn new classes and prevent forgetting of already learned classes, the class-incremental learning problem is proposed. Existing algorithms for the class-incremental learning problem mostly use knowledge distillation, dynamic network growth or sample playback to avoid forgetting. Among them, the current mainstream knowledge distillation method is based on transferring the knowledge of the old model to the new task, but it will lose the intermediate layer knowledge of the feature extractor, causing feature degradation. The current work only distills the probability distribution output from the classification layer, without considering the rich semantic information contained in the intermediate layers of the network, thereby reducing the model's ability to adapt to the feature distribution of new and old tasks. In the current widely used class-incremental learning method, it is difficult to meet the needs of privacy-sensitive scenarios.

[0003] In summary, how to alleviate the catastrophic forgetting problem of deep neural networks when learning new classes is a technical problem that needs to be solved at present. SUMMARY

[0004] Therefore, the purpose of the present application is to provide an image classification method and device based on a class-incremental learning model, which can effectively alleviate the catastrophic forgetting problem faced by deep neural networks during class-incremental learning. The specific solutions are as follows:

[0005] In a first aspect, the present application provides an image classification method based on a class-incremental learning model, applied to a computer device, comprising:

[0006] obtaining a pre-trained initial class-incremental learning model; the initial class-incremental learning model is a model trained based on historical image data corresponding to historical training tasks;

[0007] obtaining current image data corresponding to a current training task and inputting the current image data into the initial class-incremental learning model; the image class corresponding to the current image data is different from the image class corresponding to the historical image data;

[0008] In the process of processing the current image data by using the initial class incremental learning model, target historical image data is determined based on the historical image data, and task knowledge features corresponding to the target image data are determined by using a preset attention module of the initial class incremental learning model; wherein the target image data includes the current image data and the target historical image data, and the task knowledge features include task general knowledge features corresponding to the current image data and task specific knowledge features corresponding to the target historical image data.

[0009] A duplex classifier of the initial class incremental learning model is used to determine a first image classification result corresponding to the current image data based on the task general knowledge features, and a second image classification result corresponding to the target historical image data based on the task specific knowledge features.

[0010] A target classification loss is determined based on the first image classification result and the second image classification result, and model parameters of the initial class incremental learning model are updated based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, so as to determine a target image classification corresponding to a to-be-classified image by using the target class incremental learning model.

[0011] Optionally, the target historical image data is determined based on the historical image data, including:

[0012] Initial historical image data is determined from the historical image data.

[0013] The initial historical image data is subjected to rotation processing and mirror image processing to obtain a plurality of enhanced historical image data, and the enhanced historical image data is used as the target historical image data.

[0014] Optionally, before the task knowledge features corresponding to the target image data are determined by using the preset attention module of the initial class incremental learning model, the method further includes:

[0015] The current image data is converted into a corresponding first feature block sequence, and the target historical image data is converted into a corresponding second feature block sequence.

[0016] Deep semantic features of the first feature block sequence and the second feature block sequence are extracted by using a self-attention module of the initial class incremental learning model to obtain a feature map corresponding to the target image data, so as to determine the task knowledge features corresponding to the target image data by using the preset attention module of the initial class incremental learning model and based on the feature map.

[0017] The self-attention module is a module composed of a multi-head self-attention layer, a multi-layer perception mechanism, layer normalization, and a residual connection.

[0018] Optionally, before the obtaining of the pre-trained initial class-incremental learning model, the method further includes:

[0019] initializing a general knowledge vocabulary, generating a first task-specific vocabulary corresponding to the historical training task, and storing the general knowledge vocabulary and the first task-specific vocabulary in a vocabulary pool of the initial class-incremental learning model;

[0020] updating the general knowledge vocabulary in the vocabulary pool during the training of the initial class-incremental learning model using the historical training task;

[0021] Correspondingly, the determining of the task knowledge feature corresponding to the target image data based on the feature map using the preset attention module of the initial class-incremental learning model includes:

[0022] In the process of processing the feature map using the first attention module of the initial class-incremental learning model, the current general knowledge vocabulary in the vocabulary pool of the initial class-incremental learning model is taken as a target general knowledge vocabulary corresponding to the current training task;

[0023] determining a first splicing body corresponding to the feature map and the target general knowledge vocabulary, taking the feature map as a first query, and taking the first splicing body as a first key and a first value;

[0024] determining a first attention weight corresponding to the first query and the first key through scaled dot-product attention, and determining a target fusion feature based on the first attention weight and the first value;

[0025] inputting the target fusion feature into a second attention module of the initial class-incremental learning model, so as to determine the task knowledge feature corresponding to the target image data based on the target fusion feature using the second attention module.

[0026] Optionally, the determining of the task knowledge feature corresponding to the target image data based on the target fusion feature using the second attention module includes:

[0027] In the process of processing the target fusion feature using the second attention module, the first task-specific vocabulary corresponding to the historical training task and a second task-specific vocabulary corresponding to the current training task are obtained from the vocabulary pool;

[0028] determine a second splicing body corresponding to the target fusion feature, the first task-specific word item and the second task-specific word item, and take the second task-specific word item as a second query, and take the second splicing body as a second key and a second value;

[0029] determine a second attention weight corresponding to the second query and the second key through the scaled dot-product attention, and determine the task knowledge feature corresponding to the target image data based on the second attention weight and the second value.

[0030] Optionally, the dual classifier includes a first classifier and a second classifier.

[0031] Correspondingly, the dual classifier using the initial class incremental learning model determines a first image classification result corresponding to the current image data based on the task general knowledge feature, and determines a second image classification result corresponding to the target historical image data based on the task-specific knowledge feature, including:

[0032] input the task general knowledge feature into the first classifier in the dual classifier, and output the first image classification result corresponding to the current image data by using the first classifier;

[0033] input the task-specific knowledge feature into the second classifier in the dual classifier, freeze the parameters of the second classifier in the process of processing the task-specific knowledge feature by using the second classifier, and output the second image classification result corresponding to the target historical image data.

[0034] Optionally, the model parameter of the initial class incremental learning model is updated based on the preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, including:

[0035] determine a target forgetting rate based on the preset active forgetting balance mechanism, and construct a Fisher information matrix corresponding to the historical training task; the target forgetting rate is used to control the forgetting degree of the initial class incremental learning model to the historical training task;

[0036] determine a target importance of the model parameter to the historical training task based on the Fisher information matrix, and determine a target weight corresponding to the model parameter;

[0037] determine a target total loss based on the target importance, the target weight, the target forgetting rate and the target classification loss, and update the model parameter based on the target total loss to obtain the target class incremental learning model.

[0038] In a second aspect, the present application provides an image classification device based on a class-incremental learning model, applied to a computer device, comprising:

[0039] An initial class-incremental learning model acquisition module is configured to acquire an initial class-incremental learning model trained in advance; the initial class-incremental learning model is a model trained based on historical image data corresponding to historical training tasks;

[0040] A current image data acquisition module is configured to acquire current image data corresponding to a current training task and input the current image data into the initial class-incremental learning model; the image category corresponding to the current image data is different from the image category corresponding to the historical image data;

[0041] A task knowledge feature determination module is configured to determine target historical image data based on the historical image data during processing of the current image data by the initial class-incremental learning model, and determine task knowledge features corresponding to target image data by using a preset attention module of the initial class-incremental learning model; wherein the target image data includes the current image data and the target historical image data, and the task knowledge features include task general knowledge features corresponding to the current image data and task specific knowledge features corresponding to the target historical image data;

[0042] An image classification result determination module is configured to determine a first image classification result corresponding to the current image data based on the task general knowledge features by using a duplex classifier of the initial class-incremental learning model, and determine a second image classification result corresponding to the target historical image data based on the task specific knowledge features.

[0043] A target class-incremental learning model determination module is configured to determine a target classification loss based on the first image classification result and the second image classification result, and update model parameters of the initial class-incremental learning model based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class-incremental learning model, so as to determine a target image classification corresponding to a to-be-classified image by using the target class-incremental learning model.

[0044] In a third aspect, the present application provides an electronic device, comprising:

[0045] A memory is configured to save a computer program;

[0046] A processor is configured to execute the computer program to implement the image classification method based on the class-incremental learning model as described above.

[0047] In a fourth aspect, the present application provides a computer readable storage medium for storing a computer program; wherein the computer program is executed by a processor to implement the image classification method based on the class-incremental learning model.

[0048] In this application, firstly, a pre-trained initial class incremental learning model is obtained; the initial class incremental learning model is a model trained based on historical image data corresponding to historical training tasks; then, current image data corresponding to the current training task is obtained and input into the initial class incremental learning model; the image category corresponding to the current image data is different from the image category corresponding to the historical image data; subsequently, during the processing of the current image data using the initial class incremental learning model, target historical image data is determined based on the historical image data, and the task knowledge features corresponding to the target image data are determined using the preset attention module of the initial class incremental learning model; wherein, the target image data includes the current image data and the target historical image. The data, the task knowledge features include task-general knowledge features corresponding to the current image data and task-specific knowledge features corresponding to the target historical image data; then, using the duplex classifier of the initial class incremental learning model, a first image classification result corresponding to the current image data is determined based on the task-general knowledge features, and a second image classification result corresponding to the target historical image data is determined based on the task-specific knowledge features; finally, a target classification loss is determined based on the first image classification result and the second image classification result, and the model parameters of the initial class incremental learning model are updated based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, so as to determine the target image classification corresponding to the image to be classified using the target class incremental learning model. As can be seen from the above, in this application, an initial class incremental learning model is first determined based on historical image data corresponding to the historical training task, and then current image data corresponding to the current training task with image categories different from the historical image data is obtained, and the current image data is input into the initial class incremental learning model to train the model. Specifically, the preset attention module of the initial class incremental learning model determines the task-general knowledge features corresponding to the current image data and the task-specific knowledge features corresponding to the target historical image data. Subsequently, using the duplex classifier of the initial class incremental learning model, the first image classification result corresponding to the current image data is determined based on task-general knowledge features, and the second image classification result corresponding to the target historical image data is determined based on task-specific knowledge features. Then, the target classification loss is determined based on the image classification results, and the model parameters of the initial class incremental learning model are updated based on a preset active forgetting balance mechanism and the target classification loss to obtain the target class incremental learning model, thus completing the model training. The target class incremental learning model is then applied to the image classification task. In this way, this application can achieve the collaborative transfer of general knowledge and task-specific knowledge across tasks, effectively alleviating the catastrophic forgetting problem faced by deep neural networks during class incremental learning. Attached Figure Description

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings described below are only a part of the embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0050] Figure 1 A flow chart of an image classification method based on a class-incremental learning model is provided for the present application.

[0051] Figure 2 A workflow chart of a specific general attention module and task attention module is provided for the present application.

[0052] Figure 3 A training flow chart of a specific class-incremental learning model is provided for the present application.

[0053] Figure 4 A workflow chart of a specific class-incremental learning model is provided for the present application.

[0054] Figure 5 A precision change chart of a specific target class-incremental learning model on CIFAR10-10 is provided for the present application.

[0055] Figure 6 A precision change chart of a specific target class-incremental learning model on CIFAR20-20 is provided for the present application.

[0056] Figure 7 A precision change chart of a specific target class-incremental learning model on ImageNet10-10 is provided for the present application.

[0057] Figure 8 A structural schematic diagram of an image classification device based on a class-incremental learning model is provided for the present application.

[0058] Figure 9 A structural diagram of an electronic device is provided for the present application. DETAILED DESCRIPTION

[0059] The technical solutions in the embodiments of the present application will be described clearly and completely below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0060] As a core technology of computer vision, deep neural network has achieved excellent results on image classification tasks. However, in real-world dynamic scenarios, in order to realize the learning of newly added classes and prevent forgetting of already learned classes, the class-incremental learning problem is proposed. The existing algorithms for class-incremental learning problem mostly use knowledge distillation, dynamic network growth or sample playback to avoid forgetting phenomenon. Among them, the current mainstream knowledge distillation method is based on transferring the knowledge of the old model to the new task, but it will lose the intermediate layer knowledge of the feature extractor, causing the problem of feature degradation, and the current work only distills the probability distribution output from the classification layer, without considering the rich semantic information contained in the intermediate layers of the network, thereby reducing the feature distribution adaptation ability of the model to new and old tasks. In the widely used class-incremental learning method, it is difficult to meet the needs of privacy-sensitive scenarios. Therefore, the present application provides an image classification scheme based on a class-incremental learning model, which can effectively alleviate the catastrophic forgetting problem faced by deep neural networks in the class-incremental learning process.

[0061] Referring to Figure 1 The embodiment of the application discloses an image classification method based on a class-incremental learning model, applied to a computer device, which can include:

[0062] Step S11, obtaining a pre-trained initial class-incremental learning model; the initial class-incremental learning model is a model trained based on historical image data corresponding to historical training tasks.

[0063] Under the paradigm of class-incremental learning, the processing and organization mode of training samples has a great influence on the learning of the model. The process of model training can be divided into a series of tasks, and in each task, a batch of new image class samples will be used as training data. It should be noted that in the present embodiment, the learning process is divided into a series of training tasks , determining as the current training task, as the historical training task. And obtaining an initial class-incremental learning model trained by the historical training task .

[0064] To address the parameter explosion problem, this embodiment introduces a lexical pooling mechanism, compressing the specific knowledge of each task into trainable vector lexical units, rather than complete network modules. That is, before obtaining the pre-trained initial class incremental learning model, the process may further include: first, initializing general knowledge lexical units and generating first task-specific lexical units corresponding to the historical training tasks; and storing the general knowledge lexical units and the first task-specific lexical units in the lexical pool of the initial class incremental learning model; updating the general knowledge lexical units in the lexical pool during the training of the initial class incremental learning model using the historical training tasks. Specifically, the lexical pool stores two types of learnable lexical units: one general knowledge lexical unit and multiple task-specific lexical units. General knowledge lexical units Dimensions Random initialization occurs before the start of the first historical training task. During the training of each subsequent task, All will be updated, with the aim of capturing and accumulating common knowledge shared by all learned tasks. Tasks Corresponding task-specific terms Dimensions Whenever the model begins to learn a new task At any time, a new task-specific term will be initialized for the task. . In the mission It is updated during the training process to store task-specific knowledge.

[0065] Step S12: Obtain the current image data corresponding to the current training task, and input the current image data into the initial class incremental learning model; the image category corresponding to the current image data is different from the image category corresponding to the historical image data.

[0066] In this embodiment, For the current training task, For historical training tasks. Includes a batch of new image categories training samples The categories of different tasks do not overlap; that is, for Utilizing the current training task When training an initial class incremental learning model, the training data available to the model includes not only the current training task. The new image category samples also include a small subset sampled from the memory buffer of the initial class incremental learning model from historical training tasks. The samples are from the old image categories. The samples from the new image categories are mainly used to train the model to learn new knowledge, while the samples from the old image categories are mainly used to combat catastrophic forgetting.

[0067] In step S13, in the process of processing the current image data by using the initial class incremental learning model, target historical image data is determined based on the historical image data, and a preset attention module of the initial class incremental learning model is used to determine task knowledge features corresponding to the target image data; the target image data includes the current image data and the target historical image data, and the task knowledge features include task general knowledge features corresponding to the current image data and task specific knowledge features corresponding to the target historical image data.

[0068] In this embodiment, when training the initial class incremental learning model, the training samples are composed of images and image categories, the images are used as inputs, and the image categories are used as labels. In order to maintain the stable learning ability of the model on the historical training task, first, initial historical image data can be determined from the historical image data; then the initial historical image data is processed by rotation and mirroring to obtain a plurality of enhanced historical image data, and the enhanced historical image data is used as target historical image data. That is, the initial historical image data can be enhanced, and the enhancement includes rotation and mirroring of the image, for example, rotating the historical image by 180 degrees or horizontally mirroring or vertically mirroring, so that the number of input historical images is doubled, and the image is rotated 4 times, each time by 90 degrees, so that the number of input historical images is quadrupled; the rotated image is still in the horizontal and vertical direction, which does not affect the use of other enhancement processing, and at the same time, in the training process, the model can extract the rotation invariant features between images according to the enhanced historical image data, and maintain the stable learning ability of the model on the historical task.

[0069] It should be noted that the initial class incremental learning model is composed of an encoder for extracting features and a decoder designed for alleviating catastrophic forgetting and classification tasks. The encoder-feature extraction module is composed of an input module, a feature block embedding, and a self-attention module. The decoder-knowledge transfer and classification module includes a preset attention module and a duplex classifier. The preset attention module includes a general attention module and a task attention module, the general attention module can extract common knowledge across tasks by dynamically updating general knowledge vocabulary, and the task attention module retains differentiated features between tasks for task-specific vocabulary of the current training task. This design can realize the collaborative transfer of general knowledge across tasks and task-specific knowledge, and solve the problem of feature degradation caused by single knowledge transfer in traditional methods.

[0070] In the embodiment, in order to extract deep semantic features from the input image, before the preset attention module of the initial class incremental learning model determines the task knowledge features corresponding to the target image data, the method can further include: first converting the current image data into a corresponding first feature block sequence, and converting the target historical image data into a corresponding second feature block sequence; then using the self-attention module of the initial class incremental learning model to extract deep semantic features of the first feature block sequence and the second feature block sequence respectively, to obtain a feature map corresponding to the target image data, so as to use the preset attention module of the initial class incremental learning model and determine the task knowledge features corresponding to the target image data based on the feature map; wherein the self-attention module is a module composed of multi-head self-attention layers, multi-layer perceptron, layer normalization and residual connection. Specifically, for an input image , the th sample of the task , first convert the input two-dimensional image into a one-dimensional feature block sequence through a standard image block embedding operation, and the dimension is , where M is the number of feature blocks, and D is the embedding dimension of each feature block. The embedded feature block sequence passes through a plurality of self-attention modules in turn, for example, 4 self-attention modules. Each self-attention module includes a multi-head self-attention layer and a multi-layer perceptron, and uses layer normalization and residual connection. The calculation formula of each self-attention module is represented as follows:

[0071] ;

[0072] wherein represents the input feature of the lth layer in the self-attention module, represents the output feature of the lth layer in the self-attention module, represents the self-attention calculation function of the lth layer, LN represents the layer normalization operation, and + represents the residual connection operation. In the embodiment, the finally output feature map is denoted as after the self-attention module processing.

[0073] ​After that, the feature map output by the encoder of the initial class incremental learning model is sent into the general attention module, i.e., the first attention module. The above determining the task knowledge feature corresponding to the target image data based on the feature map by using the preset attention module of the initial class incremental learning model can include: in the process of processing the feature map by the first attention module of the initial class incremental learning model, first, taking the current general knowledge token in the token pool of the initial class incremental learning model as the target general knowledge token corresponding to the current training task; then determining a first concatenation corresponding to the feature map and the target general knowledge token, taking the feature map as a first query, and taking the first concatenation as a first key and a first value; then determining a first attention weight corresponding to the first query and the first key by scaled dot-product attention, and determining a target fusion feature based on the first attention weight and the first value; finally, inputting the target fusion feature into the second attention module of the initial class incremental learning model, so as to determine the task knowledge feature corresponding to the target image data based on the target fusion feature by using the second attention module. Specifically, refer to Figure 2 As shown in the figure, the general attention module receives a feature map and obtains a general knowledge token from the token pool of the initial class incremental learning model . The feature map is taken as a query Query, and and are concatenated along a certain dimension to form a first concatenation , as a key Key and a value Value, and then the following calculation is performed:

[0074] ;

[0075] wherein, is the linearly transformed query, is the linearly transformed key, is the linearly transformed value, is a learnable weight matrix, which is used to map to the dimension required for attention calculation. Then, the first attention weight is calculated by scaled dot-product attention, and is weighted according to the first attention weight to obtain the output , as shown below:

[0076] ;

[0077] wherein, is a normalization function, is a scaling factor, is a learnable weight matrix used to perform linear transformation on , is a bias term used to adjust the output result and increase the expressive ability of the model. Then the target fusion feature is obtained by integrating .

[0078] As can be seen from the above, in the general attention module, and the cross attention of enables the model to fuse the general patterns and knowledge of the historical training tasks stored in with the features of the current image data . is trainable, which can continuously absorb new general information in each training task. This design helps the model to still utilize and maintain the old general recognition ability when learning new tasks.

[0079] Subsequently, the target fusion feature output by the general attention module is sent to the task attention module, i.e., the second attention module, which aims to extract specific knowledge transfer of each historical training task and transfer it to the current training task. The above determining the task knowledge feature corresponding to the target image data based on the target fusion feature using the second attention module can include: in the process of processing the target fusion feature using the second attention module, first, the first task-specific token corresponding to the historical training task and the second task-specific token corresponding to the current training task are obtained from the token pool; then a second concatenation corresponding to the target fusion feature, the first task-specific token and the second task-specific token is determined, and the second task-specific token is taken as a second query, and the second concatenation is taken as a second key and a second value; finally, the second attention weight corresponding to the second query and the second key is determined through the scaled dot-product attention, and the task knowledge feature corresponding to the target image data is determined based on the second attention weight and the second value. Specifically, as shown in Figure 2 , the task attention module receives the target fusion feature , and obtains the task-specific token corresponding to each training task from the token pool, i.e. as input. The second task-specific token corresponding to the current training task is taken as a query Query, the target fusion feature is concatenated with all task-specific tokens along a specific dimension to form a second concatenation , as a key Key and a value Value, and then the following calculation is performed:

[0080] ;

[0081] wherein, is the linear transformed query, is the linear transformed key, is the linear transformed value, is a learnable weight matrix, which is used to map to the dimension required for attention computation. Then the second attention weight is computed by scaled dot-product attention, and is weighted by the second attention weight to get the output , which is shown as follows:

[0082] ;

[0083] wherein, is a normalization function, is a scaling factor, is a learnable weight matrix, which is used to linearly transform , is a bias term, which is used to adjust the output result and increase the expressive ability of the model. Then is integrated to get the task-specific knowledge feature with the dimension of , which condenses the knowledge of all historical training tasks and the knowledge of the current training task.

[0084] As can be seen from the above, in the general attention module serves as the query, which actively "extracts" and "associates" relevant information from the task-specific token and the target fusion feature that integrates general knowledge. This enables the model to recall and utilize the specific knowledge of historical training tasks, while obtaining the context of the current training task through and newly learned , and combining the context of the current training task with the specific knowledge of historical training tasks. This helps to distinguish the characteristics of different tasks and reduce interference between tasks.

[0085] In step S14, the duplex classifier of the initial class-incremental learning model is used to determine a first image classification result corresponding to the current image data based on the task-general knowledge feature, and to determine a second image classification result corresponding to the target historical image data based on the task-specific knowledge feature.

[0086] It can be understood that a single classifier is prone to cause feature space conflicts between new and old training tasks, and therefore a dual classifier is proposed in the embodiment and its classification process is split into a stability classifier and a plasticity classifier, which are respectively responsible for learning the feature spaces of new and old tasks. The stability classifier freezes the historical training task weights to maintain the discriminability of old image classes, and the plasticity classifier focuses on learning the features of new image classes. The dynamic fusion strategy of the outputs of the two classifiers enables the model to achieve a good and stable performance balance during the incremental process. That is, the dual classifier of the initial class incremental learning model includes a first classifier and a second classifier. The first classifier, i.e., the new class classifier, is also called the plasticity classifier; and the second classifier, i.e., the old class classifier, is also called the stability classifier.

[0087] It should be noted that the dual classifier structure of the initial class incremental learning model is composed of two parallel linear classification layers, which both receive the task knowledge features output by the task attention module as input after LN (Layer Normalization, layer normalization). That is, in the embodiment, the task general knowledge features corresponding to the current image data can be input to the plasticity classifier in the dual classifier, and the plasticity classifier is used to output the first image classification result corresponding to the current image data. Specifically, the plasticity classifier receives the task general knowledge features corresponding to the current image data , i.e., the plasticity classifier focuses on learning the current training task introduced new image classes, and the parameters of the plasticity classifier are trainable:

[0088] ;

[0089] wherein, is the output of the plasticity classifier, corresponding to the image class prediction result of the current training task , LN represents a layer normalization operation, is the weight matrix of the plasticity classifier, with a dimension of , is the number of new image classes of the current training task , is the bias term of the plasticity classifier, which serves to assist linear transformation to fit data, represents the logits after sigmoid post-processing.

[0090] Meanwhile, in this embodiment, the task-specific knowledge features corresponding to the target historical image data can be input into the stability classifier in the dual-work classifier, and in the process of processing the task-specific knowledge features by the stability classifier, the parameters of the stability classifier are frozen, and the second image classification result corresponding to the target historical image data is output. Specifically, the stability classifier receives the task-specific knowledge features corresponding to the target historical image data , that is, the stability classifier is responsible for identifying the class from the target historical image data, and the parameters of the stability classifier are frozen to retain the existing knowledge when training a new task:

[0091] ;

[0092] wherein, is the output of the stability classifier, LN represents a layer normalization operation, is the weight matrix of the stability classifier, and the dimension is , is the number of all image types in the historical training task , is the bias term of the stability classifier, represents the sigmoid post-processing logits.

[0093] In the training of the initial class incremental learning model in this embodiment, the parameters of the stability classifier are frozen, that is, fixed. When the samples of the old image class from the memory buffer pass through the model, the task-specific knowledge features corresponding to the target historical image data are input into the stability classifier, and the output thereof is used to guide the stability classifier and participate in the calculation of the knowledge distillation loss and the clustering separation loss, so as to maintain the recognition ability of the model to the old class. The parameters of the plasticity classifier are trainable. When the samples of the new image class from the current training task pass through the model, the task-general knowledge features corresponding to the current image data are input into the plasticity classifier to calculate the classification cross-entropy loss, so as to learn the features of the new class, especially to update the parameters of the plasticity classifier and the related feature extraction modules, such as part of the self-attention module, the general knowledge tokens in the token pool , and the task-specific tokens corresponding to the current training task . In the output merging stage, the outputs and of the two classifiers are spliced along the class dimension to form the final prediction for all image classes .

[0094] On one hand, when a target historical image data of an old image class, e.g. image data from a task , is input into the model, the target historical image data is passed through an encoder composed of a batch embedding and a self-attention module to obtain a feature map . Then it is input into a general attention module to interact with target general knowledge tokens of the current training task , and output . At this time, the target historical image data has already contained the accumulated general knowledge of the task . Then it is input into a task attention module to interact with task-specific tokens of the current training task and historical task-specific tokens , and output task-specific knowledge features corresponding to the target historical image data . For the target historical image data, the goal is to be correctly recognized by the stability classifier. The prediction of the old image class is mainly generated by the stability classifier . In the training phase, the parameters of the stability classifier are usually fixed, unless in a specific fine-tuning phase.

[0095] On the other hand, when a current image data of a new image class from the current training task is input into the model, the current image data is passed through an encoder composed of a batch embedding and a self-attention module to obtain a feature map . Then it is input into a general attention module to interact with target general knowledge tokens of the current training task , and output . The target general knowledge tokens will be updated in this process to incorporate new general knowledge that the current training task may contribute. Then it is input into a task attention module to interact with task-specific tokens of the current training task and historical task-specific tokens , and output task-specific knowledge features corresponding to the current image data . The task-specific tokens will be updated in this process to learn the specific knowledge of the current training task . The prediction of the new class is mainly generated by the plasticity classifier . The parameters of the plasticity classifier are trainable to adapt to the new class.

[0096] In step S15, a target classification loss is determined based on the first image classification result and the second image classification result, and a model parameter of the initial class-incremental learning model is updated based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class-incremental learning model, so as to determine a target image classification corresponding to a to-be-classified image by using the target class-incremental learning model.

[0097] It should be noted that the knowledge of the historical training task cannot be completely retained, otherwise it will bring potential interference to the learning of the current training task. For example, for new and old training tasks with conflicting feature distributions, the frozen historical weights in the stability classifier will limit the ability of the model to learn the feature distribution of the new task, resulting in a decrease in plasticity. To solve this problem, the embodiment invents an active forgetting balance mechanism under the Bayesian framework, which balances stability and plasticity by dynamically controlling the degree to which old knowledge is retained. In addition, a clustering separation loss function is also proposed in the embodiment, which constructs a four-tuple sample relationship matrix to force similar features to be clustered across tasks and dissimilar features to be separated between tasks. This can make the model cluster similar samples of the same class under different tasks in the same position, forming a clear "task-class" hierarchy in the feature space, and enhancing the robustness of the incremental learning method.

[0098] Specifically, in the embodiment, the target classification loss is calculated for the first image classification result and the second image classification result output by the dual classifier. Then, the active forgetting balance mechanism is applied to weaken the constraint of the historical training task on the parameters, which can alleviate the interference of the knowledge of the historical training task on the learning of the current training task, improve the learning probability of the current training task, and optimize the performance of continuous learning by minimizing the upper bound of the generalization error. At the same time, the clustering separation loss can be used to promote the clustering of features belonging to the same class and distinguish features between new and old tasks. Specifically, for samples from the current training task and samples from the historical training task , the clustering separation loss function will maximize the cosine distance between their feature representations, i.e., the outputs and of the model. This can guide the model to learn a mechanism to distinguish features between new and old training tasks based on the active forgetting balance mechanism, so that the knowledge specific to the training task is more diverse.

[0099] In one specific implementation, see Figure 3 and Figure 4As shown, the training process of the initial class-incremental learning model is as follows: first, read the incremental task data, batch embedding to the self-attention module to obtain the feature map, then input the feature map into the general attention module to obtain the target fusion feature, then input the target fusion feature into the task attention module to obtain the task knowledge feature, then pass through the dual classifier, and then calculate the total loss by combining the active forgetting balance mechanism, and output the prediction result. Thus, the model parameters are updated based on the prediction result, the training of the model is completed, and the target class-incremental learning model is obtained.

[0100] As can be seen from the above, in the embodiment, first, the initial class-incremental learning model trained based on the historical image data corresponding to the historical training task is determined, then the current image data corresponding to the current training task which is different from the image category of the historical image data is obtained, and the current image data is input into the initial class-incremental learning model to train the model. Specifically, the preset attention module of the initial class-incremental learning model determines the task general knowledge feature corresponding to the current image data and the task specific knowledge feature corresponding to the target historical image data. Then, the dual classifier of the initial class-incremental learning model is used to determine the first image classification result corresponding to the current image data based on the task general knowledge feature, and the second image classification result corresponding to the target historical image data based on the task specific knowledge feature. Subsequently, the target classification loss is determined based on the image classification results, and the model parameters of the initial class-incremental learning model are updated based on the preset active forgetting balance mechanism and the target classification loss to obtain the target class-incremental learning model, so as to complete the training of the model, and apply the target class-incremental learning model to the image classification task. In this way, the embodiment can realize the collaborative transfer of the general knowledge and the task specific knowledge across tasks, and can effectively alleviate the catastrophic forgetting problem faced by the deep neural network during the class-incremental learning process.

[0101] In order to enhance the robustness of the model, the training process of the target class-incremental learning model will be further described in the following embodiment.

[0102] Understandably, the core challenge of incremental learning lies in addressing the mutual interference between new and old training tasks caused by differences in data distribution. However, existing methods primarily focus on preventing catastrophic forgetting through stability protection. This embodiment designs an active forgetting balancing mechanism based on a Bayesian learning framework. This mechanism achieves a balance between stability and plasticity by tracking the probability distribution of weights under dynamically perceived input. Correspondingly, the process of updating the model parameters of the initial class incremental learning model based on the preset active forgetting balancing mechanism and the target classification loss to obtain the target class incremental learning model can include: first, determining the target forgetting rate based on the preset active forgetting balancing mechanism and constructing the Fisher information matrix corresponding to the historical training task; the target forgetting rate is used to control the degree of forgetting of the historical training task by the initial class incremental learning model; then, determining the target importance of the model parameters to the historical training task based on the Fisher information matrix and determining the target weights corresponding to the model parameters; finally, determining the target total loss based on the target importance, the target weights, the target forgetting rate, and the target classification loss, and updating the model parameters based on the target total loss to obtain the target class incremental learning model. Specifically, the incremental task learning process under the Bayesian framework is as follows:

[0103] Consider the parameters of the initial class incremental learning model There are M parameters in total, and the learning tasks are performed sequentially. and Training data and From a Bayesian perspective:

[0104] (1) Prior distribution: The initial prior is ;

[0105] (2) Task Learning: Obtain the posterior distribution through Bayesian update, as shown below:

[0106] ;

[0107] Its extreme point Corresponding task The optimal parameters;

[0108] (3) Task Learning: As the new prior, the updated posterior distribution is: , need to maximize simultaneously Used for learning new tasks and Used to retain old tasks.

[0109] To mitigate the interference of historical training task knowledge on current training task learning, a target forgetting rate is introduced. The posterior distribution is modified as

[0110] ;

[0111] where Z is a normalization constant. When , the initial class-incremental learning model tends to forget the history training tasks ; when , the initial class-incremental learning model tends to completely retain the old knowledge. Based on this, combined with the active forgetting balance mechanism and the target classification loss, the target loss function can be defined as:

[0112] ;

[0113] where is the target classification loss, represents the mth parameter in the initial class-incremental learning model; represents the optimal value of the mth parameter learned by the initial class-incremental learning model after learning the history training tasks ; represents the mth diagonal element in the Fisher information matrix corresponding to the history training tasks , reflecting the importance of the parameter to the history training tasks ; and are balance coefficients for adjusting the relative importance of each loss in the total loss, is the protection coefficient of the knowledge of the history training tasks, for controlling the strength of active forgetting and learning new knowledge. represents the selective weight of active forgetting, i.e., the target weight, which can be divided into two categories according to different implementation methods: , uniform forgetting is applied to all parameters, forcing the parameters to be close to ; , the weight is dynamically adjusted based on the Fisher information of the current training tasks ; is the reference target of active forgetting, when , , parameter contraction is performed, , , so that the parameters are close to the independent optimal solution of the current training tasks.

[0114] It should be noted that the total loss function corresponding to the initial class-incremental learning model is:

[0115] ;

[0116] where is the total loss function, a target loss determined based on the active forgetting balance mechanism and the target classification loss, a clustering separation loss. In this way, the embodiment can update the model parameters of the initial class-incremental learning model based on the target total loss through back propagation to obtain a target class-incremental learning model, which can be applied to an image classification task.

[0117] As can be seen from the above, in the embodiment, an active forgetting balance mechanism is designed under a Bayesian framework, which balances the stability and plasticity of the target class-incremental learning model by dynamically controlling the degree of retention of old knowledge. Meanwhile, in the embodiment, the target total loss is determined based on the target classification loss, the active forgetting balance mechanism and the clustering separation loss, and the model parameters are updated based on the target total loss. In this way, the model can be guided to learn to distinguish the features of new and old training tasks on the basis of active forgetting, so that the task-specific knowledge is more diversified, and a class-incremental learning model that balances stability and plasticity, has efficient parameters and can coordinate diversified knowledge transfer is realized.

[0118] In a specific implementation, the performance of the target class-incremental learning model can be tested based on three common incremental learning data sets. Referring to FIG. 8, the precision of the target class-incremental learning model is tested using CIFAR10-10; referring to FIG. 9, the precision of the target class-incremental learning model is tested using CIFAR20-20; and referring to FIG. 10, the precision of the target class-incremental learning model is tested using ImageNet10-10. As can be seen, the target class-incremental learning model has stronger anti-catastrophic forgetting ability in performance, higher classification precision, and can realize an incremental learning method with better performance for actual application scenarios under a smaller parameter amount. Figure 5 Figure 6 Figure 7 As can be seen, the target class-incremental learning model has stronger anti-catastrophic forgetting ability in performance, higher classification precision, and can realize an incremental learning method with better performance for actual application scenarios under a smaller parameter amount.

[0119] Correspondingly, referring to FIG. 11, the embodiment of the present application further provides an image classification device based on a class-incremental learning model, which can be applied to a computer device and can include: Figure 8

[0120] An initial class-incremental learning model acquisition module 11 is configured to acquire a pre-trained initial class-incremental learning model; the initial class-incremental learning model is a model trained based on historical image data corresponding to historical training tasks;

[0121] A current image data acquisition module 12 is configured to acquire current image data corresponding to a current training task and input the current image data into the initial class-incremental learning model; the image classes corresponding to the current image data are different from the image classes corresponding to the historical image data;

[0122] ​​​The task knowledge feature determination module 13 is configured to determine target historical image data based on the historical image data in a process of processing the current image data by using the initial class incremental learning model, and determine task knowledge features corresponding to the target image data by using a preset attention module of the initial class incremental learning model; wherein the target image data includes the current image data and the target historical image data, and the task knowledge features include task general knowledge features corresponding to the current image data and task specific knowledge features corresponding to the target historical image data.

[0123] The image classification result determination module 14 is configured to determine a first image classification result corresponding to the current image data based on the task general knowledge features by using a duplex classifier of the initial class incremental learning model, and determine a second image classification result corresponding to the target historical image data based on the task specific knowledge features.

[0124] The target class incremental learning model determination module 15 is configured to determine a target classification loss based on the first image classification result and the second image classification result, and update model parameters of the initial class incremental learning model based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, so as to determine a target image classification corresponding to a to-be-classified image by using the target class incremental learning model.

[0125] As can be seen from the above, in the present application, an initial class incremental learning model trained based on historical image data corresponding to a historical training task is first determined, and then current image data corresponding to a current training task different from the image classes of the historical image data is obtained, and the current image data is input into the initial class incremental learning model to train the model. Specifically, a preset attention module of the initial class incremental learning model determines task general knowledge features corresponding to the current image data and task specific knowledge features corresponding to the target historical image data. Then, a duplex classifier of the initial class incremental learning model is used to determine a first image classification result corresponding to the current image data based on the task general knowledge features, and a second image classification result corresponding to the target historical image data based on the task specific knowledge features. Subsequently, a target classification loss is determined based on the image classification results, and model parameters of the initial class incremental learning model are updated based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, so as to complete the training of the model and apply the target class incremental learning model to an image classification task. In this way, the present application can realize the collaborative transfer of general knowledge and task specific knowledge across tasks, and can effectively alleviate the catastrophic forgetting problem faced by a deep neural network in a class incremental learning process.

[0126] In some embodiments, the task knowledge feature determination module 13 can include:

[0127] an initial historical image data determination unit, configured to determine initial historical image data from the historical image data;

[0128] a target historical image data determination unit, configured to perform rotation processing and mirror processing on the initial historical image data to obtain a plurality of enhanced historical image data, and take the enhanced historical image data as the target historical image data.

[0129] In some embodiments, the image classification apparatus based on the class-incremental learning model can further include:

[0130] a feature block sequence determination module, configured to convert the current image data into a corresponding first feature block sequence, and convert the target historical image data into a corresponding second feature block sequence;

[0131] a feature map determination module, configured to extract deep semantic features of the first feature block sequence and the second feature block sequence respectively by using a self-attention module of the initial class-incremental learning model, to obtain a feature map corresponding to the target image data, so as to determine the task knowledge features corresponding to the target image data based on the feature map by using a preset attention module of the initial class-incremental learning model; wherein the self-attention module is a module composed of a multi-head self-attention layer, a multi-layer perception mechanism, layer normalization and residual connection.

[0132] In some embodiments, the image classification apparatus based on the class-incremental learning model can further include:

[0133] a general knowledge vocabulary initialization module, configured to initialize a general knowledge vocabulary, generate a first task-specific vocabulary corresponding to the historical training task, and store the general knowledge vocabulary and the first task-specific vocabulary into a vocabulary pool of the initial class-incremental learning model;

[0134] a general knowledge vocabulary updating module, configured to update the general knowledge vocabulary in the vocabulary pool in the process of training the initial class-incremental learning model by using the historical training task;

[0135] Correspondingly, the feature map determination module can include:

[0136] a target general knowledge vocabulary determination sub-module, configured to take a current general knowledge vocabulary in the vocabulary pool of the initial class-incremental learning model as a target general knowledge vocabulary corresponding to the current training task in the process of processing the feature map by using a first attention module of the initial class-incremental learning model;

[0137] The first splicing body determination sub-module is configured to determine a first splicing body corresponding to the feature map and the target general knowledge vocabulary, and take the feature map as a first query and take the first splicing body as a first key and a first value.

[0138] The target fusion feature determination sub-module is configured to determine a first attention weight corresponding to the first query and the first key by using the scaled dot-product attention, and determine a target fusion feature based on the first attention weight and the first value.

[0139] The task knowledge feature determination sub-module is configured to input the target fusion feature into a second attention module of the initial class-incremental learning model, so as to determine the task knowledge feature corresponding to the target image data based on the target fusion feature by using the second attention module.

[0140] In some embodiments, the task knowledge feature determination sub-module can include:

[0141] The task-specific vocabulary determination unit is configured to obtain, in a process of processing the target fusion feature by using the second attention module, a first task-specific vocabulary corresponding to the historical training task and a second task-specific vocabulary corresponding to the current training task from the vocabulary pool.

[0142] The second splicing body determination unit is configured to determine a second splicing body corresponding to the target fusion feature, the first task-specific vocabulary and the second task-specific vocabulary, take the second task-specific vocabulary as a second query, and take the second splicing body as a second key and a second value.

[0143] The task knowledge feature determination unit is configured to determine a second attention weight corresponding to the second query and the second key by using the scaled dot-product attention, and determine the task knowledge feature corresponding to the target image data based on the second attention weight and the second value.

[0144] In some embodiments, the dual-work classifier includes a first classifier and a second classifier.

[0145] Correspondingly, the image classification result determination module 14 can include:

[0146] The first image classification result determination unit is configured to input the task general knowledge feature into the first classifier in the dual-work classifier, and output the first image classification result corresponding to the current image data by using the first classifier.

[0147] The second image classification result determination unit is configured to input the task-specific knowledge feature into the second classifier in the dual-work classifier, freeze the parameters of the second classifier in the process of processing the task-specific knowledge feature by using the second classifier, and output the second image classification result corresponding to the target historical image data.

[0148] In some embodiments, the target class incremental learning model determination module 15 can include:

[0149] A target forgetting rate determination unit is configured to determine a target forgetting rate based on the preset active forgetting balance mechanism, and construct a Fisher information matrix corresponding to the historical training task; the target forgetting rate is used to control the forgetting degree of the initial class incremental learning model for the historical training task.

[0150] A target weight determination unit is configured to determine a target importance of the model parameters for the historical training task based on the Fisher information matrix, and determine a target weight corresponding to the model parameters.

[0151] A target class incremental learning model determination unit is configured to determine a target total loss based on the target importance, the target weight, the target forgetting rate and the target classification loss, and update the model parameters based on the target total loss to obtain the target class incremental learning model.

[0152] Further, the embodiment of the present application also discloses an electronic device, Figure 9 is the structure diagram of the electronic device 20 according to an exemplary embodiment, and the contents in the figure cannot be considered as any limitation on the use range of the present application. The electronic device 20 can specifically include at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25 and a communication bus 26. The memory 22 is used to store a computer program, and the computer program is loaded and executed by the processor 21 to realize the related steps in the image classification method based on the class incremental learning model disclosed in any of the preceding embodiments. In addition, the electronic device 20 in the embodiment can be an electronic computer.

[0153] In the embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol followed by the communication interface 24 can be any communication protocol applicable to the technical solution of the present application, which is not limited here; the input / output interface 25 is used to obtain external input data or output data to the outside world, and the specific interface type can be selected according to the specific application needs, which is not limited here.

[0154] In addition, the memory 22 can be a read-only memory, a random access memory, a magnetic disk or an optical disk, etc. as a carrier for storing resources, and the resources stored thereon can include an operating system 221, a computer program 222, etc. The storage mode can be temporary storage or permanent storage.

[0155] The operating system 221 is used to manage and control each hardware device on the electronic device 20 and the computer program 222, and can be Windows Server, Netware, Unix, Linux, etc. In addition to the computer program capable of completing the image classification method based on the class-incremental learning model executed by the electronic device 20 disclosed in any of the foregoing embodiments, the computer program 222 can further include a computer program capable of completing other specific work.

[0156] Further, the present application also discloses a computer readable storage medium for storing a computer program, wherein the computer program is executed by a processor to implement the image classification method based on the class-incremental learning model disclosed above. For the specific steps of the method, please refer to the corresponding content disclosed in the foregoing embodiments, which will not be described here.

[0157] The embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts of each embodiment can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method part.

[0158] The skilled person can further realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized by electronic hardware, computer software or a combination of the two. In order to clearly show the interchangeability of hardware and software, the components and steps of each example have been described in the above description. Whether the functions are realized by hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0159] The steps of the method or algorithm described in combination with the embodiments disclosed herein can be directly implemented by hardware, a software module executed by a processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0160] Finally, it needs to be pointed out that in this document, relational terms such as first and second and the like can only be intended to distinguish one entity or operation from another entity or operation without necessarily requiring or implying any actual such relationship or order between such entities or operations. Moreover, the terms "comprising", "including", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can also include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprising a" does not exclude the existence of additional identical elements in the process, method, article, or apparatus including the stated element.

[0161] The above detailed description of the technical solutions provided by the present application has been provided, and the principles and implementation modes of the present application have been described by applying specific examples. The above description of the examples is only for the purpose of helping to understand the method of the present application and its core idea; at the same time, for those skilled in the art, according to the idea of the present application, the specific implementation mode and application range will be changed, and the above description of the content of the specification should not be understood as a limitation of the present application.

Claims

1. An image classification method based on a class-incremental learning model, characterized in that, The application is applied to a computer device, comprising: obtaining a pre-trained initial class incremental learning model; the initial class incremental learning model is a model trained based on historical image data corresponding to a historical training task; obtain current image data corresponding to a current training task, and input the current image data into the initial class incremental learning model; the image category corresponding to the current image data is different from the image category corresponding to the historical image data; during processing of the current image data by the initial class incremental learning model, determine target historical image data based on the historical image data, and determine task knowledge features corresponding to the target image data by a preset attention module of the initial class incremental learning model; wherein the target image data includes the current image data and the target historical image data, and the task knowledge features include task general knowledge features corresponding to the current image data and task specific knowledge features corresponding to the target historical image data; determine a first image classification result corresponding to the current image data based on the task general knowledge features by a duplex classifier of the initial class incremental learning model, and determine a second image classification result corresponding to the target historical image data based on the task specific knowledge features; determine a target classification loss based on the first image classification result and the second image classification result, and update model parameters of the initial class incremental learning model based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, so as to determine a target image classification corresponding to a to-be-classified image by the target class incremental learning model. 2.The image classification method based on the class-incremental learning model according to claim 1, wherein, The target historical image data is determined based on the historical image data, comprising: determine initial historical image data from the historical image data; perform rotation processing and mirror processing on the initial historical image data to obtain a plurality of enhanced historical image data, and use the enhanced historical image data as the target historical image data. 3.The image classification method based on the class-incremental learning model according to claim 1, wherein, Before the task knowledge features corresponding to the target image data are determined by the preset attention module of the initial class incremental learning model, the method further comprises: convert the current image data into a corresponding first feature block sequence, and convert the target historical image data into a corresponding second feature block sequence; extract deep semantic features of the first feature block sequence and the second feature block sequence respectively by a self-attention module of the initial class incremental learning model to obtain a feature map corresponding to the target image data, so as to determine the task knowledge features corresponding to the target image data based on the feature map by the preset attention module of the initial class incremental learning model; wherein the self-attention module is a module composed of a multi-head self-attention layer, a multi-layer perception mechanism, layer normalization and residual connection.

4. The image classification method based on the class-incremental learning model according to claim 3, characterized in that, Before the pre-trained initial class incremental learning model is obtained, the method further comprises: initialize a general knowledge vocabulary, generate a first task-specific vocabulary corresponding to the historical training task, and store the general knowledge vocabulary and the first task-specific vocabulary in a vocabulary pool of the initial class-incremental learning model; update the general knowledge vocabulary in the vocabulary pool during training of the initial class-incremental learning model using the historical training task; Correspondingly, the initial class-incremental learning model is used by the preset attention module, and the task knowledge feature corresponding to the target image data is determined based on the feature map, which includes: In the process of processing the feature map by the first attention module of the initial class-incremental learning model, the current general knowledge vocabulary in the vocabulary pool of the initial class-incremental learning model is taken as the target general knowledge vocabulary corresponding to the current training task; determine a first splicing body corresponding to the feature map and the target general knowledge vocabulary, and take the feature map as a first query and the first splicing body as a first key and a first value; determine a first attention weight corresponding to the first query and the first key by scaling dot product attention, and determine a target fusion feature based on the first attention weight and the first value; input the target fusion feature into the second attention module of the initial class-incremental learning model, so as to determine the task knowledge feature corresponding to the target image data based on the target fusion feature by using the second attention module.

5. The image classification method based on the class-incremental learning model according to claim 4, characterized in that, The determination of the task knowledge feature corresponding to the target image data based on the target fusion feature by using the second attention module includes: In the process of processing the target fusion feature by using the second attention module, the first task-specific vocabulary corresponding to the historical training task and the second task-specific vocabulary corresponding to the current training task are obtained from the vocabulary pool; determine a second splicing body corresponding to the target fusion feature, the first task-specific vocabulary and the second task-specific vocabulary, and take the second task-specific vocabulary as a second query and the second splicing body as a second key and a second value; determine a second attention weight corresponding to the second query and the second key by scaling dot product attention, and determine the task knowledge feature corresponding to the target image data based on the second attention weight and the second value. 6.The image classification method based on the class-incremental learning model according to claim 1, wherein, The duplex classifier includes a first classifier and a second classifier; Correspondingly, the duplex classifier of the initial class-incremental learning model is used to determine a first image classification result corresponding to the current image data based on the task general knowledge feature, and to determine a second image classification result corresponding to the target historical image data based on the task-specific knowledge feature, which includes: input the task general knowledge feature into the first classifier in the duplex classifier, and output the first image classification result corresponding to the current image data by using the first classifier; The task-specific knowledge feature is input into the second classifier in the double-work classifier, and in the process of processing the task-specific knowledge feature by using the second classifier, the parameters of the second classifier are frozen, and the second image classification result corresponding to the target historical image data is output.

7. The image classification method based on the class-incremental learning model according to any one of claims 1 to 6, characterized in that, The model parameters of the initial class incremental learning model are updated based on the preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, which includes: A target forgetting rate is determined based on the preset active forgetting balance mechanism, and a Fisher information matrix corresponding to the historical training task is constructed; the target forgetting rate is used to control the forgetting degree of the initial class incremental learning model for the historical training task; The target importance of the model parameters for the historical training task is determined based on the Fisher information matrix, and the target weight corresponding to the model parameters is determined; A target total loss is determined based on the target importance, the target weight, the target forgetting rate and the target classification loss, and the model parameters are updated based on the target total loss to obtain the target class incremental learning model.

8. An image classification apparatus based on a class-incremental learning model, characterized by, Applied to a computer device, comprising: An initial class incremental learning model acquisition module is configured to acquire a pre-trained initial class incremental learning model; the initial class incremental learning model is a model trained based on historical image data corresponding to historical training tasks; A current image data acquisition module is configured to acquire current image data corresponding to a current training task and input the current image data into the initial class incremental learning model; the image category corresponding to the current image data is different from the image category corresponding to the historical image data; A task knowledge feature determination module is configured to determine target historical image data based on the historical image data in the process of processing the current image data by using the initial class incremental learning model, and determine task knowledge features corresponding to target image data by using a preset attention module of the initial class incremental learning model; wherein the target image data includes the current image data and the target historical image data, and the task knowledge features include task general knowledge features corresponding to the current image data and task-specific knowledge features corresponding to the target historical image data; An image classification result determination module is configured to determine a first image classification result corresponding to the current image data based on the task general knowledge features by using a double-work classifier of the initial class incremental learning model, and determine a second image classification result corresponding to the target historical image data based on the task-specific knowledge features; A target class incremental learning model determination module is configured to determine a target classification loss based on the first image classification result and the second image classification result, and update model parameters of the initial class incremental learning model based on a preset active forgetting balance mechanism and the target classification loss to obtain a target class incremental learning model, so as to determine a target image classification corresponding to a to-be-classified image by using the target class incremental learning model.

9. An electronic device, comprising: The electronic device comprises a processor and a memory; wherein the memory is configured to store a computer program, and the computer program is loaded and executed by the processor to implement the image classification method based on the class-incremental learning model according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored, and the computer program is executed by a processor to implement the image classification method based on the class-incremental learning model according to any one of claims 1 to 7.

Images