Image classification method and system for small sample class incremental learning based on logit adjustment

Abstract

The invention relates to the field of computer vision, in particular to an image classification method and system for small sample class incremental learning based on logit adjustment. The method comprises the following steps: constructing an image classification model, wherein the image classification model comprises a feature extraction module, a prototype similarity calculation module and a logit adjustment module which are connected in sequence; training and freezing a feature extraction module by using the base class data set; constructing pseudo new class data of a plurality of pseudo increment stages based on the preprocessed base class data set; based on the prototype of each base class and the prototype of each pseudo new class of each pseudo increment stage, training a logit adjustment module through a plurality of pseudo increment stages to obtain a trained logit adjustment module; after the trained logit adjustment module is frozen, a target image classification model is obtained; and through a target image classification model, carrying out image classification on the new class of samples in the small sample increment stage. According to the invention, the image classification precision is improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to an image classification method and system based on logit-adjusted small sample class incremental learning. Background Art

[0002] Class incremental learning (CIL) has been a key research topic in the field of computer vision in recent years. It aims to enable the model to continuously learn new knowledge from the continuously arriving data stream without forgetting the knowledge that has been learned. However, in the real world, training data usually arrives in stages in the form of data streams, and each category may not have a large number of available samples. For example, in the field of power equipment fault diagnosis and monitoring, accurate diagnosis of equipment problems and timely repair or replacement of faulty equipment can help reduce energy consumption and carbon emissions caused by reduced equipment efficiency or unplanned downtime, and ultimately promote the efficient use of green energy; through image classification technology, the state of equipment (such as transformers, transmission lines, switchgear, etc.) can be photographed by cameras or drones, and the model can detect abnormal states of equipment (such as cracks, corrosion, overheating, etc.). Over time, changes in equipment status and fault types may gradually appear, but the number of samples for each fault or abnormal phenomenon is often small, especially in some rare fault types or line problems in specific environments. Therefore, small sample class incremental learning is introduced. Small Sample Class Incremental Learning (FSCIL) uses a large amount of data to train the model only in the first stage to improve its generalization, while only a small amount of labeled data is available in subsequent stages. The purpose is to enable the model to continuously generalize and learn new knowledge from small samples and ensure that old knowledge is not forgotten.

[0003] However, one of the main challenges faced by the model in the scenario of power equipment fault diagnosis and monitoring is catastrophic forgetting. For example, when a new equipment fault type (such as transformer fault, line break, etc.) is added, the traditional neural network may significantly forget the fault type that has been learned (such as equipment overload or power fluctuation, etc.). This catastrophic forgetting is particularly serious in power systems, because power equipment usually has a variety of different faults in long-term use, and these fault types may show different characteristics in different seasons and environmental conditions. If the model fails to effectively retain the previously learned knowledge, it may cause the performance of the fault diagnosis system to seriously degrade, and even cause false positives or false negatives, thus affecting the stable operation of the power system. In order to cope with this challenge, the existing small sample class incremental learning (FSCIL) is mainly based on the "partial freezing" strategy, that is, the backbone of the model first learns sufficient information of the base class, and freezes most of the parameters of the previous backbone when learning the new class, which can effectively prevent the model from forgetting the previously learned base class information when learning the new class, greatly reducing the catastrophic forgetting of the base class. In addition, the frozen parameters can retain the knowledge of the base class, so that the model can better utilize the existing features when facing the new class, thereby achieving more effective knowledge transfer.

[0004] However, since these methods usually adopt a "partial freezing" strategy, the model only updates parameters to a certain extent when learning new classes, while most of the parameters are still fixed. This fixed approach alleviates the forgetting of base class knowledge to a certain extent, but at the same time causes insufficient learning of new class information. Since the number of samples of new classes is often small, it is difficult for the model to fully capture the subtle differences in the characteristics of the new classes, which makes the new class samples often misclassified as base classes when trained classifiers, thereby reducing the accuracy of image classification predictions. This results in the model being unable to effectively distinguish new faults from known faults in actual power equipment fault diagnosis, thereby greatly reducing the accuracy and reliability of power equipment fault diagnosis and monitoring. Summary of the invention

[0005] To this end, the technical problem to be solved by the present invention is to overcome the defect that the existing small sample class incremental learning over-learns a large number of features of the base class but ignores a small amount of information of the new class, reduces the accuracy of image classification prediction, and causes the new fault types to be unable to be effectively identified during the fault diagnosis and monitoring of power equipment, thereby increasing the risk of false alarms or missed alarms, and seriously affecting the stability and reliability of the power system.

[0006] To solve the above technical problems, the present invention provides an image classification method based on small sample class incremental learning with logit adjustment, comprising the following steps:

[0007] Obtain an image dataset, divide the image dataset into a base class dataset and new class datasets corresponding to different small sample increment stages, and perform preprocessing; wherein the base class dataset includes B base classes, each base class includes multiple samples and their true labels, and the new class dataset corresponding to each small sample increment stage includes N new classes, each new class includes multiple samples and their true labels;

[0008] Input the preprocessed base class data set into the feature extraction module, extract the feature embedding of each base class sample, train the feature extraction module, and freeze the trained feature extraction module;

[0009] The preprocessed base class data set is input into the frozen feature extraction module to extract the feature embedding of each base class sample; based on the feature embedding of each base class sample, the prototype of each base class is calculated respectively;

[0010] Based on the preprocessed base class data set, construct pseudo new class data sets corresponding to multiple pseudo incremental stages;

[0011] Input the pseudo-new class dataset of each pseudo-incremental stage into the frozen feature extraction module to extract the feature embedding of each pseudo-new class sample of each pseudo-incremental stage; based on the feature embedding of each pseudo-new class sample of each pseudo-incremental stage, calculate the prototype of each pseudo-new class of each pseudo-incremental stage respectively;

[0012] Based on the prototype of each base class and the prototype of each pseudo-new class in each pseudo-incremental stage, the logit adjustment module is trained through multiple pseudo-incremental stages to obtain a trained logit adjustment module, including:

[0013] For each pseudo-increment phase, compare the prototype of each base class with the The prototype of each pseudo-new class in the pseudo-incremental stage is input into the prototype similarity calculation module to obtain the The similarity vector of each pseudo-new class in the pseudo-incremental stage; is the total number of pseudo-incremental phases;

[0014] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is input into the logit adjustment module to obtain The balance factors of the pseudo-incremental phase include:

[0015] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is passed through the softmax function and the balance factor calculation module in turn to obtain the The balance factor of each pseudo-new class in the pseudo-incremental stage; The balance factors of all pseudo-new classes in the pseudo-incremental stage are concatenated to obtain

[0016] The balance factor vector of the pseudo-increment phase;

[0017] Set the first pseudo-increment phase to The balance factor vectors of the pseudo-incremental stages are concatenated in order of stages to obtain Balance factor for the pseudo-incremental phase;

[0018] The first The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weights of the pseudo-incremental stages;

[0019] Based on The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage,

[0020] The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage;

[0021] The first The logit of each pseudo-new class sample in the pseudo-incremental stage is passed through the softmax function to obtain The image classification results of each pseudo-new class sample in the pseudo-incremental stage;

[0022] Based on The image classification results and pseudo labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained to obtain a trained logit adjustment module;

[0023] After freezing the trained logit adjustment module, the target image classification model is obtained based on the frozen feature extraction module, the prototype similarity calculation module and the frozen logit adjustment module; the new class samples preprocessed in each small sample increment stage are input into the target image classification model in turn to obtain the image classification results of the new class samples in each small sample increment stage.

[0024] Preferably, constructing a plurality of pseudo new class data sets corresponding to pseudo incremental stages based on the preprocessed base class data set includes:

[0025] Randomly select M base classes and several samples of M base classes from the preprocessed base class data set as pseudo-new class test sets;

[0026] In addition to the M base classes of the preprocessed base class data set and the first pseudo-incremental stage to the Extracted in pseudo-incremental phase From the base classes other than the base class, randomly select 2N base classes and combine them two by two to get the first There are N category pairs in the pseudo-incremental stage, and each category pair represents a pseudo-new category;

[0027] Randomly extract K samples of the two base classes in the current category pair, and combine the samples of the first base class and the samples of the second base class in the current category pair in pairs, and then obtain K pseudo-new class samples of the current pseudo-new class through linear mixing;

[0028] Based on the total number of base class samples, the first pseudo-incremental stage to the The number of pseudo-new classes in the pseudo-incremental stage is generated The pseudo label of each pseudo new class sample in the pseudo incremental stage;

[0029] Based on All pseudo-new class samples and their pseudo-labels in the pseudo-incremental stage are constructed The pseudo new class dataset corresponds to the pseudo incremental stage.

[0030] Preferably, the total number of base class samples, the first pseudo-incremental stage to the The number of pseudo-new classes in the pseudo-incremental stage is generated The pseudo label of each pseudo-new class sample in the pseudo-incremental stage is:

[0031]

[0032] in, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, B is the total number of base classes, N is the number of pseudo-new classes in each pseudo-incremental stage, For the pseudo-incremental stage, / / is the modulus, K is the number of samples of the two base classes in each category pair randomly selected in each pseudo-incremental stage, v is the The vth pseudo-new class sample in the pseudo-incremental stage.

[0033] Preferably, the feature embedding of each pseudo-new class sample in the current pseudo-incremental stage is based on respectively calculating the prototype of each pseudo-new class in the current pseudo-incremental stage, and the formula is:

[0034]

[0035] in, For the The prototype of the cth pseudo-new class in the pseudo-incremental stage, f θ(.) is the frozen feature extraction module, is the indicator function, For the The vth pseudo-new class sample in the pseudo-incremental stage, For the The feature embedding of the vth pseudo-new class sample in the pseudo-incremental stage, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, c is the pseudo-label of the vth pseudo-new class sample, The cth pseudo-new class in the pseudo-incremental stage, N is the number of pseudo-new classes in each pseudo-incremental stage, and K is the number of samples of the two base classes in each category pair randomly selected in each pseudo-incremental stage.

[0036] Preferably, the The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weight of the pseudo-incremental stage is as follows:

[0037]

[0038] in, For the The classification weights of the pseudo-incremental stage, For the The classification weights of the pseudo-incremental stage, Concat(.) is a concatenation operation, For the The prototype of the cth pseudo-new class in the pseudo-incremental stage, N is the number of pseudo-new classes in each pseudo-incremental stage, W 0 It is the weight matrix obtained by concatenating the prototypes of all base classes.

[0039] Preferably, the The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage, The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage includes:

[0040]

[0041] in, For the The logit of the vth pseudo-new class sample in the pseudo-incremental stage, For the The vth pseudo-new class sample in the pseudo-incremental stage, For the The feature embedding of the vth pseudo-new class sample in the pseudo-incremental stage, Concat(.) is the concatenation operation, is a B-dimensional vector whose elements are all 0, B is the total number of base classes, γ is a hyperparameter, For the Balance factor for the pseudo-increment phase.

[0042] Preferably, the The image classification results and pseudo labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained, and the loss function is calculated as follows:

[0043]

[0044] Among them, L pse For the The loss function of each pseudo-incremental stage, B is the total number of base classes, N is the number of pseudo-new classes in each pseudo-incremental stage, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, For the The probability that the vth pseudo-new class sample in the pseudo-incremental stage is predicted to be the oth class, For the The balance factor of the pseudo-incremental stage, α is the regularization coefficient, For the The similarity vector of the cth pseudo-new class in the pseudo-incremental stage.

[0045] Preferably, the feature extraction module is any one of ResNet12 network, ResNet18 network, and ResNet20 network.

[0046] Preferably, the balance factor calculation module is any one of a multi-layer perceptron, a fully connected network, and a neural network.

[0047] The present invention also provides an image classification system based on logit-adjusted small sample class incremental learning, comprising:

[0048] A data acquisition module is used to acquire an image data set, divide the image data set into a base class data set and new class data sets corresponding to different small sample increment stages, and perform preprocessing; wherein the base class data set includes B base classes, each base class includes multiple samples and their true labels, and the new class data set corresponding to each small sample increment stage includes N new classes, each new class includes multiple samples and their true labels;

[0049] The feature extraction module training module is used to input the preprocessed base class data set into the feature extraction module, extract the feature embedding of each base class sample, train the feature extraction module, and freeze the trained feature extraction module;

[0050] The base class prototype acquisition module is used to input the preprocessed base class data set into the frozen feature extraction module to extract the feature embedding of each base class sample; based on the feature embedding of each base class sample, the prototype of each base class is calculated respectively;

[0051] A pseudo-new class data set construction module is used to construct pseudo-new class data sets corresponding to multiple pseudo-incremental stages based on the preprocessed base class data set;

[0052] A pseudo-new class prototype acquisition module is used to input the pseudo-new class data set of each pseudo-incremental stage into the frozen feature extraction module to extract the feature embedding of each pseudo-new class sample of each pseudo-incremental stage; based on the feature embedding of each pseudo-new class sample of each pseudo-incremental stage, the prototype of each pseudo-new class of each pseudo-incremental stage is calculated respectively;

[0053] The logit adjustment training module is used to train the logit adjustment module through multiple pseudo-incremental stages based on the prototype of each base class and the prototype of each pseudo-new class in each pseudo-incremental stage to obtain a trained logit adjustment module, including:

[0054] For each pseudo-increment phase, compare the prototype of each base class with the The prototype of each pseudo-new class in the pseudo-incremental stage is input into the prototype similarity calculation module to obtain the The similarity vector of each pseudo-new class in the pseudo-incremental stage; is the total number of pseudo-incremental phases;

[0055] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is input into the logit adjustment module to obtain The balance factors of the pseudo-incremental phase include:

[0056] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is passed through the softmax function and the balance factor calculation module in turn to obtain the The balance factor of each pseudo-new class in the pseudo-incremental stage; The balance factors of all pseudo-new classes in the pseudo-incremental stage are concatenated to obtain

[0057] The balance factor vector of the pseudo-increment phase;

[0058] Set the first pseudo-increment phase to The balance factor vectors of the pseudo-incremental stages are concatenated in order of stages to obtain Balance factor for the pseudo-incremental phase;

[0059] The first The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weights of the pseudo-incremental stages;

[0060] Based on The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage,

[0061] The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage;

[0062] The first The logit of each pseudo-new class sample in the pseudo-incremental stage is passed through the softmax function to obtain The image classification results of each pseudo-new class sample in the pseudo-incremental stage;

[0063] Based on The image classification results and pseudo labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained to obtain a trained logit adjustment module;

[0064] The classification module is used to freeze the trained logit adjustment module, and then obtain the target image classification model based on the frozen feature extraction module, the prototype similarity calculation module and the frozen logit adjustment module; the new class samples preprocessed in each small sample increment stage are input into the target image classification model in turn to obtain the image classification results of the new class samples in each small sample increment stage.

[0065] The above technical solution of the present invention has the following beneficial effects compared with the prior art:

[0066] The present invention discloses an image classification method and system for incremental learning of small sample classes based on logit adjustment. The image classification model is constructed by sequentially connecting: a feature extraction module, a prototype similarity calculation module and a logit adjustment module. The feature extraction module is trained and frozen using a preprocessed base class data set. The present invention constructs a pseudo-new class data set corresponding to a plurality of pseudo-incremental stages through the preprocessed base class data set. Based on the pseudo-new class data set of each pseudo-incremental stage, the logit adjustment module is trained through a plurality of pseudo-incremental stages to obtain a trained logit adjustment module, so that the logit adjustment module can obtain the ability to calculate a balance factor based on the similarity relationship between the base class and the pseudo-new class in advance, and is frozen during subsequent use, so as to balance the logit distribution of the base class and the real new class. The logit of each sample is corrected through the trained logit adjustment module, so as to help the image classification model to more accurately identify new fault types, effectively avoiding the problem that the model is overly dependent on the features of known fault types, resulting in the new fault type being misclassified as the base class, thereby improving the accuracy and reliability of fault diagnosis and monitoring of power equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to make the content of the present invention more clearly understood, the present invention is further described in detail below according to specific embodiments of the present invention in conjunction with the accompanying drawings, wherein:

[0068] Figure 1 It is a flowchart of the steps of an image classification method based on logit-adjusted small sample class incremental learning of the present invention.

[0069] Figure 2 It is a schematic diagram of the effect of an image classification method and system based on logit-adjusted small sample class incremental learning in the present invention.

[0070] Figure 3 is the structural diagram of the image classification model. Figure 3 (a) is the structure diagram of the feature extraction module. Figure 3 (b) is the structure diagram of the feature extraction module. Figure 3 (c) in the figure is the structure diagram of the prototype similarity calculation module.

[0071] Figure 4 It is a schematic diagram of the equilibrium result of logit distribution.

[0072] Figure 5 This is a schematic diagram of the effect of the image classification model on two public datasets, Cifar100 and CUB200. DETAILED DESCRIPTION

[0073] The present invention is further described below in conjunction with the accompanying drawings and specific embodiments so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0074] Reference Figure 1 As shown, an image classification method based on logit-adjusted small sample class incremental learning of the present invention comprises the following steps:

[0075] like Figure 2 As shown, Figure 2 This is a schematic diagram of the effect of an image classification method and system based on logit-adjusted small sample class incremental learning in the present invention.

[0076] Step S1: Obtain an image dataset, divide the image dataset into base class data and new class data in multiple small sample increment stages, and perform preprocessing;

[0077] The setting of FSCIL usually includes a basic stage and T small sample incremental stages. The basic stage includes B base classes, each of which has a large amount of training data available, while a small sample incremental stage has only N new classes (B>>N), each of which has only K sample instances available, and there is no overlap between the label spaces of all stages. The image classification model needs to be tested on the categories learned in the current stage and all previous stages. Among them, T is the total number of small sample incremental stages.

[0078] In this embodiment, the datasets used are three public datasets: mini-ImageNet, CIFAR100, and CUB200. Among them, mini-ImageNet and CIFAR100 both have 100 categories, of which 60 are base classes and 40 are new classes, each of which is divided into 8 small sample increment stages, each stage has 5 new classes, and each new class has 5 samples. CUB200 is a fine-grained bird dataset with 200 categories, of which 100 are base classes and 100 are new classes, each of which is divided into 10 small sample increment stages, each stage has 10 new classes, and each new class has 5 samples.

[0079] The image dataset is denoted as D train ={D 0 ,D 1 ,...,D T}, D 0 is the base class dataset, D t is the new class dataset in the t-th small sample increment stage, is the i-th sample in the t-th small sample increment stage, is the i-th sample in the t-th small sample increment stage The corresponding true label, C t is the label space of the tth stage. t is the data set of the tth small sample increment stage, is the i-th sample in the t-th small sample increment stage, is the i-th sample in the t-th small sample increment stage The corresponding label, C t is the label space of the t-th small sample increment stage.

[0080] The label spaces between different stages do not overlap, that is, for For any j-th stage and k-th stage (j≠k) in the interval [0,T-1], there is no intersection between the label space of the j-th stage and the label space of the k-th stage.

[0081] Each stage of training is followed by a corresponding test process, which is performed on all learned categories, that is, the test data label space of the tth stage satisfies:

[0082]

[0083] The test data label space of the tth stage It is C 0 to C t The union of all sets; is the test data label space of the tth stage, C j is the label space of the jth stage.

[0084] In the data preprocessing stage, the present invention uses operations such as scaling, cropping, and random horizontal flipping to perform data enhancement on image instances, and implements batch normalization before inputting the image classification model to facilitate subsequent processing.

[0085] Step S2: input the preprocessed base class data set into the feature extraction module, extract the feature embedding of each base class sample, train the feature extraction module, and freeze the trained feature extraction module;

[0086] Step S3: input the preprocessed base class data set into the frozen feature extraction module to extract the feature embedding of each base class sample; based on the feature embedding of each base class sample, calculate the prototype of each base class respectively;

[0087] In this embodiment, preferably, based on the feature embedding of each base class sample, the prototype of each base class is calculated respectively, and the formula is:

[0088]

[0089] in, is the prototype of the αth base class, f θ (.) is the frozen feature extraction module, is the indicator function, is the true label of the i-th base class sample, is the feature embedding of the i-th base class sample, |D 0 | is the number of samples in the base class dataset, N α is the number of samples of the αth base class, and α is the αth base class of the base class data.

[0090] 1. Basic stage

[0091] like Figure 3 As shown in (a), Figure 3 (a) is the structure diagram of the feature extraction module. In the basic stage, the feature extraction module is trained using the preprocessed base class data to obtain superior generalization ability;

[0092] The feature extraction module is trained using labeled data of multiple base classes. In the subsequent small sample incremental learning phase, the feature extraction module is kept frozen and only new category prototypes are added to the prototype similarity calculation module accordingly. This can suppress the problem of forgetting the base classes caused by the incremental process.

[0093] In this embodiment, the feature extraction module is any one of the ResNet12 network, the ResNet18 network, and the ResNet20 network.

[0094] In this embodiment, preferably, the feature extraction module adopts the ResNet12 network. The deep residual structure of the ResNet12 network effectively solves the gradient vanishing problem of the deep network, allowing the model to learn complex features at a deeper level. Secondly, it can extract rich hierarchical features, thereby improving the generalization ability of the model, especially when the number of samples is limited. In addition, ResNet12 has a relatively small number of parameters, which helps to reduce the risk of overfitting, and its training process converges quickly, which is particularly important for small sample scenarios. ResNet12 provides a powerful feature representation capability and an efficient training process for small sample incremental learning, which is conducive to capturing semantic information, thereby enhancing compensation for new classes with high similarity, adjusting the logit distribution, and correcting the prediction results.

[0095] The cross entropy loss function used in training is as follows:

[0096]

[0097] Among them, L base is the loss function of the feature extraction module, |C 0| is the number of base classes in the base class data, is the true label of the i-th sample in the base class dataset, represented by a one-hot encoding, is the probability that the i-th sample in the base class dataset is predicted to be its true label.

[0098] After training, the feature extraction module is able to well represent the features of all base class data. Freezing the feature extraction module for use in subsequent processes can help alleviate the problem of forgetting the base class.

[0099] (ii) Pseudo-training stage: Use the base class data to construct pseudo-new class instances and simulate the real incremental stage to train the logit adjustment module. This enables the logit adjustment module to acquire the ability to calculate the balance factor based on the similarity relationship between the base class and the pseudo-new class in advance, and freeze it for subsequent use to balance the logit distribution of the base class and the real new class.

[0100] The present invention additionally designs a pseudo-training phase after the basic phase, the purpose of which is to ensure that the logit adjustment module has acquired the ability to capture the similarity relationship between the new class and the base class in advance, so that the balancing factor calculated based on this has stronger generalization ability and is not easy to overfit individual new classes.

[0101] Step S4: constructing pseudo new class data sets corresponding to multiple pseudo incremental stages based on the preprocessed base class data set;

[0102] Randomly select M base classes and several samples of M base classes from the preprocessed base class data set as pseudo-new class test sets;

[0103] In addition to the M base classes of the preprocessed base class data set and the first pseudo-incremental stage to the Extracted in pseudo-incremental phase From the base classes other than the base class, randomly select 2N base classes and combine them two by two to get the first There are N category pairs in the pseudo-incremental stage, and each category pair represents a pseudo-new category;

[0104] Randomly extract K samples of the two base classes in the current category pair, and combine the samples of the first base class and the samples of the second base class in the current category pair in pairs, and then obtain K pseudo-new class samples of the current pseudo-new class through linear mixing;

[0105] Based on the total number of base class samples, the first pseudo-incremental stage to the The number of pseudo-new classes in the pseudo-incremental stage is generated The pseudo label of each pseudo new class sample in the pseudo incremental stage;

[0106] Based on All pseudo-new class samples and their pseudo-labels in the pseudo-incremental stage are constructed The pseudo new class dataset corresponds to the pseudo incremental stage.

[0107] Randomly select K samples from each of the base class m and base class n in the current category pair (m, n) and is the true label of the base class m, is the true label of base class n, is the i-th m samples, is the jth in base class n m samples, and then and Every two samples are combined to get the pseudo-incremental stage New class samples The formula is:

[0108]

[0109] Among them, λ is a random number ranging from 0.4 to 0.6. Repeat this process to finally generate N×K groups of pseudo-incremental training data. Pseudo-incremental phase The k-th new class sample.

[0110] Since each combination of two categories only occurs once, the present invention directly allows (Tag numbers start at 0).

[0111] in, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, B is the total number of base classes, N is the number of pseudo-new classes in each pseudo-incremental stage, For the pseudo-incremental stage, / / is the modulus, K is the number of samples of the two base classes in each category pair randomly selected in each pseudo-incremental stage, v is the The vth pseudo-new class sample in the pseudo-incremental stage.

[0112] (III) Incremental stage: After obtaining the small sample training data of the current stage, the prototype similarity calculation module is used to calculate the similarity between each sample instance and the category prototype, and classification is performed based on the similarity. Before the final prediction, the prototype similarity between the new class and the base class is input into the logit adjustment module to calculate the balance factor, adjust the logit distribution, reduce the confusion of the new class, and optimize the prediction results.

[0113] like Figure 3 As shown in (c), Figure 3(c) in the figure is the structure diagram of the prototype similarity calculation module. In the incremental stage, the prototype similarity calculation module is used for classification and the logit adjustment module is used to balance the prediction results.

[0114] Step S5: inputting the pseudo-new class data set of each pseudo-incremental stage into the frozen feature extraction module, extracting the feature embedding of each pseudo-new class sample of each pseudo-incremental stage; based on the feature embedding of each pseudo-new class sample of each pseudo-incremental stage, respectively calculating the prototype of each pseudo-new class of each pseudo-incremental stage;

[0115]

[0116] in, For the The prototype of the cth pseudo-new class in the pseudo-incremental stage, f θ (.) is the frozen feature extraction module, is the indicator function, For the The vth pseudo-new class sample in the pseudo-incremental stage, For the The feature embedding of the vth pseudo-new class sample in the pseudo-incremental stage, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, c is the pseudo-label of the vth pseudo-new class sample, The cth pseudo-new class in the pseudo-incremental stage, N is the number of pseudo-new classes in each pseudo-incremental stage, and K is the number of samples of the two base classes in each category pair randomly selected in each pseudo-incremental stage.

[0117] Step S6: Based on the prototype of each base class and the prototype of each pseudo-new class in each pseudo-incremental stage, the logit adjustment module is trained through multiple pseudo-incremental stages to obtain a trained logit adjustment module, including:

[0118] For each pseudo-increment phase, compare the prototype of each base class with the The prototype of each pseudo-new class in the pseudo-incremental stage is input into the prototype similarity calculation module to obtain the The similarity vector of each pseudo-new class in the pseudo-incremental stage in, is the total number of pseudo-incremental phases;

[0119]

[0120] in, For the The similarity vector of the cth pseudo-new class in the pseudo-incremental stage, sim(·) is the similarity function.

[0121] The similarity function sim(·) is any one of the Bray-Curtis similarity coefficient, the Pearson correlation coefficient, and the cosine similarity.

[0122] The similarity function sim(·) used in this embodiment is cosine similarity. Since the prototype is the most representative feature of the category, i.e., semantics, Naturally able to capture pseudo-new classes and base class The semantic similarity between them. is the prototype of the Bth base class, Is the prototype of the first base class, The prototype of the second base class.

[0123] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is input into the logit adjustment module to obtain The balance factors of the pseudo-incremental phase include:

[0124] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is passed through the softmax function and the balance factor calculation module in turn to obtain the The balance factor of each pseudo-new class in the pseudo-incremental phase;

[0125] In this embodiment, the balance factor calculation module is any one of a multi-layer perceptron, a fully connected network, and a neural network.

[0126] In this embodiment, preferably, the balance factor calculation module is a multi-layer perceptron, which has two hidden layers and uses the Relu function as the activation function. Considering that the pseudo-new class c is a mixture of two base classes, it may be highly similar to these two classes. Therefore, when using the multi-layer perceptron to calculate Before that, the similarity matrix must be normalized using the softmax function σ(·). Compared with a fully connected network, a multilayer perceptron can capture more nonlinear information, and compared with a simple neural network, a multilayer perceptron has a smaller computational complexity.

[0127] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is passed through the softmax function and the balance factor calculation module in turn to obtain the The balance factor of each pseudo-new class in the pseudo-incremental stage is as follows:

[0128]

[0129] in, For the The balance factor of the cth pseudo-new class in the pseudo-incremental phase, is a multi-layer perceptron, σ(.) is a softmax function, For the The similarity matrix of the pseudo-incremental stage.

[0130] The first The balance factors of all pseudo-new classes in the pseudo-incremental stage are concatenated to obtain The balance factor vector of the pseudo-incremental phase of the pseudo-incremental phase;

[0131] Set the first pseudo-increment phase to The balance factor vectors of the pseudo-incremental stages are concatenated in order of stages to obtain Balance factor for the pseudo-incremental phase;

[0132] The first The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weight of the pseudo-incremental stage is as follows:

[0133]

[0134] in, For the The classification weights of the pseudo-incremental stage, For the The classification weights of the pseudo-incremental stage, Concat(.) is a concatenation operation, For the The prototype of the cth pseudo-new class in the pseudo-incremental stage, N is the number of pseudo-new classes in each pseudo-incremental stage, W 0 It is the weight matrix obtained by concatenating the prototypes of all base classes.

[0135] Based on The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage, The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage;

[0136] The said The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage, The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage includes:

[0137]

[0138] Due to the confusion problem of the new class, The base class (the first B elements) is on average higher than the new class (the last In order to adjust this imbalance phenomenon, the present invention will Balance factor for pseudo-incremental phases Add according to the meaning of the elements On the logit of the new class:

[0139]

[0140] in, For the The logit of the vth pseudo-new class sample in the pseudo-incremental stage, For the The vth pseudo-new class sample in the pseudo-incremental stage, For the The feature embedding of the vth pseudo-new class sample in the pseudo-incremental stage, Concat(.) is the concatenation operation, is a B-dimensional vector whose elements are all 0, B is the total number of base classes, γ is a hyperparameter, For the Balance factor for the pseudo-increment phase.

[0141] The first The logit of each pseudo-new class sample in the pseudo-incremental stage is passed through the softmax function to obtain the first The image classification results of each pseudo-new class sample in the pseudo-incremental stage;

[0142] go through The logit adjustment will make any sample x μ The probability of predicting category γ is:

[0143]

[0144] in, For sample x μ The probability of predicting the class γ, For sample x μ The predicted score logit on class γ, For sample x μ The predicted score logit for class o.

[0145] Based on The image classification results and true labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained to obtain a trained logit adjustment module;

[0146] The calculation formula of the loss function in the pseudo training stage is:

[0147]

[0148] Among them, L pse For the The loss function of each pseudo-incremental stage, B is the total number of base classes, N is the number of pseudo-new classes in each pseudo-incremental stage, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, For the The probability that the vth pseudo-new class sample in the pseudo-incremental stage is predicted to be the oth class, For the The balance factor of the pseudo-incremental stage, α is the regularization coefficient, For the The similarity vector of the cth pseudo-new class in the pseudo-incremental stage.

[0149] After the simulation of the pseudo training process, The ability to capture similarity relationships from the class similarity matrix between the base class and the new class and calculate the appropriate class-aware balance factor β has been obtained. Subsequently, the present invention obtains an auxiliary subnetwork g with relatively stronger generalization ability through pseudo-testing. φ (·). In the subsequent real incremental learning phase, g φ (·) Freezing, only using its ability to capture similarity relations to compute class-aware β.

[0150] like Figure 4 As shown, Figure 4 It is a schematic diagram of the balance result of logit distribution. The present invention constructs pseudo-new class data in each pseudo-incremental stage through preprocessed base class data; based on the pseudo-new class data in each pseudo-incremental stage, the logit adjustment module is trained through multiple pseudo-incremental stages to obtain a trained logit adjustment module, so that the logit adjustment module pre-acquires the ability to calculate the balance factor based on the similarity relationship between the base class and the pseudo-new class, and is frozen during subsequent use, and can balance the logit distribution of the base class and the real new class. The logit of each sample is corrected through the trained logit adjustment module, which can help the image classification model to classify the new class more accurately.

[0151] like Figure 3 As shown, Figure 3 This is a structural diagram of the image classification model.

[0152] Step S7: After freezing the trained logit adjustment module, the target image classification model is obtained based on the frozen feature extraction module, the prototype similarity calculation module and the frozen logit adjustment module; the new class samples preprocessed in each small sample increment stage are input into the target image classification model (CALA-FSCIL model) in turn to obtain the image classification results of the new class samples in each small sample increment stage.

[0153] In the actual incremental learning process, taking a small sample incremental stage as an example, the balance factor of the current small sample incremental stage is also obtained by first calculating the prototype of each new class in the current small sample incremental stage, and inputting the prototype of each new class into the prototype similarity calculation module to obtain the similarity matrix of the current small sample incremental stage; the similarity matrix of the prototype of each pseudo-new class in the current pseudo-incremental stage and the prototype of all base classes is input into the logit adjustment module to obtain the balance factor of the current small sample incremental stage; the classification weight of the previous small sample incremental stage is spliced ​​with the prototype of each new class in the current small sample incremental stage to obtain the classification weight of the current small sample incremental stage; based on the balance factor of the current small sample incremental stage, the classification weight of the current small sample incremental stage, and the feature embedding of each new class sample in the current small sample incremental stage, the logit of each new class sample in the current small sample incremental stage is calculated; the logit of each new class sample in the current small sample incremental stage is passed through the softmax function to obtain the image classification result of each new class sample in the current small sample incremental stage;

[0154] γ is used to limit the degree of balance of the balance factor of the current small sample increment stage on the new class to avoid under-balance and over-balance. The new class confusion phenomenon occurs because the feature extraction module over-learns the base class data, so that the new class features extracted by it are extremely similar to the base class features in the feature space, which makes the new class more easily confused with the similar base class. The balance factor of the current small sample increment stage is calculated by the multi-layer perceptron based on the similarity matrix between the new class and the base class. Therefore, the more similar a new class is to the base class, the greater the degree of adjustment of the balance factor of the current small sample increment stage on the new class.

[0155] At this stage, the loss L can be classified by the loss function cls Fine-tune a small number of parameters of the feature extraction module or classification weight parameters.

[0156] Classification loss L cls The calculation formula is:

[0157]

[0158] Among them, L cls is the classification loss, |C t | is the number of categories in the tth small sample increment stage, is the true label of the i-th sample in the t-th small sample increment stage, represented by a one-hot encoding, p i,α is the probability that the i-th sample in the t-th small sample increment stage is predicted as its true label.

[0159] The present invention designs corresponding loss functions in three different stages. In the basic stage, L base The training feature extraction module has excellent feature representation capabilities. In the pseudo training stage, L pse Let the logit adjustment module acquire the ability to calculate the balance factor in advance to facilitate the subsequent balance of the classified logit, and finally use L in the incremental stage cls Fine-tuning parameters improves the classification effect of the prototype similarity calculation module.

[0160] This second embodiment evaluates three public dataset test tasks in image processing: mini-ImageNet, CIFAR100, and CUB200. These tasks are designed to evaluate the robustness of machine learning models when incrementally learning small sample data.

[0161] Mini-ImageNet is a subset of ImageNet, containing 100 categories, each with 600 RGB image instances of 84×84 pixels. CIFAR100 is a small image dataset with 100 categories and 60,000 samples, each with a pixel size of only 32×32. CUB200 is a fine-grained bird image dataset with 200 categories and 11,788 images, each with a pixel size of 224×224. The present invention strictly adheres to the FSCIL setting, and each stage only uses the data of the current stage for training. All comparative experimental models, including CEC, SAVC, and BiDist, are tested according to the FSCIL proposed by TOPIC.

[0162] In the second embodiment, ResNet-18 pre-trained with different parameters is used as the pre-training model for three data sets.

[0163] This second embodiment converts the small sample incremental learning (FSCIL) into a two-classification experiment, treating the base class as the positive class and the new class as the negative class, and calculating the False Positive Rate (FPR) of the classification to indicate the degree to which the new class is mistakenly classified as the base class. Compared with BiDist, this second embodiment reduces the FPR of the last stage on mini-ImageNet from 54.52% to 28.40%. The experimental results are shown in Table 1, which shows the FPR experimental results of the mini-ImageNet dataset.

[0164] Table 1

[0165]

[0166] The comparative experiment of the second embodiment is tested in stages. Compared with TEEN, the accuracy of the last stage is successfully improved by 5.4% on mini-ImageNet. The experimental results are shown in Table 2. Table 2 is the comparative experimental results of the mini-ImageNet dataset.

[0167] Table 2

[0168]

[0169] In the second embodiment, the model is used as a plug-in, which improves the harmonic mean of the accuracy of the new class and the base class in the last stage of SAVC by 3.97%. The experimental results are shown in Table 3.

[0170] Table 3

[0171]

[0172] like Figure 5 As shown, Figure 5 The figure is a schematic diagram of the effect of the image classification model on two public datasets, Cifar100 and CUB200. From the above content, it can be seen that the image classification model proposed in the present invention can balance the logit distribution of the base class and the real new class, so that the accuracy can still be maintained when classifying the new class, and effectively avoids the problem that the recognition result of the new class in the model is more biased towards the base class.

[0173] This third embodiment also provides an image classification system based on logit-adjusted small sample class incremental learning, including:

[0174] A data acquisition module is used to acquire an image data set, divide the image data set into a base class data set and new class data sets corresponding to different small sample increment stages, and perform preprocessing; wherein the base class data set includes B base classes, each base class includes multiple samples and their true labels, and the new class data set corresponding to each small sample increment stage includes N new classes, each new class includes multiple samples and their true labels;

[0175] The feature extraction module training module is used to input the preprocessed base class data set into the feature extraction module, extract the feature embedding of each base class sample, train the feature extraction module, and freeze the trained feature extraction module;

[0176] The base class prototype acquisition module is used to input the preprocessed base class data set into the frozen feature extraction module to extract the feature embedding of each base class sample; based on the feature embedding of each base class sample, the prototype of each base class is calculated respectively;

[0177] A pseudo-new class data set construction module is used to construct pseudo-new class data sets corresponding to multiple pseudo-incremental stages based on the preprocessed base class data set;

[0178] A pseudo-new class prototype acquisition module is used to input the pseudo-new class data set of each pseudo-incremental stage into the frozen feature extraction module to extract the feature embedding of each pseudo-new class sample of each pseudo-incremental stage; based on the feature embedding of each pseudo-new class sample of each pseudo-incremental stage, the prototype of each pseudo-new class of each pseudo-incremental stage is calculated respectively;

[0179] The logit adjustment training module is used to train the logit adjustment module through multiple pseudo-incremental stages based on the prototype of each base class and the prototype of each pseudo-new class in each pseudo-incremental stage to obtain a trained logit adjustment module, including:

[0180] For each pseudo-increment phase, compare the prototype of each base class with the The prototype of each pseudo-new class in the pseudo-incremental stage is input into the prototype similarity calculation module to obtain the The similarity vector of each pseudo-new class in the pseudo-incremental stage; is the total number of pseudo-incremental phases;

[0181] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is input into the logit adjustment module to obtain The balance factors of the pseudo-incremental phase include:

[0182] The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is passed through the softmax function and the balance factor calculation module in turn to obtain the The balance factor of each pseudo-new class in the pseudo-incremental stage; The balance factors of all pseudo-new classes in the pseudo-incremental stage are concatenated to obtain The balance factor vector of the pseudo-increment phase;

[0183] Set the first pseudo-increment phase to The balance factor vectors of the pseudo-incremental stages are concatenated in order of stages to obtain Balance factor for the pseudo-incremental phase;

[0184] The first The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weights of the pseudo-incremental stages;

[0185] Based on The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage,

[0186] The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage;

[0187] The first The logit of each pseudo-new class sample in the pseudo-incremental stage is passed through the softmax function to obtain The image classification results of each pseudo-new class sample in the pseudo-incremental stage;

[0188] Based on The image classification results and pseudo labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained to obtain a trained logit adjustment module;

[0189] The classification module is used to freeze the trained logit adjustment module, and then obtain the target image classification model based on the frozen feature extraction module, the prototype similarity calculation module and the frozen logit adjustment module; the new class samples preprocessed in each small sample increment stage are input into the target image classification model in turn to obtain the image classification results of the new class samples in each small sample increment stage.

[0190] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0191] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0192] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0193] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0194] Obviously, the above embodiments are merely examples for the purpose of clear explanation and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the present invention.

Claims

1. An image classification method based on incremental learning of small sample classes with logit adjustment, characterized in that: The following steps are involved: Obtain an image dataset, divide the image dataset into a base class dataset and new class datasets corresponding to different small sample increment stages, and perform preprocessing; wherein the base class dataset includes B base classes, each base class includes multiple samples and their true labels, and the new class dataset corresponding to each small sample increment stage includes N new classes, each new class includes multiple samples and their true labels; Input the preprocessed base class data set into the feature extraction module, extract the feature embedding of each base class sample, train the feature extraction module, and freeze the trained feature extraction module; The preprocessed base class data set is input into the frozen feature extraction module to extract the feature embedding of each base class sample; based on the feature embedding of each base class sample, the prototype of each base class is calculated respectively; Based on the preprocessed base class data set, construct pseudo new class data sets corresponding to multiple pseudo incremental stages; Input the pseudo-new class dataset of each pseudo-incremental stage into the frozen feature extraction module to extract the feature embedding of each pseudo-new class sample of each pseudo-incremental stage; based on the feature embedding of each pseudo-new class sample of each pseudo-incremental stage, calculate the prototype of each pseudo-new class of each pseudo-incremental stage respectively; Based on the prototype of each base class and the prototype of each pseudo-new class in each pseudo-incremental stage, the logit adjustment module is trained through multiple pseudo-incremental stages to obtain a trained logit adjustment module, including: For each pseudo-increment phase, compare the prototype of each base class with the The prototype of each pseudo-new class in the pseudo-incremental stage is input into the prototype similarity calculation module to obtain the The similarity vector of each pseudo-new class in the pseudo-incremental stage; is the total number of pseudo-incremental phases; The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is input into the logit adjustment module to obtain The balance factors of the pseudo-incremental phase include: The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is passed through the softmax function and the balance factor calculation module in turn to obtain the The balance factor of each pseudo-new class in the pseudo-incremental stage; The balance factors of all pseudo-new classes in the pseudo-incremental stage are concatenated to obtain The balance factor vector of the pseudo-increment phase; Set the first pseudo-increment phase to The balance factor vectors of the pseudo-incremental stages are concatenated in order of stages to obtain Balance factor for the pseudo-incremental phase; The first The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weights of the pseudo-incremental stages; Based on The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage, The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage; The first The logit of each pseudo-new class sample in the pseudo-incremental stage is passed through the softmax function to obtain The image classification results of each pseudo-new class sample in the pseudo-incremental stage; Based on The image classification results and pseudo labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained to obtain a trained logit adjustment module; After freezing the trained logit adjustment module, the target image classification model is obtained based on the frozen feature extraction module, the prototype similarity calculation module and the frozen logit adjustment module; the new class samples preprocessed in each small sample increment stage are input into the target image classification model in turn to obtain the image classification results of the new class samples in each small sample increment stage.

2. The image classification method based on logit-adjusted small sample class incremental learning according to claim 1, characterized in that: The method of constructing a plurality of pseudo new class data sets corresponding to pseudo incremental stages based on the preprocessed base class data set includes: Randomly select M base classes and several samples of M base classes from the preprocessed base class data set as pseudo-new class test sets; In addition to the M base classes of the preprocessed base class data set and the first pseudo-incremental stage to the Extracted in pseudo-incremental phase From the base classes other than the base class, randomly select 2N base classes and combine them two by two to get the first There are N category pairs in the pseudo-incremental stage, and each category pair represents a pseudo-new category; Randomly extract K samples of the two base classes in the current category pair, and combine the samples of the first base class and the samples of the second base class in the current category pair in pairs, and then obtain K pseudo-new class samples of the current pseudo-new class through linear mixing; Based on the total number of base class samples, the first pseudo-incremental stage to the The number of pseudo-new classes in the pseudo-incremental stage is generated The pseudo label of each pseudo new class sample in the pseudo incremental stage; Based on All pseudo-new class samples and their pseudo-labels in the pseudo-incremental stage are constructed The pseudo new class dataset corresponds to the pseudo incremental stage.

3. The image classification method based on logit-adjusted small sample class incremental learning according to claim 2, characterized in that: The total number of base class samples, the first pseudo-incremental stage to the The number of pseudo-new classes in the pseudo-incremental stage is generated The pseudo label of each pseudo-new class sample in the pseudo-incremental stage is: in, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, B is the total number of base classes, N is the number of pseudo-new classes in each pseudo-incremental stage, For the pseudo-incremental stage, / / is the modulus, K is the number of samples of the two base classes in each category pair randomly selected in each pseudo-incremental stage, v is the The vth pseudo-new class sample in the pseudo-incremental stage.

4. The image classification method based on logit-adjusted small sample class incremental learning according to claim 1, characterized in that: The feature embedding of each pseudo-new class sample in the current pseudo-incremental stage is based on respectively calculating the prototype of each pseudo-new class in the current pseudo-incremental stage, and the formula is: in, For the The prototype of the cth pseudo-new class in the pseudo-incremental stage, f θ (.) is the frozen feature extraction module, is the indicator function, For the The vth pseudo-new class sample in the pseudo-incremental stage, For the The feature embedding of the vth pseudo-new class sample in the pseudo-incremental stage, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, c is the pseudo label of the vth pseudo-new class sample, The cth pseudo-new class in the pseudo-incremental stage, N is the number of pseudo-new classes in each pseudo-incremental stage, and K is the number of samples of the two base classes in each category pair randomly selected in each pseudo-incremental stage.

5. The image classification method based on logit-adjusted small sample class incremental learning according to claim 4, characterized in that: The The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weight of the pseudo-incremental stage is as follows: in, For the The classification weights of the pseudo-incremental stage, For the The classification weights of the pseudo-incremental stage, Concat(.) is a concatenation operation, For the The prototype of the cth pseudo-new class in the pseudo-incremental stage, N is the number of pseudo-new classes in each pseudo-incremental stage, W 0 It is the weight matrix obtained by concatenating the prototypes of all base classes.

6. The image classification method based on logit-adjusted small sample class incremental learning according to claim 5, characterized in that: The The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage, The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage includes: in, For the The logit of the vth pseudo-new class sample in the pseudo-incremental stage, For the The vth pseudo-new class sample in the pseudo-incremental stage, For the The feature embedding of the vth pseudo-new class sample in the pseudo-incremental stage, Concat(.) is the concatenation operation, is a B-dimensional vector whose elements are all 0, B is the total number of base classes, γ is a hyperparameter, For the Balance factor for the pseudo-increment phase.

7. The image classification method based on logit-adjusted small sample class incremental learning according to claim 4, characterized in that: The The image classification results and pseudo labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained, and the loss function is calculated as follows: Among them, L pse For the The loss function of each pseudo-incremental stage, B is the total number of base classes, N is the number of pseudo-new classes in each pseudo-incremental stage, For the The pseudo label of the vth pseudo-new class sample in the pseudo-incremental stage, For the The probability that the vth pseudo-new class sample in the pseudo-incremental stage is predicted to be the oth class, For the The balance factor of the pseudo-incremental stage, α is the regularization coefficient, For the The similarity vector of the cth pseudo-new class in the pseudo-incremental stage.

8. The image classification method based on logit-adjusted small sample class incremental learning according to claim 1, characterized in that: The feature extraction module is any one of the ResNet12 network, the ResNet18 network, and the ResNet20 network.

9. The image classification method based on logit-adjusted small sample class incremental learning according to claim 1, characterized in that: The balance factor calculation module is any one of a multi-layer perceptron, a fully connected network, and a neural network.

10. An image classification system based on logit-adjusted small sample class incremental learning, characterized in that: include: A data acquisition module is used to acquire an image data set, divide the image data set into a base class data set and new class data sets corresponding to different small sample increment stages, and perform preprocessing; wherein the base class data set includes B base classes, each base class includes multiple samples and their true labels, and the new class data set corresponding to each small sample increment stage includes N new classes, each new class includes multiple samples and their true labels; The feature extraction module training module is used to input the preprocessed base class data set into the feature extraction module, extract the feature embedding of each base class sample, train the feature extraction module, and freeze the trained feature extraction module; The base class prototype acquisition module is used to input the preprocessed base class data set into the frozen feature extraction module to extract the feature embedding of each base class sample; based on the feature embedding of each base class sample, the prototype of each base class is calculated respectively; A pseudo-new class data set construction module is used to construct pseudo-new class data sets corresponding to multiple pseudo-incremental stages based on the preprocessed base class data set; A pseudo-new class prototype acquisition module is used to input the pseudo-new class data set of each pseudo-incremental stage into the frozen feature extraction module to extract the feature embedding of each pseudo-new class sample of each pseudo-incremental stage; based on the feature embedding of each pseudo-new class sample of each pseudo-incremental stage, the prototype of each pseudo-new class of each pseudo-incremental stage is calculated respectively; The logit adjustment training module is used to train the logit adjustment module through multiple pseudo-incremental stages based on the prototype of each base class and the prototype of each pseudo-new class in each pseudo-incremental stage to obtain a trained logit adjustment module, including: For each pseudo-increment phase, compare the prototype of each base class with the The prototype of each pseudo-new class in the pseudo-incremental stage is input into the prototype similarity calculation module to obtain the The similarity vector of each pseudo-new class in the pseudo-incremental stage; is the total number of pseudo-incremental phases; The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is input into the logit adjustment module to obtain The balance factors of the pseudo-incremental phase include: The first The similarity vector of each pseudo-new class in the pseudo-incremental stage is passed through the softmax function and the balance factor calculation module in turn to obtain the The balance factor of each pseudo-new class in the pseudo-incremental stage; The balance factors of all pseudo-new classes in the pseudo-incremental stage are concatenated to obtain The balance factor vector of the pseudo-increment phase; Set the first pseudo-increment phase to The balance factor vectors of the pseudo-incremental stages are concatenated in order of stages to obtain Balance factor for the pseudo-incremental phase; The first The classification weight of the pseudo-incremental stage is The prototypes of each base class are spliced ​​in the pseudo-incremental stage to obtain the The classification weights of the pseudo-incremental stages; Based on The balance factor of the pseudo-incremental phase, The classification weight of the pseudo-incremental stage, The feature embedding of each pseudo-new class sample in the pseudo-incremental stage is calculated to obtain the The logit of each pseudo-new class sample in the pseudo-incremental stage; The first The logit of each pseudo-new class sample in the pseudo-incremental stage is passed through the softmax function to obtain The image classification results of each pseudo-new class sample in the pseudo-incremental stage; Based on The image classification results and pseudo labels of each pseudo-new class sample in the pseudo-incremental stage are calculated by the loss function. The logit adjustment module of the pseudo-incremental stage is trained to obtain a trained logit adjustment module; The classification module is used to freeze the trained logit adjustment module, and then obtain the target image classification model based on the frozen feature extraction module, the prototype similarity calculation module and the frozen logit adjustment module; the new class samples preprocessed in each small sample increment stage are input into the target image classification model in turn to obtain the image classification results of the new class samples in each small sample increment stage.

Images