Image classification method based on sample-free class incremental learning, storage medium and equipment

Abstract

The application relates to the technical field of image classification, and particularly discloses an image classification method based on sample-free class incremental learning, equipment and a storage medium, which comprises the following steps: constructing a feature extraction backbone network and an expert hybrid classifier, wherein the expert hybrid classifier comprises an old class expert, a new class expert and a gate network; performing initial training according to a base class image dataset; performing incremental training and updating a class prototype library based on the image sample features of the newly added class and the class prototype information of the old class for each incremental step; repeating until the trained expert hybrid classifier is obtained after all incremental steps are completed; and inputting an image to be classified into the trained expert hybrid classifier to obtain a final image classification result. The image classification method based on sample-free class incremental learning improves the image classification accuracy and stability in a long-term incremental scene.

Description

Technical Field

[0001] This invention relates to the field of image classification technology, and in particular to an image classification method, computer storage medium, and electronic device based on sample-free class incremental learning. Background Technology

[0002] With the development of deep learning in the field of image classification, class incremental learning has become a key technology for dealing with dynamic category expansion scenarios. Its core requirement is to enable the model to maintain the ability to recognize existing category knowledge while continuously learning new categories, without having to retrain on the full dataset each time.

[0003] Existing incremental learning methods can be broadly categorized into two types: one is the replay-based method, which stores old class samples and trains alongside new samples to mitigate catastrophic forgetting; the other is the sample-free incremental learning method, which does not store original samples of old classes and relies solely on historical model parameters, class prototypes, or knowledge distillation to maintain knowledge of old classes. Among these, sample-free incremental learning is more suitable for privacy-sensitive and storage-constrained scenarios.

[0004] However, existing sample-less incremental learning methods still have the following drawbacks:

[0005] (1) Severe conflict between new and old categories. Most mainstream solutions use a single classifier head to handle both new and old categories. However, in scenarios without samples, the old category lacks real sample supervision, and the training signal of the new category will continue to dominate the update of classifier parameters, which will cause the discrimination boundary of the old category to be squeezed, the classifier will be biased, and the model's ability to recognize the old category will be significantly degraded.

[0006] (2) Feature space coupling leads to feature drift. The distribution of new and old class features is similar in the shared space. Training the new class will indirectly modify the representation direction of the old class features, causing the old class features to drift and the class center to become distorted, which further aggravates the confusion between the new and old classes.

[0007] (3) The classification head parameters are not updated stably. The training data in the incremental stage is mainly composed of new classes and pseudo-features. The classification head parameters are easily affected by the current data distribution and fluctuate greatly. The classification boundary is discontinuous between stages, and the robustness of the model decreases after long-term increment.

[0008] (4) The robustness of a single prediction method is insufficient. Relying solely on a parameterized classifier is susceptible to the influence of new class bias, while relying solely on a prototype classifier lacks the ability to fit complex boundaries. A single prediction method is difficult to meet the recognition needs of both new and old classes.

[0009] Therefore, how to achieve a balance between stability and plasticity in sample-less incremental learning without preserving the original samples of the old class and without significantly increasing the number of model parameters, and how to improve the image classification accuracy and robustness in the long-term incremental process, has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0010] This invention provides an image classification method, computer storage medium, and electronic device based on sample-less incremental learning, aiming to at least solve one of the problems existing in related technologies in the sample-less incremental learning scenario, such as classification conflict between old and new categories, feature space coupling, unstable classification head parameters, and insufficient robustness of single prediction.

[0011] As a first aspect of the present invention, an image classification method based on sample-free class incremental learning is provided, comprising:

[0012] Construct a feature extraction backbone network and an expert hybrid classifier, wherein the expert hybrid classifier includes old class experts, new class experts, and a gating network;

[0013] The feature extraction backbone network and the expert hybrid classifier are initially trained based on a base class image dataset that includes multiple base class categories;

[0014] For each incremental step, the expert hybrid classifier is incrementally trained based on the image sample features of the new category and the category prototype information of the old category, and the category prototype library is updated.

[0015] Repeat the above incremental training process until all incremental steps are completed to obtain a trained expert mixture classifier;

[0016] The image to be classified is obtained and then input into a trained expert mixture classifier to obtain the final classification result.

[0017] Further, initial training is performed on the feature extraction backbone network and the expert hybrid classifier based on the base class image dataset, including:

[0018] The feature extraction backbone network and initial classifier are trained based on the base class image dataset;

[0019] Freeze the network parameters of the feature extraction backbone network after training;

[0020] Calculate the feature mean for each base class in the base class image dataset;

[0021] The feature mean of each base class is used as the category prototype of the corresponding base class and stored in the category prototype library;

[0022] The old class expert is initialized based on the parameters of the initial classifier.

[0023] Furthermore, for each incremental step, the expert hybrid classifier is incrementally trained based on the image sample features of the new category and the category prototype information of the old category, and the category prototype library is updated, including:

[0024] Extract image sample features for the new category and generate pseudo features based on the category prototypes of the old categories;

[0025] A training feature set is constructed based on the image sample features of the newly added category and the pseudo-features;

[0026] The training feature set is input into the expert hybrid classifier to obtain a weighted fusion classification result;

[0027] The classification loss is calculated based on the weighted fusion classification result, and the feature orthogonality loss between the two is calculated based on the mean of the image sample features of the newly added category and the mean of the pseudo features.

[0028] Construct a joint loss based on the classification loss and the feature orthogonality loss;

[0029] The parameters of the expert hybrid classifier are optimized based on the joint loss, and the optimized parameters of the expert hybrid classifier are updated by momentum fusion with the parameter copy of the expert hybrid classifier saved at the beginning of the current incremental step.

[0030] After the current incremental step is completed, update the category prototype library, add the feature mean of the new category in the current incremental step, and keep the category prototype of the old category unchanged.

[0031] Further, the training feature set is input into the expert hybrid classifier to obtain a weighted fusion classification result, including:

[0032] The training feature set is input into the expert hybrid classifier, wherein the old class expert in the expert hybrid classifier can output the old class discrimination result, and the new class expert in the expert hybrid classifier can output the new class discrimination result. The gating network in the expert hybrid classifier calculates the fusion weight based on the features in the training feature set, and performs weighted fusion on the old class discrimination result and the new class discrimination result based on the fusion weight to obtain the weighted fused classification result.

[0033] Furthermore, the expression for the weighted fusion classification result is:

[0034] ,

[0035] in, This represents the classification result after weighted fusion. This indicates the result of the old category classification. This indicates the result of the new category classification. and Both represent the fusion weights, and .

[0036] Furthermore, the expression for the feature orthogonal loss is:

[0037] ,

[0038] in, Indicates the characteristic orthogonal loss. This represents the mean of the features of the image samples in the newly added category. This represents the mean of the pseudo-features.

[0039] Furthermore, the expression for the momentum fusion update is:

[0040] ,

[0041] in, Indicates the momentum coefficient. This represents the parameters of the optimized expert hybrid classifier. This represents a copy of the parameters of the expert hybrid classifier saved at the start of the current incremental step.

[0042] Further, the image to be classified is acquired and input into a trained expert mixture classifier to obtain the final image classification result, including:

[0043] Acquire the image to be classified and extract its image features;

[0044] The image features of the image to be classified are input into a trained expert hybrid classifier to obtain parameterized classification probabilities, and the image features of the image to be classified are input into a nearest-nearest-class mean classifier to obtain prototype-based classification probabilities.

[0045] The parameterized classification probabilities and the prototype-based classification probabilities are normalized and then merged to obtain the final image classification result.

[0046] As another aspect of the present invention, a computer storage medium is provided, wherein the computer storage medium is used to store a computer program that, when executed by a processor, implements the image classification method based on sample-free class incremental learning described above.

[0047] In another aspect of the present invention, an electronic device is provided, comprising: a memory and a processor, the memory and the processor being communicatively connected, the memory being used to store computer program instructions, and the processor being used to call and execute the computer program instructions to implement the image classification method based on sample-free class incremental learning described above.

[0048] The image classification method based on sample-free incremental learning provided by this invention constructs a feature extraction backbone network and an expert hybrid classifier. Initial training is performed on the feature extraction backbone network and the expert hybrid classifier using a base class image dataset. Then, incremental training of the expert hybrid classifier is performed for each incremental step until all incremental steps are completed, resulting in a trained expert hybrid classifier. Finally, image classification is performed on the image to be classified based on this trained expert hybrid classifier to obtain the final component result. This image classification method based on sample-free incremental learning decouples the classification tasks of new and old classes through the expert hybrid classifier. The gating network adaptively allocates weights, avoiding new class bias in a single classifier head, effectively mitigating classification conflicts between new and old classes. It achieves a balance between stability and flexibility in sample-free incremental learning without saving original samples of the old classes or significantly increasing storage costs, greatly improving the classification accuracy and robustness of images in long-term incremental scenarios. Attached Figure Description

[0049] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the following detailed description to explain the invention, but do not constitute a limitation thereof.

[0050] Figure 1 The flowchart shows the image classification method based on sample-free class incremental learning provided by this invention.

[0051] Figure 2 The flowchart for the initial training provided by this invention.

[0052] Figure 3 The flowchart for incremental training provided by this invention.

[0053] Figure 4 A flowchart for image classification in the application provided by this invention.

[0054] Figure 5 This is a structural block diagram of the electronic device provided by the present invention. Detailed Implementation

[0055] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0056] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0057] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of the invention described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0058] This embodiment provides an image classification method based on sample-free class incremental learning. Figure 1 This is a flowchart of an image classification method based on sample-free class incremental learning according to an embodiment of the present invention, as shown below. Figure 1 As shown, it includes:

[0059] S100. Construct a feature extraction backbone network and an expert hybrid classifier, wherein the expert hybrid classifier includes old class experts, new class experts and a gated network;

[0060] In this embodiment of the invention, the feature extraction backbone network can specifically use ResNet-18 (residual network - 18 layers) as the basic backbone network to extract feature vectors from the input image and output a 512-dimensional feature vector. The expert fusion classifier includes old class experts, new class experts, and a gating network. The old class experts are responsible for modeling the discrimination rules of historical categories (i.e., old categories), the new class experts are used to adapt the discrimination rules of the newly added categories, and the gating network is used to adaptively allocate the fusion weights of the old class experts and the new class experts according to the input features. In this embodiment of the invention, both the old class experts and the new class experts can be implemented using linear classification heads, and the gating network can be implemented using a 2-layer fully connected network.

[0061] S200. Initially train the feature extraction backbone network and the expert hybrid classifier based on a base class image dataset that includes multiple base class categories;

[0062] Specifically, the base class image dataset refers to the set of labeled images required in the initial training phase of the model. The number of categories and the sample size are set according to the actual application scenario. For example, 100 basic product images can be used in the product recognition scenario, and the first 50 classes of CIFAR-100 or the first 100 classes of ImageNet subset images can be used in the standard image classification dataset.

[0063] S300. For each incremental step, the expert hybrid classifier is incrementally trained based on the image sample features of the new category and the category prototype information of the old category, and the category prototype library is updated.

[0064] In this embodiment of the invention, for each incremental step, the expert hybrid classifier is incrementally trained using image sample features of the newly added category and category prototype information of the old category. After each incremental step completes the incremental training, the category prototype library is updated.

[0065] S400. Repeat the above incremental training process until all incremental steps are completed to obtain the trained expert mixture classifier.

[0066] It should be understood that by repeating step S300 above until all incremental steps are completed, the expert hybrid classifier is fully trained and a trained expert hybrid classifier is obtained.

[0067] S500: Obtain the image to be classified and input it into a trained expert hybrid classifier to obtain the final classification result.

[0068] In this embodiment of the invention, the trained expert hybrid classifier is applied based on the image to be classified, thereby obtaining the final classification result.

[0069] In summary, the image classification method based on sample-free incremental learning provided by this invention constructs a feature extraction backbone network and an expert hybrid classifier. Initial training is performed on both the feature extraction backbone network and the expert hybrid classifier using a base class image dataset. Then, incremental training of the expert hybrid classifier is performed for each incremental step until all incremental steps are completed, resulting in a trained expert hybrid classifier. Finally, image classification is performed on the image to be classified based on this trained expert hybrid classifier to obtain the final component result. This image classification method based on sample-free incremental learning decouples the new and old class classification tasks through the expert hybrid classifier. The gating network adaptively allocates weights, avoiding new class bias in a single classifier head, effectively mitigating classification conflicts between new and old classes. It achieves a balance between stability and flexibility in sample-free incremental learning without saving original samples of the old class or significantly increasing storage costs, greatly improving the classification accuracy and robustness of images in long-term incremental scenarios.

[0070] In this embodiment of the invention, the feature extraction backbone network and the expert hybrid classifier are initially trained based on the base class image dataset, such as... Figure 2 As shown, it includes:

[0071] S210. Train the feature extraction backbone network and initial classifier based on the base class image dataset;

[0072] In this embodiment of the invention, during the initial training phase, the feature extraction backbone network and the initial classifier are jointly trained using a base class dataset (i.e., an initial class set), so that the initial classifier initially possesses the ability to distinguish base class categories, and the feature extraction backbone network learns a stable basic feature extraction capability.

[0073] S220. Freeze the network parameters of the feature extraction backbone network after training is completed;

[0074] Specifically, after freezing, the feature extraction backbone network remains unchanged in subsequent incremental training phases, thus providing a stable feature space for subsequent classification. The stability of the feature space is the foundation for the effective operation of this invention. Specifically, firstly, the category prototypes (feature means) calculated based on the base class data remain valid after the feature space is fixed, requiring no correction in the incremental phase; secondly, the pseudo-features generated based on the old class prototypes and the real new class features reside in the same feature space, ensuring the accuracy of the simulation of old class knowledge during incremental training; and thirdly, in the subsequent application phase, the features extracted from the image to be classified by the backbone network and the stored prototypes are measured in the same space, ensuring the reliability of the nearest-neighbor class mean classifier.

[0075] S230. Calculate the feature mean of each base class in the base class image dataset;

[0076] Specifically, for each base class in the base class image dataset, all training sample images of that class are input into a frozen feature extraction backbone network to extract the feature vector of each sample. The extracted feature vectors are then subjected to L2 normalization, mapping each sample's feature vector to a unit hypersphere. Subsequently, the arithmetic mean of the normalized feature vectors of all samples in that class is calculated to obtain the feature mean of that class, which is then used as the class prototype for the corresponding base class. L2 normalization eliminates the influence of differences in feature vector magnitudes, allowing the class prototype to more accurately reflect the directional distribution center of that class in the feature space.

[0077] S240. Use the feature mean of each base class as the category prototype of the corresponding base class and store it in the category prototype library;

[0078] It should be noted that, in this embodiment of the invention, the category prototype plays different roles throughout multiple subsequent stages: in the incremental training stage, it is used to generate pseudo-features to simulate the old class sample distribution (since the original old class samples are not saved in the case of no samples), and it is also used to calculate the mean of the old class pseudo-features in the feature orthogonal loss. This constrains the separation of new class features from old class features; in subsequent application stages, it is used for prototype classification by the nearest-neighbor class mean classifier and integrated with the parameterized classification results of the expert hybrid classifier for prediction.

[0079] S250. Initialize the old class expert according to the parameters of the initial classifier.

[0080] In this embodiment of the invention, after the initial classifier is trained on the base class dataset, its parameters already contain the discriminative knowledge of the base class categories. These parameters are then assigned to the old class experts, enabling them to inherit the discriminative ability of the base class categories and serving as the starting point for subsequent incremental training. The new class experts are initialized to empty, waiting to learn the discriminative rules for newly added categories in subsequent incremental steps.

[0081] Therefore, based on the above initial training process, a stable feature space foundation, a complete category prototype library, and an initialized expert hybrid classifier structure are laid for subsequent sample-less incremental learning.

[0082] In this embodiment of the invention, for each incremental step, the expert hybrid classifier is incrementally trained based on the image sample features of the newly added category and the category prototype information of the old category, and the category prototype library is updated accordingly. Figure 3 As shown, it includes:

[0083] S310. Extract the image sample features of the newly added category and generate pseudo features based on the category prototype of the old category;

[0084] In this embodiment of the invention, in each incremental step, image samples of the newly added category are acquired and input into the frozen feature extraction backbone network. The feature vector of each new category sample is extracted as the image sample feature of the newly added category (i.e., the true feature). Simultaneously, the category prototypes of the old categories are read from the category prototype library, and pseudo-features are generated based on the category prototypes to simulate the feature distribution of the old category samples. The pseudo-features and the true features reside in the same feature space (because the backbone network is frozen), ensuring the accuracy of the simulation of old category knowledge during incremental training.

[0085] Specifically, the pseudo-features are generated as follows: The category prototypes (i.e., feature mean) and corresponding standard deviations of each old category are read from the category prototype library, and pseudo-features are generated according to a preset generation strategy. This invention provides the following three generation strategies:

[0086] (1) Oversampling strategy: The class prototype of the old class is directly used as the pseudo feature, and repeated according to the number of samples of each new class in the current incremental step, that is, the pseudo feature of the c-th old class. ,in The feature mean of the c-th old category;

[0087] (2) Noise strategy: Using the feature mean of the old category as the center and the feature standard deviation as the scale, sample from the Gaussian distribution to generate pseudo-features, i.e. ,in Let be the standard deviation of the c-th old category. This strategy introduces random noise to make the distribution of pseudo-features closer to the distribution of real samples.

[0088] (3) Translation strategy: Calculate the offset of the true feature vector of the newly added category relative to its category mean, and superimpose this offset onto the category prototype of the old category to generate pseudo-features, i.e. ,in For the true feature vectors of the newly added categories, To generate the feature mean of the new category, this strategy utilizes the local geometric similarity of the features of the new and old categories in the feature space, making the generated pseudo-features closer to the distribution of real samples.

[0089] The generated pseudo-features, together with the features of real image samples from the newly added categories, constitute the training feature set for the current incremental step. Since the feature extraction backbone network is frozen, the pseudo-features and real features reside in the same feature space, ensuring the accuracy of simulating old class knowledge during incremental training.

[0090] S320. Construct a training feature set based on the image sample features of the newly added category and the pseudo-features;

[0091] In this embodiment of the invention, the training feature set consists of two parts: one part is the real image sample features of the newly added category, which is used to learn the discrimination rules of the new category; the other part is the pseudo features generated based on the old category prototype, which is used to maintain the discrimination knowledge of the old category. Together, they constitute the training feature set of the current incremental step.

[0092] S330. Input the training feature set into the expert hybrid classifier to obtain the weighted fusion classification result;

[0093] Specifically, the training feature set is input into the expert hybrid classifier, wherein the old class expert in the expert hybrid classifier can output the old class discrimination result, the new class expert in the expert hybrid classifier can output the new class discrimination result, the gating network in the expert hybrid classifier calculates the fusion weight based on the features in the training feature set, and performs weighted fusion on the old class discrimination result and the new class discrimination result based on the fusion weight to obtain the weighted fused classification result.

[0094] Furthermore, before inputting the training feature set into the expert hybrid classifier, the training feature set is first processed through a reprojection layer. Specifically, the reprojection layer is positioned between the feature extraction backbone network and the expert hybrid classifier, and is used to perform a reprojection transformation on the feature vector output by the backbone network.

[0095] Structurally, the reprojection layer is implemented using a layer normalization layer, with its normalization dimension matching the dimension of the feature vector (e.g., 512 dimensions). It also enables learnable affine transformation parameters (i.e., scaling parameter γ and offset parameter β), thereby preserving the adaptive adjustment capability to the feature distribution while maintaining normalization. The expression for the reprojection layer is: , where x is the feature vector output by the backbone network, μ and σ² are the mean and variance of the feature vector in each dimension, respectively, ε is a smoothing constant to prevent division by zero, γ and β are learnable affine transformation parameters, and x' is the reprojected feature vector.

[0096] Functionally, since the feature extraction backbone network is frozen after initial training, its feature space has an inherent bias towards the old class. The reprojection layer corrects the feature bias of the frozen backbone network towards the old class by normalizing and affine transforming the feature vectors, and enhances the plasticity of the feature space towards the new class, thereby improving the overall discrimination ability of the expert hybrid classifier on the old and new classes. At the same time, the reprojection layer has a small number of parameters and will not significantly increase the storage and computational overhead of the model.

[0097] In this embodiment of the invention, the gating network calculates fusion weights by input features. .

[0098] More specifically, the expression for the weighted fusion classification result is:

[0099] ,

[0100] in, This represents the classification result after weighted fusion. This indicates the result of the old category classification. This indicates the result of the new category classification. and Both represent the fusion weights, and .

[0101] It should be noted here that, This indicates the fusion weight of experts in the old category. This indicates the fusion weight of the new type of expert.

[0102] In this embodiment of the invention, the gated network can be implemented using a two-layer fully connected network. The input is a feature vector, and the output is fused weights, which are ensured by the Softmax function. Then, the old class discrimination results and the new class discrimination results are weighted and fused according to the fusion weight to obtain the weighted fused classification result. Its expression is ,in This represents the classification result after weighted fusion. This indicates the result of the old category classification. This indicates the result of the new category classification.

[0103] S340. Calculate the classification loss based on the weighted fusion classification result, and calculate the feature orthogonality loss between the image sample features of the newly added category and the pseudo-features.

[0104] In this embodiment of the invention, the classification loss adopts the cross-entropy loss function, and the classification result after weighted fusion is calculated. The difference between the new and true labels. Feature orthogonality loss is used to constrain the feature orientations of image samples from the new category to remain orthogonal to the pseudo-feature orientations of the old category, in order to reduce the interference of new category training on the feature representations of the old category.

[0105] Specifically, the expression for the feature orthogonal loss is:

[0106] ,

[0107] in, Indicates the characteristic orthogonal loss. This represents the mean of the features of the image samples in the newly added category. This represents the mean of the pseudo-features.

[0108] To be more specific, , , Represents the set of features for a new class. This represents the set of pseudo-features of the old class. It should be understood that by minimizing the absolute value of the dot product of the means of the old and new class features, the two are forced to be directionally separated in the feature space, thus reducing feature coupling.

[0109] S350. Construct a joint loss based on the classification loss and the feature orthogonality loss;

[0110] In this embodiment of the invention, the expression for the joint loss is:

[0111] ,

[0112] in, Represents classification loss. This represents the weight coefficient of the orthogonal constraint.

[0113] Specifically, the classification loss adopts the cross-entropy loss function, the expression of which is: Where C is the total number of the current category, Let be the one-hot encoded value of the i-th component of the sample's true label, and z be the weighted fusion classification result output by the expert hybrid classifier. Let be the predicted probability of the i-th category.

[0114] Furthermore, in each incremental step, the consistency of output between the current model and the old model saved in the previous incremental step is constrained by knowledge distillation loss. Specifically, before the start of each incremental step, a copy of the parameters of the current expert hybrid classifier and its network is saved as the old model. During training, the training feature set is input into the current model and the old model respectively to obtain the current logits and the old logits. The expression for knowledge distillation loss is: Where T is the distillation temperature (T=2 in this embodiment of the invention), and z is the logits output by the current model. The output logits are the values ​​of the old model, and KL(·) represents the KL divergence. Through knowledge distillation, the old model's discriminative knowledge of the old and new classes is transferred to the current model, further mitigating the forgetting of old class knowledge during incremental training.

[0115] S360. Optimize the parameters of the expert hybrid classifier according to the joint loss, and perform momentum fusion update on the optimized parameters of the expert hybrid classifier and the parameter copy of the expert hybrid classifier saved at the beginning of the current incremental step.

[0116] In this embodiment of the invention, based on joint loss The parameters of the expert mixture classifier are optimized using the backpropagation algorithm to obtain the optimized parameters. Simultaneously, before each incremental step begins, the parameters of the current expert fusion classifier are saved as a parameter copy. After training is complete, the optimized parameters will be... With parameter copy A momentum fusion update is performed; specifically, the expression for the momentum fusion update is:

[0117] ,

[0118] in, Indicates the momentum coefficient. This represents the parameters of the optimized expert hybrid classifier. This represents a copy of the parameters of the expert hybrid classifier saved at the start of the current incremental step.

[0119] In this embodiment of the invention, the momentum coefficient m ranges from [0.9, 0.999], with a default value of 0.99. This momentum update mechanism effectively suppresses large oscillations in the classifier head parameters during incremental training, reducing parameter oscillation amplitude by 40%, enhancing the continuity of classification boundaries between stages, and significantly improving the robustness of the model after long-term incremental training.

[0120] S370. After the current incremental step is completed, update the category prototype library, add the feature mean of the new category in the current incremental step, and keep the category prototype of the old category unchanged.

[0121] Specifically, the feature mean of each new category in the current incremental step is calculated and added to the category prototype library for use in subsequent incremental steps to generate pseudo-features and for the nearest-neighbor classifier (NCM) classification in the application stage; the category prototypes of the old categories remain unchanged and are not updated (because the backbone network is frozen, the feature space is fixed, and the old category prototypes are always valid).

[0122] Finally, repeat steps S310 to S370 to complete the training of all incremental steps in sequence, obtaining the trained expert hybrid classifier. After each incremental step, the expert hybrid classifier gains discriminative ability on the new category, while maintaining the stability of recognition of the old category through the momentum update mechanism. The category prototype library is gradually expanded until it covers all existing categories.

[0123] In this embodiment of the invention, an image to be classified is acquired and input into a trained expert hybrid classifier to obtain the final image classification result, such as... Figure 4 As shown, it includes:

[0124] S510. Obtain the image to be classified and extract the image features of the image to be classified;

[0125] Specifically, the image to be classified is input into the frozen feature extraction backbone network to extract the image feature vector.

[0126] S520. Input the image features of the image to be classified into a trained expert hybrid classifier to obtain parameterized classification probabilities, and input the image features of the image to be classified into a nearest-nearest-class mean classifier to obtain prototype-based classification probabilities.

[0127] Specifically, parameterized classification probability The classification probability is derived from the output of an expert hybrid classifier (normalized by Softmax), leveraging the complex boundary fitting capabilities of the expert hybrid classifier; it is based on the prototype. The distance metric is obtained by calculating the distance between image features and each prototype in the category prototype library (normalized by Softmax), and the stability of prototype classification is utilized.

[0128] S530. The parameterized classification probability and the prototype-based classification probability are normalized and then merged to obtain the final image classification result.

[0129] Specifically, for and After normalization, summation is performed to obtain the final prediction result. = The final image classification result is output. By fully utilizing the complex boundary fitting ability of parametric classification and the stability of prototype classification through the above integrated prediction strategy, stable recognition can still be maintained in the scenario of old class feature drift, and the accuracy is improved by 2-3 percentage points compared with the single classification method.

[0130] In summary, the image classification method based on sample-less incremental learning provided in this invention decouples the classification tasks of new and old classes through an expert hybrid classifier, and uses a gating network to adaptively allocate weights, avoiding bias towards the new class in a single classifier head and effectively mitigating classification conflicts between new and old classes. It forces the separation of the mean directions of new and old class features through feature orthogonal constraints, reducing the interference of new class training on the old class representation and decreasing the drift of old class features by more than 30%. It smooths the parameter update process through a momentum update mechanism, suppressing parameter oscillation amplitude by 40% and enhancing the continuity of classification boundaries between stages. By integrating prediction and combining the advantages of parameterized classification and prototype classification, it improves prediction robustness by 2-3 percentage points. Meanwhile, the model has only 11.37M parameters, comparable to existing sample-less methods, requiring no additional storage overhead, and the core modules can be quickly implemented based on existing deep learning frameworks, making it highly applicable in engineering. Thus, without saving original samples of the old class and without significantly increasing storage costs, it achieves a balance between stability and flexibility in sample-less incremental learning, significantly improving the accuracy and robustness of image classification in long-term incremental scenarios.

[0131] The present invention will be further illustrated by a specific embodiment below.

[0132] Taking the incremental update scenario of a product recognition system as an example, the application of this invention is specifically explained:

[0133] In the initial stage, a model is trained using images of 100 basic products. ResNet-18 is used as the feature extraction backbone network, and the backbone network parameters are frozen after training. The mean features of the 100 products are calculated and stored as category prototypes. Old class experts are initialized based on the parameters of the trained initial classifier.

[0134] In incremental phase 1, 20 new products are added. Image features of the new products are extracted, and pseudo-features are generated based on the prototypes of the old products to construct a training feature set. Incremental training is performed using an expert hybrid classifier: experts of the old classes maintain the discriminative ability of the old products, while experts of the new classes learn the discrimination rules for the new products. A gating network adaptively assigns the fusion weights of the two experts. Feature orthogonal loss is used to reduce the interference of new product features on the feature space of the old products. A momentum update mechanism is used to maintain the stability of the classification parameters of the old products. After the current incremental step is completed, the category prototype library is updated, the feature mean values ​​of the 20 new products are added, and the prototypes of the old products are kept unchanged.

[0135] In incremental phase 2, 20 more products are added, and the above incremental training process is repeated. The model does not need to be retrained on historical product data; only the classification-related modules are updated.

[0136] In the application phase, any product image (including old and new products) is input, and the model outputs classification results through ensemble prediction. The recognition accuracy for old products remains stable, while the recognition accuracy for new products reaches a level close to that of joint training.

[0137] The final verification data is as follows:

[0138] On the CIFAR-100 dataset, with T=10 (10 increments, each adding 5 new classes), the final classification accuracy of this method reached 71.9%, which is 3.3 percentage points higher than the existing sample-free method FeCAM-share (68.6%) and 2.0 percentage points higher than the AMPE method (69.9%).

[0139] On the ImageNet-Subset dataset, the accuracy reaches 75.3% at T=10, which is close to the DER method with preserved samples (77.6%), and the number of parameters is only 11.37M, which is much lower than DER's 67.0M.

[0140] In the long sequence incremental scenario of the CIFAR-100 dataset T=20, the accuracy of this method is 71.7%, which is 10.2 percentage points higher than the FeTrIL method (61.5%) and 4.3 percentage points higher than FeCAM-share (67.4%).

[0141] On the TinyImageNet200 dataset with T=20, our method achieves an accuracy of 61.2%, which is 5.7 percentage points higher than the approach without momentum updates.

[0142] In summary, the image classification method based on sample-less incremental learning provided in this invention achieves a balance between stability and plasticity in sample-less incremental learning through a collaborative design that decouples the classification tasks of old and new classes using an expert hybrid classifier, reduces feature space coupling by feature orthogonal constraints, smooths the parameter update process using a momentum update mechanism, and enhances prediction robustness through integrated prediction. This significantly improves image classification accuracy and model robustness in long-term incremental scenarios without saving original samples of old classes or significantly increasing storage costs. It is effectively applicable to intelligent vision systems that require dynamically expanding categories, such as product recognition, industrial defect detection, and medical image classification.

[0143] As another embodiment of the present invention, a computer storage medium is provided, wherein the computer storage medium is used to store a computer program, which, when executed by a processor, implements the image classification method based on sample-free class incremental learning described above.

[0144] In this embodiment of the invention, a non-transitory computer-readable storage medium is provided. The computer-readable storage medium stores computer-executable instructions that can execute the image classification method based on sample-free incremental learning in any of the above method embodiments. The storage medium may be a magnetic disk, optical disk, read-only memory (ROM), random access memory (RAM), flash memory, hard disk drive (HDD), or solid-state drive (SSD), etc.; the storage medium may also include combinations of the above types of memory.

[0145] As another embodiment of the present invention, an electronic device is provided, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected, the memory is used to store computer program instructions, and the processor is used to call and execute the computer program instructions to implement the image classification method based on sample-free class incremental learning described above.

[0146] like Figure 5As shown, the electronic device 10 may include: at least one processor 11, such as a CPU (Central Processing Unit), at least one communication interface 13, a memory 14, and at least one communication bus 12. The communication bus 12 is used to enable communication between these components. The communication interface 13 may include a display screen or a keyboard; optionally, the communication interface 13 may also include a standard wired interface or a wireless interface. The memory 14 may be high-speed RAM (Random Access Memory) or non-volatile memory, such as at least one disk drive. Optionally, the memory 14 may also be at least one storage device located remotely from the aforementioned processor 11. The memory 14 stores application programs, and the processor 11 calls the program code stored in the memory 14 to execute any of the aforementioned method steps.

[0147] The communication bus 12 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The communication bus 12 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0148] The memory 14 may include volatile memory, such as random-access memory (RAM); the memory may also include non-volatile memory, such as flash memory, hard disk drive (HDD) or solid-state drive (SSD); the memory 14 may also include a combination of the above types of memory.

[0149] The processor 11 can be a central processing unit (CPU), a network processor (NP), or a combination of CPU and NP.

[0150] The processor 11 may further include a hardware chip. This hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0151] Optionally, memory 14 is also used to store program instructions. Processor 11 can invoke program instructions to implement the present invention. Figure 1 The image classification method based on sample-free class incremental learning is shown in the embodiment.

[0152] It is understood that the above embodiments are merely exemplary implementations used to illustrate the principles of the present invention, and the present invention is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and essence of the present invention, and these modifications and improvements are also considered to be within the scope of protection of the present invention.

Claims

1. An image classification method based on sample-free class incremental learning, characterized in that, include: Construct a feature extraction backbone network and an expert hybrid classifier, wherein the expert hybrid classifier includes old class experts, new class experts, and a gating network; The feature extraction backbone network and the expert hybrid classifier are initially trained based on a base class image dataset that includes multiple base class categories; For each incremental step, the expert hybrid classifier is incrementally trained based on the image sample features of the new category and the category prototype information of the old category, and the category prototype library is updated. Repeat the above incremental training process until all incremental steps are completed to obtain a trained expert mixture classifier; The image to be classified is obtained and then input into a trained expert mixture classifier to obtain the final classification result.

2. The image classification method based on sample-free class incremental learning according to claim 1, characterized in that, The initial training of the feature extraction backbone network and the expert hybrid classifier is performed based on the base class image dataset, including: The feature extraction backbone network and initial classifier are trained based on the base class image dataset; Freeze the network parameters of the feature extraction backbone network after training; Calculate the feature mean for each base class in the base class image dataset; The feature mean of each base class is used as the category prototype of the corresponding base class and stored in the category prototype library; The old class expert is initialized based on the parameters of the initial classifier.

3. The image classification method based on sample-free class incremental learning according to claim 1, characterized in that, For each incremental step, the expert hybrid classifier is incrementally trained based on the image sample features of the new category and the category prototype information of the old category, and the category prototype library is updated, including: Extract image sample features for the new category and generate pseudo features based on the category prototypes of the old categories; A training feature set is constructed based on the image sample features of the newly added category and the pseudo-features; The training feature set is input into the expert hybrid classifier to obtain a weighted fusion classification result; The classification loss is calculated based on the weighted fusion classification result, and the feature orthogonality loss between the two is calculated based on the mean of the image sample features of the newly added category and the mean of the pseudo features. Construct a joint loss based on the classification loss and the feature orthogonality loss; The parameters of the expert hybrid classifier are optimized based on the joint loss, and the optimized parameters of the expert hybrid classifier are updated by momentum fusion with the parameter copy of the expert hybrid classifier saved at the beginning of the current incremental step. After the current incremental step is completed, update the category prototype library, add the feature mean of the new category in the current incremental step, and keep the category prototype of the old category unchanged.

4. The image classification method based on sample-free class incremental learning according to claim 3, characterized in that, The training feature set is input into the expert hybrid classifier to obtain a weighted fusion classification result, including: The training feature set is input into the expert hybrid classifier, wherein the old class expert in the expert hybrid classifier can output the old class discrimination result, and the new class expert in the expert hybrid classifier can output the new class discrimination result. The gating network in the expert hybrid classifier calculates the fusion weight based on the features in the training feature set, and performs weighted fusion on the old class discrimination result and the new class discrimination result based on the fusion weight to obtain the weighted fused classification result.

5. The image classification method based on sample-free class incremental learning according to claim 4, characterized in that, The expression for the weighted fusion classification result is: ， in, This represents the classification result after weighted fusion. This indicates the result of the old category classification. This indicates the result of the new category classification. and Both represent the fusion weights, and .

6. The image classification method based on sample-free class incremental learning according to claim 3, characterized in that, The expression for the feature orthogonal loss is: ， in, Indicates the characteristic orthogonal loss. This represents the mean of the features of the image samples in the newly added category. This represents the mean of the pseudo-features.

7. The image classification method based on sample-free class incremental learning according to claim 3, characterized in that, The expression for the momentum fusion update is: ， in, Indicates the momentum coefficient. This represents the parameters of the optimized expert hybrid classifier. This represents a copy of the parameters of the expert hybrid classifier saved at the start of the current incremental step.

8. The image classification method based on sample-free class incremental learning according to any one of claims 1 to 7, characterized in that, The image to be classified is acquired and input into a trained expert mixture classifier to obtain the final classification result, including: Acquire the image to be classified and extract its image features; The image features of the image to be classified are input into a trained expert hybrid classifier to obtain parameterized classification probabilities, and the image features of the image to be classified are input into a nearest-nearest-class mean classifier to obtain prototype-based classification probabilities. The parameterized classification probabilities and the prototype-based classification probabilities are normalized and then merged to obtain the final image classification result.

9. A computer storage medium, characterized in that, The computer storage medium is used to store a computer program, which, when executed by a processor, implements the image classification method based on sample-free class incremental learning as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, include: The memory and the processor are communicatively connected. The memory is used to store computer program instructions, and the processor is used to call and execute the computer program instructions to implement the image classification method based on sample-free class incremental learning as described in any one of claims 1 to 8.

Images