Image classification method and system based on online semi-supervised incremental learning

Abstract

The application provides an image classification method and system based on online semi-supervised incremental learning, and belongs to the field of computer vision. The method acquires online new batch data and synchronously updates cache area samples; meanwhile, parameters of a feature extraction model at this time are acquired as first model parameters; double views are obtained by performing data enhancement on the new batch data and the cache area samples, feature extraction is performed, double contrast learning at instance level and pseudo label level is performed, and model parameters are updated as second model parameters; if it is the first double contrast learning, samples with confidence exceeding a corresponding category threshold are removed from negative samples, and the model parameters at this time are taken as third model parameters; thereafter, integrated learning is performed on the three model parameters after weighting by a Fisher information matrix. The application realizes efficient incremental learning on online semi-supervised data flow, improves the plasticity and stability of a semi-supervised incremental learning model in learning under online data flow, and improves the accuracy of image classification.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of computer vision, and particularly relates to an image classification method and system based on online semi-supervised incremental learning. BACKGROUND

[0002] With the rapid development of big data technology, the Internet and social media, the amount of data is showing explosive growth, and the form of data is also increasingly diversified. Especially real-time, massive online data is constantly generated, and the traditional static learning model has been unable to meet people's needs. Therefore, online incremental learning that can adapt to the continuous update of online data has become a research hotspot. Unlike the traditional method of learning all data at once, online incremental learning requires the model to learn new knowledge while receiving new data without forgetting existing knowledge.

[0003] However, in the real world, labeled data is scarce and costly to obtain, while a large amount of online data is usually unlabeled. Therefore, semi-supervised learning, that is, using a small amount of labeled data and a large amount of unlabeled online data to train the model together, has become the key to solving this problem. Online semi-supervised incremental learning, which combines online incremental learning and semi-supervised learning, has become an important research direction. It requires the model to learn new knowledge and avoid catastrophic forgetting while processing online data streams in real time, and the model can only use a single batch of online data and a small amount of past data at a time. Therefore, it is necessary to balance the acquisition of new knowledge and the preservation of knowledge in modeling. In terms of new knowledge acquisition, existing methods mostly use only one of instance-level and pseudo-label-level contrastive learning. In terms of knowledge preservation, current methods mostly use saving a part of historical data or knowledge distillation to reduce forgetting, but are inefficient when dealing with a large amount of real-time online data.

[0004] In the prior art, researchers have proposed a method that uses the powerful nonlinear partitioning needs of the nearest neighbor classifier to model the underlying data allocation flexibly - the nearest neighbor contrastive learning (NNCSL) model. The structure of the NNCSL model is as follows: Figure 1As shown, it uses the soft nearest neighbor classifier to give the sample a pseudo label, and then sums the cross entropy of the pseudo label of all unlabeled samples in two views. In order to avoid the network that may output unbalanced or even degenerative predictions, some of which are more frequent than others, the average entropy maximization value of the pseudo label is calculated. Considering that the distribution of labeled samples and unlabeled samples is different, the cross entropy of the linear classifier output result of all labeled samples in the current batch and the actual label is calculated. Finally, the cross entropy of the model parameters of the previous and subsequent tasks is calculated as a distillation loss to reduce the forgetting of old knowledge. However, only one of the instance level or the pseudo label level is used in the model to perform contrastive learning or calculate cross entropy, although this can utilize the information of unlabeled samples. However, in the face of online data streams where only one batch of data can be seen at a time and a small number of playback samples, due to the small number of samples, the weak ability of the feature extraction model in the early stage, and the use of only a single level of information, the loss function is difficult to converge during the learning process. In addition, most of the existing threshold methods for semi-supervised learning rely on a fixed number of classes or cannot adapt to the gradual strengthening of the model's ability, making it difficult to effectively eliminate the corresponding samples from the negative samples. At the same time, when the existing methods iterate the model, due to the small amount of data in a batch and the fact that most methods do not consider that different parameters are not consistent in importance for the same task, the performance of the model iteration fluctuates greatly. SUMMARY

[0005] In view of the above defects or deficiencies in the prior art, the present application aims to provide an image classification method and system based on online semi-supervised incremental learning, which designs a double contrastive learning at the instance level and the pseudo label level. In the first stage, all unlabeled samples are treated as negative samples for double contrastive learning, and in the second stage, the samples with confidence exceeding the threshold are removed from the negative samples for double contrastive learning, thereby solving the problem of difficulty in converging the loss function in the early stage of the model. Finally, the three model parameters before the first stage, after the first stage, and after the second stage are integrated by weighting using the Fisher information matrix, thereby solving the problem of inconsistent importance of different parameters for the same task and improving the image classification learning ability of the incremental learning framework in the online semi-supervised incremental learning scenario.

[0006] To achieve the above object, the embodiments of the present application adopt the following technical solutions:

[0007] In a first aspect, the embodiments of the present application provide an image classification method based on online semi-supervised incremental learning, which comprises the following steps:

[0008] Step S1, acquiring online new batch data and synchronously updating the cache area samples; at the same time, acquiring the parameters of the feature extraction model as the first model parameters;

[0009] Step S2, data augmentation is performed on the new batch of data and the buffer area samples to obtain double views;

[0010] Step S3, feature extraction is performed on the double views by a feature extraction model to obtain feature representations of the double views;

[0011] Step S4, double contrast learning is performed on the feature representations of the double views at instance level and pseudo label level, and parameters of the feature extraction model are updated, and the parameters at this time are taken as second model parameters;

[0012] Step S5, it is judged whether the double contrast learning in step S4 is the first time; if yes, step S6 is entered; if not, step S7 is entered;

[0013] Step S6, samples with confidence exceeding a corresponding category threshold are removed from negative samples, parameters of the feature extraction model at this time are taken as third model parameters, and step S4 is entered;

[0014] Step S7, integrated learning is performed on the first model parameters, the second model parameters and the third model parameters after weighting by a Fisher information matrix;

[0015] Step S8, the model after integrated learning is taken as a mature image classification model, and current images are classified; when a new batch of data is generated, step S1 is entered.

[0016] As a preferred embodiment of the present application, the buffer area samples include samples that have been saved in the past and new samples generated from the new batch of data.

[0017] As a preferred embodiment of the present application, ResNet is used to construct the feature extraction model in step S3.

[0018] As a preferred embodiment of the present application, when the double contrast learning is performed in step S4, for unlabeled samples, in addition to double views generated by data augmentation of the unlabeled samples being regarded as positive sample pairs, views of all other samples are regarded as negative sample pairs; for labeled samples, double views generated by data augmentation of the same labeled sample are regarded as positive sample pairs, and views of labeled samples from different categories and all unlabeled sample views are regarded as negative sample pairs.

[0019] As a preferred embodiment of the present application, in step S4, for instance-level contrast learning, feature representations generated by double views are taken as input embedding vectors in a loss function; for pseudo label level contrast learning, pseudo labels are taken as input embedding vectors in a loss function after pseudo labels are assigned to views by a soft nearest neighbor classifier; the parameters of the feature extraction model are adjusted by back propagation after the loss functions of the two levels are added.

[0020] As a preferred embodiment of the present application, the loss function adopts an InfoNCE loss function.

[0021] As a preferred embodiment of the present application, in step S6, the sample elimination process of completing the first double contrast learning is taken as the second stage, in order to offset the negative impact of some positive samples being wrongly classified as negative samples when the first double contrast learning is performed, an adaptive threshold algorithm is used in the second stage to eliminate samples with confidence exceeding a confidence threshold from negative samples and then perform double contrast learning.

[0022] As a preferred embodiment of the present application, the calculation method of the confidence threshold is to multiply the maximum prediction probability of each sample by the total number of the current class to calculate the confidence score thereof, then an exponential moving average algorithm is used to iteratively update the global threshold to achieve smooth adjustment of the threshold, and finally the updated threshold is normalized by maximum value to generate an adaptive confidence threshold for each class.

[0023] As a preferred embodiment of the present application, the weighting formula in step S7 is shown in formula (1):

[0024]

[0025] In formula (1), θ t-1 is the first model parameter, θ * t is the second model parameter, θ ** t is the second model parameter, F is the Fisher information matrix corresponding to the model parameter, and λ and α are hyperparameters, and θ t is the model parameter obtained after weighting.

[0026] In a second aspect, the embodiments of the present application further provide an image classification system based on online semi-supervised incremental learning, which comprises a new batch data interface, a memory replay module, a double-view enhancement module, a feature extraction module, a double contrast learning module, a first learning judgment module, a second training module, an ensemble learning module and an image classification module, wherein,

[0027] The new batch data interface is used to obtain online new batch data, and is also used to obtain the parameters of the feature extraction model at this time as the first model parameters;

[0028] The memory replay module is used to update the online new batch data obtained by the new batch data interface to the cache area, and use the samples in the cache area as memory replay data;

[0029] The double-view enhancement module is used to perform data enhancement on the new batch data and the cache area samples as memory replay data to obtain double views;

[0030] The feature extraction module is configured to perform feature extraction on the dual views by a feature extraction model to obtain feature representations of the dual views.

[0031] The dual contrast learning module is configured to perform dual contrast learning at instance level and pseudo-label level on the feature representations of the dual views, and update parameters of the feature extraction model, and take the parameters at this time as second model parameters.

[0032] The first learning judgment module is configured to judge whether the current dual contrast learning is the first dual contrast learning, and if so, start the second training module, and if not, start the ensemble learning module.

[0033] The second training module is configured to eliminate samples with confidence exceeding the corresponding class threshold from the negative samples, and take the parameters of the feature extraction model at this time as third model parameters, and start the dual contrast learning module again.

[0034] The ensemble learning module is configured to perform ensemble learning on the first model parameters, the second model parameters and the third model parameters after weighting by Fisher information matrix.

[0035] The image classification module is configured to take the model after ensemble learning as a mature image classification model to classify the current image, and start the new batch data interface when a new batch of data is generated.

[0036] The technical scheme provided by the embodiment of the application has the following beneficial effects:

[0037] The image classification method and system based on online semi-supervised incremental learning provided by the embodiment of the application fully and effectively learn the information of online semi-supervised data by dual contrast learning at instance level and pseudo-label level and by filtering part of negative samples through adaptive threshold; meanwhile, considering that the amount of data in one batch is small and the importance of different parameters to the same task is inconsistent, the application uses Fisher information matrix to weight the model parameters in the three stages for ensemble learning, so that the model is iterated smoothly and reasonably, realizes efficient incremental learning on online semi-supervised data stream, improves the plasticity and stability of the semi-supervised incremental learning model under the learning of online data stream, and improves the accuracy of image classification.

[0038] Of course, implementing any product or method of the application does not necessarily require all the advantages described above. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments description will be briefly introduced. Obviously, the drawings in the following description only represent some of the embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0040] Figure 1 is a structural schematic diagram of the NNCSL model in the prior art;

[0041] Figure 2 is a principle diagram of the image classification method based on online semi-supervised incremental learning according to the embodiments of the present application;

[0042] Figure 3 is a flowchart of the image classification method based on online semi-supervised incremental learning according to the embodiments of the present application. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings of the embodiments of the present application. Obviously, the described embodiments only represent some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict, if possible.

[0044] It should be noted that: similar labels and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in the subsequent drawings. In the description of the present application, the terms "first", "second", "third", "fourth" and the like are only used to distinguish description, and cannot be understood as indicating or implying relative importance.

[0045] For the image processing problem of online semi-supervised incremental learning in the prior art, an image classification method based on online semi-supervised incremental learning is provided to improve the incremental learning capability of the semi-supervised incremental learning framework under an online data stream. When the semi-supervised incremental learning method relies on a task-based data stream, the model can only use a single batch of data and a small amount of past data at a time, and the loss value of the model converges sub-optimally. When relying on a fixed number of classes, it cannot adapt to the scenario where the number of classes gradually increases in online incremental learning, and the model can only see a batch of data at a time, resulting in biased knowledge learned each time. The network framework is built using the deep learning framework PyTorch, and the image classification method based on online semi-supervised incremental learning is implemented in three steps, including: (1) in the first stage of the training process, all unlabeled samples are regarded as negative samples for instance-level and pseudo-label-level dual contrast learning; (2) in the second stage of training, the samples with a confidence exceeding a threshold are removed from the negative samples using an adaptive threshold, and then dual contrast learning is performed; and (3) the three model parameters before the first stage of training, after the first stage of training, and after the second stage of training are weighted and integrated.

[0046] As shown in Figures 2-3 , the image classification method based on online semi-supervised incremental learning comprises the following steps:

[0047] Step S1, acquiring online new batch data and synchronously updating the cache area samples; and acquiring the parameters of the feature extraction model at this time as the first model parameters.

[0048] In this step, under the framework of online semi-supervised incremental learning, the online data is continuously updated, and the sample quantity of learning is expanded by acquiring the online new batch data. The cache area samples include a small amount of past saved samples and new samples generated from the new batch data.

[0049] Step S2, performing data augmentation on the new batch data and the cache area samples to obtain dual views.

[0050] In this step, the data augmentation adopts methods such as random cropping, rotation, flipping, scaling, translation, and adding noise to increase the diversity of images and make the model more robust.

[0051] Step S3, performing feature extraction on the dual views by the feature extraction model to obtain the feature representation of the dual views.

[0052] In a specific embodiment, the step utilizes ResNet to construct the feature extraction model and extract the feature representation of the dual views. ResNet is a deep neural network architecture that uses residual connections to address the vanishing gradient problem in deep networks. ResNet enables the training of deeper networks by directly passing the input to the output through the intermediate layers, thereby better completing image classification.

[0053] In step S4, instance-level and pseudo-label-level dual contrast learning are simultaneously performed on the feature representation of the dual views, and the parameters of the feature extraction model are updated, and the parameters at this time are taken as the second model parameters.

[0054] In this step, when performing dual contrast learning, for unlabeled samples, in addition to the dual views generated by data augmentation of the unlabeled samples being considered as positive sample pairs, the views of all other samples are considered as negative sample pairs. By maximizing the similarity of different views of the same unlabeled sample in the feature space, the feature extraction model is encouraged to learn consistent feature representations. Since the model has not fully mastered the feature distribution of new data when it initially learns new batches of data, the generated pseudo-labels are noisy. Therefore, when performing dual contrast learning for the first time, for labeled samples, the dual views generated by data augmentation of the same labeled sample are considered as positive sample pairs, and the views of labeled samples from different categories and all unlabeled sample views are considered as negative sample pairs, to further strengthen the consistent feature representation of intra-class samples.

[0055] For instance-level contrast learning, the feature representation generated by the dual views is used as the input embedding vector in the InfoNCE loss function. For pseudo-label-level contrast learning, the pseudo-labels are used as the input embedding vector in the InfoNCE loss function after assigning pseudo-labels to the views through a soft nearest neighbor classifier. The parameters of the feature extraction model are adjusted through backpropagation after adding the loss functions of the two levels. The InfoNCE loss function, as a contrast learning loss function, can maximize the similarity of positive sample pairs (different views of the same sample) and minimize the similarity of positive samples and negative sample pairs (different samples). Through this loss, the feature extraction model can learn useful feature representations that distinguish different samples. The soft nearest neighbor classifier classifies by weighting samples based on the distance to neighbors, with closer neighbors having higher weights and more significant influence. Compared to traditional nearest neighbor classifiers, it can more flexibly reflect the similarity between samples and classes, and is suitable for classification tasks involving boundary samples.

[0056] In step S5, it is determined whether the dual contrast learning in step S4 is the first dual contrast learning. If so, proceed to step S6; if not, proceed to step S7.

[0057] In this step, due to the small amount of data in one batch in the online semi-supervised incremental learning process, the knowledge learned by the model in the dual contrast learning is biased, resulting in large fluctuations in the performance of the model after each batch training. At the same time, different parameters have different importance for the same batch of data. In order to make the model parameters iterate reasonably and smoothly, different treatments are carried out for whether it is the first time for dual contrast learning.

[0058] Step S6, eliminate samples with confidence exceeding the threshold of the corresponding category from the negative samples, and take the parameters of the feature extraction model at this time as the third model parameters, and go to step S4.

[0059] In this step, the sample elimination process of the first dual contrast learning is taken as the second stage. In order to offset the negative impact of some positive samples being incorrectly classified as negative samples during the first dual contrast learning, an adaptive threshold algorithm is used in the second stage to eliminate samples with confidence exceeding the confidence threshold from the negative samples before performing dual contrast learning again.

[0060] The calculation method of the confidence threshold is to multiply the maximum prediction probability of each sample by the total number of the current category to calculate its confidence score; then, an exponential moving average algorithm is used to iteratively update the global threshold to achieve smooth adjustment of the threshold; finally, the updated threshold is normalized by maximum value to generate an adaptive confidence threshold for each category.

[0061] Step S7, integrate learning by weighting the first model parameters, the second model parameters and the third model parameters by Fisher information matrix, and the weighting formula is shown in formula (1):

[0062]

[0063] In formula (1), θ t-1 is the first model parameter, θ * t is the second model parameter, θ ** t is the second model parameter, F is the Fisher information matrix corresponding to the model parameters, λ and α are hyperparameters, and θ t is the model parameter obtained after weighting.

[0064] In this step, considering that the amount of data in a batch is small, and different parameters are inconsistent in importance for the same task, the three model parameters before the first double contrast learning, after the first double contrast learning and after the second double contrast learning are weighted and integrated learning by Fisher information matrix, so that the model is smooth and reasonable iteration. Fisher information matrix is used to quantify the uncertainty of parameter estimation, represents the second order derivative matrix of the log-likelihood function of the parameter with respect to the parameter, and reflects the "amount of information" about the parameter contained in the data. The larger the Fisher information matrix, the smaller the variance of the parameter estimation, indicating that the estimation is more accurate.

[0065] Step S8, the integrated learning model is used as a mature image classification model to classify the current image; when a new batch of data is generated, step S1 is entered.

[0066] Based on the same idea, the embodiment of the application also provides an image classification system based on online semi-supervised incremental learning. In a specific embodiment, the technician needs to configure the computer environment before implementing the image classification system, including installing the Linux operating system, Pycharm software and PyTorch deep learning framework; in addition, because the deep learning model training process is time-consuming, it is best to train under GPU. It should be noted that the parameter setting can be set according to the need, including the batch size, the value of the learning rate, the number of iterations, etc.

[0067] The online semi-supervised incremental learning framework of the double contrast learning involved in the system is distributed to the entire online semi-supervised incremental learning-based image classification system, including: a new batch data interface, a memory playback module, a double view enhancement module, a feature extraction module, a double contrast learning module, a first learning judgment module, a second training module, an integrated learning module and an image classification module.

[0068] The new batch data interface is used to obtain online new batch data, and is also used to obtain the model parameters at this time as the first model parameters;

[0069] The memory playback module is used to update the online new batch data obtained by the new batch data interface to the cache area, and use the small amount of past samples stored in the cache area and the updated cache area samples as memory playback data;

[0070] The double view enhancement module is used to perform data enhancement on the new batch data and the cache area samples to obtain double views;

[0071] The feature extraction module is used to extract features of the double views to obtain feature representations of the double views;

[0072] The double-contrast learning module is configured to perform double-contrast learning at both the instance level and the pseudo-label level on the feature representations of the double views, and update the parameters of feature extraction, and the parameters at this time are taken as second model parameters.

[0073] The first learning judgment module is configured to judge whether the current double-contrast learning is the first double-contrast learning, and if so, start the second training module, and if not, start the ensemble learning module.

[0074] The second training module is configured to eliminate samples with confidence exceeding the corresponding category threshold from the negative samples, and take the model parameters at this time as third model parameters, and start the double-contrast learning module again.

[0075] The ensemble learning module is configured to perform ensemble learning on the first model parameters, the second model parameters and the third model parameters after weighting by the Fisher information matrix.

[0076] The image classification module is configured to take the model after ensemble learning as a mature image classification model, and perform classification on the current image, and when a new batch of data is generated, start the new batch of data interface.

[0077] The modules in the embodiment are implemented by a processor, and a memory is appropriately increased when storage is needed. The processor can be, but is not limited to, a microprocessor (MPU), a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, etc. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0078] In the above embodiments, all or part can be realized by software, hardware, firmware or any combination thereof. When realized by software, all or part can be realized in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, the computer program instructions produce the processes or functions described in the embodiments of the present application in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer readable storage medium or transferred from one computer readable storage medium to another, for example, the computer instructions can be transferred from one website, computer, server or data center to another website, computer, server or data center through wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.).

[0079] In addition, it should be noted that the online semi-supervised incremental learning-based image classification system described in the embodiments corresponds to the online semi-supervised incremental learning-based image classification method, and the description and limitation of the method also apply to the system, which will not be described here.

[0080] As can be seen from the above technical solutions, the online semi-supervised incremental learning-based image classification method and system provided by the embodiments of the present application fully and effectively learn the information of online semi-supervised data by double contrast learning at the instance level and the pseudo-label level and by filtering part of the negative samples through an adaptive threshold; at the same time, considering that the amount of data in a batch is small and the importance of different parameters to the same task is inconsistent, the present application uses the Fisher information matrix to weight the model parameters in the three stages and then performs integrated learning, so that the model is iterated smoothly and reasonably, realizes efficient incremental learning on the online semi-supervised data stream, improves the plasticity and stability of the semi-supervised incremental learning model under the online data stream, and improves the accuracy of image classification.

[0081] The above description is only the preferred embodiments of the present application and the explanation of the applied technical principles, and is not intended to limit the scope of the claimed present application, but only represents the preferred embodiments of the present application. Those skilled in the art should understand that the scope of the present application is not limited to the technical solutions formed by the specific combinations of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the inventive concept. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of the present application.

Claims

1. An image classification method based on online semi-supervised incremental learning, characterized in that, The method includes the following steps: Step S1: Obtain new batch data online and update the cached samples synchronously; at the same time, obtain the parameters of the feature extraction model at this time as the first model parameters; Step S2: Perform data augmentation on the new batch data and the cached samples to obtain a dual view; Step S3: Extract features from the dual views using a feature extraction model to obtain feature representations of the dual views; Step S4: Simultaneously perform instance-level and pseudo-label-level dual contrastive learning on the feature representation of the dual views, and update the parameters of the feature extraction model, using the current parameters as the second model parameters. Step S5: Determine whether the double contrast learning in step S4 is the first double contrast learning; If yes, proceed to step S6; otherwise, proceed to step S7. Step S6: Remove samples with confidence scores exceeding the corresponding class threshold from the negative samples, and use the parameters of the feature extraction model at this time as the parameters of the third model, then proceed to step S4. Step S7: After weighting the first model parameters, second model parameters, and third model parameters using the Fischer information matrix, perform ensemble learning. Step S8: Use the ensemble-learned model as a mature image classification model to classify the current image; when a new batch of data is generated, proceed to step S1.

2. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that, The cached samples include samples that have been saved in the past and new samples generated from new batches of data.

3. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that, Step S3 uses ResNet to build a feature extraction model.

4. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that, In step S4, when performing dual-contrast learning, for unlabeled samples, only the dual views generated by data augmentation for unlabeled samples are considered positive samples to the outside, while the views of all other samples are considered negative sample pairs; for labeled samples, the dual views generated by data augmentation for the same labeled sample are considered positive sample pairs, and the views of labeled samples from different categories and all unlabeled sample views are considered negative sample pairs.

5. The image classification method based on online semi-supervised incremental learning according to claim 4, characterized in that, In step S4, for instance-level contrastive learning, the feature representations generated by the two views are used as the input embedding vectors in the loss function; for pseudo-label-level contrastive learning, pseudo-labels are assigned to the views through a soft nearest neighbor classifier and then used as the input embedding vectors in the loss function; the loss functions of the two levels are added together and then backpropagation is performed to adjust the parameters of the feature extraction model.

6. The image classification method based on online semi-supervised incremental learning according to claim 5, characterized in that, The loss function used is the InfoNCE loss function.

7. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that, In step S6, the sample elimination process after the first double contrast learning is taken as the second stage. In order to offset the negative impact of some positive samples being incorrectly classified as negative samples during the first double contrast learning, an adaptive threshold algorithm is used in the second stage to remove samples with confidence scores exceeding the confidence score threshold from the negative samples before performing double contrast learning.

8. The image classification method based on online semi-supervised incremental learning according to claim 7, characterized in that, The confidence threshold is calculated by multiplying the maximum predicted probability of each sample by the total number of samples in the current category to calculate its confidence score. Next, for the global threshold, an exponential moving average algorithm is used for iterative updates to achieve smooth adjustment of the threshold; finally, the updated threshold is normalized to the maximum value to generate an adaptive confidence threshold for each category.

9. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that, The weighting formula in step S7 is shown in equation (1): In equation (1), θ t-1 Let θ be the first model parameter. * t For the second model parameters, θ ** t Here, λ represents the second model parameter, F is the Fischer information matrix corresponding to the model parameter, λ and α are hyperparameters, and θ is the Fischer information matrix. t These are the model parameters obtained after weighting.

10. An image classification system based on online semi-supervised incremental learning, characterized in that, The system includes: a new batch data interface, a memory playback module, a dual-view enhancement module, a feature extraction module, a dual-contrast learning module, a first-stage learning judgment module, a second-stage training module, an ensemble learning module, and an image classification module; wherein, The new batch data interface is used to obtain new batch data online, and also to obtain the parameters of the feature extraction model at this time as the first model parameters. The memory playback module is used to update the online new batch data obtained from the new batch data interface to the cache area, and use the samples in the cache area as memory playback data. The dual-view enhancement module is used to enhance the new batch data and the cache sample, which are used as memory playback data, to obtain a dual view. The feature extraction module is used to extract features from the dual views using a feature extraction model to obtain feature representations of the dual views. The dual-contrast learning module is used to perform instance-level and pseudo-label-level dual-contrast learning on the feature representation of the dual views simultaneously, and update the parameters of the feature extraction model, using the current parameters as the second model parameters. The first learning judgment module is used to determine whether the current double contrast learning is the first double contrast learning; if yes, the second training module is started; if not, the ensemble learning module is started. The second training module is used to remove samples with confidence scores exceeding the corresponding class threshold from the negative samples, and uses the parameters of the feature extraction model at this time as the parameters of the third model to restart the dual contrast learning module. The ensemble learning module is used to perform ensemble learning by weighting the first model parameters, the second model parameters, and the third model parameters using the Fischer information matrix. The image classification module is used to classify the current image by using the ensemble-learned model as a mature image classification model; when a new batch of data is generated, the new batch data interface is started.

Images