Image classification method and system based on online semi-supervised incremental learning

Abstract

The invention provides an image classification method and system based on online semi-supervised incremental learning, and belongs to the field of computer vision. The method comprises the following steps: acquiring online new batch data, and synchronously updating a cache region sample; meanwhile, acquiring parameters of the feature extraction model at the moment as first model parameters; performing data enhancement on the new batch of data and the cache region sample to obtain double views, performing feature extraction, performing double contrast learning of an instance level and a pseudo tag level, and updating the model parameter as a second model parameter; if the first-time double-contrast learning is carried out, samples of which the confidence exceeds a corresponding category threshold value are removed from negative samples, and the model parameters at the moment are used as third model parameters; and then, carrying out ensemble learning after weighting three model parameters through a Fischer information matrix. According to the method, efficient incremental learning of the online semi-supervised data stream is realized, the plasticity and stability of learning of the semi-supervised incremental learning model under the online data stream are improved, and the accuracy of image classification is improved.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision, and in particular relates to an image classification method and system based on online semi-supervised incremental learning. Background Art

[0002] With the rapid development of big data technology, the Internet and social media, the amount of data has exploded and the forms of data have become increasingly diverse. In particular, real-time, massive online data is constantly generated, and traditional static learning models can no longer meet people's needs. Therefore, online incremental learning that can adapt to the continuous updating of online data has become a research hotspot. Unlike the traditional method of learning all data at once, online incremental learning requires the model to receive new data while not forgetting existing knowledge.

[0003] However, in the real world, labeled data is scarce and expensive to obtain, while a large amount of online data is usually unlabeled. Therefore, semi-supervised learning, which uses a small amount of labeled data and a large amount of unlabeled online data to jointly train the model, has become the key to solving this problem. Online semi-supervised incremental learning, which combines online incremental learning with semi-supervised learning, has become an important research direction. It requires that in the process of real-time processing of online data streams, the model can only use a single batch of online data and a small amount of past data at a time to learn new knowledge and avoid catastrophic forgetting. Therefore, it is necessary to balance the acquisition of new knowledge and the retention of knowledge in modeling. In terms of new knowledge acquisition, most existing methods only use one of the contrastive learning at the instance level and the pseudo-label level. In terms of knowledge retention, most current methods reduce forgetting by saving a part of historical data or knowledge distillation, but they are less efficient when processing large amounts of real-time online data.

[0004] In the prior art, some researchers have proposed a method that uses the powerful function of the nearest neighbor classifier to nonlinearly divide the demand space and flexibly model the underlying data distribution - the nearest neighbor contrastive learning (NNCSL) model. The structure of the NNCSL model is as follows: Figure 1As shown in the figure, after using the soft nearest neighbor classifier to give pseudo labels to samples, the cross entropy of the pseudo labels of the two views of all unlabeled samples is calculated and summed. In order to avoid the situation where the network may output unbalanced or even degenerate predictions, in which some categories are predicted more frequently than other categories, the average entropy of the pseudo labels is calculated to maximize the value. Considering the different distributions of labeled and unlabeled samples, the cross entropy of the linear classifier output results and the actual labels of all labeled samples in the current batch is calculated. Finally, the cross entropy of the model parameters of the previous and next tasks is calculated as the distillation loss to reduce the forgetting of old knowledge. However, only one of the information at the instance level or the pseudo label level is used in this model for comparative learning or cross entropy calculation. Although this can utilize the information of unlabeled samples, when facing the scenario where only one batch of data and a small number of playback samples can be seen at a time in the online data stream, due to the small number of samples, the weak early capabilities of the feature extraction model, and the use of only a single level of information, the loss function is difficult to converge during the learning process. In addition, most of the existing threshold methods for semi-supervised learning rely on a fixed number of classes or cannot adapt to the situation where the model capabilities are gradually strengthened, making it impossible to well remove the corresponding samples from negative samples. At the same time, when existing methods perform model iteration, the model iteration performance fluctuates greatly due to the small amount of data in a batch and the fact that most methods do not take into account the inconsistent importance of different parameters for the same task. Summary of the invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, the present invention aims to provide an image classification method and system based on online semi-supervised incremental learning, and designs a double contrast learning at the instance level and the pseudo-label level. In the first stage, all unlabeled samples are first regarded as negative samples for double contrast learning. In the second stage, an adaptive threshold is used to remove samples with confidence exceeding the threshold from the negative samples and then double contrast learning is performed to solve the problem that the loss function of the above-mentioned model is difficult to converge in the early stage; finally, the three model parameters before the first stage, after the first stage and after the second stage of training are weighted by the Fisher information matrix and integrated learning is performed to solve the problems of smooth iteration of the above-mentioned model and inconsistent importance of different parameters for the same task, so as to improve the image classification learning ability of the incremental learning framework in the online semi-supervised incremental learning scenario.

[0006] In order to achieve the above purpose, the embodiment of the present invention adopts the following technical solution:

[0007] In a first aspect, an embodiment of the present invention provides an image classification method based on online semi-supervised incremental learning, the method comprising the following steps:

[0008] Step S1, obtain new batch data online and synchronously update the cache samples; at the same time, obtain the parameters of the feature extraction model at this time as the first model parameters;

[0009] Step S2, performing data enhancement on the new batch data and the cache area samples to obtain a dual view;

[0010] Step S3, extracting features from the dual views using a feature extraction model to obtain feature representations of the dual views;

[0011] Step S4, performing dual contrast learning at both the instance level and the pseudo-label level on the feature representation of the dual views, and updating the parameters of the feature extraction model, and using the parameters at this time as the second model parameters;

[0012] Step S5, determining whether the double contrast learning in step S4 is the first double contrast learning; if so, proceeding to step S6; if not, proceeding to step S7;

[0013] Step S6, removing samples whose confidence exceeds the corresponding category threshold from the negative samples, and using the parameters of the feature extraction model at this time as the third model parameters, and proceeding to step S4;

[0014] Step S7, weighting the first model parameter, the second model parameter and the third model parameter through the Fisher information matrix and performing integrated learning;

[0015] Step S8, using the integrated learning model as a mature image classification model to classify the current image; when a new batch of data is generated, proceed to step S1.

[0016] As a preferred embodiment of the present invention, the buffer area samples include samples that have been saved in the past and new samples generated from new batch data.

[0017] As a preferred embodiment of the present invention, step S3 uses ResNet to build a feature extraction model.

[0018] As a preferred embodiment of the present invention, when step S4 performs dual contrast learning, for unlabeled samples, except for the dual views generated by data enhancement of the unlabeled samples, which are regarded as positive sample pairs, the views of all other samples are regarded as negative sample pairs; for labeled samples, the dual views generated by data enhancement of the same labeled sample are regarded as positive sample pairs, and the labeled sample views from different categories and all unlabeled sample views are regarded as negative sample pairs.

[0019] As a preferred embodiment of the present invention, in step S4, for instance-level contrastive learning, the feature representation generated by the dual views is used as an input embedding vector in the loss function; for pseudo-label-level contrastive learning, pseudo-labels are assigned to views through a soft nearest neighbor classifier and the pseudo-labels are used as input embedding vectors in the loss function; the loss functions of the two levels are added together and back propagation is performed to adjust the parameters of the feature extraction model.

[0020] As a preferred embodiment of the present invention, the loss function adopts the InfoNCE loss function.

[0021] As a preferred embodiment of the present invention, in step S6, the sample elimination process of the first double contrast learning is taken as the second stage. In order to offset the negative impact caused by some positive samples being mistakenly classified as negative samples during the first double contrast learning, an adaptive threshold algorithm is used in the second stage to eliminate samples whose confidence exceeds the confidence threshold from the negative samples before double contrast learning is performed.

[0022] As a preferred embodiment of the present invention, the confidence threshold calculation method multiplies the maximum predicted probability of each sample by the total number of the current category to calculate its confidence score; then, for the global threshold, an exponential moving average algorithm is used to iteratively update to achieve smooth adjustment of the threshold; finally, the updated threshold is normalized to the maximum value to generate an adaptive confidence threshold for each category.

[0023] As a preferred embodiment of the present invention, the weighting formula in step S7 is as shown in formula (1):

[0024]

[0025] In formula (1), θ t-1 is the first model parameter, θ * t is the second model parameter, θ ** t is the second model parameter, F is the Fisher information matrix corresponding to the model parameter, λ and α are hyperparameters, θ t are the model parameters obtained after weighting.

[0026] In a second aspect, an embodiment of the present invention further provides an image classification system based on online semi-supervised incremental learning, the system comprising: a new batch data interface, a memory playback module, a dual view enhancement module, a feature extraction module, a dual contrast learning module, a first learning judgment module, a second training module, an integrated learning module and an image classification module; wherein,

[0027] The new batch data interface is used to obtain new batch data online, and is also used to obtain the parameters of the feature extraction model at this time as the first model parameters;

[0028] The memory playback module is used to update the online new batch data obtained by the new batch data interface to the cache area, and use the samples in the cache area as memory playback data;

[0029] The dual-view enhancement module is used to perform data enhancement on the new batch data and the buffer area samples as the memory playback data to obtain the dual-view;

[0030] The feature extraction module is used to extract features from the dual views through a feature extraction model to obtain feature representations of the dual views;

[0031] The dual contrast learning module is used to perform dual contrast learning at both the instance level and the pseudo-label level on the feature representation of the dual views, and update the parameters of the feature extraction model, using the parameters at this time as the second model parameters;

[0032] The first learning judgment module is used to judge whether the current double contrast learning is the first double contrast learning; if so, start the second training module; if not, start the integrated learning module;

[0033] The second training module is used to remove samples whose confidence exceeds the corresponding category threshold from the negative samples, and use the parameters of the feature extraction model at this time as the third model parameters, and start the double contrast learning module again;

[0034] The integrated learning module is used to perform integrated learning after weighting the first model parameter, the second model parameter and the third model parameter through the Fisher information matrix;

[0035] The image classification module is used to use the integrated learning model as a mature image classification model to classify the current image; when a new batch of data is generated, the new batch data interface is started.

[0036] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0037] The image classification method and system based on online semi-supervised incremental learning provided by the embodiments of the present invention fully and effectively learn the information of online semi-supervised data through double contrast learning at the instance level and the pseudo-label level and filtering some negative samples through adaptive thresholds; at the same time, considering that the amount of data in a batch is small and the importance of different parameters to the same task is inconsistent, the present invention uses the Fisher information matrix to weight the model parameters of the three stages and then performs integrated learning, so that the model can iterate smoothly and reasonably, realize efficient incremental learning of the online semi-supervised data stream, improve the plasticity and stability of the semi-supervised incremental learning model under the online data stream, and improve the accuracy of image classification.

[0038] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0040] Figure 1 It is a schematic diagram of the structure of the NNCSL model in the prior art;

[0041] Figure 2 is a schematic diagram of an image classification method based on online semi-supervised incremental learning according to an embodiment of the present invention;

[0042] Figure 3 It is a flow chart of an image classification method based on online semi-supervised incremental learning described in an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations. It should be noted that the embodiments of the present invention and the features in the embodiments can also be combined with each other without conflict.

[0044] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In the description of the present invention, the terms "first", "second", "third", "fourth", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0045] In view of the image processing problem of online semi-supervised incremental learning in the prior art, an embodiment of the present invention provides an image classification method based on online semi-supervised incremental learning to improve the ability of the semi-supervised incremental learning framework to perform incremental learning under online data flow. When the semi-supervised incremental learning method relies on task-based data flow, the model can only use a single batch of data and a small amount of past data at a time, and the loss value of the model does not converge ideally; when it relies on a fixed number of classes, it will not be able to adapt to the scenario where the number of classes in online incremental learning gradually increases. At the same time, only one batch of data can be seen at a time, resulting in deviations in the knowledge learned in each training. The embodiment of the present invention uses the deep learning framework PyTorch to build a network framework, and implements the image classification method based on online semi-supervised incremental learning in three steps. The three steps include: (1) in the first stage of the training process, all unlabeled samples are first regarded as negative samples for double contrast learning at the instance level and pseudo-label level; (2) in the second stage of the training, samples with confidence exceeding the threshold are removed from the negative samples using an adaptive threshold and then double contrast learning is performed; (3) the three model parameters before the first stage of training, after the first stage, and after the second stage are weighted by the Fisher information matrix for integrated learning.

[0046] like Figure 2-3 As shown, the image classification method based on online semi-supervised incremental learning includes the following steps:

[0047] Step S1, obtain new batch data online and synchronously update the cache area samples; at the same time, obtain the parameters of the feature extraction model at this time as the first model parameters.

[0048] In this step, under the framework of online semi-supervised incremental learning, online data is continuously updated, and the number of learning samples is expanded by acquiring new batches of online data. The cache samples include a small number of samples that have been saved in the past and new samples generated by new batches of data.

[0049] Step S2, performing data enhancement on the new batch data and the cache area samples to obtain a dual view.

[0050] In this step, the data enhancement uses methods such as random cropping, rotation, flipping, scaling, translation, and adding noise to increase the diversity of the image and make the model more robust.

[0051] Step S3, extracting features from the dual views through a feature extraction model to obtain feature representations of the dual views.

[0052] In a specific embodiment, this step uses ResNet to build a feature extraction model and extracts feature representations of the dual views. ResNet is a deep neural network architecture that uses residual connections to solve the gradient vanishing problem in deep networks. ResNet can train deeper networks by directly passing the input to the output bypassing the intermediate layer, thereby better completing image classification.

[0053] Step S4, performing dual contrast learning at both the instance level and the pseudo-label level on the feature representation of the dual views, and updating the parameters of the feature extraction model, and using the parameters at this time as the second model parameters.

[0054] In this step, when performing double contrast learning, for unlabeled samples, except for the dual views generated by data augmentation of unlabeled samples, which are regarded as positive pairs, all other views of samples are regarded as negative pairs. By maximizing the similarity of different views of the same unlabeled sample in the feature space, the feature extraction model is encouraged to learn consistent feature representations. Since the model has not fully mastered the feature distribution of new data when initially learning a new batch of data, the generated pseudo-label noise is large. Therefore, when double contrast learning is performed for the first time, for labeled samples, the dual views generated by data augmentation of the same labeled sample are regarded as positive pairs, and labeled sample views from different categories and all unlabeled sample views are regarded as negative pairs, so as to further strengthen the consistent feature representation of samples within the class.

[0055] For instance-level contrastive learning, the feature representation generated by the dual views is used as the input embedding vector in the InfoNCE loss. For pseudo-label-level contrastive learning, the pseudo-label is assigned to the view through the soft nearest neighbor classifier and the pseudo-label is used as the input embedding vector in the InfoNCE loss. The loss functions of the two levels are added and back-propagated to adjust the parameters of the feature extraction model. The InfoNCE loss function, as a loss function for contrastive learning, can maximize the similarity of positive sample pairs (different views of the same sample) and minimize the similarity of positive and negative sample pairs (different samples). Through this loss, the feature extraction model can learn useful feature representations to distinguish different samples. The soft nearest neighbor classifier performs classification by weighting the distance between the sample and the neighbor. The closer the distance, the greater the weight and the more significant the impact. Compared with the traditional nearest neighbor classifier, it can more flexibly reflect the similarity between the sample and the category, and is suitable for the classification task of boundary samples.

[0056] Step S5, determining whether the double contrast learning in step S4 is the first double contrast learning; if so, proceeding to step S6; if not, proceeding to step S7.

[0057] In this step, since the data of a batch in the online semi-supervised incremental learning process is small, the knowledge learned by the model in the double contrast learning will be biased, resulting in large fluctuations in the performance of the model after each batch training. At the same time, the importance of different parameters for the same batch of data is inconsistent. In order to make the model parameters reasonably and smoothly iterate, different treatments are performed depending on whether it is the first double contrast learning.

[0058] Step S6, remove samples whose confidence exceeds the corresponding category threshold from the negative samples, and use the parameters of the feature extraction model at this time as the third model parameters, and go to step S4.

[0059] In this step, the sample elimination process that completes the first double contrast learning is used as the second stage. In order to offset the negative impact of some positive samples being mistakenly classified as negative samples during the first double contrast learning, an adaptive threshold algorithm is used in the second stage to eliminate samples whose confidence exceeds the confidence threshold from negative samples before double contrast learning.

[0060] The confidence threshold calculation method multiplies the maximum prediction probability of each sample by the total number of the current category to calculate its confidence score; then, for the global threshold, an exponential moving average algorithm is used to iteratively update to achieve smooth adjustment of the threshold; finally, the updated threshold is normalized to the maximum value to generate an adaptive confidence threshold for each category.

[0061] Step S7, weighting the first model parameter, the second model parameter and the third model parameter by the Fisher information matrix and performing integrated learning, the weighting formula is shown in formula (1):

[0062]

[0063] In formula (1), θ t-1 is the first model parameter, θ * t is the second model parameter, θ ** t is the second model parameter, F is the Fisher information matrix corresponding to the model parameter, λ and α are hyperparameters, θ t are the model parameters obtained after weighting.

[0064] In this step, considering the small amount of data in a batch and the different importance of different parameters for the same task, the three model parameters before the first double contrast learning, after the first double contrast learning, and after the second double contrast learning are weighted by the Fisher information matrix and integrated learning is performed to make the model iterate smoothly and reasonably. The Fisher information matrix is ​​used to quantify the uncertainty of parameter estimation. It represents the second-order partial derivative matrix of the log-likelihood function of the parameter with respect to the parameter, reflecting the "amount of information" about the parameter contained in the data. The larger the Fisher information matrix, the smaller the variance of the parameter estimation, indicating that the estimation is more accurate.

[0065] Step S8, using the integrated learning model as a mature image classification model to classify the current image; when a new batch of data is generated, proceed to step S1.

[0066] Based on the same idea, an embodiment of the present invention also provides an image classification system based on online semi-supervised incremental learning. In a specific embodiment, the technician needs to configure the computer environment before implementing the image classification system, including installing the Linux operating system, Pycharm software, and PyTorch deep learning framework; in addition, because the deep learning model training process is time-consuming, it is best to train under a GPU. It should be noted that the parameter setting can be used to set the hyperparameters accordingly as needed, including the batch size, the value of the learning rate, the number of iterations, etc.

[0067] The system involves an online semi-supervised incremental learning framework of double contrast learning, which is allocated to the entire image classification system based on online semi-supervised incremental learning, including: a new batch data interface, a memory playback module, a dual view enhancement module, a feature extraction module, a double contrast learning module, a first learning judgment module, a second training module, an integrated learning module and an image classification module.

[0068] The new batch data interface is used to obtain new batch data online, and is also used to obtain the model parameters at this time as the first model parameters;

[0069] The memory playback module is used to update the online new batch data obtained by the new batch data interface to the buffer area, and use the small amount of past samples stored in the buffer area and the updated buffer area samples as memory playback data;

[0070] The dual-view enhancement module is used to perform data enhancement on the new batch data and the buffer area samples to obtain a dual view;

[0071] The feature extraction module is used to extract features from the dual views to obtain feature representations of the dual views;

[0072] The dual contrast learning module is used to perform dual contrast learning at both the instance level and the pseudo-label level on the feature representation of the dual views, and update the parameters of feature extraction, and use the parameters at this time as the second model parameters;

[0073] The first learning judgment module is used to judge whether the current double contrast learning is the first double contrast learning; if so, start the second training module; if not, start the integrated learning module;

[0074] The second training module is used to remove samples whose confidence exceeds the corresponding category threshold from the negative samples, and use the model parameters at this time as the third model parameters to start the double contrast learning module again;

[0075] The integrated learning module is used to perform integrated learning after weighting the first model parameter, the second model parameter and the third model parameter through the Fisher information matrix;

[0076] The image classification module is used to use the integrated learning model as a mature image classification model to classify the current image; when a new batch of data is generated, the new batch data interface is started.

[0077] In this embodiment, each module is implemented by a processor, and a memory is appropriately added when storage is required. Among them, the processor can be but is not limited to a microprocessor MPU, a central processing unit (CPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), other programmable logic devices, discrete gates, transistor logic devices, discrete hardware components, etc. The memory can include a random access memory (RAM) and can also include a non-volatile memory (NVM), such as at least one disk storage. Optionally, the memory can also be at least one storage device located away from the aforementioned processor.

[0078] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center.

[0079] It should also be noted that the image classification system based on online semi-supervised incremental learning described in this embodiment corresponds to the image classification method based on online semi-supervised incremental learning. The description and limitation of the method are also applicable to the system and will not be repeated here.

[0080] It can be seen from the above technical solutions that the image classification method and system based on online semi-supervised incremental learning provided by the embodiments of the present invention fully and effectively learn the information of online semi-supervised data through double contrast learning at the instance level and the pseudo-label level and filtering some negative samples through adaptive thresholds; at the same time, considering that the amount of data in a batch is small and the importance of different parameters to the same task is inconsistent, the present invention uses the Fisher information matrix to weight the model parameters of the three stages and then performs integrated learning, so that the model can be iterated smoothly and reasonably, and efficient incremental learning of the online semi-supervised data stream is realized, the plasticity and stability of the semi-supervised incremental learning model under the online data stream are improved, and the accuracy of image classification is improved.

[0081] The above description is only a preferred embodiment of the present invention and an explanation of the technical principles used. It is not intended to limit the scope of the invention claimed for protection, but only represents the preferred embodiment of the present invention. Those skilled in the art should understand that the scope of the invention involved in the present invention is not limited to the technical solution formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the inventive concept. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

Claims

1. An image classification method based on online semi-supervised incremental learning, characterized in that: The method comprises the following steps: Step S1, obtain new batch data online and synchronously update the cache samples; at the same time, obtain the parameters of the feature extraction model at this time as the first model parameters; Step S2, performing data enhancement on the new batch data and the cache area samples to obtain a dual view; Step S3, extracting features from the dual views using a feature extraction model to obtain feature representations of the dual views; Step S4, performing dual contrast learning at both the instance level and the pseudo-label level on the feature representation of the dual views, and updating the parameters of the feature extraction model, and using the parameters at this time as the second model parameters; Step S5, determining whether the double contrast learning in step S4 is the first double contrast learning; If yes, go to step S6; if no, go to step S7; Step S6, removing samples whose confidence exceeds the corresponding category threshold from the negative samples, and using the parameters of the feature extraction model at this time as the third model parameters, and proceeding to step S4; Step S7, weighting the first model parameter, the second model parameter and the third model parameter through the Fisher information matrix and performing integrated learning; Step S8, using the integrated learning model as a mature image classification model to classify the current image; when a new batch of data is generated, proceed to step S1.

2. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that: The cache samples include samples that have been saved in the past and new samples generated by new batch data.

3. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that: Step S3 uses ResNet to build a feature extraction model.

4. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that: When step S4 performs dual contrast learning, for unlabeled samples, except for the dual views generated by data enhancement of the unlabeled samples, which are regarded as positive sample pairs, the views of all other samples are regarded as negative sample pairs; for labeled samples, the dual views generated by data enhancement of the same labeled sample are regarded as positive sample pairs, and the labeled sample views from different categories and all unlabeled sample views are regarded as negative sample pairs.

5. The image classification method based on online semi-supervised incremental learning according to claim 4 is characterized in that: In step S4, for instance-level contrastive learning, the feature representation generated by the dual views is used as the input embedding vector in the loss function; for pseudo-label-level contrastive learning, the pseudo-labels are assigned to the views through a soft nearest neighbor classifier and used as the input embedding vector in the loss function; the loss functions of the two levels are added together and back-propagation is performed to adjust the parameters of the feature extraction model.

6. The image classification method based on online semi-supervised incremental learning according to claim 5, characterized in that: The loss function adopts the InfoNCE loss function.

7. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that: In step S6, the sample elimination process of the first double contrast learning is taken as the second stage. In order to offset the negative impact caused by some positive samples being mistakenly classified as negative samples during the first double contrast learning, an adaptive threshold algorithm is used in the second stage to eliminate samples whose confidence exceeds the confidence threshold from negative samples before double contrast learning is performed.

8. The image classification method based on online semi-supervised incremental learning according to claim 7, characterized in that: The confidence threshold is calculated by multiplying the maximum predicted probability of each sample by the total number of the current category to calculate its confidence score; Then, the exponential moving average algorithm is used to iteratively update the global threshold to achieve smooth adjustment of the threshold; finally, the updated threshold is normalized to the maximum value to generate an adaptive confidence threshold for each category.

9. The image classification method based on online semi-supervised incremental learning according to claim 1, characterized in that: The weighting formula in step S7 is shown in formula (1): In formula (1), θ t-1 is the first model parameter, θ * t is the second model parameter, θ ** t is the second model parameter, F is the Fisher information matrix corresponding to the model parameter, λ and α are hyperparameters, θ t are the model parameters obtained after weighting.

10. An image classification system based on online semi-supervised incremental learning, characterized in that: The system includes: a new batch data interface, a memory playback module, a dual view enhancement module, a feature extraction module, a dual contrast learning module, a first learning judgment module, a second training module, an integrated learning module and an image classification module; wherein, The new batch data interface is used to obtain new batch data online, and is also used to obtain the parameters of the feature extraction model at this time as the first model parameters; The memory playback module is used to update the online new batch data obtained by the new batch data interface to the cache area, and use the samples in the cache area as memory playback data; The dual-view enhancement module is used to perform data enhancement on the new batch data and the buffer area samples as the memory playback data to obtain the dual-view; The feature extraction module is used to extract features from the dual views through a feature extraction model to obtain feature representations of the dual views; The dual contrast learning module is used to perform dual contrast learning at both the instance level and the pseudo-label level on the feature representation of the dual views, and update the parameters of the feature extraction model, using the parameters at this time as the second model parameters; The first learning judgment module is used to judge whether the current double contrast learning is the first double contrast learning; if so, start the second training module; if not, start the integrated learning module; The second training module is used to remove samples whose confidence exceeds the corresponding category threshold from the negative samples, and use the parameters of the feature extraction model at this time as the third model parameters, and start the double contrast learning module again; The integrated learning module is used to perform integrated learning after weighting the first model parameter, the second model parameter and the third model parameter through the Fisher information matrix; The image classification module is used to use the integrated learning model as a mature image classification model to classify the current image; when a new batch of data is generated, the new batch data interface is started.

Images