Mutual Teacher-Student Training for Unlabeled Image Classification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The existing methods for training image classification models using pre-trained teacher networks to generate pseudo labels for semi-supervised learning are ineffective, leading to poor training effects and reduced accuracy and stability due to low confidence in pseudo labels, especially in the early stages of the training process.
Innovation Solution
Implement a mutual teacher-student relationship between image classification sub-models, where each sub-model provides pseudo labels to the other for semi-supervised learning, enhancing information utilization and improving the training effect by complementary learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If pre-trained teacher networks are used to generate pseudo labels for semi-supervised learning, then the training process can utilize unlabeled data, but the low confidence in pseudo labels leads to poor training effect and reduced accuracy
Solution Approach 1:
The patent merges multiple image classification sub-models into a collaborative ensemble system where each sub-model contributes to generating pseudo labels for unlabeled data. By combining the predictions of multiple sub-models, the system achieves higher confidence and accuracy in pseudo label generation compared to using a single pre-trained teacher network, thereby resolving the contradiction between utilizing unlabeled data and maintaining classification accuracy.
Solution Approach 2:
The patent introduces an intermediary mechanism where multiple image classification sub-models act as mediators between the labeled and unlabeled data. These sub-models collectively generate and refine pseudo labels through iterative collaboration, serving as an intermediate step that bridges the gap between uncertain unlabeled data and reliable classification results, thus improving both data utilization and accuracy.
2Productivity
If pre-trained teacher networks generate pseudo labels in early training stages, then semi-supervised learning can proceed, but the low confidence pseudo labels reduce training stability
Solution Approach 1:
The patent combines multiple image classification sub-models to work together during the training process, where their collective predictions stabilize the generation of pseudo labels. This merging approach ensures that even in early training stages when individual models may be uncertain, the ensemble provides more stable and reliable guidance for training, maintaining both productivity and stability.
Solution Approach 2:
The patent implements a feedback mechanism where the multiple sub-models iteratively refine their predictions on unlabeled data. The feedback loop allows the models to learn from each other's predictions and progressively improve the confidence and stability of pseudo labels, enabling stable training progression from the early stages while maintaining high productivity.
3Device complexity
If single teacher network is used for pseudo label generation, then the system structure remains simple, but the training effect is poor due to limited information utilization
Solution Approach 1:
The patent merges multiple image classification sub-models into a collaborative framework that fully exploits information from both labeled and unlabeled data. Each sub-model processes and contributes unique information, and their combined predictions capture more diverse features and patterns than a single teacher network, thereby reducing information loss while maintaining manageable system complexity through structured collaboration.
Solution Approach 2:
The patent segments the information processing task by assigning different image classification sub-models to handle different aspects or subsets of the training data. This segmentation allows each model to specialize in extracting specific features or patterns, and their results are then integrated to achieve comprehensive information utilization, reducing the overall information loss without requiring a single overly complex model.
Data Source
Figure 1~2
Figure 3~4
Figure 5~6
AI summary
The present application discloses a method for training an image classification model, an image classification method and related device. The method includes: obtaining an image set; discriminating a labeled image and an unlabeled image by a target image classification sub-model of the image classification model, and determining first classification reference information of the labeled image and first classification reference information of the unlabeled image; discriminating the unlabeled image by a non-target image classification sub-module of the image classification model, and determining second classification reference information of the unlabeled image; determining a classification loss of the target image classification sub-model according to the first classification reference information of the labeled image, the category label of the labeled image, and the second classification reference information of the unlabeled image; tuning a model parameter of the image classification model according to the classification loss of the first image classification sub-model and the classification loss of the second image classification sub-model.