Soft-Label AI Training for Many-to-Many Image-Text Correspondence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional AI model training for image classification is inefficient and inaccurate due to the use of noisy, binary ground truth labels and the inability to handle many-to-many relationships in image-text correspondences, leading to reduced classification accuracy.
Innovation Solution
A progressive self-distillation approach is employed, where a teacher AI model generates pseudo-ground truth labels with non-binary values, and a student AI model is trained using a combination of ground truth and pseudo-ground truth labels, progressively increasing the reliance on pseudo-labels to refine the model's understanding of image-text correspondences.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If binary ground truth labels are used for training, then the training process is simple and fast, but the classification accuracy is reduced due to inability to handle many-to-many relationships
Solution Approach 1:
The patent changes the parameter of ground truth labels from binary (0/1) to non-binary probability values. Instead of using hard labels, the system uses soft labels that represent probability distributions over multiple classes, allowing the model to handle many-to-many relationships and improve classification accuracy while maintaining training efficiency
Solution Approach 2:
The patent segments the training process into multiple stages: first training a teacher model with binary labels, then using the teacher model to generate pseudo-labels with probability values. This segmentation allows the system to gradually transition from simple binary labels to complex probability-based labels, resolving the contradiction between training simplicity and classification accuracy
2Ease of manufacture
If noisy binary ground truth labels are used, then data preparation is simple, but the model cannot learn from soft information leading to reduced robustness
Solution Approach 1:
The patent uses a teacher model to generate pseudo-labels that copy the structure of ground truth labels but with probability values instead of binary values. These pseudo-labels capture soft information about image-text correspondences, allowing the student model to learn from noisy data more robustly while maintaining ease of data preparation
Solution Approach 2:
The teacher model acts as an intermediary that converts simple binary labels into probability-based pseudo-labels. This intermediary layer enables the student model to access soft information and learn from noisy data, improving robustness without complicating the data preparation process
3Device complexity
If hard ground truth labels are used, then the training data is straightforward to obtain, but the model fails to capture probabilistic relationships in image-text correspondences
Solution Approach 1:
The patent changes the parameter representation from hard binary values to soft probability values. This parameter change enables the model to capture probabilistic relationships and handle many-to-many correspondences between images and texts, while the training data structure remains relatively simple through the use of pseudo-labels generated from teacher model predictions
Data Source
AI summary
Soft information can be used to train an artificial intelligence (AI) model. For example, a computer system determines first training data that includes a first image, first text, a second image, second text, a first indication of a first correspondence between the first image and the first text, and a second indication of a second correspondence between the second image and the second text. The computer system determines a likelihood of the first image and the second text corresponding to each other. The computer system generates second training data that includes the first image, the first text, the second image, the second text, the likelihood instead of the first indication, and the second indication. The computer system trains an AI model based at least in part on the second training data.