Contrastive Semi-Supervision for Limited Labeled Training Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Data collection and labeling for machine learning models is laborious, costly, and time-consuming, with limited labeled samples and abundant unlabeled samples requiring manual efforts, which increases costs and time consumption.
Innovation Solution
A semi-supervised training method using expert-guided label propagation, combining labeled and unlabeled datasets through data augmentation, latent representation spaces, and a loss function that includes mean squared error and contrastive loss, to enhance machine learning model performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual labeling is applied to unlabeled samples, then model training data quality is improved, but costs and time consumption increase
Solution Approach 1:
The system enables unlabeled samples to self-label through the contrastive learning framework. By computing latent representations and applying contrastive loss between paired samples (labeled and unlabeled), the model automatically generates label predictions for unlabeled data without manual intervention, resolving the contradiction between data quality and time consumption
Solution Approach 2:
The patent introduces latent representation spaces as an intermediary between raw data and final labels. The machine learning model transforms unlabeled samples into latent representations, which then serve as the basis for label prediction through contrastive loss computation, enabling automated quality improvement without direct manual labeling
2Quantity of substance
If manual labeling is applied to unlabeled samples, then model training data completeness is improved, but costs increase
Solution Approach 1:
The system allows the machine learning model to automatically generate labels for unlabeled samples through contrastive learning. By computing loss functions based on latent representations and iteratively updating weights, the model completes the training dataset without incurring additional manual labeling costs, resolving the contradiction between data completeness and costs
3Reliability
If more labeled samples are collected, then model training effectiveness is improved, but labor and time requirements increase
Solution Approach 1:
The patent applies contrastive loss to pairs of labeled and unlabeled samples, using partial labeling rather than requiring complete manual labeling of all samples. This partial action approach maintains model training effectiveness by leveraging the contrastive relationships between paired samples while significantly improving training efficiency
Solution Approach 2:
The latent representation space serves as an intermediary that enables effective model training with fewer labeled samples. By transforming samples into latent space and computing contrastive loss, the system achieves reliable training effectiveness without the labor and time costs of collecting extensive labeled data
Data Source
AI summary
A method includes, in response to at least one convergence criterion not being met: receiving a labeled dataset that includes a plurality of labeled samples; receiving an unlabeled dataset that includes a plurality of unlabeled samples; identifying a plurality of labeled-unlabeled sample pairs; applying a data augmentation transformation to each labeled sample and each corresponding unlabeled sample; computing, for each least one labeled-unlabeled sample pair, latent representation spaces using the machine learning model; generating, using the machine learning model, a label prediction for each unlabeled sample for each labeled-unlabeled sample pair; computing a loss function for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs based on respective latency representation spaces and respective label predictions; applying an optimization function to each respective loss function; and updating a weight value for each labeled-unlabeled sample pair of the plurality of labeled-unlabeled sample pairs responsive to applying the optimization function.