Machine Learning Module Training Without Ground Truth Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Machine learning modules in applications like automated body measurements face challenges in training and evaluation due to the unavailability or unreliability of corresponding ground truth data sets, particularly for annotation ML modules that require manual annotation of body features, which is time-consuming and costly.
Innovation Solution
A computer-implemented method and system that uses a second trainable ML module to evaluate or train the first ML module by receiving intermediate and output data sets, generating evaluation data sets, and applying a loss function based on distance metrics to assess and improve the performance of the first ML module without relying on complete ground truth data sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual annotation is used to create ground truth data sets for training annotation ML modules, then training data quality is improved, but time consumption and cost increase significantly
Solution Approach 1:
The patent uses a second ML module to generate synthetic annotation data that copies and transforms data from the first ML module's outputs. This synthetic data serves as pseudo-ground truth for training the first ML module, eliminating the need for time-consuming manual annotation while maintaining data quality through the second module's trained transformations.
Solution Approach 2:
The system enables self-service training by using the first ML module's own outputs as training data through the second ML module's transformation. The annotation ML module trains itself using synthetic data generated from its own predictions, eliminating dependency on manual annotation for training purposes.
2Measurement precision
If complete ground truth data sets are required for training and evaluation, then model accuracy is improved, but data availability becomes a limiting factor
Solution Approach 1:
The second ML module acts as an intermediary that transforms the first ML module's outputs into synthetic ground truth data. This intermediary transformation enables training and evaluation without requiring actual complete ground truth data sets, as the synthetic data generated by the second module serves as the necessary reference.
Solution Approach 2:
The system creates synthetic copies of ground truth data by transforming the first ML module's predictions through the second ML module. These synthetic copies replicate the characteristics of real ground truth data, enabling training and evaluation processes to proceed without access to actual complete ground truth data sets.
3Measurement precision
If manual annotation of body features is performed, then annotation accuracy is improved, but productivity decreases due to labor intensity
Solution Approach 1:
The second ML module generates synthetic annotation data by transforming and copying data from the first ML module's outputs. This automated copying process replaces manual annotation, maintaining annotation accuracy through the trained transformation while dramatically increasing productivity by eliminating human labor requirements.
Solution Approach 2:
The patent replaces the mechanical process of manual human annotation with an automated ML-based system. The second ML module performs the annotation transformation computationally, substituting human cognitive and manual processes with automated machine learning transformations, thereby increasing productivity while maintaining accuracy.
Data Source
AI summary
Methods and systems are disclosed for evaluating or training a machine learning module when its corresponding truth data sets are unavailable or unreliable. The methods and systems are configured for evaluating or training a target machine learning module having a first (system) input and a first output, wherein the target module is connected to a second machine learning module having an intermediate input (identical to the first output of the target module) and a second (system) output, by training the second module using received corresponding intermediate and output data sets, generating an evaluation data set using a received system input data set, and evaluating or training the target module using a loss function based on a distance metric between the evaluation data set and a received system output data set corresponding to the system input data set.