Semi-Supervised Learning Model Training with Loss and Task Modules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The data annotation bottleneck in training deep neural networks (DNNs) is significant, making it infeasible to implement many artificial intelligence systems, such as image segmentation systems, due to the laborious and time-consuming nature of annotating data, especially on end-user devices like smartphones.
Innovation Solution
A computer-implemented method for training a machine learning model using a combination of labelled and unlabelled data through semi-supervised learning, where a loss module is trained using labelled data to estimate the likelihood of labels, and a task module is trained using the loss module, labelled data, and unlabelled data to make predictions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data annotation is performed manually on end user devices, then model training can be done on-device with user data, but the annotation process becomes laborious and time-consuming
Solution Approach 1:
The system enables unlabelled user data to train the task module through semi-supervised learning, where the model self-improves by leveraging abundant unlabelled data without requiring manual annotation of each data point. The loss module automatically estimates label likelihoods, making the system self-sufficient in utilizing user data.
Solution Approach 2:
A small set of labelled data is used in advance to train the loss module, which then serves as a pre-trained component capable of estimating label likelihoods for unlabelled data. This preliminary training enables the subsequent unsupervised training phase without requiring manual annotation of all training data.
2Measurement precision
If extensive labelled data is collected for training, then model prediction accuracy improves, but the data annotation bottleneck makes it infeasible to implement AI systems
Solution Approach 1:
The invention changes the training paradigm from requiring extensive labelled data to using semi-supervised learning with a small labelled set and large unlabelled set. This parameter change in data composition and training methodology maintains prediction accuracy while making implementation feasible by reducing the annotation burden to a manageable level.
3Reliability
If all training data is labelled, then the model can be trained effectively, but the laborious annotation process creates a bottleneck that prevents system implementation
Solution Approach 1:
The training process is segmented into two distinct phases: supervised training of the loss module using a small labelled dataset, and unsupervised training of the task module using abundant unlabelled data. This segmentation allows the system to maintain training effectiveness while avoiding the complexity and cost of annotating all training data.
Data Source
AI summary
Provided is a computer-implemented method for training a machine learning (ML) model using labelled and unlabelled data, the method comprising obtaining a set or training data comprising a set of labelled data items and a set of unlabelled data items, training a loss module of the ML model using labels in the set of labelled data items, to generate a trained loss module capable of estimating a likelihood of a label for a data item, and training a task module of the ML model using the loss module, the set of labelled data items, and the set of unlabelled data items, to generate a trained task module capable of making a prediction of a label for input data.