Quality Assurance Neural Network for Semi-Supervised Training
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Semi-supervised learning for neural networks is unsatisfactory due to the difficulty in identifying accurate loss functions and the challenge of training with a small number of labeled examples, which hinders effective training with conventional cost functions.
Innovation Solution
Implementing a quality assurance neural network that learns and applies comparison metrics to adjust the parameters of a first neural network, using confidence values to select and automatically label training data, and re-train the network in an unsupervised stage, thereby improving the bootstrapping training approach.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional cost functions are used for training neural networks with a small number of labeled examples, then the training process becomes extremely difficult, but the system complexity remains low
Solution Approach 1:
The patent introduces a quality assurance neural network as an intermediary component that learns comparison metrics between predicted outputs and ground truth labels. This mediator network enables effective training with small labeled datasets by providing learned comparison metrics rather than relying on conventional cost functions, thus improving training effectiveness without requiring complex manual loss function design
Solution Approach 2:
The quality assurance neural network learns comparison metrics automatically from the data itself through self-service learning. The system performs semi-supervised learning where the QA network improves its own comparison metrics while simultaneously training the target neural network, eliminating the need for external manual intervention in loss function design and achieving reliable training with minimal labeled examples
2Reliability
If accurate loss functions are identified for semi-supervised learning, then training performance improves, but the difficulty of identifying accurate loss functions increases
Solution Approach 1:
The quality assurance neural network automatically learns comparison metrics from the training data through self-service learning. Instead of requiring researchers to manually identify and design accurate loss functions, the system enables the QA network to discover effective comparison metrics autonomously during training, significantly reducing the difficulty of loss function identification while maintaining high training accuracy
Solution Approach 2:
The system implements a feedback mechanism where the quality assurance neural network learns comparison metrics based on the performance of predicted outputs against ground truth labels. This feedback loop allows the QA network to continuously improve its comparison metrics during training, automatically adapting to the specific task and data characteristics without manual intervention
Data Source
AI summary
In some implementations initially training a first neural network includes inputting the training inputs and corresponding training labels into the first neural network to produce output labels, comparing the output labels to the corresponding training labels using a second neural network that learns and applies a comparison metric, and adjusting parameters of the first neural network based on the comparing. The device then inputs additional inputs into the first neural network to produce additional output labels and corresponding confidence values from the second neural network. The device selects, based on the confidence values, an automatically-labeled training set of data including a subset of the additional inputs and a corresponding subset of the additional output labels. During a second training stage, the device trains the first neural network and the second neural network using the automatically-labeled training set of data.