Neural Network Training with Selective Representations and Less Labeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Training neural networks for tasks like image classification in automated driving requires a vast amount of labeled training data, which is costly and difficult to acquire in diverse situations, leading to potential 'shortcut learning' that hinders generalization.
Innovation Solution
A neural network architecture with an encoder, association, and task head networks, where the association network selectively provides relevant representation components to task heads, allowing training with fewer labeled images by using unlabeled data and factor training records to learn basic skills independently.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a neural network is trained with a broad set of labeled images to improve generalization, then the classification reliability is improved, but the training cost and data acquisition effort increase significantly
Solution Approach 1:
The patent segments the training process into two distinct phases: unsupervised pre-training on unlabeled images to learn general visual features, and supervised fine-tuning on a smaller labeled dataset for specific classification tasks. This segmentation allows the network to acquire robust feature representations without requiring extensive labeled data, thereby improving classification reliability while reducing the quantity of labeled training data needed.
Solution Approach 2:
The patent applies preliminary action by performing unsupervised pre-training on unlabeled images before the supervised training phase. During this preliminary stage, the network learns fundamental visual features and patterns from abundant unlabeled data, which prepares the model for subsequent supervised learning with limited labeled data, thus reducing the overall labeled data requirement while maintaining classification reliability.
2Measurement precision
If the neural network learns from all representation components to improve task performance, then the classification accuracy is improved, but the network develops shortcut learning that hinders generalization
Solution Approach 1:
The patent extracts and removes harmful biases from the training process by using unsupervised pre-training that does not rely on labeled annotations. This preliminary training phase allows the network to learn genuine visual features without acquiring spurious correlations present in labeled data, thereby preventing shortcut learning while maintaining the ability to achieve high classification accuracy on diverse test scenarios.
Solution Approach 2:
The patent changes the training parameters by transitioning from supervised learning (with labeled data) to unsupervised learning (with unlabeled data) during the pre-training phase. This parameter change in the learning objective function allows the network to develop more robust and generalizable features by optimizing for reconstruction or prediction tasks rather than classification, thus improving adaptability while maintaining measurement precision.
Data Source
AI summary
A method for training a neural network for determining a task output with respect to a given task. The method includes: providing unlabeled and/or labelled encoder training records of measurement data; training the encoder network to map encoder training records to representations towards the goal that these representations, and/or or one or more work products derived from the representations, fulfil a self-consistency condition or correspond to ground truth; providing task training records that are labelled with ground truth; and training the association network and the task head networks towards the goal that, when a task training record is mapped to a representation using the encoder network, and the representation is mapped to a task output by the combination of the association network and the task head networks, the so-obtained task output corresponds to the ground truth with which the training record is labelled, as measured by a task loss function.