Neural Network Training With Mixed Label Granularity
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately detecting and characterizing camera blockages due to limited data availability and the difficulty in generating high-quality synthetic images that reflect real-world blockage conditions, which can impact safety and performance.
Innovation Solution
A method involving synthetic image generation using chroma keying to capture blockages on a known background, followed by superimposition onto new backgrounds, combined with a neural network training approach using both synthetic and non-synthetic images to provide pixel-level blockage annotations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If synthetic images are generated using chroma keying to capture blockages on a known background, then the availability of training data is improved, but the manufacturing precision of realistic blockage conditions deteriorates
Solution Approach 1:
The patent uses chroma keying technology as an intermediary method to separate blockage imagery from its background. By capturing blockages against a chroma key background and then compositing them onto target images, the system generates synthetic training data that maintains realistic appearance while providing controlled ground truth labels for neural network training.
2Ease of manufacture
If chroma keying is used to extract blockage imagery, then the ease of manufacture of synthetic training data is improved, but the measurement precision of blockage characterization deteriorates
Solution Approach 1:
The patent implements a feedback mechanism where the neural network is trained using both the synthetic chroma key-generated images and actual sensor images. The network's predictions are compared against ground truth labels derived from the chroma key process, and the model is iteratively refined to improve its accuracy in characterizing blockages while maintaining the efficiency of synthetic data generation.
3Reliability
If both synthetic and non-synthetic images are used for neural network training, then the reliability of blockage detection is improved, but the device complexity of the training system deteriorates
Solution Approach 1:
The patent merges synthetic chroma key-generated training data with actual sensor-collected images into a unified training dataset. This combination allows the neural network to learn from both the controlled, labeled synthetic data and the realistic variations present in actual sensor images, thereby improving detection reliability while managing system complexity through a integrated training pipeline.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enables accurate detection and characterization of camera blockages, enhancing the safety and reliability of autonomous vehicles by improving the neural network's ability to differentiate between blocked and unblocked image portions.
Implementation Method 1
performing a chroma keying operation to extract imagery of a blockage from a partial blockage image
Data Source
AI summary
This disclosure describes systems and methods for training a neural network with a training data set including data items labeled at different granularities. During training, each item within the training data set can be fed through the neural network. For items with labels of a higher granularity, weights of the network can be adjusted based on a comparison between the output of the network and the label of the item. For items with labels of a lower granularity, an output of the network can be fed through a conversion function that convers the output from the higher granularity to the lower granularity. The weights of the network can then be adjusted based on a comparison between the converted output and the label of the item.