Road Image Annotation Using 3D Projection From Vehicle Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The time-consuming and costly manual process of annotating large numbers of images for training convolutional neural networks in autonomous vehicles limits the size and quality of training datasets, affecting the performance of structure detection components.
Innovation Solution
A semi-automated image annotation method that leverages the driving behavior of a human-operated vehicle to efficiently annotate road images by projecting 3D road plane estimates into image sequences, reducing annotation time to seconds per image and requiring only inexpensive dash-cam equipment.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual hand annotation is used to create annotated training images, then annotation accuracy can be maintained, but the time and cost required increases significantly
Solution Approach 1:
The system performs preliminary actions by using the vehicle's own motion and previously captured images to pre-compute the expected road structure and generate annotation data before the actual annotation process. The vehicle's trajectory and camera motion are used to predict where road features should appear in subsequent images, creating preliminary annotations that can be quickly verified or adjusted rather than created from scratch.
Solution Approach 2:
The system makes the annotation process self-service by using the autonomous vehicle's own operational data (camera images, trajectory, motion parameters) to generate its own training annotations. The vehicle essentially annotates its own training data by leveraging its natural motion through the environment and the geometric relationships between consecutive camera views, eliminating the need for external manual annotation resources.
2Manufacturing precision
If manual hand annotation is used to create annotated training images, then annotation quality can be ensured, but the cost and resource requirements increase significantly
Solution Approach 1:
The system makes the annotation process self-service by using the autonomous vehicle's own operational data (camera images, trajectory, motion parameters) to generate its own training annotations. The vehicle essentially annotates its own training data by leveraging its natural motion through the environment and the geometric relationships between consecutive camera views, eliminating the need for external manual annotation resources.
Solution Approach 2:
The system creates copies of road structure information across multiple images by projecting the expected road geometry from one view to another. Instead of manually annotating each image independently, the system copies road feature locations and characteristics from reference images to target images based on the calculated camera motion and road structure, significantly reducing annotation effort while maintaining consistency.
3Quantity of substance
If a large number of training images are generated manually, then the training set size increases, but the time and resources required become impractical
Solution Approach 1:
The system enables continuous generation of annotated training images by processing images in sequences as the vehicle continuously moves through the environment. Rather than stopping to manually annotate each image, the system continuously captures images and automatically generates annotations in real-time or near-real-time based on the vehicle's ongoing motion, creating a continuous stream of training data without interruption.
Solution Approach 2:
The system creates copies of road structure information across multiple images by projecting the expected road geometry from one view to another. Instead of manually annotating each image independently, the system copies road feature locations and characteristics from reference images to target images based on the calculated camera motion and road structure, significantly reducing annotation effort while maintaining consistency.
4Manufacturing precision
If manual annotation processes are used, then annotation detail can be controlled, but the complexity and duration of the process increase
Solution Approach 1:
The system replaces the mechanical manual annotation process with an automated computational system that uses geometric projection and camera motion modeling. Instead of human operators physically drawing annotations on each image, the system automatically calculates expected road feature locations based on camera extrinsic parameters, intrinsic parameters, and vehicle trajectory, substituting manual mechanical annotation with automated mathematical computation.
Data Source
Figure 1
Figure 2~4
Figure 4A
AI summary
In one aspect, hierarchical image segmentation is applied to an image formed of a plurality of pixels, by classifying the pixels according to a hierarchical classification scheme, in which at least some of those pixels are classified by a parent level classifier in relation to a set of parent classes, each of which is associated with a subset of child classes, and each of those pixels is also classified by at least one child level classifier in relation to one of the subsets of child classes, wherein each of the parent classes corresponds to a category of visible structure, and each of the subset of child classes associated with it corresponds to a different type of visible structure within that category.