Road Image Annotation Using 3D Path Projection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The manual process of annotating road images for training convolutional neural networks in autonomous vehicles is time-consuming and costly, limiting the size and quality of training datasets and thereby affecting the performance of structure detection components.
Innovation Solution
A semi-automatic image annotation method that uses vehicle path reconstruction and SLAM processing to determine road structure from image data, reducing the need for human effort and enabling efficient generation of large, accurately annotated training sets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual hand annotation is used to create annotated training images, then annotation accuracy can be maintained, but annotation time and cost increase significantly
Solution Approach 1:
The system uses the autonomous vehicle's own sensor data and path information to automatically generate annotations without external human intervention. The vehicle's recorded trajectory and sensor measurements serve as the annotation source, eliminating the need for manual labeling while maintaining accuracy through the vehicle's actual driving behavior data
Solution Approach 2:
A processing system acts as an intermediary between the raw sensor data and the training images, automatically generating annotations by mapping sensor measurements to image coordinates. This intermediary system converts the vehicle's path data and sensor readings into structured annotation formats suitable for training convolutional neural networks
2Reliability
If manual hand annotation is used to create annotated training images, then annotation quality can be ensured, but the number of training images that can be produced is limited
Solution Approach 1:
The autonomous vehicle systematically collects and annotates its own operational data during normal driving, generating large volumes of training images automatically. Each driving episode produces multiple annotated images through the processing of sensor data and path reconstruction, enabling scalable dataset creation without proportional increases in manual effort
Solution Approach 2:
The system performs annotation preparation in advance by continuously recording sensor data and path information during vehicle operation. This preliminary data collection and processing creates a ready-to-use annotated dataset that can be directly applied to training convolutional neural networks without requiring subsequent manual annotation work
3Measurement precision
If more training images are produced to improve structure detection performance, then detection accuracy improves, but manual annotation effort and cost increase
Solution Approach 1:
The system automatically generates annotations using the vehicle's own operational data, eliminating the need for human annotators. The processing system converts sensor measurements and path data into annotations for multiple training images simultaneously, making the production of large datasets economically viable and significantly improving structure detection accuracy
Solution Approach 2:
The system changes the approach from manual parameter specification to automatic parameter extraction. Instead of manually defining road structures in images, the system extracts path parameters and sensor measurements automatically, transforming the annotation process into a computational task that can be performed at scale without increasing human effort
Data Source
Figure 1
Figure 2~4
Figure 4A
AI summary
A method of annotating road images, the method comprising implementing, at an image processing system, the following steps: receiving a time sequence of two dimensional images as captured by an image capture device of travelling vehicle; processing the images to reconstruct, in three-dimensional space, a path travelled by the vehicle; using the reconstructed vehicle path to determine expected road structure extending along the reconstructed vehicle path; and generating road annotation data for marking at least one of the images with an expected road structure location, by performing a geometric projection of the expected road structure in three-dimensional space onto a two-dimensional plane of that image.