3D Sensor Rendering Using 2D Image Projection for Fast Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for rendering 2D and 3D data within a 3D virtual environment in autonomous vehicles are processing-intensive and time-consuming, making it difficult to align and comprehend 3D LIDAR and 2D image data effectively.
Innovation Solution
A method that generates a 3D virtual environment, defines vehicle origins, and projects 2D images onto image planes within the environment, allowing for concurrent rendering of 3D LIDAR and 2D data, enabling quick alignment and annotation by human annotators.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If 3D LIDAR and 2D image data are stitched and aligned using conventional methods, then alignment accuracy is achieved, but processing time and computational load increase significantly
Solution Approach 1:
The patent projects 2D camera images onto 3D point cloud data by mapping 2D image pixels to corresponding 3D LIDAR points through ray-casting from the camera origin through image pixels into the 3D space. This dimensional transformation allows direct overlay of 2D and 3D data without traditional stitching, reducing processing time while maintaining alignment accuracy through geometric projection relationships.
Solution Approach 2:
The patent extracts only the necessary alignment information by defining specific origins (camera origin, LIDAR origin) and using ray-casting through image pixels to identify corresponding 3D points. This selective extraction of alignment-critical data eliminates the need for processing entire datasets, significantly reducing computational load while preserving alignment precision.
2Productivity
If 2D images are projected onto 3D point clouds with proper origin definition, then annotation efficiency improves, but system complexity increases
Solution Approach 1:
The patent pre-defines the camera origin and LIDAR origin positions before processing images and point clouds. By establishing these reference points in advance and pre-calculating projection parameters, the system enables rapid annotation without repeatedly computing transformation matrices during the annotation process, thereby improving efficiency while managing complexity through upfront setup.
3Ease of operation
If multiple 2D images are stitched into a 3D color image, then visual comprehension improves, but processing intensity and time consumption increase
Solution Approach 1:
The patent creates a visual representation by projecting 2D image data onto the 3D point cloud structure rather than stitching multiple 2D images together. This copying approach maps 2D color information onto the existing 3D geometric framework, providing intuitive visual comprehension while avoiding the computationally intensive process of 3D reconstruction from multiple 2D images.
Data Source
AI summary
One variation of a method includes: accessing a 2D color image recorded by a 2D color camera and a 3D point cloud recorded by a 3D depth sensor at approximately a first time, the 2D color camera and the 3D depth sensor defining intersecting fields of view and facing outwardly from an autonomous vehicle; detecting a cluster of points in the 3D point cloud representing a continuous surface approximating a plane; isolating a cluster of color pixels in the 2D color image depicting the continuous surface; projecting the cluster of color pixels onto the plane to define a set of synthetic 3D color points in the 3D point cloud, the cluster of points and the set of synthetic 3D color points representing the continuous surface; and rendering points in the 3D point cloud and the set of synthetic 3D color points on a display.