LiDAR Range Image Mapping for Real-Time 3D Scene Flow
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing scene flow systems for autonomous vehicles are limited by their stationary design and limited fields of view, making them incapable of real-time operation in 3D space, relying on inaccurate camera-based neural networks, noisy LiDAR, and RADAR data, which lack contextual information across multiple frames.
Innovation Solution
The system employs belief propagation for range image mapping, simplifying 3D LiDAR data to 2.5D depth flow space, passing messages between pixels to estimate noisy information, and calculating 3D motion vectors, enabling real-time LiDAR-based scene flow with a wider field of view and improved accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If existing scene flow systems operate in 3D space, then measurement precision is improved, but device complexity and computational requirements increase making real-time operation impossible
Solution Approach 1:
The patent transforms the problem from 3D space to 2D range image space by projecting 3D LiDAR data onto 2D images. This dimensionality reduction allows belief propagation to operate efficiently in 2D while maintaining the ability to recover 3D scene flow information, resolving the contradiction between measurement precision and computational complexity.
Solution Approach 2:
The patent introduces 2D range images as an intermediary representation between raw 3D LiDAR data and final scene flow output. This intermediary form enables efficient belief propagation processing while preserving the necessary information for accurate 3D motion estimation, allowing real-time operation without sacrificing measurement precision.
2Measurement precision
If LiDAR data is used for scene flow, then measurement precision is improved, but data quality deteriorates due to missing points and noise
Solution Approach 1:
The patent implements belief propagation as a feedback mechanism where information is iteratively exchanged between neighboring pixels in the 2D range image. This feedback process allows the system to infer missing depth values from contextual information and suppress noise through consistency checks, simultaneously improving measurement precision and data reliability.
Solution Approach 2:
The patent changes the parameter representation from raw 3D point cloud coordinates to 2D range image intensities and depth values. This parameter transformation enables the application of image processing techniques that are more robust to missing data and noise, improving both measurement precision and reliability through statistical inference.
3Measurement precision
If camera-based neural networks are used, then short-distance detection accuracy is improved, but long-distance detection accuracy deteriorates due to limited pixels
Solution Approach 1:
The patent creates a universal system that combines the strengths of both camera-based and LiDAR-based approaches. By using 2D range images as an intermediary, the system can apply belief propagation techniques that work effectively at both short and long distances, overcoming the distance limitation of camera-based neural networks while maintaining their computational efficiency.
4Measurement precision
If 3D LiDAR data is processed directly, then measurement precision is improved, but processing speed deteriorates preventing real-time operation
Solution Approach 1:
The patent achieves real-time processing by transforming 3D LiDAR data into 2D range images, reducing the computational dimensionality. This allows belief propagation to operate much faster in 2D space while still producing accurate 3D scene flow results, resolving the contradiction between measurement precision and processing speed.
Data Source
AI summary
In various examples, systems and methods are described that generate scene flow in 3D space through simplifying the 3D LiDAR data to “2.5D” optical flow space (e.g., x, y, and depth flow). For example, LiDAR range images may be used to generate 2.5D representations of depth flow information between frames of LiDAR data, and two or more range images may be compared to generate depth flow information, and messages may be passed—e.g., using a belief propagation algorithm—to update pixel values in the 2.5D representation. The resulting images may then be used to generate 2.5D motion vectors, and the 2.5D motion vectors may be converted back to 3D space to generate a 3D scene flow representation of an environment around an autonomous machine.