Free-Space Estimation Using 2D Optical Flow and 3D Motion Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Autonomous vehicles face challenges in accurately detecting and navigating through their surroundings due to the dynamic nature of their environment, requiring efficient free-space estimation with velocity information while managing limited computational resources.
Innovation Solution
A method utilizing optical flow in two-dimensional images combined with three-dimensional projection to estimate motion data, which is then integrated into free-space estimation, constraining motion parallel to the ground plane to reduce computational complexity and noise.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion data is derived using 3D point tracking and rate of change calculation, then velocity information is obtained for free-space estimation, but computational cost increases and temporal noise is introduced
Solution Approach 1:
The patent creates a 2D copy of the 3D scene by projecting 3D points onto the image plane to form 2D point clouds. Optical flow is then applied to this 2D representation instead of directly to 3D points, reducing computational complexity while preserving motion estimation accuracy. The 2D point cloud serves as an intermediate copy that simplifies the motion analysis.
Solution Approach 2:
The patent transforms the problem from 3D space to 2D space by projecting 3D points onto the image plane. Optical flow is naturally defined in 2D image space, so this dimensionality change allows direct application of optical flow algorithms without complex 3D motion calculations. The depth information is preserved through the projection relationship, enabling accurate 3D motion reconstruction from 2D optical flow.
2Loss of information
If 3D point position tracking is performed between time instances, then motion information is obtained, but temporal noise is introduced in velocity calculations
Solution Approach 1:
By creating a 2D copy of the scene through projection, the patent enables optical flow to track pixel intensities and gradients rather than relying solely on 3D point position matching. Optical flow uses local image gradients and intensity constraints that are more robust to temporal variations, reducing noise in motion estimation while maintaining accuracy.
3Measurement precision
If full 3D motion estimation is performed, then accurate velocity information is obtained, but computational resources are exceeded in real-time processing
Solution Approach 1:
The patent creates a simplified 2D representation (copy) of the 3D scene that is optimized for optical flow processing. This 2D point cloud in image space allows efficient parallel processing of motion vectors across all points simultaneously, achieving real-time performance while maintaining accurate velocity estimation through depth-aware projection.
Solution Approach 2:
The patent replaces complex 3D mechanical motion tracking with optical flow analysis in 2D image space. Optical flow uses image gradient and intensity information to estimate motion, substituting direct 3D point tracking with a more computationally efficient 2D image processing approach that leverages the image capturing device's native data format.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach provides a computationally favorable method for real-time free-space estimation with reduced temporal noise, enhancing the accuracy and efficiency of navigation systems in autonomous vehicles.
Implementation Method 1
applying an optical flow between the image and a subsequent image of the sequence of images, to the 2D point, thereby obtaining a subsequent 2D point
Implementation Method 2
obtaining a set of 3D points based on a depth map of the scene depicted in the image sequence. Each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene
Implementation Method 3
determining a subsequent 3D point by projecting the subsequent 2D point based on the depth map of the scene
Data Source
AI summary
The present invention relates to a method performed in a vehicle equipped with an automated driving system. The method includes: obtaining an image sequence captured by an image capturing device of the vehicle, wherein the image sequence includes a plurality of images depicting a scene at a respective time instance of a plurality of time instances; obtaining a set of 3D points based on a depth map of the scene depicted in the image sequence, wherein each 3D point of the set of 3D points is associated with a three-dimensional position of the 3D point within the scene; determining motion data associated with each 3D point of the set of 3D points,; and assigning the set of 3D points with associated motion data to a free-space estimation of the scene, based on the three-dimensional position associated with each 3D point.