Optical Flow Motion Detection via Lidar Ego Motion Subtraction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current motion detection systems for agents like robots and autonomous vehicles struggle to accurately differentiate between moving and stationary objects, especially when objects are occluded or in complex scenes, due to limitations in sensor data from lidar and camera systems.
Innovation Solution
The system employs optical flow analysis from camera images combined with lidar-based ego motion information to predict motion of points in a scene, using neural networks to generate low-level motion predictions that are object-agnostic, allowing for accurate identification of moving and stationary points, including occluded objects.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical flow analysis from camera images is used for motion detection, then motion information can be obtained, but it is difficult to differentiate between motion caused by agent movement and motion of actual objects
Solution Approach 1:
The patent uses lidar data as an intermediary to capture pure ego motion information independently of visual appearance. The lidar point cloud provides geometric flow data that represents only the agent's movement through space, which then serves as a reference to subtract from optical flow data to isolate actual object motion.
Solution Approach 2:
The patent replaces reliance on a single camera-based optical flow system with a multi-sensor system combining camera and lidar. The lidar system substitutes for the camera's motion detection capability by providing an alternative measurement modality that is insensitive to visual appearance changes, enabling separation of ego motion from object motion.
2Measurement precision
If lidar data is used for motion detection, then 3D spatial information is obtained, but it cannot capture appearance changes or occlusions effectively
Solution Approach 1:
The patent merges lidar point cloud data with camera image data to create a complementary multi-modal system. The lidar provides accurate 3D spatial positioning and ego motion, while the camera provides appearance information and occlusion detection. By combining these data sources and fusing their respective strengths, the system overcomes the limitations of each individual sensor type.
3Adaptability or versatility
If neural networks are used to generate motion predictions, then object-agnostic low-level motion information is obtained, but computational complexity increases
Solution Approach 1:
The patent segments the motion detection task into distinct processing stages: optical flow computation from images, geometric flow computation from lidar point clouds, and neural network-based motion prediction. This segmentation allows each component to be optimized independently and enables parallel processing of multiple data streams before fusion at the prediction stage.
Data Source
AI summary
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating motion detection based on optical flow. One of the methods includes obtaining a first image of a scene in an environment taken by an agent at a first time point and a second image of the scene at a second later time point. A point cloud characterizing the scene in the environment is obtained. A predicted optical flow is determined between the first image and the second image. A respective initial flow prediction for the point that represents motion of the point between the two time points is determined. A respective ego motion flow estimate for the point that represents a motion of the point induced by ego motion of the agent is determined. A respective motion prediction that indicates whether the point was static or in motion between the two time points is determined.