Adaptive Frame Stack Logic for Stationary Vehicle Depth Recovery
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Structure from motion (SfM) techniques fail to provide 3D information when a camera is stationary, as the captured 2D image sequences appear the same, leading to a lack of 3D structure recovery and inability to handle changes in the scene, such as objects moving in or out.
Innovation Solution
Adaptive frame stack update logic and optical flow tracks logic that switch between first in, first out (FIFO) and last in, first out (LIFO) methods based on vehicle motion, combined with a stationary check using camera pose and translation vector thresholds, to maintain sufficient baseline for triangulation and handle scene changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Use of energy by stationary object
If a camera is stationary, then the system saves energy and reduces motion complexity, but the captured 2D image sequences appear the same and fail to provide 3D structure information
Solution Approach 1:
The patent applies dynamics by making the frame stack update logic adaptive rather than static. The system dynamically switches between FIFO and LIFO update methods based on whether the vehicle is moving or stationary, allowing the processing approach to change according to operational conditions while the camera remains physically stationary.
Solution Approach 2:
The patent changes the parameter of frame update methodology from a fixed FIFO approach to a conditional approach that selects between FIFO and LIFO based on vehicle motion state. This parameter change enables the system to maintain 3D reconstruction capability during stationary periods by using LIFO to preserve older frames that provide necessary baseline for depth calculation.
2Device complexity
If FIFO logic is used for frame stack update, then the system maintains simplicity and consistency, but it cannot handle scene changes where objects move in or out during stationary periods
Solution Approach 1:
The system dynamically adapts the frame stack update logic based on vehicle motion detection. When the vehicle is stationary, it switches to LIFO logic to preserve historical frames for detecting scene changes, while returning to FIFO logic when moving to maintain simplicity during normal operation.
Solution Approach 2:
The system uses feedback from vehicle motion detection to control the frame stack update methodology. The motion detection module provides feedback about vehicle state, which then determines whether to apply FIFO or LIFO logic, creating a closed-loop adaptive system that responds to actual operating conditions.
3Measurement precision
If LIFO logic is used for frame stack update, then the system can maintain baseline for triangulation during stationary periods, but it increases computational complexity and processing overhead
Solution Approach 1:
The system dynamically selects the frame update logic based on vehicle motion state, applying LIFO only when necessary during stationary periods rather than continuously. This reduces overall computational complexity while maintaining depth measurement accuracy when needed.
Solution Approach 2:
The system changes the processing parameter (frame update logic) conditionally based on vehicle motion detection. By switching between FIFO and LIFO modes, the system optimizes the balance between measurement precision and processing complexity according to actual operational requirements.
4Loss of information
If background subtraction techniques are used to handle moving objects, then the system can remove 3D points where new objects have entered, but it cannot add 3D points in regions where objects have moved out
Solution Approach 1:
Instead of using background subtraction to remove points (forward approach), the patent uses LIFO frame retention to preserve historical frame data, enabling the system to detect both entering and exiting objects by comparing current frames with retained historical frames, thus inverting the traditional approach to achieve more complete 3D point cloud maintenance.
Data Source
AI summary
A vehicular structure from motion (SfM) system can store a number of image frames acquired from a vehicle-mounted camera in a frame stack according to a frame stack update logic. The SfM system can detect feature points, generate flow tracks, and compute depth values based on the image frames, the depth values to aid control of the vehicle. The frame stack update logic can select a frame to discard from the stack when a new frame is added to the stack, and can be changed from a first in, first out (FIFO) logic to last in, first out (LIFO) logic upon a determination that the vehicle is stationary. An optical flow tracks logic can also be modified based on the determination. The determination can be made based on a dual threshold comparison to insure robust SfM system performance.