Vehicle Sensor Capture Timing for Occlusion-Aware 3D Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Street-level images often suffer from obstructions by objects like vegetation, vehicles, and pedestrians, leading to incomplete 3D models when used for scene reconstruction, as these obstacles block views and prevent areas behind them from being captured.
Innovation Solution
A method and system for controlling sensors on a moving vehicle to capture data based on determined timing characteristics, identifying targets of interest, and adjusting sensor rates and frames to avoid occluders, ensuring comprehensive data capture for generating accurate 3D models.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If street-level images are captured continuously at high frame rates, then complete 3D model data can be obtained, but data redundancy and storage requirements increase significantly
Solution Approach 1:
The system dynamically changes the capture rate parameter based on detected scene content. When occluders are detected, the capture rate is reduced since these frames will be discarded. When targets of interest are detected, the capture rate increases to ensure complete data capture. This adaptive parameter adjustment resolves the contradiction by capturing only necessary data at appropriate rates.
Solution Approach 2:
The system uses onboard sensors and processing to automatically detect occluders and targets of interest, then autonomously adjusts capture rates without external intervention. The computing device processes sensor data in real-time to determine optimal capture strategies, making the system self-regulating and eliminating the need for continuous high-rate capture.
2Loss of information
If all captured images are stored and processed, then complete scene information is retained, but processing time and computational resources increase
Solution Approach 1:
The system extracts and discards useless image data captured during occluder passages. By identifying occluders and determining their occlusion periods, the system extracts only the necessary image segments for 3D modeling and discards the rest, significantly reducing processing requirements while maintaining information completeness.
Solution Approach 2:
The system performs preliminary detection of occluders and targets of interest before full image capture sequences. By pre-identifying relevant capture windows based on sensor data and predicted vehicle trajectories, the system prepares capture schedules in advance, avoiding unnecessary capture and subsequent processing of irrelevant frames.
3Measurement precision
If the capture rate is increased to account for vehicle motion, then motion blur and missed targets are reduced, but energy consumption and data redundancy increase
Solution Approach 1:
The system dynamically adjusts capture rates based on real-time vehicle speed, distance to targets, and detected occluders. Rather than using a fixed high capture rate, the system modulates the capture frequency to match actual needs, reducing energy consumption while maintaining measurement precision through adaptive timing.
Solution Approach 2:
The system uses feedback from sensors detecting vehicle position, speed, and surrounding objects to continuously optimize capture rates. This closed-loop control ensures capture accuracy is maintained only when necessary, eliminating energy waste during periods when high-rate capture would not improve measurement precision.
4Area of stationary object
If occluders are captured in the dataset, then complete visual coverage is achieved, but 3D model quality deteriorates due to obstructed views
Solution Approach 1:
The system takes preliminary action to prevent occluder data from contaminating the 3D model dataset. By detecting occluders in advance using sensors and predicting their occlusion periods, the system proactively excludes these frames from capture or marking them for discarding, thereby preventing the harmful effect of obstructed views in the final model.
Solution Approach 2:
The system converts the presence of occluders into a beneficial detection opportunity. By using sensors to detect occluders and their trajectories, the system gains precise knowledge of when and where occlusion occurs, allowing it to strategically adjust capture rates and timing to capture targets of interest during non-occluded periods, thereby turning the occluder presence into useful timing information.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The disclosure provides for a method of controlling one or more sensors on a moving vehicle that is executable by one or more computing devices. The one or more computing devices may detect a first surface at a first location and a second surface at a second location using the one or more sensors. The second surface may be classified as a target of interest. Then the one or more computing devices may determine one or more timing characteristics of the one or more sensors based on a pose or motion of the one or more sensors relative to the first location of the first surface and the second location of the second surface. Then, the one or more computing devices may control the one or more sensors to capture data according to the determined one or more timing characteristics.