ToF Data Generation Circuitry Motion Compensation via Event Correlation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Time-of-flight (ToF) cameras face limitations in frame rate and signal quality due to motion artifacts, especially when objects or the camera move during acquisition, leading to noisy and incorrect depth maps, and existing methods for motion compensation often degrade spatial resolution or introduce new artifacts.
Innovation Solution
The proposed solution involves correlating time-of-flight data streams with brightness change event data streams to generate motion-corrected ToF data frames, using techniques like intra-frame and inter-frame motion compensation, and temporal super-resolution, which improves spatial and temporal resolution while reducing motion artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If motion compensation techniques are applied to reduce motion artifacts in ToF depth maps, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The system performs preliminary action by capturing brightness change event data continuously during the ToF acquisition period. This event data is prepared in advance and used to guide the correlation process, allowing motion compensation to be applied more effectively without requiring complex post-processing of the entire depth map data set
Solution Approach 2:
The patent introduces brightness change event data as an intermediary element that mediates between the raw ToF measurements and the final depth map. This event data serves as a guide to identify regions with motion, allowing the system to apply motion compensation selectively rather than uniformly across the entire scene, thereby reducing overall processing complexity
2Measurement precision
If longer integration time is used to improve signal quality in ToF measurements, then measurement precision is improved, but speed decreases due to motion artifacts
Solution Approach 1:
The system dynamically adjusts the effective integration time on a per-pixel or per-region basis using the brightness change event data. In regions where motion is detected, the system uses shorter integration periods to avoid motion artifacts, while in static regions, it can utilize the full integration time to maximize signal quality. This dynamic adaptation allows the system to maintain high frame rates while preserving signal quality where possible
Solution Approach 2:
The patent changes the parameter of integration time from a fixed global value to a variable local value based on motion detection. By modifying this parameter dynamically according to scene content and motion characteristics, the system optimizes the trade-off between signal quality and frame rate for different regions of the image
3Measurement precision
If multiple ToF measurements are averaged to improve signal-to-noise ratio, then measurement precision is improved, but productivity decreases due to increased acquisition time
Solution Approach 1:
The system performs preliminary action by capturing brightness change event data continuously during the ToF acquisition period. This event data is prepared in advance and used to guide the correlation process, allowing motion compensation to be applied more effectively without requiring complex post-processing of the entire depth map data set
Solution Approach 2:
The patent introduces brightness change event data as an intermediary element that mediates between the raw ToF measurements and the final depth map. This event data serves as a guide to identify regions with motion, allowing the system to apply motion compensation selectively rather than uniformly across the entire scene, thereby reducing overall processing complexity
4Reliability
If existing motion compensation methods are applied, then motion artifacts are reduced, but spatial resolution is degraded or new artifacts are introduced
Solution Approach 1:
The system applies local quality by using brightness change event data to identify specific regions with motion and applying motion compensation only to those regions. This localized approach preserves the spatial resolution in static areas where no compensation is needed, while still correcting motion artifacts in dynamic regions, thereby avoiding the uniform degradation of spatial resolution that occurs with global motion compensation methods
Data Source
AI summary
The present disclosure generally pertains to time-of-flight data generation circuitry, configured to: acquire a time-of-flight (ToF) data stream using a ToF camera; acquire a brightness change event data stream using an event-based vision sensor (EVS), camera; correlate the time-of-flight data stream with the brightness change event data stream in time with each other for generating at least one time-of-flight data frame; and generate the at least one time-of-flight data frame based on the correlation.