Frame-Event Sensor Fusion for Accurate High-Speed ToF Ranging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The integration and utilization of event-based sensors with other apparatuses for improving the accuracy of distance information calculation has not been sufficiently explored.
Innovation Solution
An information processing apparatus that combines a frame-based vision sensor with an event-based sensor to calculate distance information by identifying target areas, tracking these areas based on event signals, and calculating distance using the difference between irradiation and light reception times for a predetermined light pattern.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a frame-based vision sensor is used to scan all pixels with a predetermined period, then complete image information can be obtained, but the operating speed is reduced and power consumption increases
Solution Approach 1:
The patent extracts only the necessary changes in light intensity at specific pixels rather than scanning all pixels completely. The event-based sensor detects and transmits only the delta information (changes in light intensity) from pixels where events occur, eliminating the need to process all pixel data continuously, thus achieving high-speed operation with reduced power consumption while maintaining measurement precision for dynamic objects
Solution Approach 2:
The patent segments the imaging task into two parts: the frame-based vision sensor handles static scene understanding and object identification, while the event-based sensor handles dynamic motion detection and tracking. This segmentation allows each sensor to operate optimally for its specific function, combining the advantages of both approaches to achieve both completeness and speed
2Speed
If an event-based sensor is used to detect light intensity changes asynchronously, then high-speed operation with reduced power consumption is achieved, but the accuracy of distance information calculation is insufficient
Solution Approach 1:
The patent merges the outputs of two different sensor types: the frame-based vision sensor provides complete image information and object identification, while the event-based sensor provides high-speed dynamic changes. By combining these complementary strengths, the system achieves both high operating speed from the event-based sensor and accurate distance information calculation through the integrated processing of both sensor data streams
Solution Approach 2:
The patent makes the information processing apparatus multi-functional by enabling it to perform both image acquisition (using frame-based sensor) and distance measurement (using event-based sensor with TOF principle). The apparatus can identify objects, track their motion, and calculate distance simultaneously, with the event-based sensor's asynchronous data complementing the frame-based sensor's spatial information to improve overall measurement accuracy
3Device complexity
If distance information is calculated using only a single sensor type, then the system complexity is reduced, but the accuracy and resolution of distance measurements are limited
Solution Approach 1:
The patent segments the sensing function into two specialized sensors: a frame-based vision sensor for spatial information and object identification, and an event-based sensor for temporal dynamics and motion detection. This segmentation allows each sensor to be optimized for its specific strength, and their combined output provides superior distance measurement accuracy compared to a single sensor type, while the modular architecture keeps the system manageable
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Enhances the accuracy and frame rate of distance information calculation by complementing ToF sensor data with event-based sensor information, effectively utilizing reflected light and increasing the resolution and frequency of distance measurements.
Implementation Method 1
an event-based sensor in which a pixel detecting a change in the intensity of incident light generates a signal
Implementation Method 2
a frame-based vision sensor that scans all pixels with a predetermined period, specifically, an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor)
Implementation Method 3
calculates, for each of the target areas, the distance information on the basis of a difference between an irradiation start time and a light reception time for the light with the predetermined pattern
Data Source
AI summary
There is provided an information processing apparatus that calculates distance information regarding a distance to a target irradiated with light with a predetermined pattern, the information processing apparatus including an identification section that identifies at least one area in an image acquired using a frame-based vision sensor, a tracking section that tracks a target area corresponding to the area identified by the identification section, on the basis of an event signal output by an event-based sensor, and a calculation section that calculates, for each of the target areas, the distance information on the basis of a difference between an irradiation start time and a light reception time for the light with the predetermined pattern.