Frame and Event Image Processing for Faster Subject Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing synchronous type photoelectric conversion apparatuses are limited in their ability to handle faster processing requirements, particularly in fields like transportation and robotics, due to their frame-based data acquisition.
Innovation Solution
An image processing apparatus that combines data from a first image capturing unit operating in frame format with data from a second image capturing unit that detects changes in luminance as event data, allowing for the generation of enhanced image data through predetermined processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If synchronous type photoelectric conversion apparatus is used, then image data can be acquired in synchronization with synchronizing signal, but processing speed is limited and cannot cope with faster processing requirements
Solution Approach 1:
The patent transitions from synchronous frame-based capture to asynchronous event-driven capture. The photoelectric conversion apparatus operates dynamically by triggering data acquisition based on detected events (changes in luminance) rather than fixed synchronization cycles, enabling faster processing speeds while maintaining comprehensive image data collection.
Solution Approach 2:
The patent changes the operational parameters from time-based synchronization (frame rate limited by synchronizing signal period) to event-based triggering. This parameter change allows the system to operate at higher effective frame rates by only capturing data when changes occur, rather than being constrained by the synchronizing signal period.
2Speed
If event type photoelectric conversion apparatus is used, then processing speed is improved, but the amount of information is reduced and not suitable for confirming subject from bird's-eye view
Solution Approach 1:
The patent merges two types of photoelectric conversion apparatus: a first apparatus for capturing complete frame images and a second apparatus for detecting luminance changes. By combining their outputs, the system achieves both high processing speed (from event data) and information completeness (from frame data), resolving the contradiction between speed and information loss.
Solution Approach 2:
The patent segments the image capturing function into two specialized components: one dedicated to comprehensive frame capture and another dedicated to change detection. This segmentation allows each component to optimize for its specific function while their combined output provides both speed and completeness.
3Speed
If event data is used, then processing speed is improved, but visibility and subject recognition are reduced
Solution Approach 1:
The patent introduces an intermediary processing unit that receives both frame data and event data, then synthesizes them into enhanced image data. This intermediary process combines the speed advantage of event data with the visibility advantage of frame data, producing output that achieves both high processing speed and good subject recognition.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This approach improves visibility by integrating frame data with event data, enabling better subject recognition and environmental context, even in fast-paced scenarios.
Implementation Method 1
synchronous type (frame type) photoelectric conversion apparatuses (including solid-state image pickup elements) that capture image data (frames) in synchronization with a synchronizing signal
Implementation Method 2
a second image capturing unit configured to acquire second image data which is data based on event data of a change in luminance of a subject for each pixel
Data Source
AI summary
An image processing apparatus configured to generate image data that makes it possible to improve visibility is on the basis of image data acquired by a first image capturing unit and a second image capturing unit includes the first image capturing unit configured to acquire first image data which is image data of a frame format, the second image capturing unit configured to acquire second image data which is data based on event data of a change in luminance of a subject for each pixel, and a processing unit configured to generate third image data by performing predetermined processing on the first image data on the basis of the second image data.