Event Camera Motion Blur for High-Frame-Rate Mobile Video
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mobile devices struggle to achieve cinematic motion blur due to their small lens assemblies lacking mechanical apertures or interchangeable filters, leading to choppy motion in videos with moving objects.
Innovation Solution
A system combining a frame-based camera with an event camera to simulate long exposure times by using image frames captured with short exposure times and event information, applying blur kernels based on motion detection from event frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If exposure time is shortened to achieve high frame rates on mobile devices, then frame rate is improved, but motion blur is lost resulting in choppy motion
Solution Approach 1:
The system captures events continuously before, during, and after the frame exposure period. These preliminary event captures enable the reconstruction of motion blur effects that would normally require longer exposure times, allowing high frame rates to be maintained while preserving motion continuity through computational processing of the pre-captured event data.
2Manufacturing precision
If mechanical apertures or ND filters are used to control exposure, then exposure control is improved, but device complexity increases
Solution Approach 1:
The system replaces mechanical aperture and filter mechanisms with an event-based computational approach. Instead of using physical components to control exposure, the system uses event camera data to computationally generate motion blur effects, thereby maintaining exposure control capability while eliminating the need for complex mechanical lens assemblies.
Solution Approach 2:
The event camera acts as an intermediary between the scene and the frame camera. It captures motion information that is then used to modify the frame camera output, creating the appearance of proper exposure control and motion blur without requiring direct mechanical control of the frame camera's exposure parameters.
3Reliability
If event camera data is used to generate motion blur, then motion blur simulation is improved, but computational complexity increases
Solution Approach 1:
The system segments the event data processing by creating separate event frames for different time periods (before exposure, during exposure, after exposure). This segmentation allows the computational process to handle motion blur generation in discrete, manageable steps, reducing overall computational complexity while maintaining accuracy in the motion blur simulation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system for generating motion blur comprises: a frame camera, an event camera and an accumulator for accumulating event information from a plurality of events occurring within a window around the exposure time of an image frame in a plurality of event frames. A processor determines from the events in at least a first of the plurality of event frames, one or more areas of movement within the field of view of the event camera; determines from the events in at least a second of the plurality of event frames, a direction of movement for the one or more areas of movement; and applies blur in one or more areas of the image frame corresponding to the one or more determined areas of movement in accordance with at least the direction of movement for each of the one or more areas of movement to produce a blurred image.