Optical Flow Estimation Using Event Frames for Nonlinear Motion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional optical flow estimation methods assume linear motion between adjacent image frames, leading to inaccurate estimation of optical flows to any moment between the frames due to actual non-linear motions in scenes.
Innovation Solution
An optical flow estimation method utilizing a pixel camera and an event-based camera to capture pixel and luminance changes, employing a lightweight convolutional neural network for accurate estimation of optical flows by incorporating event frames, which capture both linear and non-linear motions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If linear motion assumption is used to allocate optical flows between adjacent frames, then the calculation is simple, but the accuracy of optical flow estimation is low
Solution Approach 1:
The patent introduces event data as an intermediary element between adjacent image frames. Event data captures luminance changes at high temporal resolution and serves as a mediator to guide the allocation of optical flows, enabling accurate representation of non-linear motion without complex calculations
Solution Approach 2:
The patent changes the parameter used for optical flow allocation from uniform time-based weighting to event-driven dynamic weighting. By using event data to determine weight values, the system adapts to actual motion patterns, improving accuracy while maintaining computational efficiency
2Measurement precision
If event-based camera is used to capture luminance changes, then the accuracy of motion information is improved, but the device complexity increases
Solution Approach 1:
The patent merges data from two different sensor types (pixel camera and event-based sensor) into a unified optical flow estimation framework. By combining image frame data with event data, the system achieves high-accuracy motion capture while leveraging the complementary strengths of both sensors
Solution Approach 2:
The patent creates a multi-functional system where event data serves multiple purposes: capturing high-speed motion, guiding optical flow allocation, and providing temporal information. This universal use of event data maximizes the value of the additional sensor, justifying the increased device complexity
Data Source
Figure 1(a)~1(b)
Figure 2(a)~3
Figure 4
AI summary
This application provides an optical flow estimation method and apparatus, to improve accuracy of estimating optical flows from two adjacent image frames to any moment between the two adjacent image frames. The method may include: obtaining a first image frame and a second image frame, where the first image frame and the second image frame are any two adjacent image frames in an image sequence, and the image sequence is obtained by photographing a target scene; obtaining a first event frame, where the first event frame is used to describe a luminance change of the target scene within a time period from the first image frame to the second image frame; and determining a target optical flow based on the first image frame, the second image frame, and the first event frame, where the target optical flow is an optical flow from the first image frame to a target moment, and the target moment is any moment between the first image frame and the second image frame.