Frame Interpolation with Occluded Motion for Video Post-Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional post-processing techniques for motion blur in video capture often conflict with slow-motion and stabilization settings, leading to unnatural motion blur effects, especially when capturing high-frame-rate video intended for playback at lower frame rates.
Innovation Solution
A combination of vector blur and frame interpolation with stacking techniques is used in post-processing to generate motion blur, allowing for adaptive rendering based on motion analysis and optical flow, which preserves foreground and background information and reduces artifacts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If conventional post-processing techniques are used to add motion blur, then motion blur effect is achieved, but unnatural artifacts and ghosting occur especially in slow-motion and stabilized video
Solution Approach 1:
The patent segments the video processing into multiple layers: a stabilized layer processed with electronic image stabilization and a motion blur layer processed separately with motion blur effects. By separating these processing streams and selectively applying them to different portions of the video, the technique avoids the ghosting and artifacts that occur when conventional single-stream processing is used on slow-motion and stabilized content.
Solution Approach 2:
The patent applies electronic image stabilization preliminarily to the video frames before adding motion blur effects in post-processing. By pre-stabilizing the footage and then applying motion blur selectively to moving objects while preserving the stabilized background, the technique achieves natural-looking motion blur without the ghosting artifacts that plague conventional approaches.
2Speed
If high frame rate video is captured for slow-motion effects, then slow-motion playback is achieved, but natural motion blur is reduced due to fast shutter speed
Solution Approach 1:
The patent captures video at high frame rates preliminarily to enable slow-motion playback, then applies motion blur effects in post-processing to compensate for the reduced natural motion blur. By capturing the high-speed footage first and then adding artificial motion blur selectively, the technique achieves both slow-motion effects and natural-looking motion blur that would be impossible to capture naturally at such high frame rates.
Solution Approach 2:
The patent changes the shutter speed parameter in post-processing to simulate different exposure times and motion blur effects. By adjusting the effective shutter speed parameter during post-processing of high-frame-rate footage, the technique can create natural-looking motion blur that corresponds to what would have been captured at lower frame rates with slower shutter speeds.
3Stability of the object's composition
If electronic image stabilization is applied, then camera shake is reduced, but motion blur conflicts with stabilization settings
Solution Approach 1:
The patent segments the video into stabilized portions and moving object portions, applying electronic image stabilization to the overall footage while selectively applying motion blur only to identified moving objects. This segmentation allows the stabilized background to remain sharp and stable while moving objects receive natural-looking motion blur, resolving the conflict between stabilization and motion blur effects.
Solution Approach 2:
The patent uses motion detection and optical flow analysis as intermediary processes to identify which portions of the stabilized video should receive motion blur effects. These intermediary analysis tools detect moving objects within the stabilized footage and serve as a bridge between the stabilization process and the motion blur application, ensuring that motion blur is applied only where appropriate and not to the stabilized background.
Data Source
AI summary
Systems, apparatus, and methods adding post-processing motion blur to video and/or frame interpolation with occluded motion. Conventional post-processing techniques relied on the filmmaker to select and stage their shots. Different motion blur techniques were designed to fix certain types of footage. Vector blur is one technique that “smears” pixel information in the direction of movement. Frame interpolation and stacking attempts to create motion blur by stacking interpolated frames together. Each technique has its own set of limitations. Various embodiments use a combination of motion blur techniques in post-processing for better, more realistic outcomes with faster/more efficient rendering times. In some cases, this may enable adaptive quality post-processing that may be performed in mobile/embedded ecosystems. Various embodiments use a combination of video frame interpolation techniques for better interpolated frames with faster/more efficient rendering times.