Video Encoder Subframe Segmentation for Motion Blur and Resolution Trade-off
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-frame-rate video capture in existing technologies results in a tradeoff between capturing scene details and accurately capturing motion, due to the need for short exposure times which lead to reduced resolution and increased bandwidth, while longer exposure times cause blurring.
Innovation Solution
A method involving an image sensor with pixels divided into subframes with alternating short and long exposure times and temporal offsets, where long-exposure low-resolution images are combined with short-exposure low-resolution images to create subframe residual images, and these are encoded into a multi-layer video stream, allowing for higher resolution without the bandwidth costs of traditional high frame-rate video.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If short exposure time is used for high-frame-rate video capture, then motion accuracy is improved, but resolution deteriorates
Solution Approach 1:
The patent divides the image sensor into multiple subframes (first subframe, second subframe, etc.) that are captured at different exposure times. The first subframe uses short exposure time for motion accuracy, while the second subframe uses long exposure time for resolution. This segmentation allows both requirements to be satisfied simultaneously without compromising either aspect.
2Manufacturing precision
If long exposure time is used, then resolution is improved, but motion accuracy deteriorates due to blur
Solution Approach 1:
The image sensor is segmented into multiple subframes with different exposure times. The second subframe uses long exposure time to capture detailed scene information, while the first subframe captures motion information. This segmentation resolves the contradiction by assigning different exposure characteristics to different subframes based on their specific capture requirements.
3Measurement precision
If high-frame-rate video is captured, then motion accuracy is improved, but bandwidth cost increases
Solution Approach 1:
The patent merges the first subframe (short exposure, motion information) and the second subframe (long exposure, detail information) to form a complete video frame. This merging allows the system to achieve high-frame-rate motion accuracy while reducing overall bandwidth requirements by utilizing the complementary strengths of both subframes rather than transmitting separate high-rate streams.
4Device complexity
If single exposure speed is used, then device complexity is simplified, but tradeoff between detail capture and motion capture occurs
Solution Approach 1:
The patent implements dynamic exposure control where different subframes of the same sensor can have different exposure times. The first subframe uses short exposure time optimized for motion, while the second subframe uses long exposure time optimized for detail. This dynamic adjustment of exposure parameters across different subframes resolves the tradeoff between detail capture and motion capture without requiring completely different hardware systems.
Data Source
AI summary
An image sensor includes a plurality of pixels, each pixel belonging to one of N subframes each characterized by (a) a same exposure-time sequence that includes a short exposure-time alternating with a long exposure-time, and (b) a respective temporal offset equal to a multiple of the short exposure-time. A method for encoding a video stream captured by the image sensor includes (i) for each subframe, linearly combining a long-exposure image, captured at the long exposure-time, and a short-exposure image, captured at the short exposure-time, to yield a residual image, (ii) combining at least some of the long-exposure images from the N subframes to yield a full-frame image having a higher resolution than any long-exposure image, (iii) encoding the full-frame image into a base layer of the video stream, and (iv) encoding at least some of the residual images from the N subframes into an enhancement layer of the video stream.