Video Stream Encoding with No-Display Frames for FOV Changes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing video coding algorithms struggle with efficiently encoding video streams when the camera's field-of-view (FOV) changes, particularly during operations like zooming, leading to increased computational demands and unrealistic motion vectors.
Innovation Solution
Generate additional video frames that reference previous frames using motion vectors derived from camera FOV changes, eliminating the need for extensive search and reducing bitrate by inserting these frames as no-display frames in the encoded stream.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If motion vectors are used to encode video frames during camera zooming, then encoding efficiency is maintained, but computational time increases and motion vectors become unrealistic
Solution Approach 1:
The video stream is segmented into regular video frames and additional no-display frames during FOV changes. This segmentation allows the system to handle zooming operations differently from normal video capture, enabling efficient encoding by separating the FOV transformation step from the regular motion-compensated prediction steps.
Solution Approach 2:
Additional no-display frames act as intermediaries between frames captured at different FOVs. These intermediate frames contain the transformed image content at the new FOV, serving as a bridge that allows subsequent frames to use standard motion vector techniques without requiring excessively long search vectors across large FOV differences.
2Measurement precision
If search area is enlarged to find motion vectors during FOV changes, then motion vector accuracy improves, but computational load increases
Solution Approach 1:
The system performs preliminary FOV transformation by generating additional no-display frames that contain the image content already transformed to the new field of view. This preliminary action prepares the data in advance, so that when motion vectors are subsequently calculated, the search can be performed over a reasonable area rather than requiring an excessively large search space, thus reducing computational load while maintaining accuracy.
3Manufacturing precision
If additional video frames are inserted to account for FOV changes, then encoding accuracy during zooming improves, but bitrate increases
Solution Approach 1:
The additional frames inserted during FOV changes are marked as no-display frames, meaning they are not rendered or displayed but are used solely for encoding purposes. These frames serve their function as temporary computational aids to enable accurate motion compensation during zooming, then are discarded from the final output stream, thus improving encoding accuracy without permanently increasing the transmitted bitrate.
Data Source
AI summary
A method of encoding a video stream is provided, including obtaining a first image with a first FOV; encoding the first image as part of a first encoded video frame; obtaining a second image with a second FOV different from the first FOV; generating a first additional video frame referencing the first video frame, including motion vectors transforming image content of the first image to a FOV closer to the second FOV than the first FOV, wherein the motion vectors are formed based on a difference between the first and second FOVs; inserting the first additional video frame into the encoded video stream as a no-display frame, and encoding the second image as part of a second video frame of the encoded video stream referencing the first additional video frame. A corresponding device, computer program and computer program product are also provided.