Extendable Field of View Rendering via Camera Motion Metadata
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional video rendering methods fail to utilize camera motion effectively, resulting in lost content during playback and inefficient generation of panoramic videos, which can be resource-intensive for transmission and sharing.
Innovation Solution
The method involves generating an extended field of view composite by processing media data and sensor data to determine frame extensions based on camera motion, using metadata to adjust the field of view, and employing multiple media playback components to render the composite, allowing for efficient rendering of panoramic views without increasing video resolution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional video rendering is used, then video playback is simple, but camera motion content is lost and field of view cannot be extended
Solution Approach 1:
The system performs preliminary analysis of camera motion from sensor data and media metadata before rendering, identifying frames and portions where camera motion occurred. This preliminary action enables the system to pre-determine which frames need extension and from which source frames, so that when rendering occurs, the content recovery can be executed efficiently without complex real-time computations.
Solution Approach 2:
The rendering process is segmented into distinct components: analyzing camera motion from sensor data, identifying specific frames and portions affected by motion, determining extension parameters, and finally rendering the extended field of view composite. This segmentation allows each step to be optimized independently and simplifies the overall complex rendering task.
2Loss of information
If panorama stitching is used to generate larger resolution video, then field of view is extended, but transmission and sharing efficiency decreases due to larger file size
Solution Approach 1:
Instead of stitching all video frames to create a complete panoramic video, the system applies partial action by only extending the field of view for specific frames or portions where camera motion occurred. This selective approach recovers lost content during playback without generating a full-resolution panoramic video, thereby maintaining smaller file sizes while still providing extended field of view where needed.
Solution Approach 2:
The system applies different quality levels to different parts of the video output. Frames affected by camera motion receive extended field of view with additional recovered content, while frames without camera motion maintain standard resolution. This local quality approach ensures extended field of view coverage is provided only where necessary, optimizing the balance between information recovery and data quantity.
3Reliability
If frame extension is applied to recover lost content, then playback quality improves, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary analysis of camera motion and determines extension parameters before the actual rendering process. By pre-identifying which frames need extension, the source frames to use, and the appropriate extension parameters, the system avoids complex real-time computations during rendering, significantly reducing processing time while maintaining complete content recovery.
Solution Approach 2:
The system copies relevant portions from existing source frames to extend the field of view in target frames, rather than generating new content through complex computational methods. This copying approach, guided by pre-determined extension parameters, efficiently recovers lost content with minimal processing time while ensuring playback content completeness.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A method, apparatus and computer program product are provided for extendable field of view rendering. A method is provided for receiving media data including a plurality of video frames. The method also includes determining, with a processor, video frames corresponding to camera motion and determining speed and duration of the camera motion. The method also includes determining at least one frame extension based on the video frames corresponding to camera motion, determining a frame extension offset based on the camera motion, and generating an extended field of view metadata based on the frame extensions and frame extension offsets.