Multi-View Media Streaming with Pre-Synthesized Virtual Viewpoints
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing free-viewpoint video technologies face challenges in maintaining playback quality while conserving bandwidth resources, as stitching methods cause image quality loss and real-time synthesis requires high equipment performance.
Innovation Solution
Incorporating descriptions of real and virtual viewpoints in a Media Presentation Description (MPD) text, allowing terminal devices to download media frames efficiently and reduce view interaction delay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If stitching scheme is used to process free-viewpoint videos, then the playback quality is preserved, but the transmission bandwidth is significantly occupied
Solution Approach 1:
The patent segments the media stream into multiple independent viewpoint streams (real viewpoints and virtual viewpoints) and organizes them in the MPD text. This allows the terminal to download only the specific viewpoint data needed, rather than transmitting all viewpoint data, thereby reducing bandwidth consumption while maintaining playback quality.
Solution Approach 2:
The patent performs preliminary generation of virtual viewpoints and pre-organization of media data into viewpoint-specific segments during the encoding phase. The MPD text is prepared in advance with descriptions of all available viewpoints, enabling the terminal to efficiently select and download only the required viewpoint data, reducing real-time bandwidth requirements.
2Loss of energy
If real-time view synthesis scheme is used, then the bandwidth resources are conserved, but the playback effect cannot be ensured and high equipment performance is required
Solution Approach 1:
Virtual viewpoints are generated in advance during the media encoding phase rather than in real-time during playback. This preliminary synthesis allows complex view synthesis computations to be performed when computational resources are abundant, ensuring high playback quality without requiring high equipment performance during the playback phase.
Solution Approach 2:
The patent creates virtual viewpoint copies by synthesizing virtual views from multiple real viewpoint media streams. These virtual viewpoint copies are then included in the MPD text, allowing the terminal to select from pre-synthesized high-quality virtual viewpoints without performing real-time synthesis, thus ensuring playback quality while conserving bandwidth.
3Stability of the object's composition
If virtual viewpoints are introduced to reduce parallax, then the video jitter during view switching is reduced, but the complexity of media processing increases
Solution Approach 1:
The patent segments the media data into distinct real viewpoint streams and virtual viewpoint streams, with clear identification in the MPD text. This segmentation allows the terminal to efficiently manage and switch between different viewpoint types without complex processing, as each viewpoint is independently organized and accessible.
Solution Approach 2:
The MPD text serves as an intermediary structure that organizes and describes multiple viewpoint streams (both real and virtual). It provides a standardized interface between the complex multi-viewpoint media data and the terminal device, simplifying the terminal's task of selecting and downloading appropriate viewpoint data while maintaining view switching stability.
Data Source
AI summary
A media information processing method and apparatus, a media information playback method and apparatus are disclosed. The media information processing method may include: acquiring media of a plurality of viewpoints, the plurality of viewpoints include at least one real viewpoint and at least one virtual viewpoint, and the media of the at least one virtual viewpoint is generated according to the media of the at least one real viewpoint; generating a media stream according to the media of the plurality of viewpoints, the media stream is a media file including media information; and performing segmentation and packing of the media stream, and generating a Media Presentation Description (MPD) text of the media, the MPD text includes a description of the real viewpoints and a description of the at least one virtual viewpoint, and the MPD text is used for acquiring media stream information of a viewpoint to be played.