Composite Video Stream Generation for Volumetric Content
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in efficiently streaming and decoding volumetric videos in Virtual Reality (VR) and Augmented Reality (AR) environments due to high bandwidth and computational complexity, especially when combined with non-volumetric videos, as existing compression techniques are not optimized for streaming and require separate decoding processes.
Innovation Solution
A processor system generates a composite video stream by determining the spatial subregion of a non-volumetric video occluded by a volumetric video and replaces that region with a non-volumetric representation of the volumetric video, allowing for a single composite stream to be transmitted and decoded, reducing bandwidth and computational requirements.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If volumetric video and non-volumetric video are transmitted as separate streams, then the client device can decode both videos independently, but the bandwidth consumption and computational complexity increase significantly
Solution Approach 1:
The patent combines volumetric video data and non-volumetric video data into a single composite video stream. The server replaces occluded regions in the non-volumetric video with corresponding regions from the volumetric video, creating one integrated stream that reduces bandwidth consumption while allowing the client to process both video types through a unified decoding pipeline
2Loss of information
If volumetric video and non-volumetric video are transmitted as separate streams, then the complete video content is preserved, but the computational complexity for decoding both streams increases
Solution Approach 1:
The patent merges the decoding process by integrating volumetric and non-volumetric video into a single stream. The client device decodes one composite stream rather than managing two separate decoding processes, reducing computational complexity while preserving complete video content through the occlusion-replacement methodology
Solution Approach 2:
The patent extracts only the necessary volumetric video regions that are not occluded by the non-volumetric video, and replaces only those specific occluded regions in the composite stream. This selective extraction and replacement approach minimizes the amount of additional data processing required at the client device while maintaining complete video content
3Quantity of substance
If compression techniques are applied to volumetric video, then the data size is reduced, but the compression techniques are not optimized for streaming and require significant storage and bandwidth
Solution Approach 1:
The patent combines compressed volumetric video data with non-volumetric video data into a single composite stream during server-side processing. This merging approach allows the use of existing video compression standards for both video types while eliminating the need for specialized volumetric video streaming optimization, as the composite stream can be handled by standard video streaming infrastructure
Solution Approach 2:
The patent performs the complex volumetric video processing, occlusion detection, and region replacement operations in advance at the server before streaming. By completing these computationally intensive operations beforehand, the patent reduces the processing burden during actual streaming transmission, making the streaming process more efficient and easier to implement with standard infrastructure
Data Source
Figure 1A~1B
Figure 1C
Figure 2A~2B
AI summary
A processor system and computer-implemented method are provided for generating a composite video stream which may include a non-volumetric video and a volumetric video, at least part of which is to be displayed in front of the non-volumetric video. The volumetric video may be included in the composite video stream in the form of a non-volumetric representation of the volumetric video, for example inserted into a spatial subregion of the non-volumetric video which may be occluded by the volumetric video during display. The encoding, transmission and decoding may thus not have to be modified to support volumetric video. Signaling data may be provided which may be indicative of the composite video stream containing the non-volumetric representation of the volumetric video. A processor system and computer-implemented method may be provided for rendering the composite video stream using the signaling data.