360 Video and 3D Object Integration in XR Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies lack effective solutions for delivering and rendering 360 video media alongside 3D objects in the same space, particularly in 6G communication systems, where seamless integration and user pose-based rendering are required.
Innovation Solution
The method involves obtaining and determining 360 video data based on user pose information, composing it with scene objects to create a 3D scene, and using metadata to describe the 360 video space and components within the scene description, enabling view synthesis and support for multiple 360 videos in various rendering modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If 360 video and 3D objects are delivered and rendered separately, then existing technologies can be used, but seamless integration in the same space cannot be achieved
Solution Approach 1:
The patent merges 360 video delivery with 3D object delivery by integrating them into a unified scene description format. The 360 video is represented as a textured sphere object within the scene graph, allowing both 360 video and 3D objects to be described, delivered, and rendered through the same system infrastructure, achieving seamless integration without requiring separate processing pipelines.
2Ease of operation
If traditional video rendering is used, then rendering pipelines are simple, but user pose-based rendering for immersive experiences cannot be provided
Solution Approach 1:
The patent implements dynamic rendering by continuously tracking user pose information (head position and orientation) and adjusting the rendered viewport of the 360 video in real-time. The scene description includes viewport parameters that are dynamically updated based on user pose, enabling the rendering system to adapt to changing user perspectives while maintaining a coherent integration with 3D objects.
3Adaptability or versatility
If multiple 360 videos are supported, then immersive experiences are enhanced, but scene description complexity increases
Solution Approach 1:
The patent segments the 360 video representation into multiple textured sphere objects within the scene graph, each representing a different 360 video source or viewpoint. Each sphere object can be independently configured with its own texture, position, and viewport parameters, allowing the system to support multiple 360 videos while maintaining a modular and manageable scene description structure.
Data Source
AI summary
According to an embodiment of the disclosure, the method for supporting 360 video performed by a XR device includes obtaining a plurality of 360 video data, determining a 360 video to be displayed. based on a user pose information. determining a scene object. based on a media input and composing a 3D scene the 360 video and the scene object.