Point Cloud Video Streaming Server Rendering
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing XR devices lack the computing power necessary to process and render point cloud-based digital twin content, making it challenging to stream and play such content in real-time.
Innovation Solution
A method and apparatus for point cloud video streaming that involves receiving pose information from a user terminal, changing the pose of a virtual camera in a virtual space, rendering an image texture of the point cloud video, and transmitting video data to the user terminal, thereby enabling the playback of point cloud-based content on lightweight user devices.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Manufacturing precision
If point cloud data is processed and rendered on user terminals, then high-quality 3D visualization is achieved, but computing power requirements exceed the capabilities of existing XR devices
Solution Approach 1:
A server acts as an intermediary between the point cloud data source and the user terminal. The server performs complex point cloud decoding, rendering, and view synthesis operations, then transmits only the final rendered video streams to user terminals. This mediator approach transfers computational burden from weak user terminals to powerful servers, enabling high-quality 3D visualization on resource-constrained devices.
Solution Approach 2:
Instead of transmitting raw point cloud data and requiring local rendering, the system creates rendered video copies of the point cloud content on the server side. Multiple view-angle specific video streams are generated as copies, which are then transmitted to user terminals for direct playback. This copying approach eliminates the need for user terminals to possess point cloud processing capabilities.
2Productivity
If complex compression technology is applied to point cloud data, then data transmission efficiency is improved, but processing complexity increases significantly
Solution Approach 1:
The complex compression and rendering operations are extracted from the user terminal and concentrated on the server side. The server handles all complex point cloud decoding, view synthesis, and video encoding operations, while user terminals only perform simple video decoding and playback. This extraction of complex processing functions resolves the contradiction by centralizing complexity where computational resources are abundant.
3Speed
If real-time rendering of point cloud videos is performed, then user interaction responsiveness is improved, but computational load on user terminals becomes unmanageable
Solution Approach 1:
The server serves as a real-time rendering intermediary that receives user pose information, dynamically synthesizes new views, and streams updated video content to user terminals. This mediator architecture enables real-time interactivity by performing all computationally intensive rendering operations on the server, keeping user terminals lightweight and responsive without burdening them with heavy computational loads.
Data Source
AI summary
Disclosed herein are a method and apparatus for point cloud video streaming. According to an embodiment, the method for point cloud video streaming may include receiving pose information from a user terminal changing the pose of a virtual camera in a virtual space where a point cloud video is played, by applying the pose information to the virtual camera, rendering an image texture of the point cloud video corresponding to a viewpoint of the virtual camera with the changed pose and transmitting video data with the rendered image texture to the user terminal.