3D Streaming System Using Point Clouds for Dynamic Scene Updates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current 3D streaming technologies often result in buffering, lag, and excessive latency, as they convert 3D image data into 2D images for transmission, losing dimensionality and detail, and struggle to handle dynamic and changing 3D scenes effectively.
Innovation Solution
The system streams true 3D image data using point clouds, where points are defined in 3D space with positional and descriptive elements, allowing for real-time updating and adaptive streaming that only transmits relevant data within the user's field-of-view, minimizing data transfer and maintaining scene integrity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If 3D image data is converted to 2D images for streaming, then bandwidth requirements are reduced, but dimensionality and detail are lost
Solution Approach 1:
The patent segments the 3D scene into multiple 2D images captured from different viewpoints around the object. These segmented views are then streamed separately and reconstructed by the client device to form the complete 3D scene, thus reducing bandwidth requirements while preserving dimensionality and detail through the collective information from multiple angles.
Solution Approach 2:
The patent uses multiple 2D images from different spatial dimensions (viewpoints) to represent and reconstruct a 3D scene. By capturing and streaming images from multiple dimensional perspectives rather than a single viewpoint, the system preserves three-dimensional information while transmitting two-dimensional data, effectively resolving the contradiction between bandwidth efficiency and dimensionality preservation.
2Loss of information
If 3D mesh or polygonal models are streamed, then true 3D data is transmitted, but data volume and processing complexity increase
Solution Approach 1:
The patent creates simplified 2D image copies of the 3D scene from multiple viewpoints instead of streaming the complete 3D mesh or polygonal models. These 2D image copies serve as proxies that contain sufficient information for reconstruction, significantly reducing data volume while maintaining scene integrity through the collective representation across multiple views.
3Quantity of substance
If 2D images are streamed to represent 3D scenes, then bandwidth is reduced, but buffering and lag increase
Solution Approach 1:
The patent performs preliminary rendering of multiple viewpoint images at the server side before streaming. By pre-processing and preparing the 2D images from different angles in advance, the system reduces the computational burden on client devices and minimizes processing delays during playback, thereby reducing buffering and latency while still using efficient 2D image transmission.
4Loss of information
If complete 3D scenes are rendered and streamed, then scene detail is preserved, but transmission time and latency increase
Solution Approach 1:
The patent extracts only the essential visual information needed to represent the 3D scene by converting it into multiple 2D images from key viewpoints. This extraction process removes unnecessary data (such as complete mesh structures, hidden surfaces, and redundant information) while retaining sufficient detail for accurate scene reconstruction, thereby reducing transmission time without significant loss of scene detail.
Data Source
AI summary
Disclosed is a system that streams true three-dimensional (“3D”) image data over a data network in a manner that preserves the dimensionality and detail of a dynamic and changing 3D scene. The system generates the 3D image data to represent the 3D scene, and streams different set of the 3D image data that are within different viewing frustums requested by different devices. The system generates updates to the 3D image data based on changes occurring at different parts of the 3D scene. The system streams a first update to the first device in response to image data updated by the first update being within the first device's viewing frustum, and streams a second update to the second device in response to image data updated by the second update being within the second device's viewing frustum.