3D Environment Representation for Immersive Communication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing communication technologies do not adequately facilitate the sharing of 3D environments during communication sessions, limiting the immersive experience for users.
Innovation Solution
The method involves generating and transmitting a 3D representation of a physical environment by selectively replacing certain elements with visual features, using machine learning for semantic labeling, and supplementing with additional detailed content, allowing for a more efficient and immersive communication experience.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a complete 3D representation of the physical environment is transmitted, then the immersive experience and detail quality are improved, but the data size increases and communication efficiency deteriorates
Solution Approach 1:
The patent extracts only the essential visual features and semantic information from the complete 3D environment representation. Instead of transmitting all geometric details, it identifies and transmits only the key elements that define the environment's structure and appearance, such as wall positions, floor/ceiling planes, and major object locations, thereby reducing data size while maintaining immersive quality
Solution Approach 2:
The environment representation is segmented into distinct semantic categories (walls, floors, ceilings, furniture, decorations) and further divided into essential visual features versus optional detailed elements. This segmentation allows selective transmission of critical information that preserves environmental understanding while eliminating redundant data
2Loss of information
If all elements of the 3D representation are transmitted, then the completeness of the environment model is improved, but the data transmission time and processing load increase
Solution Approach 1:
The system extracts essential environmental structure information (walls, floors, ceilings forming the room boundary) and transmits it as geometric primitives, while excluding or simplifying non-essential decorative elements. This extraction approach maintains environmental completeness for functional understanding while dramatically reducing transmission time
Solution Approach 2:
The patent changes the representation parameters from high-detail mesh models to simplified geometric primitives (planes, boxes, cylinders) with key semantic attributes. This parameter transformation reduces data complexity and transmission requirements while preserving the essential spatial and visual characteristics of the environment
3Loss of information
If detailed 3D models of all objects are transmitted, then the visual quality and realism are improved, but the data size and rendering complexity increase
Solution Approach 1:
Instead of transmitting complete detailed 3D models of all objects, the system creates simplified visual feature copies that capture essential appearance and spatial information. These copies use basic geometric shapes with key visual attributes (color, texture, position) rather than full geometric detail, reducing rendering complexity while maintaining visual quality
Solution Approach 2:
The patent applies different levels of detail selectively to different elements based on their importance. Critical structural elements (walls, floors) receive accurate geometric representation, while less important elements use simplified primitives. This local quality differentiation maintains visual quality where needed while reducing overall rendering complexity
Data Source
AI summary
Various implementations disclosed herein include devices, systems, and methods that generate and share/transmit a 3D representation of a physical environment during a communication session. Some of the elements (e.g., points) of the 3D representation may be replaced to improve the quality and/or efficiency of the modeling and transmitting processes. A user's device may provide a view and/or feedback during a scan of the physical environment during the communication session to facilitate accurate understanding of what is being transmitted. Additional information, e.g., a second representation of a portion of the physical environment, may also be transmitted during a communication session. The second representations may represent an aspect (e.g., more details, photo quality images, live, etc.) of a portion not represented by the 3D representation.