Occlusion Simulation via Point Cloud Meshing in AR
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current virtual and augmented reality systems fail to accurately simulate the occlusion of virtual objects by physical objects, leading to a less immersive user experience as they do not replicate the interactions of physical objects in the real world effectively.
Innovation Solution
A system comprising a client device with physical processors, distancing devices, and image-forming components that generate a mesh based on point cloud data from real-world objects, allowing for accurate portrayal of physical objects occluding virtual objects by determining the position and movement of user objects and modifying virtual object images accordingly.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If virtual objects are displayed in augmented reality environments, then users can interact with virtual content, but the experience lacks realism because physical objects do not occlude virtual objects
Solution Approach 1:
The system performs preliminary actions by capturing depth information and generating meshes of physical objects before virtual objects are rendered. The distancing device continuously maps the physical environment, creating a digital representation that is ready to interact with virtual content. This preliminary preparation enables realistic occlusion effects without adding complexity during the actual rendering phase.
Solution Approach 2:
The patent introduces an intermediary mesh representation of physical objects that mediates between the real world captured by distancing devices and the virtual objects rendered by the display system. This mesh acts as a digital proxy that enables occlusion calculations, allowing virtual objects to be realistically blocked by physical objects without requiring direct complex interactions between physical and virtual elements.
2Measurement precision
If the system tracks and processes real-world object positions to simulate occlusion, then occlusion accuracy improves, but computational requirements and processing time increase
Solution Approach 1:
The system performs preliminary actions by capturing depth information and generating meshes of physical objects before virtual objects are rendered. The distancing device continuously maps the physical environment, creating a digital representation that is ready to interact with virtual content. This preliminary preparation enables realistic occlusion effects without adding complexity during the actual rendering phase.
Solution Approach 2:
The patent applies partial action by selectively processing only the portions of the physical environment that are relevant to occlusion. Rather than processing all spatial data at full resolution, the system focuses computational resources on generating meshes for objects that will interact with virtual content, reducing overall processing time while maintaining occlusion accuracy where it matters.
Data Source
AI summary
A system and method to simulate physical objects occluding virtual objects within an interactive space may obtain output signals conveying three-dimensional positions of surfaces of a user object. The three-dimensional positions may be conveyed in a point cloud. Point cloud images may be obtained from the output signals. An individual point cloud image may depict a two-dimensional representation of the point cloud. A current point cloud image may be aggregated with one or more previous aggregate point cloud images to generate a current aggregate point cloud image. A mesh may be generated based on the current aggregate point cloud image. The mesh may include a set of vertices. The mesh may be generated by assigning individual vertices to individual points in the current aggregate point cloud image.