XR Spatial Scanning With 2D-to-3D Object Positioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing extended reality (XR) systems face challenges in efficiently and computationally lightweight detection of physical objects in a real-world scene for accurate anchoring of virtual objects, as conventional 3D data processing is time-consuming and intensive.
Innovation Solution
An XR system captures video frame data and determines 3D positions of physical objects using 2D positions and depth data, optionally involving an object identification service, to provide lightweight and efficient detection.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional 3D data processing is used for object detection, then measurement precision is improved, but productivity deteriorates due to time-consuming and computationally intensive processing
Solution Approach 1:
The patent introduces an intermediary service (object identification service or XR system service) that processes video frame data to determine 3D positions of physical objects. This intermediary handles the computationally intensive 3D data processing, allowing the head-wearable apparatus to achieve accurate object detection without bearing the full computational burden, thus resolving the contradiction between detection precision and processing speed.
Solution Approach 2:
The patent replaces traditional mechanical/computational 3D data processing methods with alternative approaches including: (1) using 2D position data combined with depth data from the environment map instead of full 3D point cloud processing, (2) leveraging pre-generated environment maps to avoid real-time 3D reconstruction. This substitution reduces computational intensity while maintaining detection accuracy.
2Measurement precision
If full 3D data processing is performed, then object detection accuracy is improved, but use of energy deteriorates due to computational intensity
Solution Approach 1:
The patent extracts only the necessary information from video frame data - specifically 2D positions of physical objects combined with depth data from pre-generated environment maps - rather than processing complete 3D point clouds. This extraction approach maintains object detection accuracy while significantly reducing computational energy consumption by processing only essential data elements.
Solution Approach 2:
The patent performs preliminary processing by generating environment maps containing depth data before the actual object detection task. This preliminary action allows the system to avoid computationally intensive real-time 3D reconstruction during object detection, thereby reducing energy consumption while maintaining detection accuracy through efficient lookup and matching operations.
3Productivity
If lightweight object detection is implemented, then productivity is improved, but measurement precision deteriorates
Solution Approach 1:
The patent transitions from traditional 3D point cloud processing to a hybrid approach using 2D position data from video frames combined with depth information from pre-generated environment maps. This dimensional transformation allows the system to achieve lightweight processing (improved productivity) while maintaining accurate 3D position determination (preserved measurement precision) by leveraging depth data without performing full 3D reconstruction.
Data Source
AI summary
An extended Reality (XR) system that provides services for determining 3D data of physical objects in a real-world scene. The XR system receives a request from an application to initiate a spatial scan of a real-world scene. In response, the XR system captures video frame data of the real-world scene and captures a pose of the XR system. The XR system determines a physical object in the real-world scene and determines a 2D position of the physical object, using the video frame data. The XR system determines a depth of the physical object using the 2D position and determines a 3D position of the physical object in the real-world scene using the 2D position of the physical object, the depth of the physical object, and the pose of the XR system. The XR system communicates the 3D position data to the application.