Volumetric Plane Tracking for Sensor-Light AR Surface Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) devices face challenges in accurately estimating the locations, orientations, and dimensions of real-world objects due to computational limitations and the need for specialized depth sensors, which are not suitable for small, user-friendly AR devices.
Innovation Solution
An AR device equipped with a camera and an inertial measurement unit (IMU) uses visual inertial odometry (VIO) to estimate trajectory and orientation, generates depth maps, and employs a truncated signed distance function (TSDF) to represent 3D space, fitting local planes to voxel blocks and merging them into larger surfaces, dynamically updating as the environment changes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If depth sensors such as LIDAR are used to estimate surface characteristics, then measurement precision is improved, but device complexity and cost increase
Solution Approach 1:
The patent creates a 3D volumetric representation (copy) of the environment using only 2D camera images and IMU data. Instead of directly measuring depth with specialized sensors, the system reconstructs depth information by tracking visual features across multiple frames and integrating them with inertial measurements, producing a TSDF voxel grid that replicates the physical environment in digital space.
Solution Approach 2:
The patent replaces the mechanical/optical depth sensing system (LIDAR) with a computational approach combining vision algorithms and inertial measurement. The VIO system substitutes direct range measurement with indirect depth estimation through visual feature tracking and sensor fusion, eliminating the need for complex active depth sensing hardware.
2Measurement precision
If specialized depth sensors are used, then measurement precision is improved, but ease of manufacture worsens
Solution Approach 1:
The patent makes the camera system multi-functional by using it for both 2D visual recognition and 3D depth estimation. The same imaging sensor that captures standard photographs is also used to track visual features for depth reconstruction, eliminating the need for separate depth sensing hardware and simplifying the bill of materials and assembly process.
Solution Approach 2:
The system creates a digital copy of the physical environment using only standard imaging components. By reconstructing 3D spatial information from 2D images through computational methods, the patent avoids incorporating specialized depth sensing hardware, thereby simplifying manufacturing while maintaining depth estimation capability.
3Device complexity
If computational methods are used to estimate depth from color camera images, then device complexity is reduced, but measurement precision may worsen
Solution Approach 1:
The patent employs dynamic visual feature tracking across multiple camera frames to estimate depth. Instead of relying on static single-frame analysis, the system continuously updates depth estimates by tracking the motion of visual features through time, integrating these observations with dynamic IMU data to produce accurate and temporally consistent depth maps.
Solution Approach 2:
The VIO system uses feedback from both visual feature tracking and IMU measurements to continuously refine depth estimates. The system compares predicted feature positions based on IMU data with actual observed positions in camera images, using the discrepancy to correct depth and pose estimates, thereby improving measurement precision through iterative feedback loops.
Data Source
AI summary
A system is disclosed, including a processor and a memory. The memory stores instructions that, when executed by the processor, configure the system to perform operations. Depth estimates are used to generate distance values by applying a signed distance function to the depth estimates. A 3D representation of the environment is generated using the distance estimates. Local planes are fit to the 3D representation, and larger planes are generated by merging local planes using predefined criteria such as surface normal agreement. Larger planes are dynamically updated or removed in response to updated depth estimates.