Virtual Surface Overlay Using Stereo Vision Depth Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for overlaying 3D objects with virtual surfaces suffer from low resolution, noise, and latency, making real-time applications like video conferences difficult due to the limitations of optical reference determination and visible reference points, which restrict the accuracy and extent of the overlay.
Innovation Solution
A device using a contactless position determiner to achieve absolute position measurement of reference points, combined with stereo cameras to generate a high-resolution 3D grid structure, allowing for precise alignment and overlay of virtual surfaces with detailed surface information, enabling real-time rendering and seamless transitions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If optical reference determination with glued dots is used, then the overlay can be achieved, but the resolution is low and the reference points are visible in the overlay
Solution Approach 1:
The patent removes the need for physical reference markers (dots) from the scene by extracting the reference determination function to a separate time domain. Instead of spatially embedding markers in the 3D scene, the system uses temporal correlation of image sequences to automatically identify and track reference points, eliminating visible markers while maintaining high precision overlay.
Solution Approach 2:
The patent replaces the mechanical/optical reference marker system with an automated image processing system. Instead of using physical dots that reflect infrared light, the system uses algorithms to detect natural features in the image sequence, compute their 3D positions through stereo vision, and use these for overlay registration, achieving higher resolution without visible markers.
2Measurement precision
If facial recognition is used to track reference points, then the overlay can be adjusted, but the processing speed is slow causing latency
Solution Approach 1:
The patent segments the reference point tracking into two distinct phases: an initial comprehensive facial recognition phase to identify and establish reference points, followed by a lightweight tracking phase that only monitors the already-identified points. This segmentation allows accurate initial positioning while maintaining high-speed continuous tracking, reducing overall latency.
Solution Approach 2:
The patent performs preliminary facial recognition and reference point identification in advance before the overlay process begins. By pre-establishing the reference points and their 3D positions, the system avoids the computational burden of continuous full-face recognition during real-time overlay, enabling low-latency performance while maintaining high tracking accuracy.
3Ease of manufacture
If conventional 3D mesh animation is used with few dots, then the processing is simple, but the model lacks lifelike quality
Solution Approach 1:
The patent transitions from sparse 3D point-based animation to dense pixel-level 3D surface reconstruction. By utilizing the full image resolution and computing depth information for numerous pixels through stereo vision, the system creates high-resolution 3D surfaces that capture fine facial details, enabling lifelike overlay quality while maintaining computational efficiency through optimized processing pipelines.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution provides high-resolution, real-time overlay of virtual surfaces with minimal latency, allowing for lifelike animations and precise alignment, suitable for applications like video conferencing and virtual reality, without the need for infrared lighting or visible markers.
Implementation Method 1
The absolute position determination can be carried out, for example, using sonar or another contactless positioning method.
Implementation Method 2
The computing unit, based on the absolute position of the reference point, determines depth information from a plurality of voxels or vertices in a 3D image of the object
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device is described for superimposing at least one part of an object with a virtual surface. The device comprises a position-determining means which is designed to determine an absolute position of a reference point on the object in an environment. Furthermore, the device comprises a computing unit which determines depth information about a plurality of voxels or vertices in a 3D image of the object on the basis of the absolute position of the reference point, generates a 3D grid structure of the object, and superimposes the virtual surface of the 3D grid structure.