Pattern-Triggered AR Object Modification With Contour-Anchored Occlusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Mixed reality applications face challenges in seamlessly integrating virtual objects with physical objects, particularly when related to human bodies, leading to an 'uncanny valley' effect and inaccuracies in object movement and transformation.
Innovation Solution
A system that uses a convolutional neural network to detect and reconstruct physical objects, generating virtual representations anchored to their contours, and displays these objects in mixed reality environments, utilizing on-device and networked computing resources for enhanced accuracy and versatility.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If virtual objects are displayed to occlude physical objects, then the virtual object integration is improved, but the uncanny valley effect increases
Solution Approach 1:
The patent applies local quality by making different parts of the virtual object have different occlusion properties. The virtual object uses semi-transparent or translucent materials in certain regions to allow partial visibility of the physical object underneath, rather than complete occlusion. This creates a gradient effect where the virtual object blends more naturally with the physical environment, reducing the uncanny valley effect while maintaining integration.
Solution Approach 2:
The patent employs composite materials by combining virtual objects with translucent or transparent material properties with opaque regions. This creates a composite visual effect where the virtual object appears to be partially made of glass or translucent material, allowing the physical object to be seen through certain portions. This composite approach reduces the unnatural appearance of complete occlusion while maintaining the virtual object's presence.
2Measurement precision
If convolutional neural network is used for object detection, then the object detection accuracy is improved, but the computing resource requirements increase
Solution Approach 1:
The patent applies segmentation by dividing the computing task into multiple stages. First, a lightweight model performs initial object detection and candidate identification. Then, a more sophisticated convolutional neural network is applied only to the detected candidate objects for detailed analysis and contour determination. This segmented approach reduces the overall computing burden while maintaining high detection accuracy for the final output.
Solution Approach 2:
The patent implements preliminary action by performing preprocessing steps before the main convolutional neural network processing. This includes initial object candidate identification, contour detection, and feature extraction that prepare the data in advance. By performing these preliminary actions, the main CNN model receives pre-processed input that requires less computational effort to achieve high accuracy, thereby reducing overall computing resource requirements.
Data Source
AI summary
Provided is a system configured to obtain a set of images via a camera of the computing device, input the set of images into a neural network, and detect a target physical object with the neural network. The system may determine a contour of the target physical object and a first three-dimensional reconstruction of the target physical object. The system may generate a virtual representation and a virtual object based on attributes of the virtual representation, where a first attribute of the set of attributes includes the first three-dimensional reconstruction. The system may associate the virtual object with the virtual representation and displays the virtual object at pixel coordinates of a display that at least partially occlude at least part of the target physical object, where a position of the virtual object is computed based on the contour.


