AR Surface Painting 3D Object Mapping
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality devices struggle to maintain the accuracy and realism of user-generated paintings on objects when the perspective changes, leading to inconsistencies in size, shape, and position, which negatively impacts the user experience.
Innovation Solution
The implementation of a surface painting component that identifies and adjusts the three-dimensional space of objects within the visual scene, allowing users to paint objects persistently across different perspectives by using predefined geometric data and camera analytics to ensure accurate rendering of paintings on objects' surfaces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If paintings are applied to single captured images, then painting operation is simple, but painting accuracy and consistency across different perspectives deteriorates
Solution Approach 1:
The patent transitions from 2D image-plane painting to 3D object-surface painting by mapping painted regions onto the three-dimensional geometry of detected objects. This allows paintings to maintain accurate size, shape, and position relationships as the object is viewed from different angles, resolving the contradiction between operational simplicity and painting consistency across perspectives.
Solution Approach 2:
The system creates a virtual copy of the painted region that is bound to the 3D object model rather than the original 2D image. This virtual painting copy is then rendered consistently across multiple views by transforming it according to the object's geometry and the camera's perspective, maintaining painting accuracy while keeping the user interface simple.
2Device complexity
If paintings are applied to single captured images, then processing complexity is low, but painting consistency across multiple perspectives deteriorates
Solution Approach 1:
By elevating paintings from 2D image space to 3D object space, the system inherently achieves consistency across perspectives. The 3D object model provides a stable geometric framework that automatically handles perspective transformations, reducing the need for complex per-perspective processing while maintaining painting stability.
Solution Approach 2:
The 3D object model serves multiple functions simultaneously: it provides geometric information for perspective transformation, defines the painting target surface, and enables consistent rendering across different views. This multi-functionality reduces overall system complexity while achieving painting consistency.
3Productivity
If perspective changes are not accounted for, then rendering speed is high, but painting realism and immersion deteriorates
Solution Approach 1:
The system performs preliminary actions by detecting objects and pre-computing their 3D geometric models before painting operations. This preparation work enables fast perspective transformations during rendering, as the geometric framework is already in place, maintaining both rendering speed and painting realism.
Solution Approach 2:
The patent replaces complex real-time perspective calculation mechanics with pre-computed 3D geometric transformations. By using the object's known geometry and standard transformation matrices, the system achieves accurate perspective rendering without sacrificing rendering speed.
Data Source
AI summary
Techniques for simulating interactions using an augmented reality device are described. Embodiments receive a request to paint over portions of a visual scene. Here, the visual scene is captured using one or more camera devices of the augmented reality device and is presented on a display of the augmented reality device. A first object in the visual scene corresponding to the first location is identified. Additionally, embodiments paint at least a portion of the first object as specified by the received request, and render a series of frames depicting the first object, such that the painted at least a portion of the first object is shown as painted when viewed from different perspectives using the augmented reality device.