AR Sketching System Dynamic Object Binding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional augmented reality sketching systems are limited to displaying digital content as static or with pre-defined movements, failing to dynamically bind digital content to physical objects based on user inputs, thereby not allowing interactive behavior with the physical environment.
Innovation Solution
A computing device implements a sketch system that generates three-dimensional scene data of a physical environment and objects, tracks movements using two-dimensional coordinates, and projects user inputs as three-dimensional graphical elements, enabling dynamic binding and interaction with physical objects in real-time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If digital content is displayed as static or with pre-defined movements, then the system implementation is simple, but the digital content cannot dynamically interact with physical objects in real-time
Solution Approach 1:
The patent implements dynamic binding between digital content and physical objects by tracking the physical object's movements through video analysis and automatically updating the digital content's position and orientation in real-time. This transforms the static or pre-defined movement system into a dynamic system that adapts to actual physical object movements, resolving the contradiction between interaction capability and system complexity.
Solution Approach 2:
The system continuously captures video frames, detects physical object positions, and uses this feedback to update digital content placement. This closed-loop feedback mechanism enables real-time dynamic interaction while managing system complexity through automated tracking algorithms rather than manual programming of all possible movements.
2Reliability
If digital content is bound to physical objects for real-time tracking, then the interaction with physical environment is enhanced, but the computational requirements and processing time increase
Solution Approach 1:
The system performs preliminary actions by pre-processing video frames to identify and track physical objects before digital content needs to be placed. Object detection and tracking are initiated early in the processing pipeline, allowing the system to prepare tracking data in advance and reduce real-time processing delays, thereby maintaining high tracking accuracy while minimizing processing time impact.
Solution Approach 2:
The system applies partial tracking by focusing computational resources on tracking only the specific physical objects that have digital content bound to them, rather than tracking all objects in the scene. This selective approach maintains reliable tracking for relevant objects while reducing overall computational requirements and processing time.
3Speed
If the system processes video frames to track physical objects, then real-time tracking is achieved, but the computational load increases
Solution Approach 1:
The video processing is segmented into distinct stages: frame capture, object detection, position tracking, and digital content update. Each stage processes only necessary data independently, allowing the system to achieve real-time processing speed while reducing overall computational energy consumption by avoiding redundant processing across the entire frame for each operation.
Solution Approach 2:
The system performs video frame processing at periodic intervals rather than continuously analyzing every pixel change. By sampling frames at optimized intervals and using incremental tracking between frames, the system maintains real-time processing speed while significantly reducing computational energy consumption compared to continuous full-frame analysis.
Data Source
AI summary
In implementations of systems for augmented reality sketching, a computing device implements a sketch system to generate three-dimensional scene data describing a three-dimensional representation of a physical environment including a physical object. The sketch system displays a digital video in a user interface that depicts the physical environment and the physical object and the sketch system tracks movements of the physical object depicted in the digital video using two-dimensional coordinates of the user interface. These two-dimensional coordinates are projected into the three-dimensional representation of the physical environment. The sketch system receives a user input connecting a portion of a graphical element in the user interface to the physical object depicted in the digital video. The sketch system displays the portion of the graphical element as moving in the user interface corresponding to the movements of the physical object depicted in the digital video.