AR Procedural Guidance via Automatic Video-to-AR Conversion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current augmented reality (AR) content creation systems lack user-friendly authoring tools, requiring programming expertise and are inflexible, as they cannot easily adapt to modify procedures or recognize and categorize objects in the user's environment, limiting their ability to provide intuitive guidance for procedural tasks.
Innovation Solution
A method for operating a procedural training user interface system that uses AR devices to detect and display user interactions, providing interactive guidance by superimposing virtual hands, incorporating object recognition and haptic feedback to aid in precise manual tasks, and utilizing a procedural language system for generating AR content without requiring programming knowledge.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If current AR content creation systems are used, then AR guidance can be provided for procedural tasks, but the systems require programming expertise and lack user-friendly authoring tools
Solution Approach 1:
The system captures video recordings of expert users performing procedures and automatically converts them into AR guidance content. This copying approach eliminates the need for manual programming of AR content while preserving the procedural knowledge, making content creation accessible to non-programmers through automatic video-to-AR conversion
Solution Approach 2:
The patent replaces manual content authoring mechanisms with automatic video processing and AI-based procedure recognition systems. Instead of requiring users to programmatically create AR content, the system uses machine learning to automatically generate guidance from video recordings, substituting mechanical authoring processes with automated intelligent systems
2Adaptability or versatility
If current AR content creation systems are used, then AR guidance can be provided, but the content cannot be easily modified to support related procedures
Solution Approach 1:
The system creates dynamic and flexible AR guidance content that can be easily adapted to different procedures. By using video recordings as the base material rather than fixed programmed content, the system allows for flexible modification - new procedures can be captured by recording additional videos, and existing content can be adjusted by re-processing video files, making the system adaptable without requiring complex re-programming
Solution Approach 2:
The patent creates a universal content creation system that handles multiple procedure types through a single video-based approach. The same system infrastructure processes different procedures by capturing videos of various tasks, making the content creation mechanism universally applicable across diverse procedural domains without requiring procedure-specific authoring tools
3Adaptability or versatility
If current AR content creation systems are used, then AR guidance can be provided, but the systems lack the ability to recognize and categorize objects in the user's environment
Solution Approach 1:
The system enables AR guidance content to automatically adapt to the user's environment through AI-based object recognition. The procedure recognition system automatically identifies and categorizes objects in the user's workspace from video recordings, allowing the content to serve itself by adapting to different environments without manual configuration or programming
Solution Approach 2:
The patent replaces manual object categorization and environment setup with automated computer vision and machine learning systems. The AI automatically recognizes objects, understands their relationships, and adapts the AR guidance content accordingly, substituting manual system configuration with intelligent automated recognition
Data Source
AI summary
A method of operating a procedural training user interface system involves displaying an interactive guided process of a first user using at least one augmented reality (AR) layer through an AR device worn by a second user, where a representation of first user hands is displayed. The second user interactions may be detected during the interactive guided process, where the second user attempts to superimpose second user hands on the representation of the first user hands in the at least one AR layer. The interactive guided process of the second user may then be displayed using the AR layer through an AR device on the first user and the AR device on the second user. If the first user hands and the second user hands are not superimposed in the AR layer, the first user or the second user may be notified to take corrective action.