Digital Twin Authoring for Single-Demonstration AR/VR Instructions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for creating immersive media content for kinesthetic learning require Subject Matter Experts (SMEs) to possess multiple skill-sets, including 3D modeling, video editing, and AR/VR programming, leading to inefficiencies and increased resource requirements, as they need to demonstrate tasks multiple times for different media formats.
Innovation Solution
A multi-media instruction authoring system that allows SMEs to record a single demonstration using an AR-HMD and external sensors, generating immersive AR, VR, and 2D video content by tracking user and object poses, enabling easy 3D video editing with in-headset interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If SMEs create content for multiple media formats (AR, VR, video), then content versatility is improved, but device complexity and resource requirements increase
Solution Approach 1:
The authoring system is designed to perform multiple functions: it can capture demonstrations, automatically generate 3D avatars, create AR/VR content, and export to multiple media formats all within a single unified platform. This eliminates the need for separate tools for 3D modeling, video editing, and AR/VR programming that SMEs would otherwise need to master.
Solution Approach 2:
The system creates a digital twin (3D avatar) that copies the SME's physical demonstrations. This digital copy can then be reused across multiple media formats (AR, VR, video) without requiring the SME to physically demonstrate the same task multiple times in different formats, significantly reducing complexity and resource requirements.
2Adaptability or versatility
If SMEs demonstrate tasks multiple times for different media, then content coverage is improved, but loss of time increases
Solution Approach 1:
The system performs preliminary actions by automatically capturing the SME's demonstration once and then automatically generating all necessary media formats from that single capture. The automated processing includes avatar creation, content generation, and multi-format export, eliminating the need for repeated demonstrations.
Solution Approach 2:
The digital twin captures and stores the SME's demonstration as a reusable asset. This copy can be instantiated across multiple media formats (AR, VR, video) without requiring additional time from the SME, as the same captured data is processed into different formats automatically.
3Manufacturing precision
If multiple professionals are involved in content creation, then manufacturing precision is improved, but productivity decreases
Solution Approach 1:
The system performs self-service by automatically generating high-quality AR/VR/video content from the SME's demonstration without requiring external specialists. The automated avatar creation and content generation processes maintain quality while eliminating the need for multiple professionals, thus improving productivity.
Solution Approach 2:
The system merges the functions of multiple professionals (3D modelers, video editors, AR/VR programmers) into a single unified authoring platform that the SME can operate alone. This consolidation maintains content quality by incorporating professional-grade features while improving productivity by eliminating coordination overhead and resource bottlenecks.
Data Source
AI summary
A system and method for multi-media instruction authoring is disclosed. The multi-media instruction authoring system and method provide a unified and efficient system that supports the simultaneous creation of instructional media in AR, VR, and video-based media formats by subject matter experts. The multi-media instruction authoring system and method require only a single demonstration of the task by the subject matter expert and does not require any technical expertise of the subject matter expert. The multi-media instruction authoring system and method incorporate a 3D editing interface for free hand, in-headset interaction to create 2D videos from 3D recordings, by exploring novel virtual camera interactions.