Remote Expert Session Recording and Playback System
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality (AR) devices face challenges in providing reliable and accessible technical assistance due to unreliable connectivity and the need for a qualified expert, limiting the effectiveness of remote expert sessions for users performing tasks in various environments.
Innovation Solution
A system and method for recording and playing back interactive sessions using AR devices, where a server aggregates and synchronizes data streams from user and expert devices, generating playback content based on sensor and content data, allowing for the recreation of expert sessions for training or assistance purposes, and enabling remote expert interactions through virtual object manipulation and annotation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If real-time remote expert sessions are used, then immediate assistance is provided, but reliability deteriorates due to unstable connectivity
Solution Approach 1:
The system records the entire remote expert session including video feeds, audio, annotations, and sensor data during the initial interaction. This recorded session can then be played back later without requiring real-time connectivity, allowing users to review expert guidance at their convenience and eliminating dependency on unstable network connections.
Solution Approach 2:
The system creates a digital copy of the remote expert session by capturing and storing video, audio, and interaction data from both the expert's and user's devices. This copy preserves the complete interaction experience and can be accessed independently of the original real-time connection, ensuring reliable access to expert guidance.
2Ease of operation
If qualified experts are required for assistance, then quality of guidance is improved, but accessibility deteriorates due to limited expert availability
Solution Approach 1:
The system records expert sessions in advance during times when experts are available. These pre-recorded sessions serve as a library of expert knowledge that can be accessed by multiple users at any time, eliminating the need for real-time expert availability and expanding accessibility to expert guidance.
Solution Approach 2:
A single recorded expert session can be accessed and reused by multiple different users across different locations and times. The system transforms expert knowledge from a time-limited resource into a universally accessible resource that serves multiple users without requiring the expert's continued presence.
3Loss of information
If comprehensive session data is recorded, then playback quality is improved, but device complexity increases
Solution Approach 1:
The system divides session data into distinct segments or data streams including video feed from user device, video feed from expert device, audio data, annotation data, and sensor data. Each data type is captured and stored separately, allowing for organized management and efficient retrieval during playback without overwhelming system complexity.
Data Source
AI summary
A server receives, from a first display device of a first user, first content data, first sensor data, and a request for assistance identifying a context of the first display device. The server identifies a second display device of a second user based on the context of the first display device. The server receives second content data and second sensor data from the second display device. The first content data is synchronized with the second content data based on the first and second sensor data. Playback parameters are formed based on the context of the first display device. An enhanced playback session is generated using the synchronized first and second content data in response to determining that the first sensor data meet the playback parameters. The enhanced playback session is communicated to the first display device.