AR-Overlay Manipulation via Head Pose and Voice Commands
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality (AR) systems in medical environments require users to context-switch and look away from the surgical site to interact with remote device interfaces, disrupting their focus on the procedure.
Innovation Solution
A computer-implemented method using an AR-device that manipulates AR-overlays based on the user's head pose and viewing direction relative to the observed object, allowing users to stay focused on the area of concern while adjusting the overlay content without needing to look away.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If users interact with remote device interfaces to manipulate AR-overlay content, then the AR-overlay content can be adjusted, but the user must context-switch and look away from the surgical site, disrupting focus
Solution Approach 1:
The patent introduces voice commands as an intermediary mechanism that allows users to manipulate AR-overlay content without breaking visual focus on the surgical site. The voice recognition system acts as a mediator between the user's intent and the AR-display system, enabling context-free interaction.
Solution Approach 2:
The patent replaces the mechanical interaction model (hand movements to remote interfaces) with a vocal interaction model. This substitution eliminates the need for physical context-switching to remote displays, allowing users to maintain visual focus while manipulating AR-content through speech.
2Measurement precision
If users maintain continuous focus on the surgical site, then surgical accuracy is improved, but the ability to manipulate AR-overlay content is limited
Solution Approach 1:
The system enables self-service interaction where the AR-display adapts to the user's visual state automatically. By detecting whether the user is looking at the surgical site or remote interfaces, the system autonomously adjusts the AR-overlay content and interaction mode, eliminating the need for manual context-switching.
Solution Approach 2:
The patent implements a feedback loop where the system continuously monitors the user's visual attention and adjusts the AR-overlay presentation accordingly. This feedback mechanism ensures that surgical focus is maintained while AR-information remains accessible and manipulable through voice commands.
3Adaptability or versatility
If remote device interfaces are used for AR-overlay control, then comprehensive control is achieved, but interaction time and complexity increase
Solution Approach 1:
The patent segments the interaction model into two distinct modes: visual-focus mode for surgical observation and voice-command mode for AR-control. This segmentation allows users to switch between observation and control tasks without physical movement, reducing interaction time while maintaining comprehensive control capability.
Solution Approach 2:
The system dynamically adjusts the interaction modality based on the user's visual state and task requirements. When the user needs to control AR-content, the system transitions from passive visual display to active voice-recognition mode, enabling rapid mode switching without temporal loss.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
The present invention relates to a computer-innplennented medical method, a computer program and a medical system (1) for providing, in a medical environment, an AR-overlay that is projected into the field of view provided by an AR-device (4) as a function of the characteristics of a user's (5) view with respect to an observed object (6).