Mixed Reality Control Layer for 2D Medical Screen Interfaces
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional 2D interfaces in augmented reality environments add complexity and operational challenges for medical personnel, requiring specific inputs and design adaptations that complicate user interactions.
Innovation Solution
A device and method for controlling a 2D application in mixed reality by identifying and translating interactive elements from a 2D interface to a mixed reality display, using a processor to map user input to commands for controlling the 2D application, and integrating with a mixed reality presentation arrangement.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional 2D interfaces are used in augmented reality environments, then user interaction with 2D applications is maintained, but operational complexity and difficulty increase for medical personnel
Solution Approach 1:
The patent introduces an intermediary system consisting of a processor, detection module, and translation module that mediates between the 2D application interface and the augmented reality display. This intermediary extracts interactive elements from the 2D interface, translates them into AR-compatible commands, and maps user inputs from the AR environment back to the 2D application, thereby resolving the contradiction by adding a mediating layer that simplifies operation while managing complexity
Solution Approach 2:
The patent replaces traditional mechanical mouse/keyboard input mechanisms with augmented reality-based interaction mechanisms such as hand gestures, eye gaze tracking, and voice commands. This substitution eliminates the need for physical interface hardware in the AR environment, reducing operational complexity while maintaining interaction capability
2Adaptability or versatility
If 2D applications are integrated into augmented reality, then interaction capabilities are enhanced, but the complexity of integrating different interfaces increases
Solution Approach 1:
The patent creates a universal integration layer that can handle multiple types of 2D applications and interface elements through a single processing pipeline. The detection module can identify various interactive elements (buttons, sliders, charts), the translation module can convert different element types into AR commands, and the mapping system can handle diverse input modalities, thereby achieving broad adaptability without proportionally increasing complexity
Solution Approach 2:
The patent creates a virtual copy or representation of the 2D interface elements within the augmented reality environment. Instead of directly integrating the original 2D interface into the AR display, the system extracts interactive elements, creates AR representations of them, and maintains the functional equivalence through the mapping mechanism, allowing flexible adaptation while managing integration complexity
3Ease of operation
If specific inputs and design requirements are implemented for augmented reality, then user interface intuitiveness is improved, but the complexity of operational tasks increases
Solution Approach 1:
The patent performs preliminary extraction and translation of interactive elements from the 2D interface before the user needs to interact with the application in AR mode. The system pre-processes the interface elements, identifies their functions, and prepares the mapping rules, so that when the user activates AR interaction, the system is already configured and ready, reducing the time required for operational tasks
Data Source
AI summary
The present invention relates to controlling a 2D application in a mixed reality application. In order to provide further simplification in terms of operating devices and equipment for the medical personnel, a device (10) for controlling a 2D application in a mixed reality application is provided. The device comprises a 2D application interface (12), a processor (14), a mixed reality application interface (16) and a control interface (18). The 2D application interface receives data representative of a 2D application. The application comprises interactive elements providing commands for user control of the 2D application. The processor identifies and extracts the interactive elements based on the data representative of the 2D application. The processor also translates the interactive elements to user interface features of a mixed reality display application. The processor further generates data for presenting the mixed reality display application and maps user input to commands for user control of the 2D application. The mixed reality application interface provides the mixed reality display application to a user as mixed reality. The control interface receives input from the user for the user interface features and provides the commands to the 2D application in order to control the 2D application. Hence, augmented reality interfaces are mapped to traditional 2D screen interfaces. This allows to integrate applications that only provided the interaction on a monitor with a mouse or the like to be operated in today's augmented reality environment.