3D Medical Image Manipulation Using Gesture-Based Controller
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies for displaying and interacting with 3D medical images on extended reality headsets lack intuitive and precise control methods, making it difficult for medical professionals to effectively manipulate and analyze volumetric medical data.
Innovation Solution
The implementation of a user controller interface with a joystick and functional buttons, along with geo-registered tools such as a focal point pen, multi-function tool, and 3D cursor, enables precise control and manipulation of 3D medical images, allowing for tasks like filtering, segmentation, and annotation within a geo-registered volumetric coordinate system.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional display methods are used for 3D medical images, then device complexity is reduced, but measurement precision and manipulation control are insufficient
Solution Approach 1:
A virtual cursor acts as an intermediary between the user's hand movements and the 3D medical image data. The cursor translates physical gestures into precise digital manipulations, allowing users to navigate and analyze volumetric medical data with high precision without directly interacting with complex software controls.
Solution Approach 2:
The patent replaces traditional mechanical control interfaces (buttons, knobs, sliders) with gesture-based control through a handheld device. Hand movements in physical space are directly mapped to navigation and manipulation actions in the 3D medical image space, eliminating the need for complex mechanical control systems while improving precision.
2Ease of operation
If comprehensive control functions are added to manipulate 3D medical images, then ease of operation improves, but device complexity increases
Solution Approach 1:
The handheld device serves multiple functions: it acts as a pointer for navigation, a controller for image manipulation, and a tool for measurement. By consolidating these diverse functions into a single device, the system improves ease of operation without proportionally increasing device complexity.
Solution Approach 2:
The system automatically tracks the position and orientation of the handheld device using sensors and computer vision, eliminating the need for manual calibration or complex setup procedures. The device self-adjusts to provide intuitive control, reducing the operational burden on users.
3Measurement precision
If high-resolution display is used for medical images, then measurement precision improves, but loss of information increases due to limited field of view
Solution Approach 1:
The patent transitions from 2D screen display to 3D volumetric visualization, allowing users to perceive medical images in three dimensions. This enables simultaneous viewing of detailed structures and their spatial context, maintaining high resolution while preserving contextual information through depth perception and multi-angle exploration.
Solution Approach 2:
The system provides real-time feedback about the cursor position and selected region, along with contextual information about surrounding structures. This feedback mechanism ensures users maintain awareness of the broader anatomical context even when focusing on detailed high-resolution views.
Data Source
AI summary
This patent provides a method and apparatus for improving display of images of a virtual scene on an extended reality display. Specifically, this patent provides a method and apparatus to use different portions of the display based on a look angle of a user. For example, if a user looks at a nearby virtual object, more medially positioned portions of the display are utilized.