See-through patient visualization via RGB-D and medical image fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Medical image modalities like CT, MRI, PET, and Ultrasound provide static 3D views of patients, limiting clinicians' ability to achieve desired views, and require mental mapping to physical anatomy, as they are divorced from real-world interactions.
Innovation Solution
An RGB-D camera acquires a 3D body surface model, aligning it with scanner data using landmarks for automated registration, enabling a mobile device with positioning capabilities to interactively track and render see-through views of patients, merging body hull and internal structures at various details.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If medical image modalities (CT, MRI, PET, Ultrasound) are used to provide 3D views of patient anatomy, then anatomical visualization capability is improved, but the view flexibility and real-world spatial correspondence deteriorate
Solution Approach 1:
The patent merges medical image data (CT, MRI, PET, Ultrasound) with real-time RGB-D camera feeds by registering both to a common coordinate system defined by the patient's body surface. This combination allows clinicians to see through the patient's body in real-time while maintaining accurate anatomical visualization, thus simultaneously improving view flexibility and maintaining anatomical precision.
Solution Approach 2:
The patent introduces a mobile device with positioning capabilities as an intermediary between the medical imaging system and the clinician. This device captures RGB-D data, performs registration with medical images, and renders see-through views, thereby enabling flexible real-time exploration while preserving anatomical accuracy through the registration process.
2Measurement precision
If medical image data is acquired and displayed in traditional modalities, then anatomical detail is improved, but ease of interaction and navigation in real-time deteriorates
Solution Approach 1:
The patent replaces traditional mechanical interaction with medical images (manual manipulation of static images) with an immersive mobile-based system. The mobile device with positioning capabilities and see-through visualization allows natural interaction through movement and orientation in 3D space, making navigation through anatomical structures more intuitive while preserving detailed anatomical information through the registered medical images.
3Ease of manufacture
If static medical images are displayed, then data acquisition simplicity is improved, but productivity in clinical examination deteriorates
Solution Approach 1:
The patent transforms static medical images into dynamic see-through views that update in real-time as the mobile device moves. The system dynamically registers medical images with live RGB-D camera feeds and renders views that change according to the device's position and orientation, enabling efficient clinical examination through natural real-time interaction while maintaining the simplicity of existing medical image acquisition processes.
Data Source
AI summary
A computer-implemented method for providing a see-through visualization of a patient includes receiving an image dataset representative of anatomical features of the patient acquired using a medical image scanner and acquiring a body surface model of the patient using an RGB-D sensor. The body surface model is aligned with the image dataset in a canonical/common coordinate system to yield an aligned body surface model. A relative pose of a mobile device is determined with respect to the RGB-D sensor and a pose dependent visualization of the patient is created by rendering the image dataset at a viewpoint corresponding to the relative pose of the mobile device. Then, the pose dependent visualization of the patient may be presented on the mobile device.