Mixed Reality Surgical Imaging Projection Alignment
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional surgical imaging methods require surgeons to frequently look away from the surgical site to reference 3D virtual models displayed on screens, disrupting their focus and accuracy during procedures.
Innovation Solution
A mixed reality system using an MR device with infrared and color image capturing units, and a display lens, which generates a 3D virtual model projection directly onto the surgeon's field of view, allowing for continuous alignment with the surgical site without needing to look at a separate screen.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If a 3D virtual model is displayed on a separate screen to assist the surgeon, then the surgical imaging information is provided to the surgeon, but the surgeon must frequently turn their head to look at the screen, disrupting focus and reducing operational efficiency
Solution Approach 1:
The patent merges the 3D virtual model display with the surgeon's direct field of view by projecting the virtual model onto the patient's body surface. This combines the informational function (displaying 3D model) with the operational field (surgical site), eliminating the need for separate screen viewing and maintaining continuous surgical focus.
Solution Approach 2:
The patent introduces a projection system as an intermediary between the 3D virtual model data and the surgeon's perception. The projection device acts as a mediator that translates digital 3D model information into a visual format directly visible on the patient's body, bridging the gap between digital imaging and physical surgical field.
2Ease of operation
If the 3D virtual model is projected directly onto the MR device display, then the surgeon can maintain focus on the surgical site, but complex calibration and coordinate system transformations are required to achieve accurate alignment
Solution Approach 1:
The patent performs preliminary calibration actions by establishing projection matrices and coordinate transformations before the actual surgical procedure begins. The system pre-aligns the virtual model coordinate system with the physical patient coordinate system through calibration markers and mathematical transformations, so that during surgery, the projection is already accurate and no additional real-time calibration is needed.
Solution Approach 2:
The patent replaces complex manual mechanical alignment procedures with computational mathematics. Instead of physically adjusting and manually aligning the projection system with the patient's anatomy, the system uses coordinate geometry, projection matrices, and mathematical transformations to automatically calculate and achieve precise alignment between the virtual 3D model and the physical patient surface.
Data Source
AI summary
A method for performing surgical imaging based on mixed reality (MR) includes: obtaining a 3D virtual model of a body part of a subject, the 3D virtual model including a plurality of model reference points; continuously capturing IR images of the body part, including a plurality of IR reference points; calculating a first projection matrix based on the IR images; continuously capturing color images of the body part; calculating a second projection matrix based on the color images; in response to a calibration operation, calculating a third projection matrix; and generating a to-be-projected model using the 3D virtual model and the projection matrices.