Body Part Model Registration on a Single Display With Synthesized 3D Depth
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional navigation systems for minimally invasive surgery require extensive hardware, including multiple monitors and tracking systems, which occupy space, are costly, and introduce ergonomic issues with tracking markers, and suffer from registration errors due to assumptions about instrument rigidity and organ movement.
Innovation Solution
A device and method using a single user input device, such as a wireless 3D mouse, to register a 3D model with a real-world image by selecting reference points on a shared display, incorporating monocular depth estimation and topological pattern analysis to enhance accuracy without additional hardware, allowing registration on a single screen.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If conventional navigation systems use multiple monitors and tracking systems, then spatial anatomical information can be provided to the surgeon, but the system occupies extensive space, incurs high costs, and introduces ergonomic issues
Solution Approach 1:
The patent combines the 3D model display and real-world image display onto a single monitor, eliminating the need for multiple monitors. The tracking system is integrated with the surgical console, reducing the need for separate tracking hardware. This merging maintains the reliability of spatial anatomical information while significantly reducing device complexity and space requirements.
Solution Approach 2:
The surgical console is designed to perform multiple functions: it displays both the 3D model and real-world image, handles tracking data, and provides the user interface. This multi-functionality eliminates the need for separate dedicated hardware components, reducing overall system complexity while maintaining the reliability of navigation information.
2Measurement precision
If tracking markers are attached to surgical instruments, then instrument position can be tracked, but ergonomic issues arise and registration errors occur due to assumptions about instrument rigidity
Solution Approach 1:
The patent extracts the tracking markers from the surgical instruments and places them on the surgical console instead. This eliminates the need to attach markers to instruments, improving ergonomics by removing physical constraints on instrument manipulation while maintaining tracking precision through software-based coordinate system transformations.
Solution Approach 2:
The patent replaces the mechanical attachment of tracking markers to instruments with a software-based tracking system. The system uses camera-based optical tracking and coordinate transformations to monitor instrument positions without physical markers, eliminating ergonomic issues while maintaining measurement precision.
3Measurement precision
If extensive calibration is performed to achieve accurate registration, then registration accuracy improves, but the calibration process becomes time-consuming and complex
Solution Approach 1:
The patent performs preliminary coordinate system transformations and calibration setup during system initialization rather than requiring extensive real-time calibration. The surgical console pre-processes the relationship between the tracking system coordinate system and the display coordinate system, enabling accurate registration without time-consuming calibration procedures during surgery.
Solution Approach 2:
The system performs automatic coordinate transformations and registration calculations using the pre-established coordinate system relationships. The surgical console automatically handles the transformation of instrument position data from the tracking system coordinate system to the display coordinate system without requiring manual calibration adjustments, reducing both time and complexity.
4Reliability
If multiple coordinate systems are used for tracking and display, then accurate spatial representation is achieved, but coordinate transformation errors occur
Solution Approach 1:
The patent introduces a reference coordinate system as an intermediary between the tracking system coordinate system and the display coordinate system. The surgical console maintains transformation matrices between these coordinate systems, enabling accurate spatial representation while minimizing transformation errors through systematic coordinate reference management.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A device 20 and method for registering a model 34 of a body part and a real world image 32 of the body part. The device 20 comprises: an input interface 21 configured to receive 3D data of the model 34 and 2D data of the real world image 32, a display unit 24 configured to visualize the model 34 and the body part on a 2D display 26, a user input device 22 configured to receive user input data and to communicate the user input data to the display unit 24, which is configured to operate a mouse pointer on the 2D display 26 based on the user input data. The device 20 is further enhanced by a registration unit 30 configured to receive user input data, which, for at least a first to third pair comprising a reference P point and a corresponding reference point P*, by application of the mouse pointer, indicates a registration between the reference point P in the real world image 32 and the corresponding reference point P* in the 3D model 34, an expanding unit 36, configured to expand the 2D data of the reference points P in the real world image 32 into synthesized 3D data by adding depth information, which is derived from the 3D data of the model 34 of the body part and/or the 2D data of the real world image 32, a transformation unit 38 configured to determine a registration matrix for registering the 3D data of the model 34 of the body part and the synthesized 3D data of the real world image 32 of the body part using the 3D coordinates of the first to third pair of reference points P, P*.