Cross-Modality Image Registration via Intelligent Agent
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Registration between preoperative and intraoperative images acquired using different medical imaging modalities, such as magnetic resonance and X-ray fluoroscopy, is challenging due to differences in intensity values, contrast, resolution, and field of view, and existing AI-based solutions require large datasets and cannot be transferred across modality combinations.
Innovation Solution
A method using an intelligent agent to determine transformations between coordinate frames of image data from different modalities by receiving and processing models from various imaging modalities, segmenting data, generating polygon meshes, and applying reinforcement learning to select actions for transformation, allowing for registration across different imaging modalities without requiring extensive retraining.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual registration with fiducial markers is used to achieve accurate cross-modality registration, then registration accuracy is improved, but clinical workflow complexity and additional hardware requirements increase
Solution Approach 1:
The patent replaces manual mechanical registration processes with an automated deep learning system that uses neural networks to perform cross-modality image registration. The system automatically identifies corresponding anatomical structures between preoperative and intraoperative images without requiring manual intervention or fiducial markers, thereby maintaining high registration accuracy while eliminating workflow complexity and additional hardware requirements.
2Reliability
If preoperative imaging is performed immediately before the procedure to ensure fiducial marker consistency, then registration reliability is improved, but loss of time in clinical workflow increases
Solution Approach 1:
The patent performs image registration in advance during the preoperative planning phase, generating a transformation matrix that can be stored and applied during the actual procedure. This preliminary registration action ensures reliability by establishing the coordinate transformation before the procedure begins, while avoiding time loss during the procedure itself since the registration is already completed and can be rapidly applied when needed.
3Measurement precision
If AI-based registration systems are trained on specific modality combinations, then measurement precision for those modalities is improved, but adaptability to other modality combinations deteriorates
Solution Approach 1:
The patent develops a universal deep learning framework that can handle multiple imaging modality combinations through a single trained model. The system uses a standardized processing pipeline that accepts different input modalities (MRI, CT, ultrasound, fluoroscopy) and automatically adapts to register them with intraoperative images. This universal approach maintains high precision across different modality pairs without requiring separate training for each combination, thereby achieving both precision and adaptability.
Data Source
AI summary
The disclosure relates to a method of determining a transformation between coordinate frames of sets of image data. The method includes receiving a model of a structure extracted from first source image data, the first source image data being generated according to a first imaging modality and having a first data format, wherein the model has a second data format, different from the first data format. The method also includes determining, using an intelligent agent, a transformation between coordinate frames of the model and first target image data, the first target image data being generated according to a second imaging modality different to the first imaging modality.