Modality-Neutral Machine Learning for Multi-Modal Image Registration
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep learning image registration systems struggle to accurately align images from different imaging modalities due to significant pixel/voxel intensity distribution mismatches, and existing multi-modal training efforts have achieved limited success.
Innovation Solution
A system that generates modality-neutral representations of multi-modal images using a machine learning model, allowing deep learning image registration to align these images by transforming them into a common intensity distribution, facilitating accurate registration.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep learning image registration is trained on mono-modal training datasets, then training accuracy and speed are improved, but the ability to register multi-modal images deteriorates
Solution Approach 1:
The patent introduces a modality-neutral representation as an intermediary transformation layer. This representation acts as a mediator that converts both mono-modal and multi-modal images into a common intensity distribution space, enabling the mono-modal trained deep learning model to accurately register multi-modal images without requiring retraining on diverse modalities.
Solution Approach 2:
The patent applies parameter changes by transforming image intensity distributions through the modality-neutral representation. This transformation adjusts the intensity parameters of input images to match a common distribution profile, allowing the registration model to handle different imaging modalities as if they were the same modality.
2Productivity
If deep learning image registration is applied to multi-modal images with different imaging modalities, then processing speed is improved, but registration accuracy deteriorates due to intensity distribution mismatches
Solution Approach 1:
The patent performs preliminary action by pre-transforming the input images into modality-neutral representations before feeding them to the deep learning registration model. This preliminary intensity distribution transformation ensures that the subsequent registration process benefits from both speed and accuracy, as the model receives pre-normalized input data.
Data Source
AI summary
Systems/techniques that facilitate multi-modal image registration via modality-neutral machine learning transformation are provided. In various embodiments, a system can access a first image and a second image, where the first image can depict an anatomical structure according to a first imaging modality, and where the second image can depict the anatomical structure according to a second imaging modality that is different from the first imaging modality. In various aspects, the system can generate, via execution of a machine learning model on the first image and the second image, a modality-neutral version of the first image and a modality-neutral version of the second image. In various instances, the system can register the first image with the second image, based on the modality-neutral version of the first image and the modality-neutral version of the second image.