MR-US Prostate Image Fusion Using Semi-Supervised Learning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The challenge of automatically fusing MR and US prostate images for biopsy guidance is complicated by multi-modality fusion, low signal-to-noise ratio, different fields of view, large deformations, and the difficulty in interpreting prostate MR images, requiring a significant learning curve for radiologists to align corresponding structures.
Innovation Solution
A semi-supervised constrained learning approach is employed to automatically register MR and US images by segmenting objects of interest, extracting point clouds, and determining transformations using machine learning-based networks, reducing the reliance on manual alignment and fiducials.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual rigid registration based on fiducials is used, then alignment reliability is improved, but operation complexity and learning curve increase
Solution Approach 1:
The system performs automatic image registration using machine learning algorithms that independently identify and align anatomical structures without requiring manual fiducial placement or operator intervention. The deep learning model automatically detects prostate boundaries and performs transformation calculations, making the system self-sufficient and eliminating the need for trained operators to perform manual alignment procedures.
Solution Approach 2:
The patent replaces the manual mechanical process of fiducial placement and visual alignment with an automated computational system. Machine learning models process the medical images, automatically detect anatomical features, calculate transformation parameters, and perform registration without physical manipulation or human visual assessment, substituting the mechanical/manual system with an intelligent automated system.
2Productivity
If automatic fusion is implemented, then productivity is improved, but measurement precision may worsen due to challenges in multi-modality fusion
Solution Approach 1:
The system performs preliminary segmentation of anatomical structures from the input medical images before performing registration. By pre-identifying and extracting relevant anatomical features such as prostate boundaries and internal structures, the system prepares the data in advance, enabling faster automated processing while maintaining precision through structured feature-based alignment rather than raw image matching.
Solution Approach 2:
The patent segments the medical images to identify and extract specific anatomical structures and features before performing the registration process. This segmentation approach divides the complex multi-modality fusion problem into manageable components, allowing the machine learning model to focus on aligning specific anatomical landmarks and structures, thereby maintaining measurement precision while enabling automated high-speed processing.
3Ease of operation
If point cloud registration without correspondence is used, then ease of operation is improved, but manufacturing precision worsens due to different point cloud densities
Solution Approach 1:
The system transforms the point cloud data by converting spatial coordinates into density-based representations. Instead of directly matching individual points between different point clouds, the method computes density distributions and uses these transformed parameters for alignment. This parameter transformation allows the system to handle point clouds with different densities and point counts while maintaining alignment precision through statistical rather than point-to-point correspondence.
Data Source
AI summary
Systems and methods for automatically registering a first input medical image and a second input medical image are provided. The first input medical image in a first modality and the second input medical image in a second modality are received. One or more objects of interest are segmented from the first input medical image to generate a first segmentation map and one or more objects of interest are segmented from the second input medical image to generate a second segmentation map. A first point cloud is extracted from the first segmentation map and a second point cloud is extracted from the second segmentation map. A transformation for aligning the first point cloud and the second point cloud is determined to register the first input medical image and the second input medical image. The transformation is output.