Cross-Modality Medical Image Segmentation via Synthetic Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current medical image segmentation methods lack the ability to effectively leverage multi-modality information for improved accuracy, particularly in radiotherapy planning, where different imaging modalities like CT and MRI provide complementary information but are not efficiently integrated for single-modality image analysis.
Innovation Solution
A system and method that utilizes a learning network trained with multi-modality images, where a cross-modality learning network is jointly trained with first and second modality segmentation networks, allowing the segmentation of single-modality images to benefit from cross-modality information, using a processor and memory to select and apply the appropriate segmentation network for image segmentation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If multi-modality image segmentation is used to improve accuracy, then segmentation precision is improved, but device complexity increases due to requiring multiple imaging modalities
Solution Approach 1:
The patent creates a virtual copy of multi-modality information through synthetic data generation. A synthesis network generates synthetic images of a second modality from first modality images and corresponding segmentation masks, allowing the segmentation network to learn cross-modality relationships without requiring actual multi-modality imaging equipment during inference
Solution Approach 2:
The patent transforms the problem from requiring physical multi-modality inputs to using parameter-based synthetic generation. By changing the approach from physical device requirements to computational parameter transformations, the system achieves multi-modality segmentation benefits with single-modality equipment
2Measurement precision
If multi-modality information is integrated during training, then segmentation accuracy is improved, but training complexity and computational resources increase
Solution Approach 1:
The patent divides the training process into distinct segments: a synthesis network is trained separately to generate synthetic multi-modality images, and then a segmentation network is trained using these synthetic images. This segmentation of training stages reduces overall complexity by breaking down the multi-modality training problem into manageable independent tasks
Solution Approach 2:
The synthesis network performs preliminary action by pre-generating synthetic multi-modality images and their corresponding masks before the segmentation network training begins. This preliminary preparation creates ready-to-use training data that simplifies the subsequent segmentation network training process
3Loss of information
If cross-modality learning networks are used, then information utilization is improved, but processing time increases due to additional network operations
Solution Approach 1:
The patent applies partial action by selectively using cross-modality information only during the training phase through synthetic images. During inference, only the segmentation network processes the actual input image, avoiding the computational overhead of the synthesis network while retaining the benefits of cross-modality learning
Data Source
AI summary
Embodiments of the disclosure provide systems and methods for segmenting a medical image. The system includes a communication interface configured to receive the medical image acquired by an image acquisition device. The system also includes a memory configured to store a plurality of learning networks jointly trained using first training images of a first imaging modality and second training images of a second imaging modality. The system further includes a processor, configured to segment the medical image using a segmentation network selected from the plurality of learning networks.