Multi-level Convolutional LSTM for MR Image Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for automatic segmentation of magnetic resonance (MR) images struggle to capture spatial-temporal dynamics and account for local intensity decreases, leading to inhomogeneities and reduced reliability in segmented images, particularly in heart tissue images with conditions like myocardial thinning and lesions.
Innovation Solution
A computer-implemented method using a convolutional neural network (CNN) and recurrent neural network (RNN) blocks to segment MR images by producing feature maps at multiple levels of resolution, with a RNN block processing these maps to generate output tensors and a segmentation block producing probability maps, accounting for spatial continuity and temporal dependencies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If typical neural network approaches are used for segmentation, then the method is simple and fast, but spatial-temporal dynamics between consecutive frames cannot be captured
Solution Approach 1:
The patent combines CNN and RNN into a hybrid architecture where CNN extracts spatial features from individual frames while RNN captures temporal dependencies across consecutive frames. This merging of two different neural network types enables the system to simultaneously process spatial and temporal information, resolving the contradiction between maintaining simple processing and capturing complex spatio-temporal dynamics.
Solution Approach 2:
The patent transitions from processing single images in isolation to processing sequences of images by adding the temporal dimension. The RNN component processes frames in sequence, maintaining hidden states that carry information across time steps, thereby capturing temporal dynamics without sacrificing the spatial processing capabilities of the CNN component.
2Reliability
If standard segmentation methods are applied, then processing is straightforward, but local intensity decreases due to tissue thinning and lesions are not accounted for
Solution Approach 1:
The patent implements local quality enhancement by applying specific preprocessing operations针对不同 regions of the image. Motion compensation and intensity normalization are applied locally to correct for tissue thinning and lesions in specific areas, allowing the segmentation algorithm to handle local intensity variations while maintaining overall processing efficiency.
Solution Approach 2:
The patent performs preliminary actions by preprocessing the images before segmentation to correct for motion artifacts and intensity variations. Motion compensation and intensity normalization are applied in advance to remove harmful effects, so that the subsequent segmentation process operates on corrected images, improving reliability without adding complexity during the main segmentation task.
3Measurement precision
If multi-level feature maps are processed through RNN blocks, then spatial-temporal characteristics are captured accurately, but computational complexity increases
Solution Approach 1:
The patent segments the feature extraction process into multiple levels of resolution, with different RNN blocks processing feature maps at different scales. This hierarchical segmentation allows the system to capture spatial-temporal characteristics at multiple levels of detail while distributing computational load across different resolution levels, reducing overall computational energy requirements compared to processing all features at maximum resolution.
Data Source
AI summary
Approaches for the automatic segmentation of magnetic resonance (MR) images. Machine learning models segment images to identify image features in consecutive frames at different levels of resolution. A neural network block is applied to groups of MR images to produce primary feature maps at two or more levels of resolution. The images in a given group of MR images may correspond to a cycle and have a temporal order. A second RNN block is applied to the primary feature maps to produce two or more output tensors at corresponding levels of resolution. A segmentation block is applied to the two or more output tensors to produce a probability map for the MR images. The first neural network block may be a convolutional neural network (CNN) block. The second neural network block may be a convolutional long short-term (LSTM) block.