Multi-Prong 3D-UNet for Explainable Biomedical Image Inference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing convolutional neural networks (CNNs) face challenges in clinical environments due to variations in diseases and technology, lack of explainability, and difficulty in performing multiple tasks concurrently, particularly in medical image analysis such as cardiac MRI, which requires simultaneous localization, segmentation, and synthesis of objects.
Innovation Solution
A hybrid, multi-pronged CNN architecture, named Triton-Net, is developed to concurrently perform tasks like regression of values, segmentation of cardiac chambers, and image synthesis, providing explainability and adaptability through continuous learning.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If a single CNN architecture is used to perform multiple tasks (segmentation, classification, synthesis), then productivity and diagnostic value are improved, but device complexity increases
Solution Approach 1:
The patent implements a multi-pronged CNN architecture where a single network performs multiple functions including image segmentation, classification, and synthesis tasks simultaneously through shared feature extraction layers, reducing the need for separate models while maintaining specialized capabilities for each task
Solution Approach 2:
The network is divided into distinct prongs or branches after shared feature extraction, with each prong specialized for specific tasks (segmentation, classification, synthesis), allowing modular complexity management while achieving multi-functionality through a unified architecture
2Reliability
If pixel-wise segmentation CNNs are used, then explainability is improved, but data annotation requirements and training complexity increase
Solution Approach 1:
The patent combines classification and segmentation tasks within a single unified architecture, allowing the model to leverage both image-wide contextual information and pixel-level detail simultaneously, improving explainability while reducing the burden of granular annotation through shared learning
3Ease of manufacture
If image-wide classification CNNs are used, then ease of training is improved, but explainability and ability to detect subtle findings deteriorate
Solution Approach 1:
The multi-pronged architecture simultaneously performs classification and segmentation functions, allowing the system to benefit from the training efficiency of classification approaches while maintaining the explainability and subtle finding detection capabilities of segmentation approaches through integrated multi-task learning
Data Source
AI summary
A neural network architecture and method for analysis of time series images from an image source employs a 3D-UNet convolutional neural network (CNN) configured to receive the time series images and generate spatiotemporal feature maps therefrom. Multiple sub-convolutional neural network output prongs based on an SRNet architecture receive the feature maps and simultaneously generate inferences for image segmentation, regression of values, and multi-landmark localization.