3D Volume Prediction via Slice Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for classifying three-dimensional medical images using deep neural networks face challenges such as overfitting due to small datasets and the issue of features not appearing in all slices of a 3D volume, leading to inefficient training and reduced performance in detecting abnormalities like hemangiomas.
Innovation Solution
A system that partitions a 3D volume into slices, generates slice features using a 2D neural network, and merges these features to create a 3D feature volume, allowing for efficient training by combining 2D and 3D prediction losses, which reduces overfitting and improves performance by using an auxiliary loss for regularization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a 3D neural network is used to classify three-dimensional medical images directly, then the model can capture 3D contextual information, but the training efficiency decreases and overfitting occurs due to small dataset sizes
Solution Approach 1:
The patent divides the 3D volume into multiple 2D slices, allowing the use of efficient 2D neural networks for feature extraction while maintaining the ability to capture 3D contextual information through subsequent fusion of slice features. This segmentation approach improves training efficiency by using smaller 2D inputs while preserving detection accuracy through 3D feature aggregation.
Solution Approach 2:
The patent transitions from direct 3D processing to a 2D-3D hybrid approach by extracting features from 2D slices and then fusing them to reconstruct 3D feature volumes. This dimensionality change enables efficient 2D feature extraction while maintaining 3D contextual awareness, resolving the contradiction between training efficiency and detection accuracy.
2Productivity
If 2D slices are used for training, then training efficiency improves, but features that do not appear in all slices are missed, reducing detection performance
Solution Approach 1:
The patent merges features extracted from multiple 2D slices to reconstruct a comprehensive 3D feature volume. This merging process ensures that features appearing in any slice are captured and integrated, maintaining detection accuracy while benefiting from the training efficiency of 2D slice-based processing.
Solution Approach 2:
The patent uses 2D slices for efficient feature extraction and then combines these 2D features to reconstruct 3D feature volumes. This approach leverages the efficiency of 2D processing while ensuring that 3D contextual information and features not present in all slices are preserved through the fusion process.
3Measurement precision
If only 3D prediction loss is used for training, then the model optimizes for volume-level accuracy, but overfitting occurs on small datasets
Solution Approach 1:
The patent introduces slice-level prediction loss as an auxiliary feedback mechanism during training. This additional feedback signal provides regularisation by enforcing consistency between slice-level and volume-level predictions, preventing overfitting while maintaining volume prediction accuracy through the combined loss function.
Solution Approach 2:
The patent modifies the training objective by adding slice prediction loss to the standard 3D prediction loss, creating a composite loss function. This parameter change in the training methodology introduces regularisation that improves generalization performance while preserving volume-level detection accuracy.
Data Source
AI summary
An example system includes a processor to receive a three-dimensional (3D) volume. The processor can partition the 3D volume into slices. The processor can generate, via a two-dimensional (2D) neural network, slice features based on the slices. The processor can generate, via a three-dimensional (3D) neural network, a three-dimensional (3D) feature volume based on the slice features. The processor can generate, via a volume predictor, a volume prediction based on the 3D feature volume.