Hartley Frequency-Domain Self-Attention for High-Resolution Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Convolutional neural networks (CNNs) used for medical image segmentation face challenges in capturing long-range spatial correlations due to local receptive fields, leading to suboptimal performance when applied on original resolution, and self-attention approaches in Transformers require significant computational resources for large images.
Innovation Solution
A Hartley Multi-Head Attention (HartleyMHA) model is introduced, utilizing the Hartley transform with shared model parameters in the frequency domain, residual connections, and deep supervision to create a resolution-robust and parameter-efficient network architecture for image segmentation, applying multi-head self-attention efficiently in the frequency domain.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If self-attention approaches in Transformers are used for image segmentation, then long-range spatial correlations can be captured, but computational resources required increase significantly for large images
Solution Approach 1:
The patent transforms the image data from spatial domain to frequency domain using Hartley transform, changing the representation parameters of the input data. This allows the model to capture global correlations more efficiently in the frequency domain, reducing computational complexity while maintaining segmentation accuracy.
Solution Approach 2:
The patent replaces the standard Transformer self-attention mechanism with a Hartley-based frequency domain attention mechanism. This substitution uses mathematical transform properties instead of direct pairwise comparisons, significantly reducing computational resources required for large images.
2Productivity
If Convolutional neural networks are used for medical image segmentation, then computational efficiency is maintained, but long-range spatial correlations cannot be captured due to local receptive fields
Solution Approach 1:
The patent changes the domain representation from spatial to frequency using Hartley transform, enabling the model to capture global correlations without sacrificing computational efficiency. The frequency domain representation allows global information to be accessed more efficiently.
Solution Approach 2:
The Hartley transform acts as an intermediary that bridges the gap between local convolution operations and global correlation capture. By transforming to frequency domain, the model can implicitly capture global correlations through frequency components without requiring explicit long-range interactions.
3Measurement precision
If standard Transformers with self-attention are applied to high-resolution images, then segmentation accuracy improves, but the number of model parameters increases significantly
Solution Approach 1:
The patent changes the operational domain from spatial to frequency, which fundamentally alters how the model processes image data. This parameter change enables the model to achieve high segmentation accuracy with fewer parameters by exploiting frequency domain properties and shared parameter mechanisms.
Solution Approach 2:
The Hartley-based frequency domain attention mechanism serves multiple functions simultaneously: it captures global correlations, reduces parameter count through shared parameters across frequencies, and maintains computational efficiency. This multi-functional approach resolves the contradiction between accuracy and complexity.
Data Source
AI summary
Embodiments of the present invention provide computer-implemented methods, computer program product, and computer systems. One or more processors access an image file. The one or more processors input the image file into a deep learning model, where the deep learning model includes multiple blocks, each block of the multiple blocks including a Hartley transform, mixings of features in the frequency domain with a set of learnable parameters to produce new features, and an inverse of the Hartley transform. The one or more processors output another image file containing segmentation results of the accessed image file.