A cardiac image segmentation method integrating multi-receptive field convolution and attention shifting

Abstract

This invention belongs to the field of medical image processing technology. Addressing the critical technical challenge of constructing Transformer variants with global modeling capabilities and controllable computational complexity in medical image segmentation, this invention proposes a cardiac image segmentation method integrating multi-receptive field convolution and shifted attention. The method includes the following steps: S1, preprocessing the training data to obtain processed training data; S2, constructing a multimodal cardiac image segmentation model; S3, training the multimodal cardiac image segmentation model using the processed training data to obtain a fully trained segmentation model; S4, segmenting the target cardiac image using the fully trained segmentation model to obtain the segmentation result. This invention significantly improves the accuracy of multimodal cardiac image segmentation and enhances the model's adaptability to complex anatomical structures by deeply integrating convolutional neural networks with a global self-attention mechanism.

Description

Technical Field

[0001] This invention belongs to the field of medical image processing technology, specifically relating to a cardiac image segmentation method that integrates multi-receptive field convolution and attention shifting. Background Technology

[0002] Precise segmentation of cardiac structures is crucial for disease diagnosis, surgical planning, and prognostic assessment, playing an irreplaceable role, especially in clinical applications such as interventional cardiac surgery, radiofrequency ablation, and artificial heart implantation. Three-dimensional cardiac CT and MRI images can provide millimeter-resolution information on cardiac chambers, myocardium, and vascular structures, but they face several challenges:

[0003] 1. Anatomical Complexity: The heart comprises seven major anatomical structures with highly dynamic deformation characteristics. For example, the right ventricular wall is only 3-5 mm thick and undergoes dramatic morphological changes during systole, leading to significant errors in traditional threshold segmentation methods. Furthermore, the blurred boundaries between cardiac structures, such as the mitral valve region between the left ventricle and left atrium, and the transitional regions between myocardium and heart chambers, greatly complicate accurate segmentation. During the cardiac cycle, the ventricular wall thickness can change by 30%-40%, requiring segmentation algorithms to possess dynamic adaptability.

[0004] 2. Modal Differences: CT images exhibit low soft tissue contrast (the difference in HU values ​​between myocardium and cardiac chambers is approximately 20-50), while MRI images, based on T1-weighted and T2-weighted imaging, show significant differences in structural visualization between these two imaging sequences. For example, myocardium appears as a moderate signal intensity in T1-weighted images, while it appears as a high signal intensity in T2-weighted images. This modal difference makes developing a universal segmentation algorithm extremely challenging. Furthermore, artifacts from metallic implants in CT images and motion artifacts in MRI images both affect segmentation accuracy.

[0005] 3. Data Limitations: 3D medical image annotation requires professional physicians to draw the images layer by layer. Annotating a single CT heart image takes approximately 4-6 hours, and due to patient privacy concerns, publicly available datasets are limited in size. Public datasets such as the Automated Heart Diagnosis Challenge (ACDC) contain only 100 annotated cases, far from sufficient for training deep learning models. Furthermore, differences in scanning equipment and parameters used by different medical institutions lead to inconsistent data distribution, further increasing the difficulty of model generalization.

[0006] 4. Computational complexity: Standard 3D cardiac CT images can reach sizes of 512×512×300 voxels. Directly applying Transformer global attention will result in computational complexity. Computational complexity ( This computational complexity far exceeds the GPU's video memory capacity. For example, the current mainstream NVIDIA A100 GPU, with its 80GB of video memory, can only process data of approximately 256×256×128 voxels. This computational complexity not only limits the training efficiency of the model but also affects its real-time application in clinical practice.

[0007] The rapid development of deep learning technology has driven a paradigm shift in medical image segmentation methods from traditional algorithms to deep learning. Current research mainly focuses on three types of technical architectures: convolutional neural networks (CNNs), Transformers, and their hybrid architectures.

[0008] Since AlexNet's landmark breakthrough in the 2012 ImageNet competition, CNNs have gradually become the core technology paradigm in the field of image segmentation due to their superior feature learning capabilities. This architecture achieves multi-level feature extraction from local texture to global semantics through a combination of stacked convolutional and pooling layers, and completes classification or regression tasks through fully connected layers. In the field of medical image segmentation, CNN technology has been successfully applied to the accurate segmentation of key anatomical structures such as the heart, brain tissue, and lung nodules, with its advantages mainly reflected in its autonomous representation capabilities of complex biomedical features. Fully Convolutional Networks (FCNs) achieved a complete architectural innovation for end-to-end semantic segmentation for the first time, overcoming the limitations of input image size and directly outputting pixel-level predictions by replacing the fully connected layers of traditional CNNs with convolutional layers. This innovation laid the methodological foundation for the evolution of subsequent medical image segmentation frameworks. As a classic architecture for medical image segmentation, U-Net significantly improves segmentation accuracy and computational efficiency through its encoder-decoder symmetric structure and cross-layer skip connection mechanism. Its encoder extracts abstract features through stepwise downsampling, while the decoder restores the spatial resolution of the feature map through deconvolution. Skip connections effectively alleviate information decay by fusing shallow details with deep semantic information. Building upon the U-Net architecture, R2U-Net innovatively integrates residual modules and recurrent convolution mechanisms, achieving performance improvements while maintaining the same number of parameters. Residual connections improve gradient propagation efficiency by establishing cross-layer information pathways, while recurrent convolution enhances the representational ability of local features through temporal feature reuse. LadderNet further enhances the ability to capture multi-scale features by constructing a feature pyramid through cascading multiple U-Net units. For the processing needs of 3D medical images, V-Net, through 3D convolution kernels and spatial pooling operations, achieves an end-to-end volumetric data segmentation framework for the first time. DeepIGeoS proposes an interactive segmentation paradigm, introducing a manual feedback correction mechanism to reduce the number of user interactions while improving the clinical applicability of the segmentation results. It is worth noting that although CNNs have made significant progress in medical image segmentation, their inherent local receptive field characteristics still pose a fundamental constraint on global context modeling. This structural characteristic leads to theoretical bottlenecks in the modeling of long-range dependencies and the fusion of multi-scale features, which has become a key factor restricting further improvement in segmentation accuracy.

[0009] With the groundbreaking advancements of Visual Transformer (ViT) in computer vision, its application in medical image segmentation is rapidly emerging. This architecture constructs a global pixel association model through a self-attention mechanism, overcoming the limitations of the local receptive field in traditional CNNs and significantly improving the segmentation accuracy of complex anatomical structures. In terms of architectural innovation, Swin-Unet constructs a Transformer-based symmetric encoding and decoding structure by encoding the input image into blocks as visual tokens. Its core innovation lies in the dynamic fusion of local detail features and global semantic information through a sliding window multi-head self-attention mechanism. Building upon this, DS-TransUNet proposes a dual-branch Swin Transformer architecture, integrating a hierarchical feature learning mechanism simultaneously into the encoding and decoding process for the first time: a progressive feature abstraction strategy is adopted in the encoding stage, and a multi-scale feature fusion scheme is implemented in the decoding stage, thereby significantly improving the segmentation robustness of medical images of different modalities. However, the application of Transformer in medical image segmentation still faces significant challenges. Although the self-attention mechanism has global modeling capabilities, its computational complexity is quadratically related to the input resolution, making memory usage and computation time particularly prominent when processing high-resolution 3D medical images such as CT and MRI. This computational efficiency bottleneck severely restricts the application of Transformers in clinical scenarios such as real-time surgical navigation. How to achieve a balance between global modeling capabilities and computational efficiency has become a key research breakthrough direction.

[0010] Given the complementary advantages of CNNs in local feature extraction and Transformers in long-range dependency modeling, the field of medical image segmentation is currently witnessing a wave of innovation in CNN-Transformer hybrid architectures. UNETR innovatively uses the Transformer as the core module of the encoder, establishing a voxel-level association model through a global attention mechanism, followed by multi-resolution feature reconstruction via a U-shaped decoder, significantly improving the segmentation accuracy of organ boundaries. For the characteristics of 3D medical images, TransBTS embeds a Transformer module into a 3D CNN framework, achieving joint modeling of spatial and channel dimensions through an axial attention mechanism. Its cascaded feature fusion strategy demonstrates excellent performance in brain tumor segmentation tasks. VT-Unet further optimizes the encoder design, constructing a cascaded self-attention module. Through the synergistic effect of local window attention and global cross attention, it achieves hierarchical representation of anatomical structures. TransFuse proposes a dual-branch parallel architecture, where the CNN branch is responsible for extracting high-resolution spatial details, and the Transformer branch constructs global semantic associations. A gating fusion module achieves feature complementarity, improving computational efficiency while maintaining network depth. 3DUX-Net introduces three-dimensional dynamic convolutional units, which adapt to the morphological features of different anatomical structures through deformable convolutional kernels, and combine with hierarchical Transformers to achieve multi-scale volume feature extraction.

[0011] SwinUNETR integrates the sliding window mechanism of the Swin Transformer with the voxel reconstruction module of UNETR, reducing computational complexity while maintaining long-range dependency modeling capabilities, making it particularly suitable for high-resolution CT image segmentation. MISSFormer proposes a multi-scale self-attention module that generates feature maps of differentiated receptive fields through multiple sets of deformable convolutional kernels in parallel, and then achieves adaptive feature weighting through channel attention. Attention U-Net designs a spatial-channel dual-dimensional attention gating mechanism, dynamically suppressing irrelevant background regions through learnable parameters, demonstrating outstanding performance in the segmentation of small organs such as the pancreas. nnFormer employs an interleaved convolution-attention block design, capturing a wide range of contextual information through dilated convolutions, and then refining edge features through local attention, achieving collaborative modeling of volumetric data. D-Former innovatively introduces a dilated Transformer module, effectively expanding the receptive field and reducing computational complexity by embedding a spatial interval sampling strategy in attention calculation. Notably, CoTr proposes a deformable feature interaction mechanism that dynamically adjusts the geometric distribution of the attention region using learnable offsets, enhancing local feature alignment accuracy while maintaining global modeling capabilities. BiTr-Unet constructs a bidirectional temporal attention mechanism, achieving context-aware optimization of anatomical structures through a forward-backward feature propagation path. Despite these significant advancements in hybrid models, two key challenges remain: first, existing methods underdevelop the multi-scale feature extraction potential of CNN modules, lacking a systematic strategy for combining convolutional layers with different receptive fields; second, the relationship between the computational complexity of the Transformer module and the square of the input resolution remains unresolved, especially when processing 3D medical images, facing the dual pressures of GPU memory and computation time.

[0012] While the Transformer architecture has demonstrated significant advantages in the field of medical image segmentation, its inherent limitations... Computational complexity remains a significant constraint on its application in high-resolution medical imaging. To address this, researchers have proposed several lightweight improvement schemes. The SOFT model innovatively proposes a Softmax-free attention mechanism, using a Gaussian kernel function to replace traditional softmax calculations, reducing computational complexity to a linear level while maintaining global modeling capabilities. EfficientVit achieves balanced extraction of local and global features through a collaborative design of linear attention and depthwise separable convolutions. FlattenTransformer builds upon this by introducing focused linear attention and a rank recovery module, effectively mitigating feature degradation through a low-rank approximation strategy. SwinTransformer constructs a hierarchical window attention architecture, alternating between local window self-attention (W-MSA) and sliding window self-attention (SW-MSA) to ensure cross-window information interaction while achieving linear growth in computational complexity. PVT proposes a progressive sparse attention mechanism, significantly reducing memory consumption during image processing by gradually reducing the number of key-value pairs in the deep network stages. However, existing lightweight schemes generally face a trade-off between modeling capabilities and computational efficiency. While windowing strategies reduce computational overhead, they weaken the ability to model long-range dependencies; linear approximation methods are prone to feature representation degradation; and sparsification may result in the loss of key anatomical context information.

[0013] Therefore, how to construct a Transformer variant with global modeling capabilities and controllable computational complexity remains a core problem that urgently needs to be solved in the field of medical image segmentation. Summary of the Invention

[0014] The purpose of this invention is to overcome the shortcomings of the prior art and provide a cardiac image segmentation method that integrates multi-receptive field convolution and attention shifting.

[0015] This invention proposes a multimodal cardiac image segmentation (MRFTA) model that integrates multi-receptive field convolution and transfer attention for multimodal 3D cardiac image segmentation. This framework significantly improves the segmentation accuracy and computational efficiency of the entire cardiac structure by integrating multi-receptive field feature extraction with an efficient global modeling mechanism. The multimodal cardiac image segmentation model adopts a symmetric encoder-decoder architecture, following a classic U-shaped topology. The encoder consists of multiple multi-receptive field depth convolutional downsampling modules (MRFDD), effectively capturing local contextual information without increasing computational cost. This invention innovatively proposes a 3D transfer attention Transformer module (TA3D), which introduces learnable transfer tokens. T Avoid query vectors Q With key vector K Direct computation between them. This mechanism reduces computational complexity from... Reduce to At the same time, it does not compromise global modeling capabilities, ensuring the accuracy of model segmentation. The decoder consists of multiple multi-receptive-field depthwise convolutional upsampling modules (MRFDU), which combine multi-scale encoder features passed through skip connections to progressively reconstruct high-resolution segmented images.

[0016] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0017] A cardiac image segmentation method integrating multi-receptive-field convolution and attention shifting includes the following steps:

[0018] S1. Preprocess the original cardiac image training data to generate a preprocessed training dataset.

[0019] S2. Construct a multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention shifting;

[0020] S3. Use the preprocessed training dataset to train the constructed multimodal cardiac image segmentation model to obtain a fully trained segmentation model.

[0021] S4. Use the well-trained segmentation model to segment the heart image under test and output the segmentation results.

[0022] Preferably, step S1 includes the following steps:

[0023] S11. Map the non-continuous anatomical structure identifiers in the original cardiac image training data into an 8-dimensional binary vector and output the binary label data.

[0024] S12. Perform anatomical structure space standardization on the original cardiac image training data and binary label data respectively, and adjust all data to the RAS coordinate system through affine transformation to obtain image data and label data.

[0025] S13. Process the image data using bilinear interpolation and the label data using nearest neighbor interpolation. Then, resample the spatial resolution of the processed image data and label data to 1mm×1mm×1mm to obtain standardized data.

[0026] S14. The voxel values ​​of the standardized data are linearly scaled to the range of [0,1], and then processed by the contrast-limited adaptive histogram equalization algorithm to obtain the preprocessed data.

[0027] S15. Extract the three-dimensional region of interest from the preprocessed data to obtain the extracted data;

[0028] S16. Perform random intensity shift and random spatial transformation on the extracted data to obtain the preprocessed training dataset.

[0029] Preferably, in step S2, the constructed multimodal cardiac image segmentation model that integrates multi-receptive field convolution and transfer attention includes a TransUnet-based symmetric encoder-decoder architecture, a linear projection module, a feature mapping module, multiple multi-receptive field depth convolution downsampling modules, multiple multi-receptive field depth convolution upsampling modules, and multiple three-dimensional transfer attention Transformer modules.

[0030] Multiple multi-receptive-field depth convolutional downsampling modules are sequentially set in the encoder path of the TransUnet-based symmetric encoder-decoder architecture to progressively extract local-long-range features, thereby achieving spatial dimension compression and feature abstraction.

[0031] The linear projection module is set in the connection layer between the encoder output and the bottleneck part of the TransUnet-based symmetric encoder-decoder architecture, and is used to transform the convolutional feature map sequence into a Transformer-compatible vector sequence.

[0032] Multiple 3D transfer attention Transformer modules are sequentially placed in the bottleneck part of the TransUnet-based symmetric encoder-decoder architecture to model remote spatial dependencies through a 3D transfer attention mechanism.

[0033] The feature mapping module is located in the interface layer between the bottleneck part and the decoder path of the TransUnet-based symmetric encoder-decoder architecture. It is used to remap the Transformer output sequence into a spatial feature map structure.

[0034] Multiple multi-receptive-field depth convolutional upsampling modules are sequentially set in the decoder path of the TransUnet-based symmetric encoder-decoder architecture. They are used to gradually restore spatial resolution through deconvolution and skip connections, and to generate segmentation maps by fusing multi-scale contextual information.

[0035] Preferably, each multi-receptive-field depth convolution downsampling module includes two connected multi-receptive-field depth convolution blocks, and each multi-receptive-field depth convolution block includes a batch normalization layer, an activation layer, and a multi-receptive-field depth convolution module connected in sequence.

[0036] Preferably, the working method of each multi-receptive field depthwise convolutional module is as follows:

[0037] Step 1: On the preprocessed training dataset, use depthwise separable convolutions with kernel sizes of 3 and 5 in parallel to extract features, and obtain feature extraction data for two receptive fields.

[0038] Step 2: Preprocess the feature extraction data of the two receptive fields by passing them through a batch normalization layer and an activation layer respectively, and output the regularized feature data of the two receptive fields.

[0039] Step 3: After concatenating the regularized feature data of the two receptive fields, the topological connection relationship between the feature channels is dynamically reorganized through channel shuffling operation to promote the interaction and fusion of cross-receptive field feature information and obtain the output of the multi-receptive field deep convolution module.

[0040] Preferably, each 3D transfer attention Transformer module contains interconnected transfer attention modules and a feedforward network;

[0041] The attention transfer module is used to extract global features;

[0042] The feedforward network is used to perform nonlinear transformation and information processing on the extracted global features to obtain the output of the 3D attention transfer Transformer module.

[0043] The preferred method for the attention shifting module is as follows:

[0044] Step 1, Global Aggregation: Define a new transfer token as the query. The transfer token is obtained by pooling the original query tokens. Apply the Softmax attention operation to the transfer token to aggregate information from various values ​​and obtain the transfer features.

[0045] Step 2, Global Broadcast: Using the transition feature as the value and the transition token as the key, perform a Softmax attention operation with the query matrix to broadcast global information from the transition feature to each query token, achieving an effective mapping from global context to local features, and obtaining global features.

[0046] Preferably, each multi-receptive-field depth convolution upsampling module includes a multi-receptive-field depth convolution module, a first convolutional layer, a first batch normalization layer, a first activation layer, a second convolutional layer, a second batch normalization layer, and a second activation layer connected in sequence.

[0047] Preferably, in step S3, during the training of the constructed multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention transfer, the training process of the multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention transfer is constrained by a loss function.

[0048] Preferably, the loss function is composed of Dice loss function and TopK The composition of the loss function;

[0049] The expression for the loss function is:

[0050] ;

[0051] in, For loss function, for Dice loss function for TopK Loss function.

[0052] Compared with the prior art, the advantages of this invention are as follows:

[0053] (1) The multimodal cardiac image segmentation model established in this invention integrates multi-receptive field convolution and shifted attention. By deeply integrating CNN with global self-attention mechanism, it effectively combines the local feature extraction capability of CNN with the global modeling advantage of self-attention mechanism. It not only improves the accuracy of cardiac multimodal image segmentation, but also enhances the adaptability of the model to complex anatomical structures. Simulation data shows that the average Dice coefficient is 93.65%, the Jaccard coefficient is 93.1%, and the Hausdorff distance is 42.02 mm. It also provides a new idea for the transfer application of cross-modal medical image tasks.

[0054] (2) By designing a multi-receptive-field deep convolution module, this invention effectively captures information at different scales in the image by using convolution kernels with different receptive fields without increasing computational complexity. This not only preserves the inherent inductive bias of convolution but also significantly enhances the network's ability to model long-distance spatial dependencies. It improves segmentation accuracy while maintaining computational efficiency. Simulation data shows that this invention compresses the number of parameters and computational complexity by 11.94% and 21.49%, respectively, demonstrating excellent lightweight characteristics.

[0055] (3) This invention innovatively introduces a transfer token through the design of a three-dimensional attention transfer Transformer module. T as query vector Q and key vector K A medium for transmitting information between them, replacing traditional Q and K By directly performing similarity calculations, the high-complexity operations in Transformer are avoided. This module achieves effective modeling of global information with low computational cost, solves the problem of excessive resource consumption in 3D medical image processing, and makes the model more suitable for actual clinical scenarios.

[0056] (4) The simulation results show that the multimodal cardiac image segmentation model proposed in this invention, which integrates multi-receptive field convolution and attention shifting, significantly outperforms existing methods in cardiac multimodal segmentation tasks. In particular, it excels in the segmentation accuracy of edge details and complex structures (such as the boundaries of the atria and ventricles), providing a more reliable automated tool for clinical cardiac image analysis. Attached Figure Description

[0057] Figure 1 This is a framework diagram of the multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention transfer constructed in this invention.

[0058] Figure 2 The diagrams show the frameworks of existing linear attention mechanisms, Softmax attention mechanisms, and the attention transfer module proposed in this invention; (a) shows the linear attention mechanism; (b) shows the Softmax attention mechanism; and (c) shows the attention transfer module.

[0059] In this context, the blue dashed box represents the global aggregation stage, the black dashed box represents the global broadcast stage, and the red dashed box represents the kernel function within the generalized linear attention.

[0060] Figure 3 A visual comparison chart of the performance of different models;

[0061] The size of the circle represents the number of floating-point operations per second.

[0062] Figure 4 This is a visualization comparison of the segmentation results of the MRFTA model proposed in this invention and other methods on the MM-WHS 2017 dataset;

[0063] Among them, red, green, dark blue, yellow, cyan, light purple and white represent the left ventricular myocardium (Myo), left atrial chamber (LA), left ventricular chamber (LV), right atrial chamber (RA), right ventricular chamber (RV), ascending aorta (AA) and pulmonary artery (PA), respectively. Detailed Implementation

[0064] The following will be described in conjunction with embodiments of the present invention. Figures 1 to 4 The technical solutions in the embodiments of the present invention are clearly and completely described herein. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0065] like Figure 1 As shown in the figure, this invention proposes a cardiac image segmentation method that integrates multi-receptive field convolution and attention shifting, including the following steps:

[0066] S1. Preprocess the original cardiac image training data to generate a preprocessed training dataset, which includes the following steps:

[0067] S11. Map the non-continuous anatomical structure identifiers in the original cardiac image training data into an 8-dimensional binary vector and output the binary label data.

[0068] S12. Perform anatomical structure space standardization on the original cardiac image training data and binary label data respectively, and adjust all data to the RAS coordinate system through affine transformation to obtain image data and label data.

[0069] S13. Process the image data using bilinear interpolation and the label data using nearest neighbor interpolation. Then, resample the spatial resolution of the processed image data and label data to 1mm×1mm×1mm to obtain standardized data.

[0070] S14. The voxel values ​​of the standardized data are linearly scaled to the range of [0,1], and then processed by the contrast-limited adaptive histogram equalization algorithm to obtain the preprocessed data.

[0071] S15. Extract the three-dimensional region of interest from the preprocessed data to obtain the extracted data;

[0072] S16. Perform random intensity shift and random spatial transformation on the extracted data to obtain the preprocessed training dataset.

[0073] S2. Construct a multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention shifting;

[0074] In this embodiment of the invention, the constructed multimodal cardiac image segmentation model that integrates multi-receptive field convolution and transfer attention includes a TransUnet-based symmetric encoder-decoder architecture, a linear projection module, a feature mapping module, multiple multi-receptive field depth convolution downsampling modules, multiple multi-receptive field depth convolution upsampling modules, and multiple three-dimensional transfer attention Transformer modules.

[0075] Multiple multi-receptive-field depth convolutional downsampling modules are sequentially set in the encoder path of the TransUnet-based symmetric encoder-decoder architecture to progressively extract local-long-range features, thereby achieving spatial dimension compression and feature abstraction.

[0076] The linear projection module is set in the connection layer between the encoder output and the bottleneck part of the TransUnet-based symmetric encoder-decoder architecture, and is used to transform the convolutional feature map sequence into a Transformer-compatible vector sequence.

[0077] Multiple 3D transfer attention Transformer modules are sequentially placed in the bottleneck part of the TransUnet-based symmetric encoder-decoder architecture to model remote spatial dependencies through a 3D transfer attention mechanism.

[0078] The feature mapping module is located in the interface layer between the bottleneck part and the decoder path of the TransUnet-based symmetric encoder-decoder architecture. It is used to remap the Transformer output sequence into a spatial feature map structure.

[0079] Multiple multi-receptive-field depth convolutional upsampling modules are sequentially set in the decoder path of the TransUnet-based symmetric encoder-decoder architecture. They are used to gradually restore spatial resolution through deconvolution and skip connections, and to generate segmentation maps by fusing multi-scale contextual information.

[0080] Specifically, each multi-receptive-field depthwise convolutional downsampling module contains two connected multi-receptive-field depthwise convolutional blocks, and each multi-receptive-field depthwise convolutional block contains a batch normalization layer, an activation layer (i.e., a ReLU6 activation layer), and a multi-receptive-field depthwise convolutional module connected in sequence. The activation function of the activation layer is ReLU6.

[0081] Specifically, the working method of each multi-receptive field depthwise convolutional module is as follows:

[0082] Step 1: On the preprocessed training dataset, use depthwise separable convolutions with kernel sizes of 3 and 5 in parallel to extract features, and obtain feature extraction data for two receptive fields.

[0083] Step 2: Preprocess the feature extraction data of the two receptive fields by passing them through a batch normalization layer and an activation layer respectively, and output the regularized feature data of the two receptive fields.

[0084] Step 3: After concatenating the regularized feature data of the two receptive fields, the topological connection relationship between the feature channels is dynamically reorganized through channel shuffling operation to promote the interaction and fusion of cross-receptive field feature information and obtain the output of the multi-receptive field deep convolution module.

[0085] Specifically, each 3D attention transfer Transformer module contains interconnected attention transfer modules and a feedforward network. The attention transfer module is responsible for extracting global features, and the feedforward network is responsible for performing nonlinear transformations and information processing on the extracted global features. Linear layers, activation functions, and random deactivation operations are used to enhance the model's expressive power and prevent overfitting, ultimately yielding the output of the 3D attention transfer Transformer module. The working method of the attention transfer module is as follows:

[0086] Step 1, Global Aggregation: Define a new transfer token as the query. The transfer token is obtained by pooling the original query tokens. Apply the Softmax attention operation to the transfer token to aggregate information from various values ​​and obtain the transfer features.

[0087] Step 2, Global Broadcast: Using the transition feature as the value and the transition token as the key, perform a Softmax attention operation with the query matrix to broadcast global information from the transition feature to each query token, achieving an effective mapping from global context to local features, and obtaining global features.

[0088] Specifically, each multi-receptive-field depthwise convolutional upsampling module comprises a multi-receptive-field depthwise convolutional module, a first convolutional layer, a first batch normalization layer, a first activation layer (i.e., a first ReLU activation layer), a second convolutional layer, a second batch normalization layer, and a second activation layer (i.e., a second ReLU activation layer) connected in sequence. The activation function of the first activation layer and the second activation layer is ReLU.

[0089] S3. Use the preprocessed training dataset to train the constructed multimodal cardiac image segmentation model to obtain a fully trained segmentation model.

[0090] In this embodiment of the invention, during the training of the constructed multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention transfer, the training process of the multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention transfer is constrained by a loss function.

[0091] Specifically, the loss function is composed of Dice loss function and TopK The composition of the loss function;

[0092] The expression for the loss function is:

[0093] ;

[0094] in, For loss function, for Dice loss function for TopK Loss function.

[0095] S4. Use the well-trained segmentation model to segment the heart image under test and output the segmentation results.

[0096] The technical solution of the present invention will be described in detail below:

[0097] 1. Data preprocessing:

[0098] To accurately segment multimodal whole-heart structures, preprocessing of the raw cardiac image training data is necessary. Based on international cardiac image segmentation standards, seven target anatomical structures are defined: left ventricular myocardium (Myo), left atrial chamber (LA), left ventricular chamber (LV), right atrial chamber (RA), right ventricular chamber (RV), ascending aorta (AA), and pulmonary artery (PA). The raw label data is encoded using seven discontinuous voxel values ​​(205, 420, 500, 550, 600, 820, 850), where a voxel value of 0 represents the background region. First, to adapt to the training requirements of deep learning models, a one-hot encoding strategy is used to map the discontinuous anatomical structure identifiers in the raw cardiac image training data into 8-dimensional binary vectors (0, 1, 2, 3, 4, 5, 6, 7). Secondly, anatomical spatial standardization is performed on the original image data and binary label data. Affine transformation is used to uniformly adjust the data to the RAS coordinate system, resulting in image data and label data. Since the image data is a continuous variable, smooth interpolation (bilinear interpolation) is used to generate smoothly transitioning intensity values ​​to maintain the realism of the anatomical structure. Because the label data consists of discrete categories, nearest neighbor interpolation is used to ensure that the binary nature of the segmentation labels is not destroyed, avoiding non-integer results. The spatial resolution of the processed image data and label data is uniformly resampled to 1mm × 1mm × 1mm to obtain standardized data. Subsequently, intensity normalization is performed on the standardized image data, linearly scaling the voxel values ​​to [0,1] to obtain normalized data. To address the image quality optimization issue, and to maximize the extraction of useful information while minimizing interference from device differences and noise, an improved contrast-limited adaptive histogram equalization (CLAHE) algorithm was employed for the normalized data. An 8×8 checkerboard partition was set along the axial slice direction, with a histogram cropping threshold of 2.0 for each sub-region. Bilinear interpolation was used for smoothing to eliminate block artifacts, resulting in contrast-optimized data. To extract key components from the data, a 3D region of interest (ROI) extraction was performed. The contrast-optimized data was cropped to a 96×96×96 effective anatomical region to obtain the extracted data. This process removed irrelevant background areas while preserving the complete cardiac structure, significantly improving the efficiency of subsequent processing. Finally, data augmentation operations were performed, including random intensity shift enhancement (applying a global intensity perturbation of ±10% with a probability of 0.5) and random spatial transformation enhancement (around the Z-axis). (Random rotation of radians and anisotropic scaling ±10%), finally obtaining the preprocessed training dataset.

[0099] 2. Construct a multimodal cardiac image segmentation model that integrates multi-receptor field networks and attention shifting:

[0100] like Figure 1As shown, the constructed multimodal cardiac image segmentation (MRFTA) model that integrates multi-receptive field convolution and transfer attention includes a TransUnet-based symmetric encoder-decoder architecture, a linear projection module, a feature mapping module, multiple multi-receptive field depth convolution downsampling modules, multiple multi-receptive field depth convolution upsampling modules, and multiple 3D transfer attention Transformer modules.

[0101] Specifically, multiple multi-receptive-field depth convolutional downsampling modules are sequentially set in the encoder path of the TransUnet-based symmetric encoder-decoder architecture to progressively extract local and long-range features, thereby achieving spatial dimension compression and feature abstraction.

[0102] Specifically, the linear projection module is set in the connection layer between the encoder output and the bottleneck part of the TransUnet-based symmetric encoder-decoder architecture, and transforms the convolutional feature map sequence into a Transformer-compatible vector sequence.

[0103] Specifically, multiple 3D attention-transferring Transformer modules are sequentially placed at the bottleneck part of the TransUnet-based symmetric encoder-decoder architecture to model remote spatial dependencies through a 3D attention-transferring mechanism;

[0104] Specifically, the feature mapping module is set in the interface layer between the bottleneck part of the TransUnet-based symmetric encoder-decoder architecture and the decoder path, and remaps the Transformer output sequence into a spatial feature map structure.

[0105] Specifically, multiple multi-receptive-field depth convolutional upsampling modules are sequentially set in the decoder path of the TransUnet-based symmetric encoder-decoder architecture. Spatial resolution is gradually restored through deconvolution and skip connections, and multi-scale contextual information is fused to generate a segmentation map.

[0106] 2.1 Encoder:

[0107] To fully leverage the strengths of CNNs in extracting local features and Transformers in modeling long-range dependencies, the network uses multi-layered 3D CNNs to extract local features in the lower layers of the encoder, and 3D Transformers in the deeper layers to model the global relationships between features. Specifically, when the model receives image data ( After input, the spatial and depth information of the original data is parsed through multiple multi-receptive-field depthwise convolutional downsampling (MRFDD) modules. At the same time, 3×3×3 convolutions with a stride of 2 are applied for downsampling to generate feature maps with rich local information. , Where K=128. To ensure a comprehensive representation of each volume, the generated feature map will be... By using a linear projection module, specifically convolutional layers for linear projection, the channel dimension is increased from K=128 to k=512. Since the Transformer layer requires a sequence as input, this invention compresses the spatial and depth dimensions into one dimension, resulting in a feature map. , ,in, To preserve important positional information during segmentation, this invention introduces learnable position embedding. , And it is added to the feature map by direct addition. The feature embedding is obtained by fusion. Feature mapping and feature embedding The expression is as follows:

[0108] (1)

[0109] (2)

[0110] in, It is a linear projection operation. For feature maps, For position embedding.

[0111] Ultimately, Input to A low-complexity 3D transferred attention Transformer module (TA3D) captures long-range spatial dependencies and generates the final feature embedding. .

[0112] 2.1.1 Multi-receptive-field depthwise convolutional downsampling module (MRFDD):

[0113] To better extract multi-scale feature information and rich contextual relationships from images, while improving the flexibility and efficiency of the model, this invention introduces a multi-receptive-field depthwise convolutional downsampling module (MRFDD) in the encoder section. This module extracts multi-scale information by utilizing convolutional kernels of different sizes and reduces resource consumption during multiple convolutions through depthwise convolution. Each MRFDD contains two multi-receptive-field depthwise convolutional blocks (MRFDB). In the MRFDB of this invention, preprocessing is first performed using batch normalization layers and ReLU6 activation layers. Then, a multi-receptive-field depthwise convolutional module (MRFD) is used to capture multi-scale and multi-resolution contextual information. Specifically, this invention first uses depthwise separable convolutions with kernel sizes of 3 and 5 in parallel for feature extraction, and then applies batch normalization layers and ReLU6 activation layers respectively for processing. Finally, since depthwise convolution ignores the relationships between channels, channel shuffling is used to reintegrate the correlations between channels. The expressions for MRFDB, MRFD, and depthwise convolutional blocks in this embodiment are as follows:

[0114] (3)

[0115] (4)

[0116] (5)

[0117] in, For multi-receptive field deep convolutional blocks, For multi-receptive field depthwise convolutional modules, and These are batch normalization and ReLU6 activation, respectively. For feature splicing, For mixed washing of channels, For a depthwise convolution with a kernel size of s, The input feature map.

[0118] 2.1.2 3D Attention Transfer Transformer Module (TA3D):

[0119] To reduce the computational complexity of the model without compromising remote modeling capabilities, this invention introduces a 3D Transfer Attention Transformer module (TA3D). TA3D consists of a Transfer Attention module (TA) and a Feedforward Network (FFN). The output of a TA3D can be expressed by the following formula:

[0120] (6)

[0121] (7)

[0122] (8)

[0123] in, This indicates the attention shifting module. Representation layer normalization, Indicates a feedforward network. For the first In each Transformer layer, the intermediate feature representation is processed by the transferred attention module and then supplemented with residual connections. For the first The output of each Transformer layer For the first The output of each Transformer layer This is the index of the current layer in the Transformer module stack. This is a random deactivation operation. Let Gaussian error be the activation function of the linear unit. and This is a two-way linear projection.

[0124] (1) Attention Shifting Module (TA):

[0125] The fundamental reason for the high complexity of Transformer is that the Softmax attention mechanism in Transformer has... The computational complexity, such as Figure 2 As shown in (b) in the figure. And having linear complexity (i.e. Linear attention (with its computational cost) is generally considered to be far less expressive than Softmax attention in terms of model representation. Figure 2 As shown in (a) above. Most people consider these two attention mechanisms as independent methods, with researchers either attempting to reduce the computational cost of Softmax attention or to improve the performance of linear attention. However, according to most current research results, the existing optimized linear attention still falls short of the model expressiveness of Softmax attention, and existing methods for reducing the computational cost of Softmax attention suffer some loss in long-range modeling capability. Therefore, this invention introduces a new attention mechanism, namely, transferred attention, which cleverly combines the advantages of Softmax attention and linear attention, maintaining linear computational complexity without sacrificing model expressiveness, such as... Figure 2 As shown in (c) in the figure.

[0126] Specifically, the expressions for Softmax attention and linear attention are as follows:

[0127] (9)

[0128] (10)

[0129] in, This represents Softmax attention. Represents linear attention. These represent the query, key, and value, respectively. The dimension representing the key. Represents the kernel function.

[0130] The attention shifting proposed in this invention effectively combines Softmax attention and linear attention; specifically, it does so by introducing a new shift token. T Avoid and The expression for directly calculating similarity and shifting attention is as follows:

[0131] (11)

[0132] Specifically, the attention-shifting mechanism of this invention consists of two phases: a global aggregation phase and a global broadcast phase. First, in the global aggregation phase, a new attention-shifting token is defined. T As a query, a Softmax attention computation is applied to it to aggregate the values. V Information to obtain transfer characteristics Subsequently, during the global broadcast phase, the transfer features will be... As a value, the transfer token T As a key, in the query matrix A second Softmax attention calculation is performed, broadcasting global information from the transition features to each query token, thus achieving an efficient mapping from global context to local features.

[0133] Shifting attention by introducing transfer tokens T Successfully bypassed and The requirement for pairwise similarity calculation. Setting the number of transfer tokens. n As a hyperparameter, thus achieving This invention achieves linear complexity while effectively preserving the model's long-range modeling capability. For acquiring transfer tokens, a pooling strategy is chosen. Furthermore, the attention transfer process establishes a generalized linear attention paradigm through two Softmax attention calculations, where the two Softmax similarity calculations serve as kernel functions.

[0134] 2.2 Decoder:

[0135] To accommodate the input dimensions of a 3D CNN decoder, this invention first applies a feature mapping module to project the sequence data into a standard feature map, and simultaneously uses convolutional blocks to reduce the channel dimension from... k Reduce to K Specifically, it involves processing the output sequence of the Transformer. Remodeling into feature maps Following this, multiple multi-receptive-field depthwise convolutional upsampling modules (MRFDU) and 3×3×3 convolution operations are used to refine the feature maps generated by the encoder. Detailed feature extraction and upsampling are performed to generate refined segmentation results. Furthermore, skip connections are applied to fuse encoder and decoder features.

[0136] Multi-receptive-field depthwise convolutional upsampling module (MRFDU):

[0137] To improve the performance of the model in generating high-resolution images while maintaining computational efficiency, this invention introduces a multi-receptive-field depth convolutional upsampling (MRFDU) module. In MRFDU, multi-scale feature extraction is first performed using MRFD (the MRFD process in this part is the same as the MRFD process in MRFDD), followed by two rounds of convolutional layers, a batch normalization layer, and a ReLU activation layer. The formula for MRFDU is as follows:

[0138] (12)

[0139] (13)

[0140] in, For multi-receptive-field depthwise convolutional upsampling modules, For multi-receptive field depthwise convolutional modules, For ordinary convolution, and These are batch normalization and ReLU activation, respectively.

[0141] 3. Training a multimodal cardiac image segmentation model that integrates multi-receptive field convolution and attention shifting:

[0142] To effectively train the model proposed in this invention, two complementary loss functions are combined in a 1:1 ratio, namely... Dice loss function and TopK Loss function. Dice Loss functions are used to mitigate the problem of data imbalance, while TopK The loss function focuses on the model's performance on the most challenging samples.

[0143] DiceThe loss function is defined as:

[0144] (14)

[0145] in, for Dice loss function and They are the first The predicted results and the actual results for each sample. N It is the total number of samples. It is a very small constant to avoid division by zero.

[0146] TopK Loss functions focus on the most challenging one sample, and TopK The loss function is defined as:

[0147] (15)

[0148] in, for TopK loss function For the first The sample (corresponding to the first) The prediction results for the most difficult-to-segment sample. K This represents the number of the most difficult samples to consider.

[0149] Therefore, the final loss function of this invention is defined as:

[0150] (16)

[0151] in, For loss function, for Dice loss function for TopK Loss function.

[0152] To verify the effectiveness and feasibility of this invention, the proposed method was trained and tested on the CT and MRI modalities of the MM-WHS 2017 dataset and compared with other methods. This dataset, an open-source dataset provided by the 2017 Multimodal Whole Heart Challenge, contains 60 cardiac CT images and 60 cardiac MRI images, covering the entire cardiac substructure. The goal of this challenge is to accurately segment multimodal whole cardiac structures, including: left ventricular myocardium (Myo), left atrial chamber (LA), left ventricular chamber (LV), right atrial chamber (RA), right ventricular chamber (RV), ascending aorta (AA), and pulmonary artery (PA). Since only 20 sets of publicly labeled data were available, to avoid overfitting, this invention borrows the method of generating pseudo-labels in semi-supervised training, generating pseudo-labels for 40 unlabeled images using a 3D UX-NET network. 32 of these pseudo-labeled images were used as the training set, and 8 were used as the validation set. Finally, 20 images with publicly labeled information were used as the test set.

[0153] The effects of this invention can be further illustrated by the following simulation experiments:

[0154] 1. Simulation conditions:

[0155] This invention was simulated using the PyTorch 1.13.1 framework on an Ubuntu 18.04.1 operating system with an Intel® Xeon® Platinum 8260Y CPU with a clock speed of 2.40GHz, 24GB of memory per GPU, an NVIDIA GeForce RTX 3090 GPU, and a CPU with a CPU of 18.04.1. The experimental code was written in Python 3.8.

[0156] 2. Simulation details:

[0157] During training, the AdamW algorithm is used with a batch size of 2, weight decay of 0.0005, an initial learning rate of 0.0001, and a maximum of 40,000 iterations. The best-performing model is saved based on the Dice coefficient. In the multi-receptive-field deep convolution module, the kernel size is set to [3, 5]. Furthermore, in TA3D, this invention transfers the number of tokens... n Set it to 64.

[0158] 3. Simulation content:

[0159] To fully demonstrate the feasibility and effectiveness of the present invention, the simulation includes whole-heart structure segmentation experiments, comparative experiments, and ablation experiments on the CT and MRI modalities of the MM-WHS 2017 dataset.

[0160] 4. Evaluation Criteria:

[0161] This invention uses three commonly used metrics to evaluate the performance of MRFTA: Dice coefficient, Jaccard coefficient, and Hausdorff distance.

[0162] The Dice coefficient is a metric that measures the similarity between two sets and is commonly used to evaluate the accuracy of image segmentation. Its calculation formula is:

[0163] (17)

[0164] in, A and B These represent the predicted segmentation result and the actual segmentation result, respectively. Dice The coefficient ranges from 0 to 1. The closer the value is to 1, the higher the overlap between the segmentation result and the true result, and the better the segmentation effect.

[0165] The Jaccard coefficient, also known as the Intersection over Union (IoU), is another commonly used segmentation evaluation index. Its calculation formula is as follows:

[0166] (18)

[0167] in, A and B These represent the predicted segmentation result and the actual segmentation result, respectively.

[0168] The Jaccard coefficient also ranges from 0 to 1; the closer the value is to 1, the higher the overlap between the segmentation result and the true result. Compared to the Dice coefficient, the Jaccard coefficient is more sensitive to the boundaries of the segmentation result.

[0169] Hausdorff distance is a metric that measures the maximum mismatch between two point sets. It is often used to evaluate the boundary accuracy of segmentation results, and is defined as follows:

[0170] (19)

[0171] in, Indicates the Hausdorff distance. D Point a and points b The distance between them; the smaller the Hausdorff distance, the closer the boundary of the segmentation result is to the true boundary.

[0172] 5. Experimental Results:

[0173] (1) Comparison of experimental results:

[0174] The performance comparison results of different models on the MM-WHS 2017 CT dataset are shown in Table 1 below.

[0175] Table 1. Performance comparison of different models on the MM-WHS 2017 CT dataset:

[0176] ;

[0177] As shown in Table 1, the MRFTA model of this invention achieves state-of-the-art results across all three performance metrics: an average Dice coefficient of 93.65%, a Jaccard coefficient of 93.1%, and a Hausdorff distance of 42.02 mm. Furthermore, the MRFTA model obtains optimal Dice coefficients for each cardiac substructure.

[0178] Table 2 below shows the performance comparison results of the MRFTA model and other methods on the MM-WHS 2017 MRI dataset.

[0179] Table 2. Performance comparison of different models on the MM-WHS 2017 MRI dataset:

[0180] ;

[0181] As shown in Table 2, while the MRFTA model did not achieve the optimal Jaccard coefficient, it did achieve the best average Dice coefficient of 81.21% and the smallest Hausdorff distance of 32.94 mm. Furthermore, the MRFTA model also obtained the best Dice coefficients for five cardiac substructures: Myo, LA, LV, RV, and PA. Moreover, although the SwinUNETR model achieved suboptimal segmentation results on the CT dataset, its performance on the MRI dataset was unsatisfactory. In contrast, the MRFTA model of this invention achieved optimal results on both CT and MRI datasets, demonstrating its superior robustness. Compared to the CT dataset, the superior performance of the MRFTA model on noisier MRI images further indicates its stronger generalization ability.

[0182] like Figure 3 As shown, while the VT-Unet model is highly efficient, its segmentation accuracy is far inferior to that of the MRFTA model of this invention. Compared with other existing methods, the MRFTA model of this invention achieves the highest Dice coefficient while maintaining a relatively small model size and low computational complexity.

[0183] (2) Results of the whole heart structure segmentation experiment:

[0184] Figure 4This visually demonstrates the whole-heart segmentation results of the MRFTA model compared to other segmentation models. Figure 4 Based on the generated sliced ​​images and 3D segmented images, the segmentation results of the MRFTA model of this invention are closer to reality. In particular, the MRI segmentation results clearly show that the segmentation results of the MRFTA model are smoother.

[0185] (3) Ablation test results:

[0186] 1. Ablation study of kernel size in MRFD:

[0187] To more effectively optimize the receptive field size in MRFD, this invention conducted ablation experiments with different convolutional kernel sizes, and the results are shown in Table 3 below.

[0188] Table 3. Ablation effects of MRFD with different kernel sizes on the MM-WHS 2017 CT dataset:

[0189] ;

[0190] Based on the results in Table 3, using a single receptive field for depthwise separable convolution leads to poor segmentation performance. Therefore, this invention chooses to use combinations of receptive fields of different sizes simultaneously to capture image features at multiple scales. After experimenting with various combinations, we determined the optimal convolutional kernel size to be [3, 5]. This choice ensures optimal segmentation performance while maintaining manageable model complexity.

[0191] 2. Ablation study of MRFD sites:

[0192] Further experiments were conducted to explore the position of MRFD in the model. The results are shown in Table 4 below. The first group is the model obtained by replacing all MRFD in MRFDD with ordinary convolutions with a kernel size of 3, and deleting MRFD in MRFDU. The second group is the model obtained by replacing MRFD in MRFDD with ordinary convolutions with a kernel size of 3. The third group is the model obtained by deleting MRFD in MRFDU. The fourth group is the model obtained by adding ordinary convolutions with a kernel size of 3 after each MRFD in MRFDD. The fifth group is the MRFTA proposed in this invention.

[0193] Table 4. Location of MRFD in the model structure in ablation experiments on the MM-WHS 2017 CT dataset:

[0194] ;

[0195] As shown in Table 4, using MRFD alone in either the encoder or decoder does not achieve optimal performance. Directly using MRFD in the encoder to replace traditional convolution for feature extraction can even lead to a performance degradation. However, when combined with ordinary convolution to form MRFDU, optimal performance is achieved. Furthermore, the results in Group 4 verify the superiority of using MRFDD, i.e., using MRFD to replace ordinary convolution and combining it with MRFDU, thus confirming the effectiveness of the MRFTA model structure proposed in this invention.

[0196] 3. Ablation studies of different modules in MRFTA:

[0197] To verify the effectiveness of the MRFTA architecture, the present invention conducted ablation experiments on its core modules on the MM-WHS 2017 CT dataset, and the results are shown in Table 5 below.

[0198] Table 5. Ablation study results of different modules of MRFTA on the MM-WHS 2017 CT dataset:

[0199] ;

[0200] The results in Table 5 show that both TA3D and MRFD improve the segmentation performance of the model. Although TA3D slightly increases the number of parameters in the model, it also reduces the model's FLOPs by 9.36%. MRFD, through structural optimization, successfully compresses the number of parameters and computational complexity by 11.94% and 21.49%, respectively, demonstrating excellent lightweight characteristics. Experimental data show that each module achieves synergistic optimization of parameter efficiency and computational resource consumption while improving segmentation accuracy.

[0201] 4. Ablation studies of different attention mechanisms:

[0202] To verify the effectiveness of TA3D, this invention conducted ablation experiments with different attention mechanisms, and the results are shown in Table 6 below.

[0203] Table 6 Performance evaluation results of different attention mechanisms on the MM-WHS 2017 CT dataset:

[0204] ;

[0205] As shown in Table 6, the experimental results demonstrate that the multi-head self-attention mechanism based on Softmax attention achieves higher segmentation accuracy compared to Softmax-free attention. Particularly noteworthy is that the proposed transfer attention mechanism achieves optimal performance on the Dice coefficient, Jaccard coefficient, and Hausdorff distance metrics, highlighting the significant advantages of global context modeling. In terms of computational efficiency, TA3D achieves the highest segmentation accuracy while maintaining computational complexity comparable to linear attention. These experimental results fully demonstrate that TA3D, through its innovative attention transfer strategy, achieves Pareto optimality between model accuracy and computational efficiency.

[0206] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method for cardiac image segmentation fusing multi-receptive field convolution and shift attention, characterized in that, The method comprises the following steps: S1, preprocessing the original heart image training data to generate preprocessed training data set; S2, constructing a multi-modal heart image segmentation model fusing multi-receptive field convolution and shift attention; The constructed multi-modal heart image segmentation model fusing multi-receptive field convolution and shift attention comprises a symmetric encoder-decoder architecture based on TransUnet, a linear projection module, a feature mapping module, a plurality of multi-receptive field deep convolution down-sampling modules, a plurality of multi-receptive field deep convolution up-sampling modules, and a plurality of three-dimensional shift attention Transformer modules; Each three-dimensional shift attention Transformer module comprises a shift attention module and a feedforward network connected in sequence; The shift attention module is used for extracting global features; The feedforward network is used for nonlinear transformation and information processing of the extracted global features to obtain the output of the three-dimensional shift attention Transformer module; The working method of the shift attention module is as follows: Step 1, global aggregation: define a new shift token as a query, the shift token is obtained by pooling the original query token, and a Softmax attention calculation operation is applied to the shift token to aggregate information from each value to obtain a shift feature; Step 2, global broadcast: taking the shift feature as a value and the shift token as a key, a Softmax attention calculation operation is performed on the query matrix to broadcast global information from the shift feature to each query token, realizing effective mapping from global context to local features, and obtaining global features; The plurality of multi-receptive field deep convolution down-sampling modules are sequentially arranged in the encoder path of the symmetric encoder-decoder architecture based on TransUnet; The linear projection module is arranged in the connection layer between the output end of the encoder of the symmetric encoder-decoder architecture based on TransUnet and the bottleneck part; The plurality of three-dimensional shift attention Transformer modules are sequentially arranged in the bottleneck part of the symmetric encoder-decoder architecture based on TransUnet; The feature mapping module is arranged in the interface layer between the bottleneck part of the symmetric encoder-decoder architecture based on TransUnet and the decoder path; The plurality of multi-receptive field deep convolution up-sampling modules are sequentially arranged in the decoder path of the symmetric encoder-decoder architecture based on TransUnet; S3, training the constructed multi-modal heart image segmentation model using the preprocessed training data set to obtain a trained segmentation model; S4, segmenting the to-be-tested heart image using the trained segmentation model to output a segmentation result.

2. The method of claim 1, wherein the fused multi-receptive field convolution and attention- guided cardiac image segmentation method further comprises: Step S1 comprises the following steps: S11, mapping the non-continuous anatomical structure identifier in the original heart image training data into an 8-dimensional binary vector to output binary label data; S12, performing anatomical structure space standardization processing on the original heart image training data and the binary label data respectively, and uniformly adjusting all data to the RAS coordinate system through affine transformation to obtain image data and label data; S13, the image data is processed by bilinear interpolation, and the label data is processed by nearest neighbor interpolation, and the spatial resolution of the processed image data and label data is uniformly resampled to 1mm*1mm*1mm to obtain standardized data; S14, linearly scale the voxel value of the standardized data to the range of [0, 1], and then process it through the contrast limited adaptive histogram equalization algorithm to obtain preprocessed data; S15, three-dimensional region of interest extraction is performed on the preprocessed data to obtain extracted data; S16, the extracted data is processed by random intensity offset and random spatial transformation to obtain preprocessed training data set.

3. The method of claim 1, wherein the method further comprises: Each multi-receptive field deep convolution down-sampling module comprises two multi-receptive field deep convolution blocks connected in series, and each multi-receptive field deep convolution block comprises a batch normalization layer, an activation layer and a multi-receptive field deep convolution module connected in series.

4. The method of claim 3, wherein the fused multi-receptive field convolution and attention- transferred cardiac image segmentation method is characterized by, The working method of each multi-receptive field deep convolution module is as follows: Step one, the preprocessed training data set is input, and a depth separable convolution with a convolution kernel size of 3 and 5 is used in parallel for feature extraction to obtain two receptive field feature extraction data; Step two, the two receptive field feature extraction data are sequentially preprocessed by a batch normalization layer and an activation layer to output two receptive field regularization feature data; Step three, after feature splicing of the two receptive field regularization feature data, the topological connection relationship between the feature channels is dynamically reorganized by channel shuffling operation to promote cross-receptive field feature information interaction and fusion to obtain the output of the multi-receptive field deep convolution module.

5. The method of claim 1, wherein, Each multi-receptive field deep convolution up-sampling module comprises a multi-receptive field deep convolution module, a first convolution layer, a first batch normalization layer, a first activation layer, a second convolution layer, a second batch normalization layer and a second activation layer connected in series.

6. The method of claim 1, wherein the method further comprises: In step S3, during the training of the constructed multi-receptive field convolution and attention transfer multi-modal cardiac image segmentation model, the training process of the multi-receptive field convolution and attention transfer multi-modal cardiac image segmentation model is constrained by a loss function.

7. The method of claim 6, wherein the fused multi-receptive field convolution and attention- guided cardiac image segmentation method further comprises: The loss function is composed of Dice a loss function and TopK a loss function. The expression of the loss function is: ； wherein, is a loss function, is Dice a loss function, is TopK a loss function.

Images