Three-dimensional heart image segmentation method and device, equipment and medium

Abstract

The invention relates to the technical field of medical image processing, and discloses a three-dimensional heart image segmentation method and device, equipment and a medium. According to the method, a trained whole heart segmentation model is adopted to segment a three-dimensional heart image, feature extraction is carried out on the three-dimensional heart image through a double encoder with two encoding branches of pyramid pooling Transformer and lightweight large kernel convolution, and features and context information of different scales of the three-dimensional heart image are extracted. A segmentation mask and a residual mask of a three-dimensional heart image are generated through a double decoder with two decoding branches of a segmentation decoder and a residual decoder according to features of different scales extracted by the double decoder and context information, the segmentation mask indicates different areas in the three-dimensional heart image, and the residual mask indicates different areas in the three-dimensional heart image. The residual mask indicates the difference between the context slices of each region in the three-dimensional heart image, and segmentation is performed according to the segmentation mask and the residual mask, so that the segmentation precision of the heart image is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a three-dimensional cardiac image segmentation method, device, equipment and medium. Background Art

[0002] Cardiac pathology, including coronary artery stenosis, myocardial infarction, and heart valve disease, poses significant challenges and therapeutic needs for modern medicine. In this field, accurate segmentation of cardiac structures is crucial for lesion detection, quantitative assessment, and the design of personalized treatment plans.

[0003] Convolutional Neural Networks (CNN) have performed outstandingly in medical image processing, including tasks such as classification, segmentation, detection, and localization. Traditional CNN models are limited in their ability to model spatial context when processing three-dimensional medical images, especially in complex cardiac structures, where key tasks such as pulmonary artery boundary segmentation have limited performance. Based on this, a network structure specifically for 3D medical images is proposed, but due to the limitations of the local receptive field of traditional CNNs, it is difficult to directly capture long-range dependencies. In cardiac structures with complex branching structures, the segmentation of pulmonary artery boundaries still faces significant challenges. In addition, pure CNN models usually rely on specially designed network architectures trained for specific datasets and tasks, lacking versatility and reproducibility.

[0004] The Transformer model is a neural network architecture that uses an attention mechanism as its core and is used to process sequence data. Its multi-head self-attention mechanism helps capture global correlations and can make up for the limitations of the CNN receptive field. However, the computational complexity of the Transformer model's multi-head self-attention module grows exponentially with the increase in image size, and the Transformer's ability to represent local high-level semantic information is limited, resulting in low efficiency and accuracy in the segmentation of three-dimensional medical images. Summary of the invention

[0005] The object of the present invention is to provide a three-dimensional cardiac image segmentation method, device, equipment and medium, which can improve the efficiency and accuracy of three-dimensional cardiac image segmentation.

[0006] To solve the above technical problems, an embodiment of the present invention provides a three-dimensional cardiac image segmentation method, comprising the following steps:

[0007] Obtain three-dimensional cardiac images;

[0008] Input the 3D heart image into the trained whole heart segmentation model; the whole heart segmentation model uses a network structure including: a dual encoder with two encoding branches, pyramid pooling Transformer and lightweight large kernel convolution, and a dual decoder with two decoding branches, a segmentation decoder and a residual decoder;

[0009] The pyramid pooling transformer is used to apply multiple average pooling layers of different scales on the three-dimensional cardiac image to generate pyramid feature maps of different scales, and the multi-head self-attention mechanism in the pyramid pooling transformer is used to extract features from the pyramid feature maps of different scales; the context information in the pyramid feature maps of different scales is captured through lightweight large kernel convolution; the features extracted by the pyramid pooling transformer and the context information captured by the lightweight large kernel convolution are fused to obtain the output features of the dual encoder;

[0010] The output features of the dual encoder are upsampled by a segmentation decoder, and the upsampled features are fused with the output features of the dual encoder to generate a segmentation mask for indicating multiple different regions in the three-dimensional cardiac image; the residual context features between multiple slices corresponding to each region in the segmentation mask are extracted by a residual decoder to generate a residual mask for indicating the difference between the context slices of each region in the three-dimensional cardiac image;

[0011] The three-dimensional heart image is segmented according to the segmentation mask and the residual mask of the three-dimensional heart image.

[0012] In some optional embodiments, the multi-head self-attention mechanism in the pyramid pooling Transformer is used to extract features from pyramid feature maps of different scales, including:

[0013] Encode the corresponding positions in pyramid feature maps of different scales through deep convolution;

[0014] Flattening the pyramid feature maps of different scales respectively, and connecting the pyramid feature maps of different scales according to the encoding of the corresponding positions in the pyramid feature maps of different scales, so as to obtain a sequence of a preset length from the pyramid feature maps of different scales;

[0015] A multi-head self-attention mechanism is used to extract features from sequences of preset length.

[0016] In some optional embodiments, the capturing of context information in pyramid feature maps of different scales by lightweight large kernel convolution includes:

[0017] For each scale of the pyramid feature map, a large kernel with a kernel size of 5×5×5 is used for depthwise convolution, and the number of channels of the pyramid feature map is adjusted to 4 times through two 1×1×1 point convolution channel scaling blocks to capture the contextual information in pyramid feature maps of different scales.

[0018] In some optional embodiments, the fusion of features extracted by pyramid pooling Transformer and context information captured by lightweight large kernel convolution includes:

[0019] The features of the pyramid feature map of each scale extracted by the pyramid pooling Transformer are fused with the contextual information in the pyramid feature map of the corresponding scale captured by the lightweight large kernel convolution.

[0020] In some optional embodiments, before segmenting the three-dimensional cardiac image according to the segmentation mask and the residual mask of the three-dimensional cardiac image, the method further includes:

[0021] Segmenting a three-dimensional cardiac image into multiple slices;

[0022] For each area in the three-dimensional cardiac image indicated by the segmentation mask, the residual information between context slices in multiple slices corresponding to the area is generated by the residual decoder, and the generated residual information is returned to the segmentation decoder through attention guidance, so that the segmentation decoder generates a segmentation mask of the three-dimensional cardiac image according to the residual information.

[0023] In some optional embodiments, the residual decoder includes multiple feature extraction layers, and the residual information between context slices in multiple slices corresponding to the region is generated by the residual decoder, including:

[0024] For each slice among the multiple slices corresponding to the region, respectively obtain residual information between the slice and two slices adjacent to the slice, and add the residual information between the slice and the two slices adjacent to the slice to obtain residual information extracted by a feature extraction layer of the residual decoder;

[0025] For each feature extraction layer of the residual decoder, the residual information extracted by the feature extraction layer is added to the residual information extracted by the feature extraction layer of the previous layer to obtain the residual information between the context slices in the multiple slices corresponding to the region.

[0026] In some optional embodiments, during the training process, the residual decoder uses the ground truth residual mask to train the generated residual mask based on the deep supervision mechanism.

[0027] An embodiment of the present invention further provides a three-dimensional cardiac image segmentation device, comprising:

[0028] An image acquisition module, used for acquiring three-dimensional cardiac images;

[0029] An image input module is used to input the three-dimensional heart image into the trained whole heart segmentation model; the whole heart segmentation model adopts a network structure including: a dual encoder with two encoding branches, pyramid pooling Transformer and lightweight large kernel convolution, and a dual decoder with two decoding branches, a segmentation decoder and a residual decoder;

[0030] The image segmentation module is used to generate pyramid feature maps of three-dimensional cardiac images of different scales by applying multiple average pooling layers of different scales on the three-dimensional cardiac images through the pyramid pooling transformer, and extract features from the pyramid feature maps of different scales using the multi-head self-attention mechanism in the pyramid pooling transformer; capture the context information in the pyramid feature maps of different scales through lightweight large kernel convolution; and fuse the features extracted by the pyramid pooling transformer and the context information captured by the lightweight large kernel convolution to obtain the output features of the dual encoder;

[0031] The output features of the dual encoder are upsampled by a segmentation decoder, and the upsampled features are fused with the output features of the dual encoder to generate a segmentation mask for indicating multiple different regions in the three-dimensional cardiac image; the residual context features between multiple slices corresponding to each region in the segmentation mask are extracted by a residual decoder to generate a residual mask for indicating the difference between the context slices of each region in the three-dimensional cardiac image;

[0032] The three-dimensional heart image is segmented according to the segmentation mask and the residual mask of the three-dimensional heart image.

[0033] An embodiment of the present invention also provides a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned three-dimensional cardiac image segmentation method.

[0034] An embodiment of the present invention further provides a computer-readable storage medium storing a computer program, wherein the computer program implements the above-mentioned three-dimensional cardiac image segmentation method when executed by a processor.

[0035] The three-dimensional cardiac image segmentation method provided by the present invention has at least the following beneficial effects:

[0036] The three-dimensional heart image is segmented by a trained whole heart segmentation model. The network structure adopted by the whole heart segmentation model includes: a dual encoder with two encoding branches, pyramid pooling transformer (i.e., P2T) and lightweight large kernel convolution, and a dual decoder with two decoding branches, segmentation decoder and residual decoder. Among them, the pyramid pooling transformer of the dual encoder is a transformer model for visual scene understanding. It introduces the pyramid pooling technology into the multi-head self-attention module of the visual transformer, generates pyramid feature maps of three-dimensional heart images of different scales through pyramid pooling, and then extracts the features of three-dimensional heart images through the multi-head self-attention mechanism of the transformer model, thereby realizing the feature extraction of three-dimensional heart images at different scales, reducing the complexity of image calculation and improving the efficiency of image processing. The lightweight large kernel convolution can capture the contextual information in the pyramid feature maps of different scales. The fusion of the two can obtain comprehensive and detailed three-dimensional heart image features, which is helpful to improve the subsequent segmentation accuracy of three-dimensional heart images.

[0037] In addition, while the segmentation decoder in the dual decoder obtains a segmentation mask indicating multiple different areas in the three-dimensional cardiac image for image segmentation, the residual decoder is also used to extract the residual context features of the segmentation mask, and generates a residual mask for indicating the differences between context slices of each area in the three-dimensional cardiac image. The three-dimensional cardiac image is segmented based on these two, which can further improve the segmentation accuracy of the three-dimensional cardiac image. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] One or more embodiments are exemplarily described by the pictures in the corresponding drawings, and these exemplary descriptions do not constitute limitations on the embodiments.

[0039] Figure 1 is a flow chart of a three-dimensional cardiac image segmentation method provided according to an embodiment of the present invention;

[0040] Figure 2 is a schematic diagram of a network structure of a whole heart segmentation model provided according to an embodiment of the present invention;

[0041] Figure 3 is a schematic structural diagram of a dual encoder provided according to an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of segmentation and comparison of a three-dimensional cardiac image provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0043] In order to make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the embodiments of the present invention, many technical details are proposed in order to enable the reader to better understand the present invention. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present invention can be implemented. The division of the following embodiments is for the convenience of description and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined and referenced with each other without contradiction.

[0044] Firstly, the related technologies of the present invention are introduced below:

[0045] 1. Pyramid pooling:

[0046] Pyramid pooling is a key technology widely used in computer vision tasks, including semantic segmentation, object detection, image classification, etc. It significantly improves model performance by capturing contextual information from feature maps at multiple scales. Pyramid pooling is introduced into deep convolutional neural networks for image classification and object detection, and robust visual recognition is achieved by aggregating feature maps of different scales to form a fixed-size feature representation. Some studies apply the concept of pyramid pooling to the input feature maps of the multi-head self-attention mechanism (MHSA) and incorporate pyramid pooling into the visual transformer structure. It is worth noting that the pyramid visual transformer and multiscale visual transformer (MViT) first use a single-layer pooling operation to reduce the number of markers involved in the calculation of the MHSA module and model the relationship between "marked regions". However, these methods have limitations in capturing finer-grained semantic relationships between markers. In contrast, Pyramid Pooling Transformer (P2T) is a Transformer model designed for visual scene understanding tasks. It proposes a method to build a pyramid structure, which solves the challenges associated with complex calculations on long sequences by aggregating the keys and values ​​in MHSA into feature information of different scales, while retaining the global relationship modeling features of Transformer and providing practical backbone network support for downstream tasks.

[0047] At present, strategies such as pyramid pooling transformer have been verified in general visual tasks, and their application in medical imaging tasks has been explored. For example, hierarchical features are constructed using a multi-head spatial pyramid attention mechanism to achieve segmentation of two-dimensional skin lesion images; P2T is used as a backbone network to provide robust background features for the downstream polyp segmentation network. However, compared with single-case images such as skin lesions, the heart is a complex structure composed of atria, ventricles, and arteries. There are complex anatomical relationships between these components, especially in three-dimensional images, where it is crucial to consider the dependencies of global contextual information. Therefore, completing the three-dimensional substructure segmentation of the entire heart is a more challenging task. However, existing research has mainly focused on two-dimensional images, and the exploration of applying pyramid pooling transformers to 3D complex images has not been fully developed.

[0048] 2. CNN:

[0049] Convolutional neural networks (CNNs) with large convolution kernels have attracted much attention in medical image segmentation tasks. The U-Net approach, which combines large convolution kernels with an attention mechanism, has been shown to be feasible in various segmentation tasks and improves model performance by expanding the receptive field. Based on this, the effect of large convolution kernels was re-evaluated using deep convolution with large kernels, and 3D UX-Net was proposed, which uses evolutionary techniques to replace the multilayer perceptron in the Transformer to achieve a larger receptive field. However, the computational efficiency of the deep convolution operation in this module still needs to be improved. To further optimize the model, the convolution block of the U-Net is enhanced with large convolution kernels and depthwise separable convolutions, which significantly reduces the number of model parameters while retaining the context representation ability. Nevertheless, it is still worth exploring whether the context representation ability of large kernel convolution networks can compare favorably with the global modeling achieved by the Transformer.

[0050] 3. Transformer-CNN:

[0051] In recent years, segmentation methods based on CNN, Transformer and Transformer-CNN have made significant progress in the field of computer vision. For example, an attention mechanism that learns the contextual relationship between slices effectively improves the segmentation accuracy of slices; by introducing localization to reduce the calculation of non-interested regions, accurate and efficient whole-heart segmentation is achieved; by learning volumes through local and global volume self-attention mechanisms, the shortcomings of Transformer in spatial perception are compensated; by alternating local and global self-attention, the receptive field is expanded and the performance is improved while reducing the computational cost.

[0052] In order to fully explore the potential of CNN and Transformer, related technologies combine them to fully tap their potential. For example, by building a deformable Transformer to model the long-term dependency of features extracted by CNN, good segmentation performance was achieved on the organ classification dataset; the brain tumor segmentation task was redefined as a sequence prediction problem, using Swin Transformer as an encoder to provide long-range features for the fully convolutional network decoder; by introducing an efficient paired attention mechanism, high-quality segmentation masks were achieved with lower computational cost and faster inference speed. Although these methods have made progress in improving performance, it is currently difficult to effectively integrate the comprehensive global and local feature expression capabilities of the 3D pyramid pooling Transformer backbone network and the large-core CNN. In addition, it is difficult for them to capture the basic contextual residual information between slices, especially when dealing with the challenging task of segmenting complex 3D whole heart images. This limitation becomes particularly prominent.

[0053] Based on this, the present invention proposes a three-dimensional cardiac image segmentation method, which can solve the problems existing in the above-mentioned related technologies, thereby improving the segmentation accuracy and efficiency of three-dimensional cardiac images.

[0054] An embodiment of the present invention relates to a three-dimensional cardiac image segmentation method. The implementation details of the three-dimensional cardiac image segmentation method of this embodiment are specifically described below. The following content is only the implementation details provided for easy understanding and is not necessary for implementing this solution.

[0055] The specific process of the three-dimensional cardiac image segmentation method of this embodiment can be as follows: Figure 1 As shown, including:

[0056] Step 101, obtaining a three-dimensional cardiac image.

[0057] Step 102, input the three-dimensional heart image into the trained whole heart segmentation model; wherein the network structure adopted by the whole heart segmentation model includes: a dual encoder with two encoding branches, a pyramid pooling Transformer and a lightweight large kernel convolution, and a dual decoder with two decoding branches, a segmentation decoder and a residual decoder.

[0058] Step 103, applying multiple average pooling layers of different scales on the three-dimensional cardiac image through the pyramid pooling transformer to generate pyramid feature maps of different scales of the three-dimensional cardiac image, and using the multi-head self-attention mechanism in the pyramid pooling transformer to extract features from the pyramid feature maps of different scales; capturing context information in the pyramid feature maps of different scales through lightweight large kernel convolution; fusing the features extracted by the pyramid pooling transformer and the context information captured by the lightweight large kernel convolution to obtain the output features of the dual encoder.

[0059] Specifically, when using the multi-head self-attention mechanism to extract features from pyramid feature maps of different scales, the corresponding positions in the pyramid feature maps of different scales are first encoded through deep convolution, and then the pyramid feature maps of different scales are flattened respectively, and according to the encoding of the corresponding positions in the pyramid feature maps of different scales, the pyramid feature maps of different scales are connected to obtain a sequence of preset length from the pyramid feature maps of different scales, and finally the multi-head self-attention mechanism is used to extract features from the sequence of preset length.

[0060] When capturing contextual information in pyramid feature maps of different scales through lightweight large kernel convolution, for each scale of the pyramid feature map, a large kernel with a kernel size of 5×5×5 is used for depth convolution, and two 1×1×1 point convolution channel scaling blocks are used to adjust the number of channels of the pyramid feature map to 4 times to capture contextual information in pyramid feature maps of different scales.

[0061] When fusing the features extracted by the 3D pyramid pooling Transformer and the contextual information captured by the lightweight large kernel convolution, the features of the pyramid feature map of each scale extracted by the pyramid pooling Transformer are fused with the contextual information in the pyramid feature map of the corresponding scale captured by the lightweight large kernel convolution.

[0062] Step 104, upsampling the output features of the dual encoder through a segmentation decoder, and fusing the upsampled features with the output features of the dual encoder to generate a segmentation mask for indicating multiple different regions in the three-dimensional cardiac image; extracting residual context features between multiple slices corresponding to each region in the segmentation mask through a residual decoder to generate a residual mask for indicating the difference between context slices of each region in the three-dimensional cardiac image.

[0063] In one example, a three-dimensional cardiac image is first segmented into multiple slices, and then for each region in the three-dimensional cardiac image indicated by a segmentation mask, residual information between context slices in multiple slices corresponding to the region is generated by a residual decoder, and the generated residual information is returned to the segmentation decoder through attention guidance, so that the segmentation decoder generates a segmentation mask of the three-dimensional cardiac image according to the residual information.

[0064] In a specific implementation, the residual decoder includes multiple feature extraction layers. When the residual information between context slices in the multiple slices corresponding to the region is generated by the residual decoder, for each slice in the multiple slices corresponding to the region, the residual information between the slice and the two slices adjacent to the slice is obtained respectively, and the residual information between the slice and the two slices adjacent to the slice is added to obtain the residual information extracted by a feature extraction layer of the residual decoder; for each feature extraction layer of the residual decoder, the residual information extracted by the feature extraction layer is added to the residual information extracted by the feature extraction layer of the previous layer to obtain the residual information between the context slices in the multiple slices corresponding to the region. In the training process, the residual decoder uses the ground truth residual mask to train the generated residual mask based on the deep supervision mechanism.

[0065] Step 105 , segmenting the three-dimensional heart image according to the segmentation mask and the residual mask of the three-dimensional heart image.

[0066] In this embodiment, the three-dimensional heart image is segmented by a trained whole heart segmentation model, and the network structure adopted by the whole heart segmentation model includes: a dual encoder with two encoding branches, namely, a pyramid pooling transformer (i.e., P2T) and a lightweight large kernel convolution, and a dual decoder with two decoding branches, namely, a segmentation decoder and a residual decoder. Among them, the pyramid pooling transformer of the dual encoder is a transformer model for visual scene understanding, which introduces the pyramid pooling technology into the multi-head self-attention module of the visual transformer, generates pyramid feature maps of three-dimensional heart images of different scales by pyramid pooling, and then extracts the features of the three-dimensional heart image by the multi-head self-attention mechanism of the transformer model, thereby realizing the feature extraction of the three-dimensional heart image at different scales, reducing the complexity of image calculation, and improving the image processing efficiency, and the lightweight large kernel convolution captures the context information in the pyramid feature maps of different scales, and the two are integrated to obtain comprehensive and detailed three-dimensional heart image features, which helps to improve the segmentation accuracy of subsequent three-dimensional heart images. In addition, while the segmentation decoder in the dual decoder obtains a segmentation mask indicating multiple different areas in the three-dimensional cardiac image for image segmentation, the residual decoder is also used to extract the residual context features of the segmentation mask, and generates a residual mask for indicating the differences between context slices of each area in the three-dimensional cardiac image. The three-dimensional cardiac image is segmented based on these two, which can further improve the segmentation accuracy of the three-dimensional cardiac image.

[0067] In order to understand how the above three-dimensional heart image is segmented in the whole heart segmentation model, the training process of the whole heart segmentation model is introduced in detail below:

[0068] 1. Data preprocessing:

[0069] Before training the network model, all training data (i.e., all three-dimensional heart images) need to be preprocessed to accurately segment the three-dimensional heart images (i.e., the multi-modal whole heart structure). The whole heart structure includes: left ventricular (Myo) myocardium, left atrial blood cavity (LA), left ventricular blood cavity (LV), right atrial blood cavity (RA), right ventricular blood cavity (RV), ascending aorta (AA), pulmonary artery (PA). Among them, the label corresponding to each structure is represented by 7 voxel values: 205, 420, 500, 550, 600, 820 and 850, and the voxel value of 0 represents the background. In this embodiment, the 7 voxel values ​​of all medical image labels are converted offline to 1 to 7 in sequence, and the size of the three-dimensional heart image is cropped to (80, 160, 160), and irrelevant background areas are removed to better focus on the area of ​​interest.

[0070] 2. Construction of the network structure used in the whole heart segmentation model:

[0071] The network structure used in the whole heart segmentation model (i.e., P2TC) is as follows: Figure 2 As shown in the figure, it includes: a dual encoder with two encoding branches, pyramid pooling Transformer and lightweight large kernel convolution, a dual decoder with two decoding branches, segmentation decoder and residual decoder, and a bidirectional residual perception module (BRP). The P2TC network adopts a dual encoder-dual decoder architecture, in which the encoder consists of two parallel networks, pyramid pooling Transformer (i.e. 3D P2T) and lightweight large kernel convolution CNN (i.e. L2C). The dual decoder also contains two branches for generating segmentation masks and highlighting positive and negative changes in inter-slice contextual residuals.

[0072] (1) Dual encoder:

[0073] The P2TC architecture uses a dual encoder strategy to process 3D medical images at multiple scales, namely the three-dimensional cardiac images described in this embodiment. Figure 3 As shown in the figure, it contains a pyramid pooling Transformer branch and a lightweight large kernel volume convolution branch, which work together to capture more comprehensive spatial information. The pyramid pooling Transformer branch uses a multi-head self-attention mechanism to achieve feature extraction of keys and values ​​at different scales, thereby effectively reducing the computational complexity associated with long sequences. The lightweight large kernel volume convolution branch expands the perception range of the global context. Then, the outputs of the two branches are combined through a fusion module to provide a comprehensive and detailed input representation for the model. This design significantly improves the modeling and analysis capabilities of complex cardiac anatomy and enhances the expression and semantic understanding of three-dimensional medical images. In addition, the architecture also helps to achieve better performance in various three-dimensional vision tasks, including target detection, segmentation, and reconstruction. Before entering the dual encoder, the input image is first processed by a convolution layer with 32 channels and a single residual block, providing a richer feature representation for the subsequent convolution branch. Among them, the convolution layer operation uses a convolution kernel of size 3×3×3, and the number of output channels is 32.

[0074] About Pyramid Pooling Transformer (3D P2T):

[0075] The input image first passes through the patch embedding module, which is mapped, flattened and normalized. Subsequently, multiple 3D P2T blocks are stacked together. The entire branch has three stages, each with a feature dimension Ci (i = {1, 2, 3}). Projection techniques are used to complete the volume size conversion between stages. The feature map sizes of the three stages are as follows: and These feature maps are denoted by {B1, B2, B3} respectively. At each stage, the features obtained are fused with the features of the convolutional branches and used for skip connections.

[0076] Among them, the 3DP2T block: the input features are processed by a multi-head self-attention module based on pyramid pooling. Subsequently, the processed results are added to the original input and normalized by LayerNorm (LN). Next, a traditional transformer module is applied to project the features using a feed-forward network (FFN). Finally, residual connections and layer norms are applied again to further process the features.

[0077] The above process is as follows:

[0078] X att =LNX+P 3D -MHSAX

[0079] X out =LNX att +FFNX att

[0080] In the formula, X, Xatt represent P 3D -MHSA input and output, Xout represents the output of 3DP2T block.

[0081] P 3D -MHSA: First, the input X is reshaped into three-dimensional space, and then multiple average pooling layers of different scales are applied to the reshaped X to generate a pyramid feature map, denoted as P i The above process is as follows:

[0082] P i =AvgPool3D i X,i=1,2,3

[0083] At each stage, three pooling ratios are introduced, namely {8, 10, 12}, {5, 6, 7} and {2, 3, 4}. These pyramid feature maps are relatively positionally encoded through deep convolution and then flattened and concatenated. When the pooling rate is large enough, a shorter sequence P can be obtained from the input X, which contains the contextual semantic information of the input X. This sequence P can replace X as the input of the multi-head self-attention module. In the traditional multi-head self-attention module, the query, key and value are represented by Q, K and V respectively. The specific calculation method is shown in formula (6):

[0084]

[0085] Q,K,V=XW q ,PW k ,PWv

[0086] In the formula, Represents P after encoding the relative position i .W q ,W k and W v Represents the weight matrix for generating linear queries, key and value transformations.

[0087] About lightweight large kernel convolution (L2C):

[0088] The lightweight large kernel convolution branch consists of three L2C blocks, processing the features generated by the convolutional layers and residual blocks. Before each L2C block, a downsampling block is used to capture a more comprehensive context range. The branch is further divided into three stages, denoted by {E1, E2, E3}. In addition, the features of the 3DP2T branch are fused with the features of each stage to achieve skip connections. Large kernel convolution is used to extract feature representations that can capture information of a specific block size. This approach is similar to the sliding window strategy adopted by the non-local MHSA in SwinTransformer

[17] . Therefore, the global receptive field can be generated across layers, allowing features to be refined corresponding to non-overlapping windows. Based on these considerations, each stage of this embodiment uses two L2C blocks to learn dense volume segmentation while accommodating a large receptive field.

[0089] Taking the L2C module as an example, in order to improve performance while reducing the order of the model, a large kernel depthwise convolution (DWC) with a kernel size of 5×5×5 is used to simulate the window shift characteristics of the Swin Transformer

[17] . Subsequently, two 1×1×1 point convolution (PWC) channel scaling blocks are used to adjust the number of channels to 4 times. This method enriches the feature representation and eliminates redundant information between context channels. In order to reduce memory usage without affecting performance, a lightweight Hardswish activation function is used. The Hardswish activation function provides the best performance while occupying less memory.

[0090] About the Fusion Module:

[0091] In the encoder path, the output B from the 3DP2T and L2C branches is i and E i To perform fusion, a fusion function F(·) is used to generate skip connection features at each stage. Specifically, in the activated B i and E i A multiplication operation is performed between them to guide the fusion feature map in learning E i The global features are based on B iThe aggregated output features of the fusion block are used as the jump connection of the decoder to compensate for the information loss that may occur during the path transmission process. The above process is as follows:

[0092] F i =E i *1+σ(B i ),i=1,2,3

[0093] In the formula, z i represents the incoming feature map of the skip connection at the i-th stage, and σ(·) represents the Sigmoid activation function.

[0094] (2) Dual decoder:

[0095] P2TC adopts a dual decoder structure, including a segmentation decoder and a residual decoder, which are used to generate segmentation and residual masks. It is divided into three stages, and a residual context module is introduced in each stage to bridge the gap between the two decoders. The segmentation decoder upsamples its feature map through trilinear interpolation technology, and then combines it with the low-level features passed by the encoder through element-wise summation to achieve feature fusion. The fused features are input into the residual decoder. The residual decoder synthesizes the residual context features of the segmentation decoder and generates a residual mask through a refinement process. In addition, the residual decoder generates residual information of inter-slice context, which is returned to the segmentation decoder through attention guidance to enhance its perception of three-dimensional context. Finally, the model generates a segmentation mask and a corresponding inter-slice context residual mask for each input.

[0096] (3) Bidirectional residual perception module:

[0097] Considering the trend of residual variation between adjacent slices, simply relying on the residual information of adjacent slices may not be enough to provide sufficient information. Secondly, for simplicity, directly assigning zeros to the residual input layer may lead to the loss of some boundary information, which may affect the performance of the model in some cases. Therefore, we propose a bidirectional residual-aware module to enhance the model's ability to learn the residual information between positive and negative slices.

[0098] In the segmentation path, the output segmentation feature O seg It is achieved by combining the segmentation features I from the previous layer seg The feature I of the current layer is added. These segmentation features maintain the same ratio when they are passed from the encoder to the next feature layer. In the context residual path, the context residual feature I of the previous layer res Generate the corresponding output feature O res In addition, in order to solve the limitation of unidirectional extraction of residuals, the BRP module is introduced to enhance the model's ability to understand the heart edge through bidirectional learning. The process can be described as follows:

[0099] G=ConvI seg +I skip

[0100]

[0101] O seg =G*1+σO res

[0102] Where Conv(·) represents an operation including convolution operation, group normalization and Relu activation F function. σ(·) represents the Sigmoid function, and d, h and w of the feature map G represent the depth, height and width respectively. front It is used to calculate the residual between the current slice and the previous slice, and H back It is used to calculate the residual between the current slice and the next slice. These two residuals are then added together as the output of the bidirectional residual perception module and I res Combined to form the output O of the context residual path res In order to improve the sensitivity of the segmentation output to the residual, O res It is considered as an attention mechanism.

[0103] In order to speed up the training convergence of the residual decoder, a deep supervision mechanism is introduced and the ground truth residual mask Y is used res This method includes an additional convolutional layer to generate a residual mask for each deep supervision operation. Considering the unbalanced distribution of foreground and background voxels in each context residual mask, the foreground voxels are weighted in the binary cross entropy loss function, defined as:

[0104]

[0105] In the formula, i represents the residual prediction layer of the i-th layer, and the class weight is V k represents the voxel belonging to class k. The total loss of the residual decoder is defined as:

[0106]

[0107] By carefully observing the loss curve on the validation set, λ is set to 0.5. The loss function of the segmentation decoder is optimized by combining the cross entropy loss and the dice loss, which is defined as:

[0108]

[0109] In the formula, l bce is the binary cross entropy loss, and are the predicted segmentation labels and true segmentation labels at (s, h, w), and e are the smoothing factors of the dice loss.

[0110] Therefore, the total loss function is defined as follows:

[0111] L=L res +L seg

[0112] Based on the above network structure, the 3D cardiac image segmentation method of this embodiment has the following beneficial effects:

[0113] (1) Different from the traditional network architecture that relies on specific datasets and tasks, this paper adopts a more general design that effectively integrates the comprehensive global and local feature expression capabilities of the 3D pyramid pooling Transformer backbone network and the large-core CNN. It not only performs well in the whole heart segmentation task, but can also be effectively migrated to other medical image processing tasks, thus expanding its application scope.

[0114] (2) The pyramid pooling Transformer was first proposed and used as one of the encoders in the whole heart segmentation task, providing powerful contextual information representation and improving segmentation accuracy while maintaining low computational requirements.

[0115] (3) In order to overcome the problem of high computational complexity of existing methods when processing three-dimensional image data, a lightweight large-kernel volume convolution module is designed. This module not only maintains the segmentation accuracy but also has the characteristics of being lighter, reducing the time cost of training and inference, and improving the practicality of the method.

[0116] (4) In order to improve the accuracy of pulmonary artery boundary segmentation in three-dimensional whole heart images, a bidirectional residual perception module is introduced. This module can capture the differences between multiple slices and effectively learn the contextual residual map and attention map that accurately describe the residual change trend between slices. This enables the segmentation model to more accurately capture the characteristics of the pulmonary artery boundary, thereby improving the segmentation accuracy.

[0117] In a specific example, in order to verify the effectiveness and feasibility of the three-dimensional cardiac image segmentation method of the present invention, training and testing were performed on the computed tomography (CT) and magnetic resonance imaging (MRI) modalities of the dataset, and compared with other methods. The dataset is an open source dataset, which includes 60 cardiac CT and 60 cardiac MRI, covering the entire cardiac substructure. The goal of this example is to accurately segment the multi-modal whole heart structure, including: (1) left ventricular (Myo) myocardium; (2) left atrial blood cavity (LA); (3) left ventricular blood cavity (LV); (4) right atrial blood cavity (RA); (5) right ventricular blood cavity (RV); (6) ascending aorta (AA); (7) pulmonary artery (PA). The 40 groups of data were used as training sets, and the 20 groups of data with real annotations were randomly divided into 15 groups for testing and 5 groups for verification.

[0118] Since only 20 sets of publicly labeled data are available, the dataset needs to be repartitioned to avoid overfitting. For the original 40 sets of unlabeled data, we first use the 20 sets of known labels to train the model to predict the pseudo labels of the 40 sets of unlabeled data, and then use the evaluation tool to measure the accuracy of the pseudo labels. In the end, the IoU score of the 40 sets of CT test sets was about 0.81, and the IoU score of the 40 sets of MRI test sets was about 0.78. Next, these two sets of data were used as training sets in this study, and the remaining 20 sets of labeled data were randomly divided into 15 test sets, and the remaining 5 sets were used as validation sets. Similarly, the MRI dataset was divided using the same strategy.

[0119] The following simulation experiment is carried out:

[0120] 1. Simulation conditions:

[0121] The simulation was performed using the Pytorch 1.13.1 framework on an Intel(R) Core(TM) i7-9800X CPU @ 3.80 GHz, 62 GB of memory, two Nvidia RTX 3090 GPUs, and an Ubuntu 18.04.1 operating system. The experimental code was written in Python 3.8.16.

[0122] 2. Simulation details:

[0123] During the training process, the weight decay (Adam+WeightDecay, AdamW) algorithm is used, the batch size is 1, the weight decay is 0.0005, and the initial learning rate is 0.0001. Online data enhancement techniques such as random scaling and flipping are also used to reduce overfitting.

[0124] 3. Simulation content:

[0125] The simulation content includes whole heart structure segmentation experiments, comparative experiments, and ablation experiments on the CT and MRI modalities of the MM-WHS2017 dataset.

[0126] 4. Evaluation Criteria

[0127] Dice and Hausdorff distance are used as evaluation metrics. Dice quantifies the overlap between the segmentation prediction (P) and the corresponding ground truth (T). It is defined as follows:

[0128]

[0129] The Hausdorff distance evaluates the quality of the segmentation boundary by calculating the maximum distance between the ground truth and the predicted result. It is defined as follows:

[0130]

[0131] In the formula, and Represent the surface point sets of P and T respectively. The higher the Dice value or the smaller the Hausdorff distance, the more accurate the segmentation result.

[0132] 5. Experimental results:

[0133] Table 1 compares the performance of the P2TC method with eight state-of-the-art segmentation models on the dataset. These models are divided into three groups according to their architecture: CNN, Transformer, and CNN-Transformer. We evaluate the segmentation performance of these models on MM-WHS2017 CT images for seven cardiac substructures, using Dice score and Hausdorff distance as evaluation metrics. The results show that the P2TC model achieves an average Dice score of 92.6% in the entire cardiac segmentation task, which is 1.5% higher than the baseline model ConResNet. Among them, the Myo, LV, RV, AA, and PA structures have the highest Dice scores, while the two-stage segmentation methods CFUN+ and SwinUNETR have the highest Dice scores for LA and RA. Compared with the baseline, our method improves the Dice score by about 1% by guiding with a bidirectional residual perception module, which improves the edge segmentation accuracy. Despite a slight increase in parameter count and model size compared to the baseline, our method outperforms other excellent 3D medical image segmentation models by a considerable margin. Experimental results show that the Transformer structure is robust on tasks with fewer structures, but is slightly insufficient in the complex, multi-structure whole heart segmentation task. Therefore, our proposed P2TC model successfully solves the problem of accurate segmentation of the whole heart structure by making full use of the unique features of P2T and L2C.

[0134] Table 2 shows that P2TC achieved an average Dice score of 88.1% on MRI images, providing the best Dice score for segmenting 6 of the cardiac substructures. Compared with CFUN+, it only had a slight disadvantage of 0.1% in RA. Compared with other competing methods, P2TC showed excellent performance in all Dice score metrics. In terms of Hausdorff distance score, the P2TC method was slightly higher than the baseline, indicating that the bidirectional residual perception module had a positive impact on the edge segmentation accuracy of the residual information between slices in different image modalities. Despite the slight defect in edge segmentation accuracy, the P2TC method performed well in Dice score compared with the other eight methods, highlighting the superior performance and higher segmentation accuracy of the present invention.

[0135] Figure 4 The whole heart segmentation results of the P2TC method are shown in comparison with other excellent segmentation models on CT and MRI modalities. Regardless of the modality used, the segmentation results are closer to the true value labels, especially in left ventricle (LV) segmentation. In addition, the present invention solves the problem of rough boundaries caused by inappropriate inter-slice residual activation, making the model significantly improved over the baseline model, confirming the continuous performance improvement of the P2TC method in the whole heart segmentation task. Figure 4 In each modality, the first row is the visualization of the segmentation effect on the axial slice, the second row is the corresponding local magnified image, and the third row is the 3D visualization. Among them, Myo, LA, LV, RA, RV, AA and PA are represented in red, yellow, orange, blue, purple, light pink and off-white, respectively, and PA is not shown.

[0136] Table 1 (the default evaluation index is Dice)

[0137]

[0138]

[0139] Table 2 (the default evaluation index is Dice)

[0140]

[0141] Ablation experiment:

[0142] 1. Ablation experiments on model architecture strategies: Table 3 discusses in detail the impact of different encoder strategies and fusion strategies on the model evaluation index Dice and the number of model parameters, and verifies the interpretability of the P2TC model architecture. When the same fusion strategy is adopted, the accuracy of using the L2C branch alone is about 1.6% higher than that of using the 3DP2T branch alone as an encoder. In addition, the use of dual encoders is 0.2% higher than the accuracy of using only the L2C branch. These findings show that the lightweight large-kernel convolution branch (L2C) significantly improves the accuracy of overall heart segmentation, and the 3D pyramid pooling Transformer branch plays a positive role in capturing long-range features and enhancing segmentation performance. In addition, the present invention designs fusion strategies 3, 4, and 5 to experiment and explore how to provide information-rich feature maps for the decoding path. The results presented in Table 3 confirm the important contribution of the long-range feature information extracted by the pyramid pooling Transformer encoder in supplementing the local features of the heart obtained by the convolutional network.

[0143] 2. Ablation experiment of improved modules: In order to ensure the consistent effectiveness of the overall model structure, ablation experiments were performed by modifying the specified modules. Table 4 shows that the Dice scores of the two sets of baseline modules are 0.7% lower than those of our proposed method. This result proves that the improved L2C block and BRP module have important contributions to the segmentation performance of the entire heart segmentation task.

[0144] Table 3

[0145]

[0146] Among them, E i and B i denote the output feature maps of the L2C branch and the 3DP2T branch at each stage, and z i is the fused feature map, and σ(·) represents the sigmoid function.

[0147] Table 4

[0148]

[0149] The step division of the various methods above is only for clear description. When implemented, they can be combined into one step or some steps can be split and decomposed into multiple steps. As long as they include the same logical relationship, they are all within the protection scope of the present invention. Adding insignificant modifications or introducing insignificant designs to the algorithm or process without changing the core design of the algorithm and process are all within the protection scope of the invention.

[0150] Another embodiment of the present invention relates to a three-dimensional cardiac image segmentation device. The implementation details of the three-dimensional cardiac image segmentation device of this embodiment are specifically described below. The following content is only for the convenience of understanding the implementation details provided, and is not necessary for implementing this solution. The three-dimensional cardiac image segmentation device of this embodiment includes:

[0151] An image acquisition module, used for acquiring three-dimensional cardiac images;

[0152] An image input module is used to input the three-dimensional heart image into the trained whole heart segmentation model; the whole heart segmentation model adopts a network structure including: a dual encoder with two encoding branches, pyramid pooling Transformer and lightweight large kernel convolution, and a dual decoder with two decoding branches, a segmentation decoder and a residual decoder;

[0153] The image segmentation module is used to generate pyramid feature maps of three-dimensional cardiac images of different scales by applying multiple average pooling layers of different scales on the three-dimensional cardiac images through the pyramid pooling transformer, and extract features from the pyramid feature maps of different scales using the multi-head self-attention mechanism in the pyramid pooling transformer; capture the context information in the pyramid feature maps of different scales through lightweight large kernel convolution; and fuse the features extracted by the pyramid pooling transformer and the context information captured by the lightweight large kernel convolution to obtain the output features of the dual encoder;

[0154] The output features of the dual encoder are upsampled by a segmentation decoder, and the upsampled features are fused with the output features of the dual encoder to generate a segmentation mask for indicating multiple different regions in the three-dimensional cardiac image; the residual context features between multiple slices corresponding to each region in the segmentation mask are extracted by a residual decoder to generate a residual mask for indicating the difference between the context slices of each region in the three-dimensional cardiac image;

[0155] The three-dimensional cardiac image is segmented according to the segmentation mask and the residual mask of the three-dimensional cardiac image. It is not difficult to find that the present embodiment is a system embodiment corresponding to the above-mentioned method embodiment, and the present embodiment can be implemented in conjunction with the above-mentioned method embodiment. The relevant technical details and technical effects mentioned in the above-mentioned embodiments are still valid in the present embodiment, and in order to reduce repetition, they are not repeated here. Accordingly, the relevant technical details mentioned in the present embodiment can also be applied in the above-mentioned embodiments.

[0156] It is worth mentioning that all modules involved in this embodiment are logic modules. In practical applications, a logic unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present invention, this embodiment does not introduce units that are not closely related to solving the technical problem proposed by the present invention, but this does not mean that there are no other units in this embodiment.

[0157] Another embodiment of the present invention relates to a computer device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional cardiac image segmentation method in the above-mentioned embodiments.

[0158] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor is transmitted on a wireless medium via an antenna, and further, the antenna also receives data and transmits the data to the processor.

[0159] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory can be used to store data used by the processor when performing operations.

[0160] Another embodiment of the present invention relates to a computer-readable storage medium storing a computer program, which implements the above method embodiment when executed by a processor.

[0161] That is, those skilled in the art can understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a program, and the program is stored in a storage medium, including a number of instructions for a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, referred to as: ROM), random access memory (Random Access Memory, referred to as: RAM), disk or optical disk and other media that can store program codes.

[0162] Those skilled in the art will appreciate that the above embodiments are specific embodiments for implementing the present invention, and in actual applications, various changes may be made thereto in form and detail without departing from the spirit and scope of the present invention.

Claims

1. A three-dimensional cardiac image segmentation method, characterized in that: include: Obtain three-dimensional cardiac images; Input the 3D heart image into the trained whole heart segmentation model; the whole heart segmentation model uses a network structure including: a dual encoder with two encoding branches, pyramid pooling Transformer and lightweight large kernel convolution, and a dual decoder with two decoding branches, a segmentation decoder and a residual decoder; The pyramid pooling transformer is used to apply multiple average pooling layers of different scales on the three-dimensional cardiac image to generate pyramid feature maps of different scales, and the multi-head self-attention mechanism in the pyramid pooling transformer is used to extract features from the pyramid feature maps of different scales; the context information in the pyramid feature maps of different scales is captured through lightweight large kernel convolution; the features extracted by the pyramid pooling transformer and the context information captured by the lightweight large kernel convolution are fused to obtain the output features of the dual encoder; The output features of the dual encoder are upsampled by a segmentation decoder, and the upsampled features are fused with the output features of the dual encoder to generate a segmentation mask for indicating multiple different regions in the three-dimensional cardiac image; the residual context features between multiple slices corresponding to each region in the segmentation mask are extracted by a residual decoder to generate a residual mask for indicating the difference between the context slices of each region in the three-dimensional cardiac image; The three-dimensional heart image is segmented according to the segmentation mask and the residual mask of the three-dimensional heart image.

2. The three-dimensional cardiac image segmentation method according to claim 1, characterized in that: The multi-head self-attention mechanism in the pyramid pooling Transformer is used to extract features from pyramid feature maps of different scales, including: Encode the corresponding positions in pyramid feature maps of different scales through deep convolution; Flattening the pyramid feature maps of different scales respectively, and connecting the pyramid feature maps of different scales according to the encoding of the corresponding positions in the pyramid feature maps of different scales, so as to obtain a sequence of a preset length from the pyramid feature maps of different scales; A multi-head self-attention mechanism is used to extract features from sequences of preset length.

3. The three-dimensional cardiac image segmentation method according to claim 2, characterized in that: The context information in pyramid feature maps of different scales is captured by lightweight large kernel convolution, including: For each scale of the pyramid feature map, a large kernel with a kernel size of 5×5×5 is used for depthwise convolution, and the number of channels of the pyramid feature map is adjusted to 4 times through two 1×1×1 point convolution channel scaling blocks to capture the contextual information in pyramid feature maps of different scales.

4. The three-dimensional cardiac image segmentation method according to claim 3, characterized in that: The features extracted by the fusion pyramid pooling transformer and the context information captured by the lightweight large kernel convolution include: The features of the pyramid feature map of each scale extracted by the pyramid pooling Transformer are fused with the contextual information in the pyramid feature map of the corresponding scale captured by the lightweight large kernel convolution.

5. The three-dimensional cardiac image segmentation method according to claim 1, characterized in that: Before segmenting the three-dimensional cardiac image according to the segmentation mask and the residual mask of the three-dimensional cardiac image, the method further includes: Segmenting a three-dimensional cardiac image into multiple slices; For each area in the three-dimensional cardiac image indicated by the segmentation mask, the residual information between context slices in multiple slices corresponding to the area is generated by the residual decoder, and the generated residual information is returned to the segmentation decoder through attention guidance, so that the segmentation decoder generates a segmentation mask of the three-dimensional cardiac image according to the residual information.

6. The three-dimensional cardiac image segmentation method according to claim 5, characterized in that: The residual decoder includes multiple feature extraction layers, and the residual information between context slices in multiple slices corresponding to the region is generated by the residual decoder, including: For each slice among the multiple slices corresponding to the region, respectively obtain residual information between the slice and two slices adjacent to the slice, and add the residual information between the slice and the two slices adjacent to the slice to obtain residual information extracted by a feature extraction layer of the residual decoder; For each feature extraction layer of the residual decoder, the residual information extracted by the feature extraction layer is added to the residual information extracted by the feature extraction layer of the previous layer to obtain the residual information between the context slices in the multiple slices corresponding to the region.

7. The three-dimensional cardiac image segmentation method according to claim 1, characterized in that: During the training process, the residual decoder uses the ground truth residual mask to train the generated residual mask based on the deep supervision mechanism.

8. A three-dimensional cardiac image segmentation device, characterized in that: include: An image acquisition module, used for acquiring three-dimensional cardiac images; An image input module is used to input the three-dimensional heart image into the trained whole heart segmentation model; the whole heart segmentation model adopts a network structure including: a dual encoder with two encoding branches, pyramid pooling Transformer and lightweight large kernel convolution, and a dual decoder with two decoding branches, a segmentation decoder and a residual decoder; The image segmentation module is used to generate pyramid feature maps of three-dimensional cardiac images of different scales by applying multiple average pooling layers of different scales on the three-dimensional cardiac images through the pyramid pooling transformer, and extract features from the pyramid feature maps of different scales using the multi-head self-attention mechanism in the pyramid pooling transformer; capture the context information in the pyramid feature maps of different scales through lightweight large kernel convolution; and fuse the features extracted by the pyramid pooling transformer and the context information captured by the lightweight large kernel convolution to obtain the output features of the dual encoder; The output features of the dual encoder are upsampled by a segmentation decoder, and the upsampled features are fused with the output features of the dual encoder to generate a segmentation mask for indicating multiple different regions in the three-dimensional cardiac image; the residual context features between multiple slices corresponding to each region in the segmentation mask are extracted by a residual decoder to generate a residual mask for indicating the difference between the context slices of each region in the three-dimensional cardiac image; The three-dimensional heart image is segmented according to the segmentation mask and the residual mask of the three-dimensional heart image.

9. A computer device, characterized in that: include: at least one processor; And, a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the three-dimensional cardiac image segmentation method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the three-dimensional cardiac image segmentation method according to any one of claims 1 to 7 is implemented.

Images