Automatic segmentation method of left atrium and left atrial scar based on cardiac medical images

Abstract

The present invention belongs to the field of medical image processing technology. Aiming at the technical problem that the accuracy of left atrium and left atrial scar segmentation in the prior art is not very high, the present invention proposes an automatic segmentation method for left atrium and left atrial scar based on cardiac medical images, comprising the following steps: S1, preprocessing training data and test data to obtain processed training data and test data; S2, training the Boundary Loss Constrained Mutil‑Attention U‑Net model using the processed training data to obtain 5 trained segmentation models; S3, segmenting the processed test data using the segmentation model to obtain 5 segmentation results, and taking the average of the segmentation results as the final segmentation result. The present invention can alleviate the problem of imbalanced categories between left atrium and scar, and significantly improve the segmentation ability of the baseline model.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical image processing, and in particular relates to a method for automatically segmenting a left atrium and a left atrial scar based on a cardiac medical image. Background Art

[0002] Atrial fibrillation (AF) is a common arrhythmia characterized by irregular atrium structure and frequent rapid contractions. Catheter ablation is a minimally invasive procedure for the treatment of AF that uses targeted energy to selectively destroy abnormal cardiac tissue that causes arrhythmia and restore normal heart rhythm. After catheter ablation, corresponding atrial scars are formed. Accurate quantification of left atrial (LA) and left atrial scars is a key prerequisite and indicator for evaluating the quality of catheter ablation treatment. In recent years, late gadolinium enhancement (LGE) cardiac magnetic resonance (CMR) imaging has become a promising tool for scar assessment and visualization. However, manual annotation of left atrial and left atrial scars requires a lot of time and effort from clinicians. Therefore, realizing an automatic segmentation algorithm for left atrial and left atrial scars on LGE CMR images is crucial for the evaluation of catheter ablation procedures and the treatment of AF.

[0003] Traditional segmentation methods, such as probabilistic atlas, mean clustering, region growing, and support vector machines, are widely used for the segmentation of the left atrium and left atrial scar. In addition, some researchers have used auxiliary CMR images from the same patient to obtain additional structural information, thereby improving the segmentation performance of LGE CMR. However, the performance of these traditional methods depends largely on the quality of the selected features, the experience of the set parameters, and the additional auxiliary MRI, resulting in problems such as low segmentation accuracy, complex actual operation, and poor generalization ability in actual clinical applications. In recent years, with the development of deep learning (DL) technology in the field of medical image processing, many deep learning-based algorithms have been proposed for the automatic segmentation of the left atrium and left atrial scar in LGE CMR. For example, Borra et al. used 2D and 3D U-Net networks to perform left atrium segmentation tests respectively. First, the region of interest (ROI) was located, and then the cropped ROI was finely segmented. The experimental results show that the performance of 3D U-Net is significantly better than that of 2D U-Net. Punithakumar et al. adjusted the geometric orientation of the input image and label based on the nnU-Net framework to obtain three-dimensional geometric information and improved the segmentation performance of the left atrial scar. Although some progress has been made in the fully automatic segmentation of the left atrium and left atrial scar, there are still many challenges in practical applications: 1) The number of training samples is scarce. Due to the high cost of manual data annotation and medical data privacy issues, it is difficult to obtain sufficient training data, especially three-dimensional data, which poses a major challenge to training robust models. 2) The left atrium has a complex shape and the distribution of left atrial scars is small and scattered, which increases the difficulty of automatic segmentation. 3) The boundary between the atrium and the scar is blurred and the contrast is low, which easily causes the network to misclassify the boundary pixels, affecting the segmentation accuracy. 4) The background pixels of the three-dimensional cardiac data usually occupy a large part of the entire image, and the total number of pixels in proportion to the atrium and scar is very small, which is a serious class imbalance problem that affects the optimal result of network convergence. Summary of the invention

[0004] The purpose of the present invention is to overcome the deficiencies of the prior art and provide a method for automatic segmentation of the left atrium and left atrial scar based on cardiac medical images. The method improves the segmentation accuracy of the left atrium and scar by adopting a single-stage model.

[0005] To achieve the above object, the technical solution adopted by the present invention is:

[0006] The method for automatically segmenting the left atrium and left atrial scar based on cardiac medical images comprises the following steps:

[0007] S1. Preprocess the training data and the test data respectively to obtain processed training data and test data;

[0008] S2. Train the Boundary LossConstrained Mutil-Attention U-Net (BMAU-Net) model using the 5-fold cross-validation training method on the processed training data to obtain 5 trained segmentation models;

[0009] S3. Use the five trained segmentation models to segment the processed test data respectively to obtain five segmentation results, and take the average of the five segmentation results as the final segmentation result.

[0010] Preferably, step S1 comprises the following steps:

[0011] S11, resampling the spatial resolutions of the training data and the test data to 0.625 mm×0.625 mm×2.5 mm, respectively, to obtain first training data and first test data;

[0012] S12, normalizing the intensity values ​​of the first training data and the first test data respectively to obtain second training data and second test data;

[0013] S13, randomly cropping the second training data and the second test data into blocks with a resolution of 128×128×32 respectively, to obtain processed training data and test data.

[0014] Preferably, in step S2, the BMAU-Net model includes a three-dimensional Swin UNETER model, a Multi-Orientation Attention Block (MOAB) module, a Multi-Scale Boundary Loss Block (MSBLB) module, and a Multi-Level Parameter Sharing Pyramid Pooling Module (MPASPP) module;

[0015] A multi-directional attention module, placed in the encoder path of the 3D Swin UNETER model;

[0016] The multi-scale boundary loss module is placed at the back of the decoder path of the 3D Swin UNETER model;

[0017] A multi-level parameter-sharing pyramid pooling module is placed on the skip connection path between the encoder and decoder of the 3D Swin UNETER model.

[0018] Preferably, the multi-directional attention module includes a first branch, a second branch, and a third branch, and the working method of the multi-directional attention module is as follows:

[0019] S211, the first branch is the original axis feature branch, which directly outputs the original features extracted by the SwinTransformer module in the three-dimensional Swin UNETER model to obtain the output of the first branch;

[0020] S212, the second branch and the third branch rotate the original features extracted by the Swin Transformer module in the three-dimensional Swin UNETER model along different axes to obtain two sets of rotation features;

[0021] S213, extracting the two groups of rotation features obtained in step S212 through the deep attention module and the spatial attention module respectively, to obtain the deep attention coefficients and spatial attention coefficients of the two groups;

[0022] S214, performing bitwise addition operation, product operation and rotation operation on the depth attention coefficient and the spatial attention coefficient of each group obtained in step S213, respectively, to obtain the output of the second branch and the output of the third branch;

[0023] S215. Superimpose the output of the first branch, the output of the second branch, and the output of the third branch to obtain the output of the multi-directional attention module.

[0024] Preferably, the multi-scale boundary loss module is operated by the prediction end and the annotation end respectively, and the working method of the multi-scale boundary loss module is as follows:

[0025] S221. At the prediction end, the decoder prediction result of the 3D Swin UNETER model is enlarged to the real annotated image size through bilinear interpolation and used as the prediction input of the multi-scale boundary loss function.

[0026] S222, at the annotation end, using two-dimensional Laplace convolution kernels with different convolution step sizes to process each slice of the real annotated image, the convolution step sizes are set to 1, 2, 4, and 8, respectively, to obtain edge features of 1, 1 / 2, 1 / 4, and 1 / 8 times the size;

[0027] S223, using bilinear interpolation method to enlarge the edge features obtained in step S222 to 1 / 2, 1 / 4 and 1 / 8 times the original size respectively to obtain three edge features;

[0028] S224, fusing the 1-fold edge feature obtained in step S222 and the three edge features obtained in step S223 through a 1×1 convolution kernel to obtain a multi-scale edge feature as a true label of the multi-scale boundary loss function;

[0029] S225. During the BMAU-Net model training process, the predicted input obtained in step S221 is compared with the true label obtained in step S224, and multiple multi-scale boundary loss functions are calculated. , , and ;

[0030] S226, using the multiple multi-scale boundary loss functions obtained in step S225 , , and Constrain the training process of the BMAU-Net model.

[0031] Preferably, the multi-level parameter-sharing pyramid pooling module is obtained by first modifying the hollow pyramid pooling block into a three-dimensional form, and then adding a residual connection outside the hollow pyramid pooling block; and the multi-level parameter-sharing pyramid pooling modules of different layers share the same network parameters.

[0032] Preferably, the number of layers of the pyramid pooling module with multi-level parameter sharing is set to 3.

[0033] Preferably, in step S2, during the training of the BMAU-Net model, the training process of the BMAU-Net model is constrained by a class imbalanced mixed loss function.

[0034] Preferably, the class imbalance hybrid loss function consists of a multi-scale boundary loss function, an online difficult sample mining cross entropy loss function, a generalized Dice loss function, and a Focal loss function;

[0035] The expression of the class imbalanced mixed loss function is:

[0036] ;

[0037] in, is the class imbalanced mixed loss function, Cross entropy loss function for online difficult sample mining, is the generalized Dice loss function, is the Focal loss function, is the multi-scale boundary loss function, , .

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] (1) The BMAU-Net model established in the present invention can alleviate the problem of class imbalance between the left atrium and scars by enhancing the extraction and utilization of multi-scale feature maps, and significantly improve the segmentation ability of the baseline model. Specifically, the MOAB module is first used to capture the depth and spatial attention information of multiple views during the downsampling process, thereby improving the segmentation performance of the BMAU-Net model for the complex structure of the left atrium and the discretely distributed left atrial scar. The MSBLB module is then used to calculate the predicted and truly annotated scar boundaries from multiple scales, thereby enhancing the BMAU-Net model's ability to distinguish pixels between the left atrium and the scar boundary. In addition, through the design of a class-imbalanced mixed loss function, the learning coefficient of the scar area can be adaptively reweighted, thereby alleviating the significant difference in pixel counts between the scar and the left atrium, and maintaining the accuracy and efficiency of scar segmentation. Finally, the encoder and decoder of the BMAU-Net model are connected through the MPASPP module, thereby reducing the number of downsampling operations, increasing the size of the receptive field within the layer, promoting the flow of information between layers, retaining detailed information about the characteristics of small scars, and significantly improving the performance of the baseline model.

[0040] (2) Compared with other two-stage methods based on deep learning technology, the present invention can segment the left atrium and scar into two categories by using only a single-stage BMAU-Net model. In addition, through the joint design of the MOAB module, the MSBLB module, the MPASPP module and the class-imbalanced mixed loss function, the segmentation performance of the left atrium and scar is better than that of a single segmentation model that only segments a certain category. In actual medical applications, the training and deployment are more cost-effective. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 A flowchart of a method for automatically segmenting the left atrium and left atrial scar based on cardiac medical images provided by an embodiment of the present invention;

[0042] Figure 2 Schematic diagram of the structure of the BMAU-Net model in an embodiment of the present invention;

[0043] Figure 3 Schematic diagram of the structure of the MOAB module in an embodiment of the present invention;

[0044] Figure 4 Schematic diagram of the structure of the MSBLB module in the embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of the structure of the MPASPP module in an embodiment of the present invention;

[0046] Figure 6Visualization of the segmentation results of the MOBA module ablation study; (a) without using SAM and DAM; (b) using only SAM; (c) using only DAM; (d) using both SAM and DAM;

[0047] The green area in the figure represents LA, the orange area represents scar, and the red circle represents the misclassification when only DAM is used;

[0048] Figure 7 Visualization of the segmentation results of the MSBLB module ablation study; (a) is Scale=0; (b) is Scale=1; (c) is Scale=2; (d) is Scale=3; (e) is Scale=4;

[0049] Figure 8 Visualization of the segmentation results of the MPASPP module ablation study; (a) does not use the MPASPP module; (b) the MPASPP module has 1 layer; (c) the MPASPP module has 2 layers; (d) the MPASPP module has 3 layers. DETAILED DESCRIPTION

[0050] The following is a description of the embodiments of the present invention. Figures 1 to 8 , the technical solutions in the embodiments of the present invention are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] like Figure 1 As shown, an embodiment of the present invention provides a method for automatically segmenting the left atrium and left atrial scar based on cardiac medical images, comprising the following steps:

[0052] S1. Preprocessing the training data and the test data respectively to obtain the processed training data and the test data includes the following steps:

[0053] S11, resampling the spatial resolutions of the training data and the test data to 0.625 mm×0.625 mm×2.5 mm, respectively, to obtain first training data and first test data; the training data and the test data are both left atrium and scar data;

[0054] S12, normalize the intensity values ​​of the first training data and the first test data respectively according to the mean variance normalization method to obtain second training data and second test data; specifically, for each slice in the data according to the formula Normalize the pixel values, where Indicates The normalized value of pixels, , and Respectively represent the pixel value, minimum pixel value and maximum pixel value;

[0055] S13, randomly crop the second training data and the second test data into blocks with a resolution of 128×128×32, respectively, to obtain processed training data and test data. Due to the high resolution of three-dimensional images, in order to reduce the requirements of training on computing resources, a random cropping method is used during training, and the second training data is randomly cropped into blocks with a resolution of 128×128×32, and then sent to the BMAU-Net model for training. During testing, a sliding window prediction method is used, and the window size is also set to 128×128×32.

[0056] S2, training the BMAU-Net model using the processed training data through a 5-fold cross-validation training method to obtain 5 trained segmentation models;

[0057] During the specific training, the processed training data is evenly divided into 5 parts for five-fold cross-training, specifically training data 1, training data 2, training data 3, training data 4 and training data 5; that is, 4 parts are selected as training data without repetition each time training, and the remaining 1 part is used as verification data. After the model converges or the training rounds are set, 5 trained segmentation models are obtained. They are sub-model 1, sub-model 2, sub-model 3, sub-model 4 and sub-model 5.

[0058] like Figure 2 As shown, the BMAU-Net model in the embodiment of the present invention includes a three-dimensional Swin UNETER model, a multi-directional attention (MOAB) module, a multi-scale boundary loss (MSBLB) module, and a multi-level parameter sharing pyramid pooling (MPASPP) module;

[0059] A multi-directional attention module, placed in the encoder path of the 3D Swin UNETER model;

[0060] The multi-scale boundary loss module is set in the decoder path of the 3D Swin UNETER model;

[0061] A multi-level parameter-sharing pyramid pooling module is placed on the skip connection path between the encoder and decoder of the 3D Swin UNETER model.

[0062] The BMAU-Net model proposed in the embodiment of the present invention is improved based on the three-dimensional Swin UNETER model. Specifically, after each Swin Transformer module, the MOAB module uses spatial attention and deep attention mechanisms to extract multi-view features along three axes, thereby improving the segmentation performance of the BMAU-Net model for complex structures of the left atrium and discretely distributed left atrial scars. In addition, the present invention can calculate the boundary loss between the generated features and the corresponding true annotations through the design of the MSBLB module. By comparing the boundary structure information, the MSBLB module helps to improve the classification accuracy of the BMAU-Net model for the fuzzy boundary pixels between the left atrium and the left atrial scar.

[0063] In an embodiment of the present invention, a multi-directional attention (MOAB) module includes a first branch, a second branch, and a third branch, and the working method of the multi-directional attention module is as follows:

[0064] S211, the first branch is the original axis feature branch, which does not perform any operation on the original features extracted by the SwinTransformer module in the three-dimensional Swin UNETER model, and directly outputs them to obtain the output of the first branch;

[0065] S212, the second branch and the third branch rotate the original features extracted by the Swin Transformer module in the three-dimensional Swin UNETER model along different axes to obtain two sets of rotation features;

[0066] S213, extracting the two groups of rotation features obtained in step S212 through the deep attention module and the spatial attention module respectively, to obtain the deep attention coefficients and spatial attention coefficients of the two groups;

[0067] S214, performing bitwise addition operation, product operation and rotation operation on the depth attention coefficient and the spatial attention coefficient of each group obtained in step S213, respectively, to obtain the output of the second branch and the output of the third branch;

[0068] S215. Superimpose the output of the first branch, the output of the second branch, and the output of the third branch to obtain the output of the multi-directional attention module.

[0069] Figure 3This is the structure diagram of the MOAB module. The MOAB module is designed to capture the spatial information of the left atrium and left atrial scar on three different axes. It consists of the first branch, the second branch, and the third branch: the first branch is the original axis feature branch, which does not perform any operation on the original features extracted by the Swin Transformer module; the second and third branches rotate the input features (i.e., the original features extracted by the Swin Transformer module) along different axes, respectively, and then extract them through the Deep Attention Module (DAM) and the Spatial Attention Module (SAM). After extraction, the features are rotated back to the original axis to obtain multi-directional attention features. Finally, the features in the first branch, the second branch, and the third branch are superimposed together to obtain the final output. In particular, DAM extracts attention along the depth direction of the rotated features to capture the cross-relationships between different slices. In addition, SAM extracts attention within each image slice, focusing on extracting the relationship between the left atrium and the left atrial scar. As shown in Figure 3 As shown in the second row, when features are input to DAM, they first pass through two parallel branches (max pooling layer and average pooling layer) to compress the spatial dimension, generating two one-dimensional spatial features of the same size. The shared multi-layer perceptron (MLP) processes the attention of these two one-dimensional features respectively. After ReLU activation and superposition operation, the final depth attention coefficient is obtained through the sigmoid function. On the other hand, SAM first performs the max pooling and average pooling operations to compress the depth dimension to 1, and then splices these features along the depth direction, and uses a convolutional network for feature extraction. Then, after Sigmoid activation, the final spatial attention coefficient is obtained. Finally, after multiplying the spatial attention coefficient with the depth attention coefficient obtained by the previous DAM, the final output of one branch is obtained.

[0070] In the embodiment of the present invention, the working method of the multi-scale boundary loss (MSBLB) module is as follows:

[0071] S221. At the prediction end, the decoder prediction result of the 3D Swin UNETER model is enlarged to the real annotated image size through bilinear interpolation and used as the prediction input of the multi-scale boundary loss function.

[0072] S222, at the annotation end, using two-dimensional Laplace convolution kernels with different convolution step sizes to process each slice of the real annotated image, the convolution step sizes are set to 1, 2, 4, and 8, respectively, to obtain 1, 1 / 2, 1 / 4, and 1 / 8 times the edge features;

[0073] S223, using bilinear interpolation method to enlarge the edge features obtained in step S222 by 1 / 2, 1 / 4 and 1 / 8 times to the original size respectively, to obtain three edge features;

[0074] S224, fusing the 1-fold edge feature obtained in step S222 and the three edge features obtained in step S223 through a 1×1 convolution kernel to obtain a multi-scale edge feature as a true label of the multi-scale boundary loss function;

[0075] S225. During the BMAU-Net model training process, the predicted input obtained in step S221 is compared with the true label obtained in step S224, and multiple multi-scale boundary loss functions are calculated. , , and ;

[0076] S226, using the multiple multi-scale boundary loss functions obtained in step S225 , , and Constrain the training process of the BMAU-Net model.

[0077] For patients with atrial fibrillation who have undergone catheter ablation, the scar produced is spatially located at the edge of the left atrial cavity, and the scar consists of multiple small areas scattered on the left atrial wall. This greatly increases the difficulty of the model learning the spatial relationship between the left atrium and the left atrial scar. To solve this problem, the present invention introduces the MSBLB module to extract the edge features and true labels of CMR, and then calculates the loss by comparing the edge features with the feature maps generated at different decoding stages, such as Figure 4 As shown. Specifically: Use a two-dimensional Laplacian convolution kernel to process each slice of the label, and set the stride to 1, 2, 4, and 8 respectively to obtain edge features of 1, 1 / 2, 1 / 4, and 1 / 8 times the size. Then, use bilinear interpolation to enlarge the three small-scale edge features to the original size. Finally, use a 1×1 convolution kernel to fuse multi-scale edge features. During the training process, the feature maps generated at different decoder stages are compared with the fused edge features and the loss function is calculated. Combine multiple boundary loss functions , , and Constrain the model training process. The boundary loss function of each layer is expressed as: , which consists of the Cross Entropy Loss function and the Dice Loss function, and in order to reduce the difficulty of hyperparameter adjustment, are all set to the same value of 1.

[0078] In the three-dimensional Swin UNETER model, the feature map passes through four image block merging modules (i.e., image block merging layers) and a 2-step downsampling operation, the size of the output feature is halved in the three-dimensional space, and the number of channels is doubled. The image block merging module is similar to the pooling layer in the Convolutional Neural Network (CNN), which is used to expand the receptive field. However, the pooling layer will cause the loss of small target information during the downsampling process. Similarly, the image block merging module simply takes the features at specific positions within each attention window as the input of the next layer, which will also lead to the loss of small targets during the feature extraction process. Therefore, inspired by the Atrous Spatial Pyramid Pooling (ASPP), the present invention designs a multi-level parameter sharing pyramid pooling (MPASPP) module to handle the loss of small scar information caused by the image block merging module in the BMAU-Net model, such as Figure 5 As shown. With the introduction of the MPASPP module, the general two-dimensional ASPP block is extended to a 3D MPASPP module, and the MPASPP module is placed on the jump connection path between the encoder and the decoder. In addition, by adding residual connections, the ASPP features are merged with the input features to obtain rich semantic information. At the same time, the encoder and decoder structures of the BMAU-Net model are reduced by one layer, retaining more details of small scars. In the experiment, the expansion factors in the hole pyramid pooling block were set to 2, 4, and 8, respectively. In order to improve information perception and reduce model parameters, the MPASPP module shares the network weights across different levels of connections, that is, the MPASPP modules of different layers share the same network parameters. The number of MPASPP modules in this embodiment is set to 3 layers.

[0079] In the embodiment of the present invention, during the training of the BMAU-Net model in step S2, the training process of the BMAU-Net model is constrained by a class imbalanced mixed loss function, and the class imbalanced mixed loss function in the embodiment of the present invention is composed of a multi-scale boundary loss function, an online difficult sample mining cross entropy loss function, a generalized Dice loss function, and a Focal loss function;

[0080] In the embodiment of the present invention, the multi-scale boundary loss function is the loss from the MSBLB module, and the expression of the multi-scale boundary loss function is:

[0081] ;

[0082] in, is the cross entropy loss function, is the Dice loss function;

[0083] The standard cross entropy loss is usually difficult to handle the class imbalance problem in image segmentation. When the difference between positive and negative samples is too large, for example, when negative samples significantly exceed positive samples, the loss function value will be dominated by negative samples. During the training process, the model will tend to learn a larger number of negative samples, which makes it difficult to accurately segment a smaller number of positive samples. Therefore, the present invention uses the Online Hard Example Mining (OHEM) method and develops the OHEM cross entropy (OHEMCE) loss function. The core idea of ​​the OHEM algorithm is to mine difficult samples that affect the classification accuracy based on the loss value of the sample, and learn these selected samples in subsequent training. In addition, the present invention also uses the generalized Dice loss (Generalized Dice, GDice) to replace the commonly used Dice loss to assign adaptive weights to the left atrium and left atrial scar, thereby reducing the correlation between the regional size of the positive sample and the Dice score. The Dice and GDice loss functions for multiple classes are:

[0084] ;

[0085] ;

[0086] in , and Respectively represent Category The predicted and true values ​​of pixels, It is a very small smoothing coefficient that ensures that the denominator is not 0.

[0087] The Focal loss function is also a common loss function used to solve the class imbalance problem. It is an improvement on the cross entropy loss. It introduces a modulation factor to adjust the relative importance of easy-to-classify and difficult-to-classify samples. The formula of the Focal loss function is expressed as:

[0088] ;

[0089] in, Indicates low The predicted value of the class, is the adjustment weight of the positive sample, represents the hyperparameter, Adjust the ratio of easy-to-classify and difficult-to-classify samples. Set to 2, Set to 1. Then combine the above loss functions to get the class imbalance mixed loss function for training the BMAU-Net model, that is, the class imbalance mixed loss function is the sum of the multi-scale boundary loss function, the online difficult sample mining cross entropy loss function, the generalized Dice loss function and the Focal loss function;

[0090] The expression of the class imbalanced mixed loss function is:

[0091] ;

[0092] in, is the class imbalanced mixed loss function, Cross entropy loss function for online difficult sample mining, is the generalized Dice loss function, is the Focal loss function, is the multi-scale boundary loss function, , .

[0093] S3. Use the five trained segmentation models to segment the processed test data respectively, and obtain five segmentation results. Take the average of the five segmentation results as the final segmentation result, that is, the segmentation result of the left atrium and scar. Finally, calculate the effect of the final segmentation result based on indicators such as the Dice coefficient, average surface distance (Average Surface Distance, ASD) and Hausdorff distance (Hausdorff Distance, HD).

[0094] The effect of the automatic segmentation method of the left atrium and left atrial scar based on cardiac medical images provided by the embodiment of the present invention is simulated and verified as follows:

[0095] 1. Specific design of the scheme:

[0096] (1) The performance of the BMAU-Net model proposed in this embodiment of the present invention was evaluated on the Left Atrial and Scar Quantification & Segmentation Challenge 2022 (LAScarQS2022) dataset.

[0097] The training data consisted of 3D LGE CMR images of 60 patients undergoing catheter ablation procedures, each of which included substructures of the entire heart, with the left atrium and left atrial scar manually labeled by clinical experts. The data had two different resolution sizes, 576×576×44 and 640×640×44. The test data consisted of 3D LGE CMR images of 10 patients with a spatial resolution of 1.0mm×1.0mm×1.0mm. In the data preprocessing stage, all training and test data were first resampled to a spatial resolution of 0.625mm×0.625mm×2.5mm, and the intensity was normalized using the mean variance normalization method. In order to reduce the computational requirements, the training data was randomly cropped into blocks with a resolution of 128×128×32 and fed into the BMAU-Net model. During the testing process, a sliding window prediction method was used, and the window size was also set to 128×128×32.

[0098] (2) Training data division: The above 60 training data were divided into 5 parts, each with 12 CMR image data. During training, 4 parts were selected without duplication, totaling 48 cases as training data, and the remaining part, totaling 12 cases, was used as validation data. This operation was performed 5 times to obtain 5 different sets of training and validation data.

[0099] (3) Training BMAU-Net model: The above five groups of data were trained separately to obtain five different trained segmentation models. AdamW was used as the network optimizer. The batch size was set to 2 and the initial learning rate was set to 0.001. In addition, a linear warm-up and cosine annealing decay strategy was adopted, with a warm-up time of 50 rounds. The training process adopted five-fold cross validation, and the final model adopted the average output. Each model was trained for 500 rounds.

[0100] (4) LGE CMR test: 10 images in the test data were used for testing. Five trained segmentation models were used for segmentation, and the average of the five segmented images was taken as the final result. The Dice index, average surface distance (ASD), and Hausdorff distance (HD) were used to evaluate the segmentation effect of the left atrium. In addition, since the HD and ASD of discrete scars are difficult to calculate, the Dice index and Sensitivity index were used as evaluation indicators for scar segmentation.

[0101] 2. Simulation conditions:

[0102] The present invention uses the Pytorch 1.1 framework for simulation on an Intel (R) i9-10900X 3.7GHz CPU, 125G memory, an NvidiaRTX3090 GPU, and an Ubuntu 20.04 operating system. The data set uses the above-mentioned LAScarQS 2022.

[0103] 3. Simulation results:

[0104] (1) Comparison with other advanced models: The comparison results of the BMAU-Net model proposed in the embodiment of the present invention and other advanced methods are shown in Table 1 below.

[0105] Table 1 Comparison of Dice indicators between the present invention and other competitors in the LAScarQS 2022 challenge:

[0106] ;

[0107] From the results in Table 1, it can be seen that, combined with the efforts to alleviate class imbalance and multi-scale boundary information fusion, the average Dice score of the BMAU-Net model of the present invention is 0.778, ranking second. Among them, Punithakumar et al. trained the nnUNet segmentation model by changing the geometric direction of the input image. Although there are better scar and average segmentation Dice indicators, the training method based on multiple inputs of images in different directions requires large computing resources and takes a long time to train. Jiang et al. adopted a two-stage segmentation method, combined with a curriculum learning method, to segment the left atrium and left atrial scar separately, which has the disadvantages of higher training cost and longer prediction time. Similarly, Lefebvre et al. first locate and extract the region, and then perform regional segmentation. Since the second stage does not take into account the class imbalance and detail loss of scar and left atrium, the scar segmentation result is poor. Zhang et al. introduced the TopK loss function to obtain more accurate prediction of the left atrium and scar boundaries, but did not take into account that the scar will lose detail information during the downsampling process, so the TopK loss cannot accurately guide the training of the model. Mazher et al. proposed a semi-supervised two-stage segmentation method to segment LA and scars. This model takes a long time to train and has a large number of parameters. Arega et al. introduced polynomial cross entropy loss and uncertainty information to improve the segmentation effect of scars, but did not alleviate the loss of scar details, and the final segmentation effect was not ideal.

[0108] (2) Ablation experiment:

[0109] This section verifies the effectiveness of the modules proposed in this invention (MOAB, MSBLB and MPASPP modules) and the proposed class-imbalanced hybrid loss function. The specific results are shown in Tables 2 to 5 below. SAM and DAM represent the spatial attention module and deep attention module in the MOAB module, respectively.

[0110] Table 2 Ablation experiment results of MOAB module:

[0111] ;

[0112] According to the results in Table 2, the network with SAM achieved better performance on HD and ASD compared with the network without SAM (regardless of the presence or absence of DAM), proving that SAM effectively extracts the spatial relationship between the left atrium and scar. Although DAM can improve the Dice score without the influence of SAM, using DAM alone will lead to some misclassifications, such as Figure 6 Middle (a) - Figure 6 As shown in (d). Using DAM and SAM (i.e., MOAB module) simultaneously can overcome these shortcomings. The above ablation experiments prove that the MOAB module can fuse the deep features and spatial features in 3D LGE CMR, so that the BMAU-Net model can model the global structural information and obtain better segmentation results.

[0113] Table 3 and Figure 7 Ablation study results of the MSBLB module, where Scale represents the number of boundary losses adopted.

[0114] Table 3 Ablation experiment results of MSBLB module:

[0115] ;

[0116] Through Table 3 and Figure 7 Middle (a) - Figure 7 From the results in (e), we can see that as the scale increases, the relative area of ​​the boundary loss increases, and more scale features are used to calculate the boundary loss. When the scale reaches 4, the Dice indicator has the best effect. For the left atrium region, HD and ASD usually decrease with the increase in the number of scales, and reach the best value when the scale = 3. This shows that as the multi-scale boundary loss constraint increases, the BMAU-Net model learns the precise boundary spatial relationship between the left atrium and the scar, thereby achieving reliable classification of blurred edge pixels. The results of the ablation experiment of the class imbalanced mixed loss function are shown in Table 4 below.

[0117] Table 4 Ablation experiment results of class imbalanced mixed loss function:

[0118] ;

[0119] It can be seen from Table 4 that with the application of OHEMCE, GDice and Focal loss functions, the segmentation performance is significantly improved. 1 and L 2 As a result, the present invention observed that online difficult sample mining improved the model's ability to distinguish scars. 1 and L 3 As a result, the GDice loss function enables the model to focus on the small scar area and balances the pixel difference between the left atrium and the scar. In addition, the Focal loss function also improves the segmentation performance. In summary, the class imbalance hybrid loss function proposed in the present invention effectively solves the class imbalance problem and improves the segmentation performance of the left atrium and scar.

[0120] In addition, the present invention proposes a multi-level parameter sharing pyramid pooling (MPASPP) module to reduce the downsampling frequency of the BMAU-Net model and alleviate the severity of class imbalance in the feature extraction process. The ablation experimental results of the MPASPP module are shown in Table 5 and Figure 8 As shown, the first column of Table 5 indicates the number of layers where the MPASPP module is applied, which starts from the bottom layer of the BMAU-Net skip connection and grows.

[0121] Table 5 Ablation experiment results of MPASPP module:

[0122] ;

[0123] Through Table 5 and Figure 8 Middle (a) - Figure 8 As can be seen in (d), when the number of layers of the MPASPP module is set to 3, the best segmentation effect is achieved for both the left atrium and the scar. It is worth noting that after using the MPASPP module, the Dice score of the scar is significantly improved compared with the left atrium. This shows that the model can retain more fine-grained scar feature information during feature extraction and improve its ability to handle class imbalance problems.

[0124] In summary, the automatic segmentation method of left atrium and left atrial scar based on cardiac medical images proposed in the present invention can realize the accurate segmentation of left atrium and left atrial scar on LGE CMR images. Specifically, based on the three-dimensional SwinUNETER model, the present invention first introduces a multi-directional attention module in the encoder path, and extracts feature information in multiple directions such as depth, width and height; in addition, a multi-scale boundary loss module is proposed to enhance the model's ability to recognize the boundaries of left atrium and left atrial scar. Previous technologies mainly focused on the use of multi-scale information of cardiac structures. Therefore, the present invention further improves the loss function and optimizes the network structure to compensate for the huge gap in the number of pixels between the two categories of left atrium and left atrial scar; before back propagation, some difficult-to-segment samples are selected as loss input, and Focal Loss function and generalized Dice loss function are used to automatically balance the learning weights between the two categories; at the same time, the present invention modifies the dilated pyramid pooling into a multi-level shared structure (i.e., a multi-level parameter-sharing pyramid pooling module), which expands the scale of the receptive field within the layer and reduces the number of downsampling times to retain the characteristic details of the left atrial scar.

[0125] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A method for automatic segmentation of left atrium and left atrial scar based on cardiac medical images, characterized in that: The following steps are involved: S1. Preprocess the training data and the test data respectively to obtain processed training data and test data; S2. Train the Boundary Loss ConstrainedMutil-Attention U-Net model using the 5-fold cross-validation training method on the processed training data to obtain 5 trained segmentation models; The Boundary Loss Constrained Mutil-Attention U-Net model includes a three-dimensional Swin UNETER model, a multi-directional attention module, a multi-scale boundary loss module, and a multi-level parameter sharing pyramid pooling module; A multi-directional attention module, placed in the encoder path of the 3D Swin UNETER model; The multi-directional attention module includes a first branch, a second branch, and a third branch, and the working method of the multi-directional attention module is as follows: S211, the first branch is the original axis feature branch, which directly outputs the original features extracted by the Swin Transformer module in the three-dimensional Swin UNETER model to obtain the output of the first branch; S212, the second branch and the third branch rotate the original features extracted by the SwinTransformer module in the three-dimensional Swin UNETER model along different axes to obtain two sets of rotation features; S213, extracting the two groups of rotation features obtained in step S212 through the deep attention module and the spatial attention module respectively, to obtain the deep attention coefficients and spatial attention coefficients of the two groups; S214, performing bitwise addition operation, product operation and rotation operation on the depth attention coefficient and the spatial attention coefficient of each group obtained in step S213, respectively, to obtain the output of the second branch and the output of the third branch; S215, superimposing the output of the first branch, the output of the second branch, and the output of the third branch to obtain the output of the multi-directional attention module; The multi-scale boundary loss module is placed at the back of the decoder path of the 3D Swin UNETER model; The multi-scale boundary loss module operates through the prediction end and the annotation end respectively, and the working method of the multi-scale boundary loss module is as follows: S221. At the prediction end, the decoder prediction result of the 3D Swin UNETER model is enlarged to the real annotated image size through bilinear interpolation and used as the prediction input of the multi-scale boundary loss function. S222, at the annotation end, using two-dimensional Laplace convolution kernels with different convolution step sizes to process each slice of the real annotated image, the convolution step sizes are set to 1, 2, 4, and 8, respectively, to obtain 1, 1 / 2, 1 / 4, and 1 / 8 times the edge features; S223, using bilinear interpolation method to enlarge the edge features obtained in step S222 by 1 / 2, 1 / 4 and 1 / 8 times to the original size respectively, to obtain three edge features; S224, fusing the 1-fold edge feature obtained in step S222 and the three edge features obtained in step S223 through a 1×1 convolution kernel to obtain a multi-scale edge feature as a true label of the multi-scale boundary loss function; S225. During the training of the Boundary Loss Constrained Mutil-Attention U-Net model, the predicted input obtained in step S221 is compared with the true label obtained in step S224, and multiple multi-scale boundary loss functions are calculated. , , and ; S226, using the multiple multi-scale boundary loss functions obtained in step S225 , , and The training process of the Boundary Loss Constrained Mutil-Attention U-Net model; A multi-level parameter-sharing pyramid pooling module is placed on the skip connection path between the encoder and decoder of the 3D Swin UNETER model; S3. Use the five trained segmentation models to segment the processed test data respectively to obtain five segmentation results, and take the average of the five segmentation results as the final segmentation result.

2. The method for automatic segmentation of left atrium and left atrial scar based on cardiac medical images according to claim 1, characterized in that: Step S1 comprises the following steps: S11, resampling the spatial resolutions of the training data and the test data to 0.625 mm×0.625 mm×2.5 mm, respectively, to obtain first training data and first test data; S12, normalizing the intensity values ​​of the first training data and the first test data respectively to obtain second training data and second test data; S13. Randomly crop the second training data and the second test data into blocks with a resolution of 128×128×32, respectively, to obtain processed training data and test data.

3. The method for automatic segmentation of left atrium and left atrium scar based on cardiac medical images according to claim 1, characterized in that: The multi-level parameter-sharing pyramid pooling module is obtained by first modifying the hollow pyramid pooling block into a three-dimensional form, and then adding a residual connection outside the hollow pyramid pooling block; and the multi-level parameter-sharing pyramid pooling modules of different layers share the same network parameters.

4. The method for automatic segmentation of left atrium and left atrium scar based on cardiac medical images according to claim 3, characterized in that: The number of layers of the pyramid pooling module with multi-level parameter sharing is set to 3.

5. The method for automatic segmentation of left atrium and left atrial scar based on cardiac medical images according to claim 1, characterized in that: In step S2, during the training of the Boundary Loss Constrained Mutil-Attention U-Net model, the training process of the Boundary Loss Constrained Mutil-Attention U-Net model is constrained by a class-imbalanced hybrid loss function.

6. The method for automatic segmentation of left atrium and left atrium scar based on cardiac medical images according to claim 5, characterized in that: The class imbalance hybrid loss function consists of a multi-scale boundary loss function, an online difficult sample mining cross entropy loss function, a generalized Dice loss function, and a Focal loss function; The expression of the class imbalanced mixed loss function is: ； in, is the class imbalanced mixed loss function, Cross entropy loss function for online difficult sample mining, is the generalized Dice loss function, is the Focal loss function, is the multi-scale boundary loss function, , .

Images