Semi-supervised 3D left atrium segmentation method based on Double W-Net

Abstract

The invention discloses a DoubleW-Net-based semi-supervised 3D left atrium segmentation method, and relates to the field of image segmentation, and the method comprises the steps: obtaining a to-be-detected 3D left atrium magnetic resonance image; constructing a training data set based on the labeled left atrium magnetic resonance image data and the unlabeled left atrium magnetic resonance image data; constructing a semi-supervised segmentation neural network, and training by adopting a segmentation loss function based on the training data set to obtain a trained semi-supervised segmentation neural network; the semi-supervised segmentation neural network is of a parallel Double W-Net structure, and comprises an upper layer W network constructed by integrating a channel attention module and a global attention module, and a lower layer W network; and inputting the 3D left atrium magnetic resonance image to be detected into the trained semi-supervised segmentation neural network to obtain a left atrium magnetic resonance image segmentation result, so that efficient feature interaction and extraction are realized, and the accuracy and generalization ability of a convolutional neural network model in the field of atrium segmentation are remarkably improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image segmentation, in particular to a semi-supervised 3D left atrium segmentation method based on DoubleW-Net. BACKGROUND

[0002] Medical image segmentation is a key technology for assisting doctors in disease diagnosis. Although the full-supervised method based on convolutional neural network has high accuracy, its performance is severely dependent on a large number of expert-labeled data. In the medical field, it is time-consuming, labor-intensive and costly to obtain such labeled data, which has become a major bottleneck for practical application. In contrast, unlabeled medical image data is easy to obtain, therefore, the semi-supervised learning method which can effectively utilize unlabeled data has become a research hotspot.

[0003] The current mainstream semi-supervised method mainly relies on pseudo-label and consistency regularization. The pseudo-label method uses the initial model to generate the prediction of unlabeled data as the proxy true value for self-training, but its performance is severely limited by the quality of the initial pseudo-label. Low-quality pseudo-labels can lead to error accumulation and cause the confirmation bias problem of model performance decline. The consistency regularization method improves the robustness of the model by applying perturbation to the unlabeled data and constraining the model output to be consistent. However, the existing methods mostly use single network or simple perturbation, and the constraint is limited, which makes it difficult to fully tap the potential of unlabeled data, and the model lacks generalization ability when facing data distribution differences. The existing advanced semi-supervised method can achieve competitive performance using only a small amount of labeled data, which proves the feasibility of this path. However, its performance still lags behind the full-supervised method, and there is still room for improvement in segmentation accuracy and robustness, which cannot fully meet the stringent requirements of clinical diagnosis for high accuracy and stability.

[0004] Therefore, the present application provides a semi-supervised 3D left atrium segmentation method based on DoubleW-Net to solve the above problems. SUMMARY

[0005] In order to solve the above problems, the present application provides a semi-supervised 3D left atrium segmentation method based on DoubleW-Net, which constructs a differentiated parallel network with 3D convolution in the upper layer and depth separable convolution in the lower layer, and combines the channel attention module and global attention module used only for the upper network, to realize efficient feature interaction and extraction, and significantly improve the accuracy and generalization ability of the convolutional neural network model in the field of atrial segmentation.

[0006] To achieve the above purpose, the present application provides a semi-supervised 3D left atrium segmentation method based on DoubleW-Net, comprising the following steps: S1: obtaining a 3D left atrium magnetic resonance image to be tested and pre-processing; S2: constructing a training data set based on the labeled 3D left atrial magnetic resonance image data and the unlabeled 3D left atrial magnetic resonance image data; S3: constructing a semi-supervised segmentation neural network, and training the semi-supervised segmentation neural network based on the training data set and using a segmentation loss function to obtain a trained semi-supervised segmentation neural network; the semi-supervised segmentation neural network is a parallel DoubleW-Net structure, including an upper W network constructed by an integrated channel attention module and a global attention module, and a lower W network; S4: inputting the preprocessed 3D left atrial magnetic resonance image to be tested into the trained semi-supervised segmentation neural network to obtain a left atrial magnetic resonance image segmentation result.

[0007] Preferably, the preprocessing in S1 specifically includes: The information region around the 3D left atrial magnetic resonance image to be tested is removed, and the middle image part is retained, the image is cropped to 112*112*80, the edge information region in the image is removed, and the cropped data is saved in an h5 file.

[0008] Preferably, the upper W network uses a standard 3*3*3 convolution kernel for feature extraction, and the first layer of the network input end integrates a channel attention module, and a global attention module is integrated in the jump connection stage; the lower W network uses a depth separable convolution to construct an encoder-decoder, and does not integrate a channel attention module and a global attention module.

[0009] Preferably, in step S3, the channel attention module obtains channel weights through parallel adaptive average pooling and adaptive maximum pooling, and after ReLU activation function processing, the channel weights are multiplied with the original feature map channel by channel, and then added with the input feature map in residual, so as to emphasize or suppress different feature channels according to the requirements of the task.

[0010] Preferably, in step S3, the global attention module is a 3D convolution-based Transformer-like structure, and integrates position encoding, which is used to capture the long-distance dependence relationship of the feature map in the jump connection stage, and enhance the global context feature representation capability.

[0011] Preferably, the upper W network and the lower W network both use transpose convolution for upsampling, and the feature maps in the upsampling stage of the lower W network and the feature maps in the upsampling stage of the upper W network are added element by element, which is used to obtain more rich features.

[0012] Preferably, after all the upsampling stages are completed, a 1*1*1 convolution block is used to fuse the feature maps output by the upper W network and the lower W network to obtain the final segmentation result.

[0013] Preferably, the semi-supervised segmentation neural network adopts a channel reduction strategy to reduce the parameter quantity of the network.

[0014] Preferably, the segmentation loss function in step S3 is composed of the segmentation loss of the upper W network and the segmentation loss of the lower W network.

[0015] Preferably, the segmentation loss function is represented as: ; Wherein, is the segmentation loss of the upper W network, is the segmentation loss of the lower W network.

[0016] Therefore, the present application adopts the above-mentioned one kind based on DoubleW-Net's semi-supervised 3D left atrium segmentation method, has the following beneficial effects: (1) The present application introduces a DoubleW-Net network architecture, and combines with the designed channel attention module and global attention module, and applies the two modules only in the upper W network, not in the lower network, the purpose is to make the two networks different, a weak network, a strong network, to make the two networks learn and train each other, greatly improve the training ability and feature extraction ability of the network.

[0017] (2) The present application designs a channel attention module and a global attention module, introduces the channel attention block into the first layer of the input part of the segmentation network, introduces the global attention block into the skip connection stage of the upper network, and introduces a position coding at the same time, so that the model can pay more attention to the spatial position information of the image.

[0018] (3) The present application adopts a channel reduction strategy to reduce the parameter quantity of the network.

[0019] (4) The present application is based on various improvements, compared with other methods in the prior art, in the semi-supervised left atrium segmentation task on the LA database, a new advanced performance is realized, and has strong robustness.

[0020] (5) The network in the present application can be easily combined with other shape constraint models to enhance the segmentation result, and has strong generalization ability.

[0021] The technical solutions of the present application will be further described in detail below by means of the drawings and examples. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 is a flowchart of the present application one kind based on DoubleW-Net's semi-supervised 3D left atrium segmentation method; Figure 2 is a DoubleW-Net network model structure schematic diagram in the embodiment of the present application; Figure 3 This is a schematic diagram of the CAB module structure in an embodiment of the present invention; Figure 4 This is a schematic diagram of the GAB module structure in an embodiment of the present invention. Detailed Implementation

[0023] The following detailed description of embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0024] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains.

[0025] The terms "comprising" or "including" as used in this invention mean that the element preceding the term encompasses the element listed after the term, and do not exclude the possibility of encompassing other elements. Terms such as "inner," "outer," "upper," and "lower" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings, and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. When the absolute position of the described object changes, the relative positional relationship may also change accordingly. In this invention, unless otherwise explicitly specified and limited, the term "attached" and similar terms should be interpreted broadly. For example, it can refer to a fixed connection, a detachable connection, or an integral part; it can refer to a direct connection or an indirect connection through an intermediate medium; it can refer to the internal communication of two elements or the interaction relationship between two elements. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0026] Example A semi-supervised 3D left atrial segmentation method based on DoubleW-Net, such as Figure 1 As shown, it includes the following steps: S1: Acquire and preprocess the 3D left atrial magnetic resonance image to be tested; Preprocessing specifically includes: The information region around the left atrial MRI image to be tested was removed, retaining only the central image portion, and the image was cropped to... Remove edge information areas from the image and save the cropped data in an h5 file.

[0027] S2: constructing a training data set based on the labeled 3D left atrial magnetic resonance image data and the unlabeled 3D left atrial magnetic resonance image data; Specifically, the training data set is subjected to data augmentation, including performing random rotation, random cropping and random flipping operations on the 3D left atrial magnetic resonance image, and the same data augmentation strategy is adopted for the labeled samples and the unlabeled samples.

[0028] S3: constructing a semi-supervised segmentation neural network, as shown in Figure 2 , and based on the training data set, training is performed using a segmentation loss function to obtain the trained semi-supervised segmentation neural network; the semi-supervised segmentation neural network is a parallel DoubleW-Net structure, including an upper W network constructed by integrating a channel attention module and a global attention module, and a lower W network; The upper W network uses a standard 3x3x3 convolution kernel for feature extraction, and a first layer of a channel attention block (CAB) module at the input end of the network, and a global attention module at the stage of jump connection; the lower W network uses a depth separable convolution to construct an encoder-decoder, and does not integrate a channel attention module and a global attention module.

[0029] The channel attention module obtains the weight of each channel through parallel adaptive average pooling and adaptive maximum pooling, and after ReLU activation function processing, it is multiplied with the original feature map channel by channel, and is added with the input feature map in residual, so as to emphasize or suppress different feature channels according to the requirements of the task.

[0030] Specifically, as shown in Figure 3 , an adaptive average pooling and an adaptive maximum pooling are used in parallel to obtain the importance of each channel of the feature map, and then a weight value is given to each feature according to the importance, and then the weight value is activated through a ReLU function. After that, the dimension of the tensor y is expanded to be the same as that of x, and the data in y is copied, then the learned weight y is multiplied by the original feature map x, and finally added with the input map to obtain a more useful feature map x.

[0031] The global attention (Global Attention Mechanism, GAB) module is a 3D convolution-based Transformer-like structure, as shown in Figure 4 , and integrates position encoding, which is used to capture the long-distance dependence of the feature map at the stage of jump connection, and enhance the global context feature representation ability.

[0032] Specifically, the GAB module is a 3D convolution style Transformer-like block. Specifically, when performing a skip connection, each layer of the encoding stage enters this module, adds position encoding, so as to help inject spatial position information and enable the model to learn the position relationship.

[0033] The upper W network and the lower W network both use transpose convolution for upsampling. The feature maps of the upper W network and the lower W network in the upsampling stage are added element by element to obtain more rich features.

[0034] After all the upsampling stages are completed, a 1x1x1 convolution block is used to fuse the feature maps output by the upper W network and the lower W network to obtain the final segmentation result.

[0035] The semi-supervised segmentation neural network adopts a channel reduction strategy to reduce the parameter amount of the network.

[0036] The segmentation loss function is composed of the segmentation loss of the upper W network and the segmentation loss of the lower W network. The segmentation loss function is represented as: ; Among them, is the segmentation loss of the upper W network, is the segmentation loss of the lower W network.

[0037] Finally, the network is adjusted according to the input and the output prediction result to adapt to specific application scenarios. The number of pixel-level labels is reduced, and the network performance is observed by repeatedly experimenting.

[0038] S4: input the preprocessed 3D left atrial magnetic resonance image to be tested into the trained semi-supervised segmentation neural network to obtain the left atrial magnetic resonance image segmentation result.

[0039] Embodiment 1 In order to further verify that the method provided in the application has high segmentation accuracy, dice score, intersection over union, 95% Hausdorff distance and average surface distance are used as evaluation indexes to measure the performance of the algorithm provided in the application and existing semi-supervised and fully supervised segmentation algorithms in the field of atrial segmentation, and the network performance is evaluated. The specific results are shown in Table 1 as follows: Table 1 Evaluation result table

[0040] As can be seen from Table 1, under the condition of using only 10% labeled data, the method proposed in the application reaches the highest in the two key indicators of Dice score and intersection over union, which are 88.86% and 80.35% respectively, and the 95% Hausdorff distance and the average surface distance are also the lowest in the same group, which are 9.18 and 2.49 respectively, showing that the comprehensive segmentation performance is better than that of the existing semi-supervised and fully supervised methods. Further, under the condition of 20% labeled data, the method continues to lead in the Dice score and the intersection over union, which are 91.58% and 83.71% respectively, and the 95% Hausdorff distance and the average surface distance are further reduced to 5.61 and 1.59 respectively, which is the best among all the comparison methods, effectively verifying the effectiveness and advancement of the method in the left atrium segmentation task.

[0041] Therefore, the application adopts the above-mentioned semi-supervised 3D left atrium segmentation method based on DoubleW-Net, introduces a channel attention mechanism module into the DoubleW-Net network architecture, adjusts the feature representation of the subsequent network channels by learning the first layer of the input part of the upper layer network, synchronously extracts features in multiple aspects, and adopts a network parallel mode for feature interaction, thereby avoiding the problem of information loss in the process of extracting features by the network. Further, a global attention block similar to the self-attention of the Transformer is introduced, which can better focus on global information in the stage of skip connection, thereby improving the accuracy and generalization ability of the convolutional neural network model in the field of atrium segmentation. By improving the accuracy of medical image segmentation, a new way is opened up for computer-aided diagnosis technology.

[0042] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application and not to limit it, although the application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the application can still be modified or replaced by equivalents, and these modifications or equivalent replacements cannot make the modified technical solutions deviate from the spirit and scope of the technical solutions of the application.

Claims

1. A semi-supervised 3D left atrium segmentation method based on DoubleW-Net, characterized in that, The method comprises the following steps: S1: obtaining and preprocessing a 3D left atrium magnetic resonance image to be tested; S2: constructing a training data set based on labeled 3D left atrium magnetic resonance image data and unlabeled 3D left atrium magnetic resonance image data; S3: constructing a semi-supervised segmentation neural network, and training the semi-supervised segmentation neural network based on the training data set by using a segmentation loss function to obtain a trained semi-supervised segmentation neural network; the semi-supervised segmentation neural network is a parallel DoubleW-Net structure, comprising an upper W network constructed by an integrated channel attention module and a global attention module, and a lower W network; S4: inputting the preprocessed 3D left atrium magnetic resonance image to be tested into the trained semi-supervised segmentation neural network to obtain a left atrium magnetic resonance image segmentation result.

2. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 1, wherein: The preprocessing in S1 specifically comprises: The information region around the 3D left atrium magnetic resonance image to be tested is removed, and the middle image part is reserved, the image is cropped to 112*112*80, the edge information region in the image is removed, and the cropped data is saved in an h5 file.

3. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 1, characterized in that: The upper W network uses a standard 3*3*3 convolution kernel for feature extraction, and an integrated channel attention module at the first layer of the network input end, and an integrated global attention module at the jump connection stage; The lower W network uses a depth separable convolution to construct an encoder-decoder, and does not integrate a channel attention module and a global attention module.

4. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 3, characterized in that: The channel attention module in step S3 obtains channel weights through parallel adaptive average pooling and adaptive maximum pooling, performs ReLU activation function processing, and then multiplies each channel of the original feature map, and adds a residual to the input feature map, so as to emphasize or suppress different feature channels according to the requirements of the task.

5. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 4, characterized in that: The global attention module in step S3 is a Transformer-like structure based on 3D convolution, and integrates position encoding, which is used to capture the long-distance dependence relationship of the feature map at the jump connection stage, and enhance the global context feature representation capability.

6. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 5, characterized in that: Both the upper W network and the lower W network use transpose convolution for upsampling, and add the feature maps at the upsampling stage of the lower W network and the feature maps at the upsampling stage of the upper W network element by element, which is used to obtain more rich features.

7. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 6, characterized in that: After completing all the upsampling stages, a 1*1*1 convolution block is used to fuse the feature maps output by the upper W network and the lower W network to obtain the final segmentation result.

8. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 7, characterized in that: The semi-supervised segmentation neural network uses a channel reduction strategy to reduce the parameter quantity of the network.

9. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 8, characterized in that: The segmentation loss function in step S3 is composed of the segmentation loss of the upper W network and the segmentation loss of the lower W network.

10. The semi-supervised 3D left atrium segmentation method based on DoubleW-Net according to claim 9, characterized in that: The segmentation loss function is represented as: ； wherein, is the segmentation loss for the upper W network, is the segmentation loss for the lower W network.

Images