Semi-supervised segmentation method of cardiac images based on global-local information fusion LPV-Net and 3D-EDA

Abstract

A semi-supervised cardiac image segmentation method based on global-local information fusion LPV-Net and 3D-EDA relates to the field of medical image segmentation technology. It combines the global-local information fusion LPV-Net branch with the 3D-EDA branch. This framework utilizes two different attention mechanisms to capture different dependencies in medical images to improve model performance. It adopts a 3D-EDA discriminator model and leverages adversarial learning to enhance model performance, particularly for unlabeled data. The overall loss function consists of a supervised loss and an unsupervised loss. Training is performed through backpropagation and an optimizer to gradually adjust model parameters to minimize the total loss. This method demonstrates excellent performance and can achieve accurate segmentation of the left atrium. This method improves the accuracy and efficiency of cardiac image segmentation.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a semi-supervised segmentation method for cardiac images based on global-local information fusion LPV-Net and 3D-EDA. Background Art

[0002] Cardiac segmentation can provide accurate cardiac structural information, assisting doctors in diagnosis, treatment planning, and surgical evaluation, thereby significantly improving clinical treatment outcomes and reducing complications. However, cardiac segmentation technology still faces many challenges in practical applications. Traditional manual segmentation methods are not only time-consuming and labor-intensive, but also highly subjective. Although fully supervised cardiac segmentation methods have achieved certain results, they rely on a large amount of labeled data. Existing semi-supervised segmentation methods still perform less than ideally when dealing with complex cardiac structures and pathological changes, making them difficult to apply stably in actual clinical settings. Summary of the Invention

[0003] In order to overcome the shortcomings of the above technologies, the present invention provides a semi-supervised cardiac image segmentation method based on global-local information fusion LPV-Net and 3D-EDA to improve the accuracy and efficiency of cardiac image segmentation.

[0004] The technical solution adopted by the present invention to overcome the technical problems is:

[0005] A semi-supervised cardiac image segmentation method based on global-local information fusion LPV-Net and 3D-EDA, comprising:

[0006] a) Acquisition A dataset of cardiac MRI images with or without labels , , is a labeled cardiac MRI image dataset, is an unlabeled cardiac MRI image dataset, , For the A labeled cardiac MRI image, , , For the An unlabeled cardiac MRI image, , is the number of labeled cardiac MRI images, is the number of unlabeled cardiac MRI images, ;

[0007] b) For the dataset Perform preprocessing to obtain the preprocessed data set , , is a preprocessed labeled cardiac MRI image dataset, It is a preprocessed unlabeled cardiac MRI image dataset;

[0008] c) The preprocessed dataset Divide into training set , test set ;

[0009] d) Establish the GLB-Net model. The GLB-Net model consists of the LPV-Net branch and the 3D-EDA branch. The LPV-Net branch consists of the encoder, LinformerPerformerBlock, and decoder.

[0010] e) The training set The image in is input to the encoder of the LPV-Net branch, and the output is the feature map ;

[0011] f) The feature map Input to the LinformerPerformerBlock of the LPV-Net branch and output the feature map ;

[0012] g) The feature map Input into the decoder of the LPV-Net branch and output the segmentation map ;

[0013] h) Segment the map Input into the 3D-EDA branch to obtain the unsupervised loss function and the supervised loss function;

[0014] i) Calculate the total loss based on the unsupervised loss function and the supervised loss function , using the Adam optimizer through the total loss Train the GLB-Net model to obtain the optimized GLB-Net model;

[0015] j) Test set The image in is input into the LPV-Net branch of the optimized GLB-Net model, and the segmentation map is output. .

[0016] Furthermore, in step a), we extract the Cardiac MRI images of the left atrium with or without annotations.

[0017] Furthermore, step b) comprises the following steps:

[0018] b-1) A labeled cardiac MRI image The area to be segmented is marked as 1, and the rest of the area is marked as 0. Use the np.nonzero function to find the A labeled cardiac MRI image The coordinates of all non-zero pixels in the label are calculated using the np.min function to obtain the minimum value of the coordinates of all non-zero pixels, and the maximum value of the coordinates of all non-zero pixels is calculated using the np.max function. The minimum rectangular frame containing all cardiac structures is obtained based on the minimum and maximum values ​​of the coordinates;

[0019] b-2) Use standard Numpy slicing operations to slice the first A labeled cardiac MRI image Perform cropping to obtain the cropped image ;

[0020] b-3) The cropped image The image is obtained by RandomCrop function transformation , the image Perform normalization operation to obtain the preprocessed image , preprocessed labeled cardiac MRI image dataset ;

[0021] b-4) An unlabeled cardiac MRI image The cropped image is obtained by randomly cropping the image using RandomCrop ;

[0022] b-5) The cropped image The preprocessed image is obtained by randomly rotating 0 degrees, 90 degrees, 180 degrees or 270 degrees , preprocessed unlabeled cardiac MRI image dataset .

[0023] Furthermore, from the pre-processed labeled cardiac MRI image data Select Zhang preprocessed images and unlabeled cardiac MRI image data from preprocessed images Select The training set is obtained by preprocessing the images , , from the preprocessed unlabeled cardiac MRI image dataset Select The training set is obtained by preprocessing the images , .

[0024] Furthermore, step e) comprises the following steps:

[0025] e-1) The encoder of the LPV-Net branch consists of the first ConvBlock block, the first DownsamplingConvBlock block, the second ConvBlock block, the second DownsamplingConvBlock block, the third ConvBlock block, the fourth DownsamplingConvBlock block, and the fifth ConvBlock block;

[0026] e-2) The first ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. The Preprocessed image Input into the first ConvBlock block and output the feature map The first DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input into the first DownsamplingConvBlock block and output the feature map ;

[0027] e-3) The second ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the second ConvBlock block and output the feature map The second DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the second DownsamplingConvBlock block and output the feature map ;

[0028] e-4) The third ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the third ConvBlock block and output the feature map The third DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the third DownsamplingConvBlock block and output the feature map ;

[0029] e-5) The fourth ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fourth ConvBlock block and output the feature map , the fourth DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function, which converts the feature map Input to the fourth DownsamplingConvBlock block and output the feature map ;

[0030] e-6) The fifth ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fifth ConvBlock block and output the feature map .

[0031] Furthermore, step f) comprises the following steps:

[0032] The LinformerPerformerBlock of the LPV-Net branch in f-1) consists of the Linformer attention mechanism, the Performer attention mechanism, and the LayerNorm layer;

[0033] f-2) The feature map Input to the Linformer attention mechanism of LinformerPerformerBlock and output the feature map ;

[0034] f-3) The feature map Input to the Performer attention mechanism of LinformerPerformerBlock and output the feature map ;

[0035] f-4) The feature map Input to the LayerNorm layer of LinformerPerformerBlock and output the feature map .

[0036] Further, step g) comprises the following steps:

[0037] g-1) The decoder of the LPV-Net branch consists of a first UpsamplingDeconvBlock block, a first ConvBlock block, a second UpsamplingDeconvBlock block, a second ConvBlock block, a third UpsamplingDeconvBlock block, a third ConvBlock block, a fourth UpsamplingDeconvBlock block, and a fourth ConvBlock block;

[0038] g-2) The first UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolutional layer and a ReLU activation function. Input to the first UpsamplingDeconvBlock block and output the feature map ;

[0039] g-3) The first ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the first ConvBlock block and output the feature map The second UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the second UpsamplingDeconvBlock block and output the feature map ;

[0040] g-4) The second ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the second ConvBlock block and output the feature map The third UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the third UpsamplingDeconvBlock block and output the feature map ;

[0041] g-5) The third ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the third ConvBlock block and output the feature map , the fourth UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function, which converts the feature map Input to the fourth UpsamplingDeconvBlock block and output the feature map ;

[0042] g-6) The fourth ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fourth ConvBlock block and output the feature map , the feature map Use the tanh function to get the horizontal distance graph , the horizontal distance map Input into the softmax function and output the segmentation map .

[0043] Further, step h) comprises the following steps:

[0044] h-1) The 3D-EDA branch of the GLB-Net model consists of the first convolutional layer, the second convolutional layer, the third convolutional layer, the first LeakyReLU activation function, the first dropout layer, the fourth convolutional layer, the second LeakyReLU activation function, the second dropout layer, the fifth convolutional layer, the third LeakyReLU activation function, the third dropout layer, the CARELayer module, the pooling layer, and the fully connected layer;

[0045] h-2) The segmentation graph Input into the first convolutional layer of the 3D-EDA branch and output the feature map , the training set Middle Preprocessed images Input into the second convolutional layer of the 3D-EDA branch and output the feature map , the feature map With feature map Perform addition operation to obtain feature map ;

[0046] h-3) The feature map It is sequentially input into the third convolutional layer, the first LeakyReLU activation function, and the first dropout layer of the 3D-EDA branch, and the feature map is output. ;

[0047] h-4) Feature map The output is the fourth convolution layer, the second LeakyReLU activation function, and the second dropout layer of the 3D-EDA branch, and the feature map is obtained. ;

[0048] h-5) Feature map The output is the fifth convolutional layer, the third LeakyReLU activation function, and the third dropout layer of the 3D-EDA branch, and the feature map is obtained. ;

[0049] h-6) The CARELayer module of the 3D-EDA branch consists of an average pooling layer, a first fully connected layer, a second fully connected layer, and an attention mechanism. Input to the average pooling layer of the CARELayer module and output the vector , the vector Input into the first fully connected layer of the CARELayer module and output the weight vector , the weight vector Input into the second fully connected layer of the CARELayer module and output the weight vector , the weight vector Input into the attention mechanism of the CARELayer module and output the enhanced feature map ;

[0050] h-7) The feature map The data is input into the pooling layer and the fully connected layer of the 3D-EDA branch in sequence, and the binary classification result is output.

[0051] Furthermore, in step i), the formula Calculate the total loss , where is the unsupervised loss function, , is the dice loss function, is the regulating factor, is the MSE loss function, is the consistency loss function, is the unsupervised loss function calculated based on the binary classification results.

[0052] Preferably, when training the GLB-Net model, the initial learning rate is set to 0.01, the batch_size is set to 4, the epoch is set to 16000, and the lr is set to 0.001.

[0053] The beneficial effects of the present invention are: combining the global-local information fusion LPV-Net branch and the 3D-EDA branch. The framework utilizes two different attention mechanisms to capture different dependencies in medical images to improve model performance. The discriminator model of 3D-EDA is adopted, and the concept of adversarial learning is used to enhance the performance of the model, especially for the processing of unlabeled data. The overall loss function consists of supervised loss and unsupervised loss. Training is performed through backpropagation and an optimizer to gradually adjust the model parameters to minimize the total loss. It has good performance and can achieve accurate segmentation of the left atrium. It improves the accuracy and efficiency of cardiac image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a structural diagram of the GLB-Net model of the present invention;

[0055] Figure 2 This is a structural diagram of the LPV-Net branch of the present invention. DETAILED DESCRIPTION

[0056] The following is combined with Figure 1 , Attachment Figure 2 The present invention is further described.

[0057] A semi-supervised cardiac image segmentation method based on global-local information fusion LPV-Net and 3D-EDA, comprising:

[0058] a) Acquisition A dataset of cardiac MRI images with or without labels , , is a labeled cardiac MRI image dataset, is an unlabeled cardiac MRI image dataset, , For the A labeled cardiac MRI image, , , For the An unlabeled cardiac MRI image, , is the number of labeled cardiac MRI images, is the number of unlabeled cardiac MRI images, .

[0059] b) For the dataset Perform preprocessing to obtain the preprocessed data set , , is a preprocessed labeled cardiac MRI image dataset, It is a preprocessed unlabeled cardiac MRI image dataset.

[0060] c) The preprocessed dataset Divide into training set , test set .

[0061] d) Establish the GLB-Net model. The GLB-Net model consists of the LPV-Net branch and the 3D-EDA branch. The LPV-Net branch consists of the encoder, LinformerPerformerBlock, and decoder.

[0062] e) The training set The image in is input to the encoder of the LPV-Net branch, and the output is the feature map .

[0063] f) The feature map Input to the LinformerPerformerBlock of the LPV-Net branch and output the feature map .

[0064] g) The feature map Input into the decoder of the LPV-Net branch and output the segmentation map .

[0065] h) Segment the map Input into the 3D-EDA branch to obtain the unsupervised loss function and the supervised loss function.

[0066] i) Calculate the total loss based on the unsupervised loss function and the supervised loss function , using the Adam optimizer through the total loss Train the GLB-Net model to obtain the optimized GLB-Net model.

[0067] j) Test set The image in is input into the LPV-Net branch of the optimized GLB-Net model, and the segmentation map is output. .

[0068] Table 1. Comparison of experimental results

[0069]

[0070] As shown in Table 1, to verify the reliability of this patented method, a series of experiments were conducted in which different models were used to evaluate the Dice coefficient, Jaccard coefficient, 95% HD coefficient, and ASD coefficient on the LA dataset. In our experimental approach, we compared this model with seven state-of-the-art semi-supervised methods, including consistency-based (MC-Net, DTC), uncertainty-based (UA-MT, DUWM), constraint-based (SASSNet), low-entropy-based (SS-Net), and multi-scale-based (URPC). The results were respectively based on fully supervised model training results, 10% labeled data, and 20% labeled data. It is worth noting that, with only labeled training data, the V-Net model achieved an average Dice coefficient of 89.01% and a Jaccard coefficient of 82.13%. The proposed GLB-Net framework achieved a Dice of 90.39%, a Jaccard of 82.54%, and an ASD of 1.77 mm on the 20% labeled data of the left atrial dataset, which outperformed the upper limit of other methods. However, its 95% HD effect was lower than that of V-Net.

[0071] In one embodiment of the present invention, step a) extracts the left atrium segmentation challenge dataset from 2018. Cardiac MRI images of the left atrium with or without annotations.

[0072] In one embodiment of the present invention, step b) comprises the following steps:

[0073] b-1) A labeled cardiac MRI image The area to be segmented is marked as 1, and the rest of the area is marked as 0. Use the np.nonzero function to find the A labeled cardiac MRI image The coordinates of all non-zero pixels in the label are calculated using the np.min function to obtain the minimum value of the coordinates of all non-zero pixels, and the maximum value of the coordinates of all non-zero pixels is calculated using the np.max function. The minimum rectangular frame containing all cardiac structures is obtained based on the minimum and maximum values ​​of the coordinates.

[0074] b-2) Use standard Numpy slicing operations to slice the first A labeled cardiac MRI image Perform cropping to obtain the cropped image .

[0075] b-3) The cropped image The image is obtained by RandomCrop function transformation , the image Perform normalization operation to obtain the preprocessed image , preprocessed labeled cardiac MRI image dataset .

[0076] b-4) An unlabeled cardiac MRI image The cropped image is obtained by randomly cropping the image using RandomCrop .

[0077] b-5) The cropped image The preprocessed image is obtained by randomly rotating 0 degrees, 90 degrees, 180 degrees or 270 degrees , preprocessed unlabeled cardiac MRI image dataset .

[0078] In one embodiment of the present invention, the pre-processed labeled cardiac MRI image data Select Zhang preprocessed images and unlabeled cardiac MRI image data from preprocessed images Select The training set is obtained by preprocessing the images , , from the preprocessed unlabeled cardiac MRI image dataset Select The training set is obtained by preprocessing the images , .

[0079] In one embodiment of the present invention, step e) comprises the following steps:

[0080] e-1) The encoder of the LPV-Net branch consists of the first ConvBlock block, the first DownsamplingConvBlock block, the second ConvBlock block, the second DownsamplingConvBlock block, the third ConvBlock block, the fourth DownsamplingConvBlock block, and the fifth ConvBlock block.

[0081] e-2) The first ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. The Preprocessed images Input into the first ConvBlock block and output the feature map , the ReLU activation function performs nonlinear changes. The first DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the first DownsamplingConvBlock block, and output the feature map with reduced spatial resolution .

[0082] e-3) The second ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the second ConvBlock block and output the feature map , the ReLU activation function performs nonlinear changes, and the second DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function in sequence, Input to the second DownsamplingConvBlock block, and output the feature map with reduced spatial resolution .

[0083] e-4) The third ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the third ConvBlock block and output the feature map , the ReLU activation function performs nonlinear changes, and the third DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function in sequence, Input to the third DownsamplingConvBlock block, and output the feature map with reduced spatial resolution .

[0084] e-5) The fourth ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fourth ConvBlock block and output the feature map , the ReLU activation function performs nonlinear changes, and the fourth DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function in sequence, Input to the fourth DownsamplingConvBlock block, and output the feature map with reduced spatial resolution .

[0085] e-6) The fifth ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fifth ConvBlock block and output the feature map , the ReLU activation function performs nonlinear changes.

[0086] In one embodiment of the present invention, step f) comprises the following steps:

[0087] The LinformerPerformerBlock of the LPV-Net branch in f-1) consists of the Linformer attention mechanism, the Performer attention mechanism, and the LayerNorm layer.

[0088] f-2) The feature map Input to the Linformer attention mechanism of LinformerPerformerBlock, perform low-rank decomposition on the length of the input sequence to reduce the complexity, and output the feature map .

[0089] f-3) The feature map Input into the Performer attention mechanism of LinformerPerformerBlock to reduce the complexity of self-attention calculation and output the feature map .

[0090] f-4) The feature map Input into the LayerNorm layer of LinformerPerformerBlock to accelerate convergence and output the feature map .

[0091] In one embodiment of the present invention, step g) comprises the following steps:

[0092] g-1) The decoder of the LPV-Net branch consists of a first UpsamplingDeconvBlock block, a first ConvBlock block, a second UpsamplingDeconvBlock block, a second ConvBlock block, a third UpsamplingDeconvBlock block, a third ConvBlock block, a fourth UpsamplingDeconvBlock block, and a fourth ConvBlock block.

[0093] g-2) The first UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolutional layer and a ReLU activation function. Input to the first UpsamplingDeconvBlock block, and output the feature map of spatial resolution restoration .

[0094] g-3) The first ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the first ConvBlock block and output the feature map The purpose of the ReLU activation function is to perform linear transformation. The second UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the second UpsamplingDeconvBlock block, and output the feature map with reduced spatial resolution .

[0095] g-4) The second ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the second ConvBlock block and output the feature map , the purpose of the ReLU activation function is to perform linear transformation. The third UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the third UpsamplingDeconvBlock block, and output the feature map with reduced spatial resolution .

[0096] g-5) The third ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the third ConvBlock block and output the feature map , the purpose of the ReLU activation function is to perform linear transformation. The fourth UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the fourth UpsamplingDeconvBlock block, and output the feature map with reduced spatial resolution .

[0097] g-6) The fourth ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fourth ConvBlock block and output the feature map , the feature map Use the tanh function to get the horizontal distance graph , the horizontal distance map Input into the softmax function and output the segmentation map .

[0098] In one embodiment of the present invention, step h) comprises the following steps:

[0099] h-1) The 3D-EDA branch of the GLB-Net model consists of the first convolutional layer, the second convolutional layer, the third convolutional layer, the first LeakyReLU activation function, the first dropout layer, the fourth convolutional layer, the second LeakyReLU activation function, the second dropout layer, the fifth convolutional layer, the third LeakyReLU activation function, the third dropout layer, the CARELayer module, the pooling layer, and the fully connected layer.

[0100] h-2) The segmentation graph Input into the first convolutional layer of the 3D-EDA branch and output the feature map , the training set Middle Preprocessed image Input into the second convolutional layer of the 3D-EDA branch and output the feature map , the feature map With feature map Perform addition operation to obtain feature map .

[0101] h-3) The feature map It is sequentially input into the third convolutional layer, the first LeakyReLU activation function, and the first dropout layer of the 3D-EDA branch, and the feature map is output. .

[0102] h-4) Feature map The output is the fourth convolution layer, the second LeakyReLU activation function, and the second dropout layer of the 3D-EDA branch, and the feature map is obtained. .

[0103] h-5) Feature map The output is the fifth convolution layer, the third LeakyReLU activation function, and the third dropout layer of the 3D-EDA branch, and the feature map is obtained. .

[0104] h-6) The CARELayer module of the 3D-EDA branch consists of an average pooling layer, a first fully connected layer, a second fully connected layer, and an attention mechanism. Input to the average pooling layer of the CARELayer module and output the vector , the vector Input into the first fully connected layer of the CARELayer module and output the weight vector , the weight vector Input into the second fully connected layer of the CARELayer module and output the weight vector , the weight vector Input into the attention mechanism of the CARELayer module and output the enhanced feature map .

[0105] h-7) The feature map The data is input into the pooling layer and the fully connected layer of the 3D-EDA branch in sequence, and the binary classification result is output.

[0106] In one embodiment of the present invention, in step i), the formula Calculate the total loss , where is the unsupervised loss function, , is the dice loss function, Is an adjustment factor used to adjust the weight of segmentation loss and shape loss in the total loss, is the MSE loss function, is a consistency loss function, which is used to measure the consistency of the output of the model in different situations. is the unsupervised loss function calculated from the binary classification results, which is used to measure the difference between the output of the beam 3D-EDA and the real data. In this embodiment, the initial learning rate is set to 0.01, the batch_size is set to 4, the epoch is set to 16000, and the lr is set to 0.001 when training the GLB-Net model.

[0107] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A semi-supervised cardiac image segmentation method based on global-local information fusion LPV-Net and 3D-EDA, characterized by: include: a) Acquisition labeled or unlabeled cardiac MRI images, constituting a dataset , , is a labeled cardiac MRI image dataset, is an unlabeled cardiac MRI image dataset, , For the A labeled cardiac MRI image, , , For the An unlabeled cardiac MRI image, , is the number of labeled cardiac MRI images, is the number of unlabeled cardiac MRI images, ; b) For the dataset Perform preprocessing to obtain the preprocessed data set , , is a preprocessed labeled cardiac MRI image dataset, It is a preprocessed unlabeled cardiac MRI image dataset; c) The preprocessed dataset Divide into training set , test set ; d) Establish the GLB-Net model. The GLB-Net model consists of the LPV-Net branch and the 3D-EDA branch. The LPV-Net branch consists of the encoder, LinformerPerformerBlock, and decoder. e) The training set The image in is input to the encoder of the LPV-Net branch, and the output is the feature map ; f) The feature map Input to the LinformerPerformerBlock of the LPV-Net branch and output the feature map ,LinformerPerformerBlock of LPV-Net branch consists of Linformer attention mechanism, Performer attention mechanism, and LayerNorm layer; g) The feature map Input into the decoder of the LPV-Net branch and output the segmentation map ; h) Segment the map Input into the 3D-EDA branch to obtain the unsupervised loss function and the supervised loss function. The 3D-EDA branch consists of the first convolutional layer, the second convolutional layer, the third convolutional layer, the first LeakyReLU activation function, the first dropout layer, the fourth convolutional layer, the second LeakyReLU activation function, the second dropout layer, the fifth convolutional layer, the third LeakyReLU activation function, the third dropout layer, the CARELayer module, the pooling layer, and the fully connected layer. The CARELayer module consists of the average pooling layer, the first fully connected layer, the second fully connected layer, and the attention mechanism in sequence; i) Calculate the total loss based on the unsupervised loss function and the supervised loss function , using the Adam optimizer through the total loss Train the GLB-Net model to obtain the optimized GLB-Net model; j) Test set The image in is input into the LPV-Net branch of the optimized GLB-Net model, and the segmentation map is output. .

2. The method for semi-supervised cardiac image segmentation based on global-local information fusion LPV-Net and 3D-EDA according to claim 1, characterized in that: Step a) extracts the data from the 2018 Left Atrium Segmentation Challenge dataset Cardiac MRI images of the left atrium with or without annotations.

3. The cardiac image semi-supervised segmentation method based on global-local information fusion LPV-Net and 3D-EDA according to claim 1, characterized in that: Step b) comprises the following steps: b-1) A labeled cardiac MRI image The area to be segmented is marked as 1, and the rest of the area is marked as 0. Use the np.nonzero function to find the A labeled cardiac MRI image The coordinates of all non-zero pixels in the label are calculated using the np.min function to obtain the minimum value of the coordinates of all non-zero pixels, and the maximum value of the coordinates of all non-zero pixels is calculated using the np.max function. The minimum rectangular frame containing all cardiac structures is obtained based on the minimum and maximum values ​​of the coordinates; b-2) Use standard Numpy slicing operations to slice the first A labeled cardiac MRI image Perform cropping to obtain the cropped image ; b-3) The cropped image The image is obtained by RandomCrop function transformation , the image Perform normalization operation to obtain the preprocessed image , preprocessed labeled cardiac MRI image dataset ; b-4) An unlabeled cardiac MRI image The cropped image is obtained by randomly cropping the image using RandomCrop ; b-5) The cropped image The preprocessed image is obtained by randomly rotating 0 degrees, 90 degrees, 180 degrees or 270 degrees , preprocessed unlabeled cardiac MRI image dataset .

4. The method for semi-supervised cardiac image segmentation based on global-local information fusion LPV-Net and 3D-EDA according to claim 1, characterized in that: From the pre-processed labeled cardiac MRI image data Select Zhang preprocessed images and unlabeled cardiac MRI image data from preprocessed images Select The training set is obtained by preprocessing the images , , from the preprocessed unlabeled cardiac MRI image dataset Select The training set is obtained by preprocessing the images , .

5. The method for semi-supervised segmentation of cardiac images based on global-local information fusion LPV-Net and 3D-EDA according to claim 4, characterized in that: Step e) comprises the following steps: e-1) The encoder of the LPV-Net branch consists of the first ConvBlock block, the first DownsamplingConvBlock block, the second ConvBlock block, the second DownsamplingConvBlock block, the third ConvBlock block, the fourth DownsamplingConvBlock block, and the fifth ConvBlock block; e-2) The first ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. The Preprocessed images Input into the first ConvBlock block and output the feature map The first DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input into the first DownsamplingConvBlock block and output the feature map ; e-3) The second ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the second ConvBlock block and output the feature map The second DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the second DownsamplingConvBlock block and output the feature map ; e-4) The third ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the third ConvBlock block and output the feature map The third DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the third DownsamplingConvBlock block and output the feature map ; e-5) The fourth ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fourth ConvBlock block and output the feature map , the fourth DownsamplingConvBlock block of the encoder is composed of a three-dimensional convolution layer and a ReLU activation function, which converts the feature map Input to the fourth DownsamplingConvBlock block and output the feature map ; e-6) The fifth ConvBlock block of the encoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fifth ConvBlock block and output the feature map .

6. The method for semi-supervised segmentation of cardiac images based on global-local information fusion LPV-Net and 3D-EDA according to claim 1, characterized in that: Step f) comprises the following steps: The LinformerPerformerBlock of the LPV-Net branch in f-1) consists of the Linformer attention mechanism, the Performer attention mechanism, and the LayerNorm layer; f-2) The feature map Input to the Linformer attention mechanism of LinformerPerformerBlock and output the feature map ; f-3) The feature map Input to the Performer attention mechanism of LinformerPerformerBlock and output the feature map ; f-4) The feature map Input to the LayerNorm layer of LinformerPerformerBlock and output the feature map .

7. The method for semi-supervised segmentation of cardiac images based on global-local information fusion LPV-Net and 3D-EDA according to claim 1, characterized in that: Step g) comprises the following steps: g-1) The decoder of the LPV-Net branch consists of a first UpsamplingDeconvBlock block, a first ConvBlock block, a second UpsamplingDeconvBlock block, a second ConvBlock block, a third UpsamplingDeconvBlock block, a third ConvBlock block, a fourth UpsamplingDeconvBlock block, and a fourth ConvBlock block; g-2) The first UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolutional layer and a ReLU activation function. Input to the first UpsamplingDeconvBlock block and output the feature map ; g-3) The first ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the first ConvBlock block and output the feature map The second UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the second UpsamplingDeconvBlock block and output the feature map ; g-4) The second ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the second ConvBlock block and output the feature map The third UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function. Input to the third UpsamplingDeconvBlock block and output the feature map ; g-5) The third ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the third ConvBlock block and output the feature map , the fourth UpsamplingDeconvBlock block of the decoder is composed of a three-dimensional convolution layer and a ReLU activation function, which converts the feature map Input to the fourth UpsamplingDeconvBlock block and output the feature map ; g-6) The fourth ConvBlock block of the decoder is composed of a three-dimensional convolutional layer, a BN layer, and a ReLU activation function. Input into the fourth ConvBlock block and output the feature map , the feature map Use the tanh function to get the horizontal distance graph , the horizontal distance map Input into the softmax function and output the segmentation map .

8. The method for semi-supervised segmentation of cardiac images based on global-local information fusion LPV-Net and 3D-EDA according to claim 1, characterized in that: Step h) comprises the following steps: h-1) The 3D-EDA branch of the GLB-Net model consists of the first convolutional layer, the second convolutional layer, the third convolutional layer, the first LeakyReLU activation function, the first dropout layer, the fourth convolutional layer, the second LeakyReLU activation function, the second dropout layer, the fifth convolutional layer, the third LeakyReLU activation function, the third dropout layer, the CARELayer module, the pooling layer, and the fully connected layer; h-2) The segmentation graph Input into the first convolutional layer of the 3D-EDA branch and output the feature map , the training set Middle Preprocessed images Input into the second convolutional layer of the 3D-EDA branch and output the feature map , the feature map With feature map Perform addition operation to obtain feature map ; h-3) The feature map It is sequentially input into the third convolutional layer, the first LeakyReLU activation function, and the first dropout layer of the 3D-EDA branch, and the feature map is output. ; h-4) Feature map The output is the fourth convolution layer, the second LeakyReLU activation function, and the second dropout layer of the 3D-EDA branch, and the feature map is obtained. ; h-5) Feature map The output is the fifth convolutional layer, the third LeakyReLU activation function, and the third dropout layer of the 3D-EDA branch, and the feature map is obtained. ; h-6) The CARELayer module of the 3D-EDA branch consists of an average pooling layer, a first fully connected layer, a second fully connected layer, and an attention mechanism. Input to the average pooling layer of the CARELayer module and output the vector , the vector Input into the first fully connected layer of the CARELayer module and output the weight vector , the weight vector Input into the second fully connected layer of the CARELayer module and output the weight vector , the weight vector Input into the attention mechanism of the CARELayer module and output the enhanced feature map ; h-7) The feature map The data is input into the pooling layer and the fully connected layer of the 3D-EDA branch in sequence, and the binary classification result is output.

9. The method for semi-supervised segmentation of cardiac images based on global-local information fusion LPV-Net and 3D-EDA according to claim 8, characterized in that: In step i), the formula Calculate the total loss , where is the unsupervised loss function, , is the dice loss function, is the regulating factor, is the MSE loss function, is the consistency loss function, is the unsupervised loss function calculated based on the binary classification results.

10. The method for semi-supervised segmentation of cardiac images based on global-local information fusion LPV-Net and 3D-EDA according to claim 8, characterized in that: When training the GLB-Net model, the initial learning rate is set to 0.01, batch_size is set to 4, epoch is set to 16000, and lr is set to 0.001.

Images