A semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning

Abstract

The present invention discloses a semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning, including data preprocessing, model building and hybrid loss function, characterized in that a stroke lesion segmentation dataset is used in the data preprocessing to extract a diffusion-weighted image for constructing a stroke segmentation dataset based on semi-supervised learning; the model building designs a 3D ASPP, and introduces it into the V-Net model to construct an STF-Net that is friendly to small target segmentation, and uses three STF-Nets to construct a semi-supervised stroke lesion segmentation model based on mutual consistency learning, and performs unsupervised learning by utilizing the noise output by the three models. The semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning disclosed by the present invention is not only effective in the segmentation of dense small lesions, but also performs well in the segmentation of large-scale stroke lesions, that is, conventional large targets.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image processing, and in particular to a semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning. Background Art

[0002] Stroke has become one of the top killers worldwide. It can be divided into ischemic stroke and hemorrhagic stroke, of which ischemic stroke accounts for 85%. Ischemic stroke is one of the major diseases that threaten human health and survival. It is a common disease among middle-aged and elderly people and is becoming increasingly prevalent among younger people. It has high morbidity, mortality, disability and recurrence rates, and is currently a disease that is being focused on for prevention and treatment. The survival rate for first-time patients is 70%, and the disability rate for surviving patients is 70%. For ischemic stroke, which has a high incidence, grasping the time for emergency treatment and standardized treatment is the "prescription". Thrombolytic therapy within the golden 6 hours after onset, especially 3 to 4.5 hours, can significantly benefit most patients and even recover to near normal.

[0003] Diffusion-weighted imaging is a new MR functional imaging technology and one of the most commonly used sequences for cranial MR imaging. It has a fast imaging speed and plays a very important role in the diagnosis of many diseases. In its initial clinical applications, it was mainly used to diagnose early ischemic stroke. Half an hour after cerebral vascular embolism, local brain tissue develops cytotoxic edema due to ischemia and hypoxia, the diffusion of free water is restricted, the ADC value decreases, and it appears as an obvious high signal on the DWI image. Conventional T2-weighted imaging is difficult to diagnose three hours after ischemia. Magnetic resonance diffusion-weighted imaging is currently the only non-invasive method that can detect the diffusion movement of water molecules in living tissues. It is very sensitive to cerebral ischemia, especially acute cerebral ischemia. Summary of the Invention

[0004] The present invention discloses a semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning, aiming to solve the technical problems in the background technology.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions:

[0006] A semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning, comprising 1. data preprocessing, model building and hybrid loss function, characterized in that a stroke lesion segmentation dataset is used in the data preprocessing to extract diffusion-weighted images for constructing a stroke segmentation dataset based on semi-supervised learning; a 3D ASPP (Atrous Spatial Pyramid Pooling) is designed in the model building, and it is introduced into the V-Net model with CBAM (Convolutional Block Attention Module) to construct an STF-Net (Small Target Friendly The proposed method uses three STF-Nets to construct a semi-supervised stroke lesion segmentation model based on mutual consistency learning. The noise output by the three models is used for unsupervised learning, and the update direction of the model parameters is constrained by a small number of labels to achieve the purpose of semi-supervised learning with limited labels. The hybrid loss function is mainly divided into two parts, namely the supervised loss function and the unsupervised loss function. For the supervised loss function, if the input has a corresponding label, the label and the feature map with the same scale as the input image output by the three models are used to calculate the MSE loss function. If unlabeled data is input, the 4 feature maps of 12 different scales output by the three models are used to calculate the consistency loss function. Finally, the test results are numerically evaluated and analyzed. The evaluation content includes the comparison of Dice coefficient, Jaccard coefficient, 95% Hausdorff distance and ASD to determine whether accurate segmentation of stroke lesions is achieved.

[0007] In a preferred scheme, diffusion-weighted images of the hyperacute phase and paired manual labels are used to construct a semi-supervised segmentation dataset of stroke lesions based on consistency learning. The small target segmentation-friendly STF-Net is constructed using void convolution spatial pyramid pooling and convolutional attention modules. The output is used to calculate the consistency loss function to provide rich learnable features for semi-supervised learning. At the same time, hyperparameters are introduced to balance the supervised loss and the consistency loss. The plasticity of the loss function is improved, and the proportion of the two can be adjusted for different tasks to obtain different results. The acquired features are learned to obtain the optimal network parameters, and the segmentation results are evaluated and analyzed using numerical methods.

[0008] In a preferred scheme, in the diffusion-weighted images of the hyperacute phase, the method is not only effective in the segmentation of dense small lesions, but also performs well in the segmentation of large-scale stroke lesions, that is, conventional large targets. At the same time, a backbone network and loss function that are friendly to small target segmentation are constructed to provide rich learnable features for the small target segmentation network. Since the current application of semi-supervised segmentation in medical images mostly focuses on organ segmentation, it is still relatively difficult to segment lesions with diverse morphologies and positions and small target volumes. A new type of semi-supervised segmentation network based on mutual consistency learning is constructed to accurately segment stroke lesions in diffusion-weighted images of the hyperacute phase.

[0009] From the above, it can be seen that the semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning provided by the present invention is compared with other semi-supervised segmentation models, and the test results are numerically evaluated and analyzed. The evaluation content includes the comparison of Dice coefficient, Jaccard coefficient, 95% Hausdorff distance, and ASD to determine whether accurate segmentation of stroke lesions is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 This is a schematic diagram of the overall structure of a semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning proposed in the present invention.

[0011] Figure 2 Schematic diagram of a small-target-friendly segmentation network for a semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning proposed in the present invention.

[0012] Figure 3 This is the segmentation result of dense small lesions of a semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning proposed in the present invention.

[0013] Figure 4 This is a segmentation result map of large-scale lesions, i.e., conventional large targets, for a semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning proposed in the present invention. DETAILED DESCRIPTION

[0014] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0015] A semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning includes a data preprocessing part. Using a stroke lesion segmentation dataset, diffusion-weighted images (DWI) are extracted to construct a stroke segmentation dataset based on semi-supervised learning. The training set includes 222 MRI images and the test set includes 20 annotated MRI images.

[0016] For model construction, we designed a 3D ASPP (Atrous Spatial Pyramid Pooling) and introduced it together with CBAM (Convolutional Block Attention Module) into the V-Net model to construct STF-Net (Small Target Friendly Network) which is friendly to small target segmentation.

[0017] From the overall model perspective, the downsampling strategy is slightly changed, and three STF-Nets are used to construct a semi-supervised stroke lesion segmentation model based on mutual consistency learning. Unsupervised learning is performed by utilizing the noise output by the three models, and the direction of model parameter update is constrained by a small number of labels, so as to achieve the purpose of semi-supervised learning with limited labels.

[0018] The network structure diagram used in the present invention is as follows Figure 1 As shown in the figure. The small target friendly segmentation network STF-Net proposed in this invention is as follows Figure 2 shown.

[0019] The hybrid loss function consists of two main parts: a supervised loss function and an unsupervised loss function. For the supervised loss function, if the input is labeled, the labels and the feature maps output by the three models at the same scale as the input image are used to calculate the Mean Sequence Error (MSE) loss function. If the input is unlabeled, the four feature maps output by the three models at 12 different scales are used to calculate the consistency loss function.

[0020] The results were evaluated and compared with other segmentation models based on semi-supervised learning. The test results were evaluated and analyzed numerically. The evaluation content included comparison of Dice coefficient, Jaccard coefficient, 95% Hausdorff distance, and ASD to determine whether accurate segmentation of stroke lesions was achieved.

[0021] Supervised loss function , consistency learning loss function And the mixed loss function As shown below:

[0022]

[0023]

[0024]

[0025] in, represents the j-th output feature map of the i-th scale of the main model, represents the j-th output feature map of the i-th scale of the h-th auxiliary model, represents the j-th output pseudo label of the i-th scale of the main model, represents the j-th output pseudo label of the i-th scale of the h-th auxiliary model.

[0026] The segmentation results of dense small lesions are shown in the attached Figure 3 As shown in the figure, the segmentation results of large-scale lesions are shown in the attached figure. Figure 4 shown.

[0027] Table 1: Evaluation metrics for 10% and 20% labels

[0028]

[0029] Table 2: Segmentation results of dense small lesions and large lesions

[0030]

[0031] The Dice coefficient and Jaccard coefficient values ​​of the experimental results in Tables 1 and 2 compared to the original images are large, while the 95% Hausdorff distance and ASD values ​​are small, demonstrating that this method can achieve significant results in the task of segmenting stroke lesions. The semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning performs well, accurately segmenting stroke lesions in diffusion-weighted images, and significantly reducing missed and mis-segmented instances. The four evaluation indicators also demonstrate that the present invention has good results.

[0032] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0033] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

[0034] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.

Claims

1. A semi-supervised stroke lesion segmentation method based on multi-scale mutual consistency learning, including data preprocessing, model building and hybrid loss function, characterized in that: The data preprocessing uses a stroke lesion segmentation dataset to extract a diffusion-weighted image for constructing a stroke segmentation dataset based on semi-supervised learning; The model design and construction of a 3D ASPP (Atrous Spatial Pyramid Pooling) is introduced into the V-Net model with CBAM (Convolutional Block Attention Module) to construct an STF-Net (Small Target Friendly Network) that is friendly to small target segmentation. Three STF-Nets are used to build a semi-supervised stroke lesion segmentation model based on mutual consistency learning. Unsupervised learning is performed by utilizing the noise output by the three models, and the direction of model parameter update is constrained by a small number of labels to achieve the purpose of semi-supervised learning with limited labels. The hybrid loss function is mainly divided into two parts, namely the supervised loss function and the unsupervised loss function. For the supervised loss function, if the input has a corresponding label, the label and the feature map with the same scale as the input image output by the three models are used to calculate the MSE loss function. If unlabeled data is input, the four types of feature maps with a total of 12 different scales output by the three models are used to calculate the consistency loss function. Finally, the test results are numerically evaluated and analyzed. The evaluation content includes comparison of the Dice coefficient, Jaccard coefficient, 95% Hausdorff distance, and ASD to determine whether accurate segmentation of stroke lesions is achieved.

2. The method for semi-supervised stroke lesion segmentation based on multi-scale mutual consistency learning according to claim 1, characterized in that: A semi-supervised segmentation dataset of stroke lesions based on consistency learning was constructed using diffusion-weighted images of the hyperacute phase and paired manual labels.

3. The method for semi-supervised stroke lesion segmentation based on multi-scale mutual consistency learning according to claim 1, characterized in that: Use dilated convolution spatial pyramid pooling and convolutional attention modules to build a small object segmentation-friendly STF-Net.

4. The method for semi-supervised stroke lesion segmentation based on multi-scale mutual consistency learning according to claim 1, characterized in that: The output is used to calculate the consistency loss function, providing rich learnable features for semi-supervised learning. At the same time, hyperparameters are introduced to balance the supervised loss and consistency loss. The plasticity of the loss function is improved, and the proportion of the two can be adjusted for different tasks to obtain different results.

5. The method for semi-supervised stroke lesion segmentation based on multi-scale mutual consistency learning according to claim 1, characterized in that: The acquired features are learned to obtain the optimal network parameters, and the segmentation results are evaluated and analyzed using numerical methods.

6. The method for semi-supervised stroke lesion segmentation based on multi-scale mutual consistency learning according to claim 1, characterized in that: At the same time, a backbone network and loss function that are friendly to small object segmentation are constructed to provide rich learnable features for the small object segmentation network.

7. The method for semi-supervised stroke lesion segmentation based on multi-scale mutual consistency learning according to claim 1, characterized in that: Supervised loss function , consistency learning loss function And the mixed loss function As shown below: in, represents the j-th output feature map of the i-th scale of the main model, represents the jth output feature map of the i-th scale of the h-th auxiliary model, represents the j-th output pseudo label of the i-th scale of the main model, represents the j-th output pseudo label of the i-th scale of the h-th auxiliary model.

Images