Construction method and application of choroid segmentation model in OCT images combined with prior mask

Abstract

The present invention discloses a method for constructing a choroidal segmentation model for OCT images in combination with a priori masks and its application. The present invention includes data preprocessing, through cropping, adjusting the scale, and then cropping to unify the data size; then generating two-stage choroidal masks through annotation; to solve the problem of complex image background, a cascaded multi-stage segmentation network is designed. In the first stage, the part below the BM layer is segmented to remove the upper background interference, and the second stage is to finely segment the choroid; to solve the problem of uneven choroidal texture and blurred lower boundary, a multi-scale context aggregation module is designed; finally, to optimize the training process of the cascade network, an adaptive cross-stage feature fusion module is designed. The present invention fully utilizes the information of the optical coherence tomography image itself and can effectively capture the multi-scale information and contextual information of the image, thereby improving the choroidal segmentation effect of the optical coherence tomography image.

Description

Technical Field

[0001] The present invention relates to a method for constructing an OCT image choroid segmentation model combined with a priori mask and its application. Background Art

[0002] The choroid is located between the retina and the sclera. It is a soft, smooth, elastic, and vascular brown film that can provide oxygen and nutrition to the retina. Studies have shown that the morphology of the choroid is highly correlated with major eye diseases such as glaucoma, age-related macular degeneration (AMD), pathological myopia, and diabetic retinopathy (DR). Traditional fundus photography, laser scanning fundus photography, and other examinations are two-dimensional imaging modes that cannot accurately reflect the pathological conditions of the choroid. Optical coherence tomography (OCT) imaging is a living tomographic high-resolution imaging technology that can reflect the three-dimensional anatomical structure information of the retina and greatly improve the visualization of the details of choroidal lesions. OCT imaging uses the principle of low-coherence light interference to obtain retinal tomographic images and can perform three-dimensional reconstruction of the retina. Doctors can clearly distinguish the various layers of the retina by analyzing OCT images.

[0003] Traditional choroid segmentation methods are mostly based on graph search, which is limited by the selection of appropriate graph edge weights. Once the selection is inappropriate, the segmentation results will be inaccurate. Therefore, manual inspection and correction are often required, which is very tedious. With the rise of deep learning technology in recent years, convolutional neural networks have been proven to have powerful image segmentation capabilities. However, despite this, the development of choroid automatic segmentation technology is still immature, mainly due to the following defects and deficiencies: (1) the subchoroidal boundary is relatively vague, making direct segmentation difficult; (2) the choroid region has vascular structures, which makes it have uneven texture and intensity. Summary of the Invention

[0004] In view of the shortcomings of the existing technology, the present invention proposes a method for constructing a choroid segmentation model for OCT images combined with a priori mask and its application.

[0005] To achieve the above object, the present invention comprises the following steps:

[0006] One aspect of the present invention provides a method for constructing a choroid segmentation model for OCT images in combination with a priori masks, the method comprising the following steps:

[0007] Step 1) unifying the OCT image size through preprocessing;

[0008] Step 2) mask generation, including first-stage mask and second-stage mask;

[0009] Step 3) Construct a cascade segmentation network based on the encoder-decoder structure;

[0010] The cascade segmentation network consists of two encoding-decoding networks, which are connected by a semantic prediction guidance module. The first encoding network consists of two residual convolution modules and three multi-scale context aggregation modules, and the second encoding network consists of five convolution modules. The decoding networks are composed of four upsampling convolution modules.

[0011] Step 4) The image data processed in step 1) and the mask generated in step 2) are enhanced and input into the cascade segmentation network, and trained using the ADAM algorithm to obtain a choroidal segmentation network model that can be used for optical coherence tomography images.

[0012] Furthermore, the method further includes extracting the model trained in step 4), testing it with the data processed in step 1), and evaluating the segmentation results using accuracy, precision, intersection-over-union ratio, Dice coefficient, and Hausdorff distance.

[0013] Furthermore, the skip connections at each scale between the second encoding network and the second decoding network are inserted into an adaptive cross-stage fusion module to fuse the features of the previous stage.

[0014] Furthermore, the multi-scale context aggregation module groups feature maps and then uses residual connections between groups to generate features of different scales; the features of different scales are expanded into one-dimensional features and positionally encoded; the encoded features are spliced ​​and sent to the deformable self-attention layer to model contextual relationships of different scales.

[0015] Furthermore, the adaptive cross-stage fusion module fuses feature maps from different stages by using a channel attention mechanism to select appropriate attention weights for features from different stages and using a certain proportion of Dropout to avoid overfitting.

[0016] Another aspect of the present invention provides an application of the above-constructed OCT image choroidal segmentation model combined with a priori mask in choroidal segmentation.

[0017] Beneficial effects of the present invention:

[0018] This paper decomposes the choroid segmentation task into a cascade of tasks for joint learning, solving the problem of the difficulty of directly segmenting the choroid region. At the same time, this cascade can greatly avoid the interference of the background above the choroid, allowing the network to focus on identifying the area below, thereby enhancing the model's boundary perception ability.

[0019] The multi-scale context aggregation module used in the present invention can effectively capture the multi-scale context information of OCT images and can still perform good recognition in choroidal areas with uneven texture and intensity.

[0020] The adaptive cross-stage fusion module adopted in the present invention can effectively fuse the features of the two stages, effectively alleviate the problem of gradient attenuation in the cascade network, and effectively optimize the training process of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Generate a flow chart for the mask;

[0022] Figure 2 This is the overall network structure diagram;

[0023] Figure 3 Schematic diagram of the structure of the multi-scale context aggregation module (MSCA module);

[0024] Figure 4 It is an adaptive cross-stage feature fusion module (ACSFF module structure diagram);

[0025] Figure 5 The choroid segmentation results of different segmentation networks. DETAILED DESCRIPTION

[0026] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0027] Prior to the present invention, the existing technology of U-net achieved image segmentation through encoding and decoding, and has been proven to perform well in many medical image segmentation tasks. However, the present invention takes into account that due to the low contrast of OCT images, the choroidal texture is uneven and the lower boundary is relatively fuzzy, which makes U-net perform poorly in this segmentation task. To adapt to the particularity of this segmentation task, the present invention combines the OCT image structural information and designs a cascade segmentation model structure, which decomposes the choroidal segmentation task into two stages of segmentation. The first stage segments the part below Bruch's membrane (BM). The second stage uses the results of the first stage to finely segment the choroid. By going from easy to difficult, the network can better learn the characteristics of the image.

[0028] The embodiment of the present invention includes the following steps:

[0029] 1) Since the original collected OCT images have different sizes and contain a lot of irrelevant information, they cannot be directly input into the network for training. Therefore, they are preprocessed to a uniform size.

[0030] First, the fundus image was cropped and then adjusted according to the original image ratio. Finally, the image data of 224×224 size was cropped and scaled centered on the central pixel of the image. The upper and lower boundaries of the choroid were then annotated using ImageJ software. The dataset consists of 1062 OCT images from patients with high myopia containing gold standards.

[0031] 2) Use the canny operator to extract the boundaries of the labeled data.

[0032] First, the position of the upper boundary BM is extracted, and then the image is binarized to change the pixel value of the area above the upper boundary to 0 and the pixel value of the area below the upper boundary to 1, thereby generating the first stage mask.

[0033] The second stage mask is obtained by binarizing the upper and lower boundaries extracted from the labeled data. The pixels within the upper and lower boundaries are assigned a value of 1, and the pixels outside the boundaries are assigned a value of 0 to generate the second stage mask. The mask generation process is as follows: Figure 1 shown.

[0034] 3) Construct a cascade segmentation network based on the encoder-decoder structure.

[0035] The cascade segmentation network includes two encoder-decoder networks (such as Figure 2 As shown in the figure), represented by E1-D1 and E2-D2. The two encoding-decoding networks are connected by a semantic prediction guidance module (SPG). E1 is mainly composed of 2 residual convolution modules (Res2Block) and 3 multi-scale context aggregation modules (MSCA), and E2 is composed of 5 convolution modules. Both D1 and D2 are composed of 4 upsampling convolution modules, and an adaptive cross-stage fusion module (ACSFF) is inserted into the skip connection of each scale between E2-D2 to fuse the features of the previous stage. In the first stage, the cascade segmentation network first segments the part below the BM layer to remove the background interference above, and then finely segments the choroid in the second stage.

[0036] In the cascade segmentation network, the multi-scale context aggregation module (such as Figure 3 As shown in the figure, the feature maps output by the previous encoder are first grouped. Residual connections are then used between the groups to generate features of different scales. These features are then expanded into one-dimensional features and positionally encoded. The encoded features are then concatenated and fed into a deformable self-attention layer to model and fuse contextual relationships at different scales. The generated multi-scale information contains different receptive fields. Smaller receptive fields have strong geometric representation capabilities, enabling precise perception of boundaries, while larger receptive fields have strong semantic representation capabilities, enabling the approximate extent of objects. The introduced self-attention mechanism effectively integrates the information represented by these different receptive fields, making segmentation more accurate and addressing the uneven intensity and texture within the choroid.

[0037] In order to further optimize the network training process and alleviate the attenuation of gradients, an adaptive cross-stage feature fusion module (such as Figure 4As shown in Figure 2, this module first fuses the encoder features of E1 and the decoder features of D1. The resulting fused features are then fused with the encoder features of E2 to obtain the final features, which are then concatenated with the decoder features of D2 via skip connections. Taking the fusion process of the encoder features of E1 and the decoder features of D1 as an example, the two features are first accumulated and globally pooled to obtain channel features. The channel features are then encoded through a fully connected layer to obtain two sets of features, which are then normalized using the softmax function. Finally, the original features are recalibrated using the obtained attention weights.

[0038] 4) The image data is enhanced using Gaussian filtering and histogram equalization. This is then fed into the cascade network along with the mask generated in step 2) for training. The following loss function is used during training:

[0039]

[0040] in and Represent the losses of the true value labels and prediction results of stage one and two respectively, and α and β are the weights used for control. and And by as well as composition:

[0041]

[0042]

[0043] in represents the binary cross entropy loss, represents the Dice coefficient loss, represents the prediction result of the i-th stage, Represents the true value label of the i-th stage; the gradient is calculated according to the loss value during the back propagation process, and Adam is used as the optimizer with an initial learning rate of 1e-3. The network weight parameters are optimized through continuous iteration.

[0044] In order to verify the effectiveness of this embodiment, 1062 OCT images of patients with high myopia were used to extract the model trained in step 1), and the data processed in step 1) was used for testing. Comparative experiments were conducted with multiple networks. The comparative experiments all adopted 4-fold cross validation, and the accuracy (ACC), precision (Precision), intersection over union (IoU), Dice coefficient and Hausdorff distance (Hausdorff) were used to evaluate the segmentation results. The network of this embodiment is denoted as PMGT-Net, and compared with the existing FCN-8, U-Net, UNet++, Attention UNet (AttUNet), CE-Net, SCA-UNet, SwinUnet, MedT, and TransUNet. The comparison results are shown in Table 1, which is a comparison table of the cascade segmentation network PMGT-Net of this embodiment and other segmentation networks:

[0045] Table 1 Comparison results of the cascade segmentation network PMGT-Net of this embodiment with other segmentation networks

[0046] Segmentation Network ACC (%) Precision(%) IoU (%) Dice (%) Hausdorff FCN-8 99.19 88.25 81.97 89.84 4.6424 U-Net 99.24 90.74 83.81 90.70 4.4848 U-Net++ 99.34 91.85 85.19 91.85 4.3026 CE-Net 99.32 91.00 84.61 91.51 4.3900 SwinUnet 99.09 88.88 80.42 88.85 4.7384 Medt 99.22 89.97 83.07 90.52 4.4984 TransUNet 99.30 91.45 84.49 91.42 4.3871 PMGT-Net 99.38 92.25 85.93 92.31 4.2234

[0047] As shown in Table 1, the PMGT-Net proposed in this embodiment achieves the best performance in comparison with other segmentation networks. Compared with FCN-8, the proposed PMGT-Net improves Precision, IoU, Dice, and Hausdorff indicators by 4%, 3.96%, 2.47%, and 0.419, respectively. Figure 5 The choroid segmentation results of PMGT-Net and 5 other segmentation networks are shown. Figure 5 As can be seen, the PMGT-Net proposed in this example has good adaptability to the location and shape of the choroid, effectively suppressing false positives and false negatives, and achieving optimal segmentation results for images with different shapes and interference. The implementation environment for this example is: GPU: TITAN XP; runtime environment: Python 3.6, PyTorch 1.7, and Ubuntu 16.04.

[0048] The embodiments of the present invention described above do not limit the scope of protection of the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included in the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a choroidal segmentation model for OCT images using a priori mask, characterized in that: The method comprises the following steps: Step 1) Unify the OCT image size through preprocessing; Step 2) Mask generation, including first-stage mask and second-stage mask; The mask generation method in the first stage is as follows: first, the position of the upper boundary BM is extracted, and then the image is binarized to change the pixel value of the area above the upper boundary to 0 and the area below to 1; The second stage mask generation method is as follows: the upper and lower boundaries extracted from the annotation data are binarized, where pixels within the upper and lower boundaries are assigned a value of 1, and pixels outside the boundaries are assigned a value of 0; Step 3) Construct a cascade segmentation network based on the encoder-decoder structure; The cascade segmentation network consists of two encoding-decoding networks, which are connected by a semantic prediction guidance module. The first encoding network consists of two residual convolution modules and three multi-scale context aggregation modules, and the second encoding network consists of five convolution modules. The decoding networks are composed of four upsampling convolution modules. Step 4) The image data processed in step 1) and the mask generated in step 2) are enhanced and input into the cascade segmentation network, and trained using the ADAM algorithm to obtain a choroidal segmentation network model that can be used for optical coherence tomography images.

2. The method for constructing an OCT image choroidal segmentation model in combination with a priori mask according to claim 1, characterized in that: It also includes extracting the model trained in step 4), testing it with the data processed in step 1), and evaluating the segmentation results using accuracy, precision, intersection-over-union ratio, Dice coefficient, and Hausdorff distance.

3. The method for constructing an OCT image choroidal segmentation model in combination with a priori mask according to claim 1 or 2, characterized in that: The skip connections at each scale between the second encoding network and the second decoding network are inserted into an adaptive cross-stage fusion module to fuse the features of the previous stage.

4. The method for constructing an OCT image choroidal segmentation model in combination with a priori mask according to claim 1 or 2, characterized in that: The multi-scale context aggregation module groups the feature maps and then uses residual connections between the groups to generate features of different scales; Features of different scales are expanded into one-dimensional features and positionally encoded; the encoded features are concatenated and sent to a deformable self-attention layer to model contextual relationships of different scales.

5. The method for constructing an OCT image choroidal segmentation model in combination with a priori mask according to claim 3, characterized in that: The adaptive cross-stage fusion module fuses feature maps from different stages by using a channel attention mechanism to select appropriate attention weights for features from different stages and using a certain proportion of Dropout to avoid overfitting.

6. An application of the OCT image choroid segmentation model constructed in accordance with claim 1 and combined with a priori mask in choroid segmentation.

Images