Visual cup and optic disk synchronous segmentation method and system based on improved SegFormer
By adding EVC module to the SegFormer model, the edge feature acquisition capability of the view cup disc is enhanced, and the uncertainty and edge blurring problems in the view cup disc segmentation are solved, achieving higher segmentation accuracy and efficiency.
Patent Information
- Application Number
- CN202411073389.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-22
Smart Images

Figure CN119006485A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of image segmentation, and in particular relates to an improved SegFormer optic cup and optic disc synchronous segmentation method and system. Background Art
[0002] Glaucoma is an eye disease that damages the optic nerve and can lead to vision loss. Early prevention and treatment of glaucoma can effectively reduce the risk of blindness in patients. Color fundus photography is a non-invasive retinal imaging method. The popularity of color fundus images has made optic disc (OD) and optic cup (OC) examination an important early screening method for glaucoma. The cup to disc ratio (CDR) in fundus images can be calculated to assist in the diagnosis of glaucoma. Figure 1 and Figure 2 The fundus images of normal people and glaucoma patients were compared. In clinical practice, medical workers segment the boundaries of OD and OC based on experience and estimate CDR to determine the severity of glaucoma in patients. However, manual discrimination methods are affected by differences in experience among medical workers and have significant uncertainty and subjectivity. In addition, the generation of massive and heterogeneous fundus images also poses challenges to manual discrimination methods for glaucoma. Therefore, it is of great significance to use computer-assisted interpretation to study the automatic segmentation method of OD and OC in fundus images, which can improve the interpretation efficiency of fundus images and facilitate large-scale early screening of glaucoma.
[0003] Many scholars have applied the theory and methods of deep learning to the segmentation of optic disc and optic cup. For example, Sreng et al. proposed GARDNet, which consists of three different convolutional neural networks, and trained them for images of original size, images cropped to focus on the optic disc area, and cropped images after polar coordinate transformation. Bian et al. proposed a CDR-based optic disc and optic cup segmentation model, and applied cascade neural network technology to the segmentation process of optic disc and optic cup. Bengani et al. proposed an automatic optic disc image segmentation method for retinal fundus images by applying transfer learning and semi-supervised learning techniques. Liu et al. used a densely connected deep separable convolutional network as the backbone network, added a multi-scale image pyramid at the input to broaden the network, and used image morphology to post-process the optic disc and optic cup segmentation results. Hu et al. proposed a Transformer-based glaucoma prediction network for irregularly sampled continuous fundus images, and used convolutional neural networks to embed the input into a specific dimension to solve the problem that the Transformer architecture requires large-scale training data. The above methods improve the segmentation accuracy of OC / OD in fundus images. However, in fundus images, the fuzziness of the boundary between OC and OD, the irregularity of the morphology, and the overlap are still important factors that restrict the segmentation accuracy and efficiency of OC / OD in fundus images.
[0004] In view of the above analysis, the technical problems that urgently need to be solved in the existing technology are: the manual identification method is affected by the differences in experience of medical workers and has significant uncertainty and subjectivity; the generation of massive and heterogeneous fundus images also poses a challenge to the manual identification method of glaucoma; the ambiguity of the boundary between OC and OD, the irregularity of the morphology, and the overlap restrict the accuracy and efficiency of OC / OD segmentation. Summary of the invention
[0005] In view of the problems existing in the prior art, the present invention provides an improved SegFormer optic cup and optic disc synchronous segmentation model (EVC-SegFormer), aiming to improve the segmentation accuracy and efficiency of the optic cup and optic disc in fundus images.
[0006] The EVC-SegFormer optic cup and optic disc synchronous segmentation model of the present invention is implemented in this way: MiT-B1 is selected as the backbone network of the segmentation model, which takes into account the segmentation efficiency while extracting deeper optic cup and optic disc information features; the SegFormer network is improved by using the EVC module to strengthen the extraction of global and local feature information and improve the segmentation accuracy of the optic cup and optic disc.
[0007] S1, uses the encoder to extract the features of the optic cup and optic disc in the fundus image, and outputs high-resolution shallow features and low-resolution fine features;
[0008] S2, the EVC module learns the edge features of the optic cup and disc through low-resolution fine features;
[0009] S3, uses the decoder to fuse and refine feature maps to complete the segmentation task.
[0010] Further, use SegFormer to segment the optic cup and optic disc simultaneously:
[0011] SegFormer consists of Encoder and Decoder. Encoder is a multi-layer Transformer encoder composed of four layers of Transformerblock, which can produce high-resolution shallow features and low-resolution fine features. Compared with other Transformerblocks, SegFormer eliminates the position encoding module and uses 3×3 convolution to represent position information. Decoder is responsible for receiving multi-scale encoding features. Its structure is a lightweight decoder consisting only of upsampling and multi-layer perceptrons.
[0012] Furthermore, the EVC module is added to improve the model's ability to obtain the features of the optic cup and the edge of the optic cup and optic disc:
[0013] An EVC module is added between the encoder and decoder of SegFormer. EVC consists of two parallel parts: one is a lightweight multi-layer perceptron (MLP) to capture global information, and the other is a learnable visual center (LVC) to aggregate local area features within the layer. The resulting feature maps of the lightweight MLP and LVC are concatenated along the channel dimension as the output of EVC. The calculation process is shown in formula (1).
[0014] X=cat(MLP(X in );LVC(X in ))#(1)
[0015] X in It is the input of the EVC module.
[0016] Furthermore, lightweight MLP helps the model understand the complex relationship between the optic cup, optic disc and background, thereby enhancing the understanding of the optic cup and optic disc structure without significantly increasing the computational burden, improving the model's understanding of the relationship between the optic cup and optic disc background in the image that is far apart, and improving the model's ability to obtain global information; lightweight MLP is mainly composed of two residual modules: a module based on DW convolution and a module based on channel MLP. Both modules will be followed by channel scaling and DropPath operations to improve generalization and robustness;
[0017] For the DW convolution-based module, firstly transform X in Perform group normalization, then input to the DW convolution layer, and finally perform X in residual connection; Compared with traditional spatial convolution, DW convolution can improve feature representation ability and reduce computational cost. The calculation process is shown in formula (2);
[0018] For the channel MLP-based module, we first convert Perform group normalization, then implement channel MLP, and finally perform Compared with spatial MLP, channel MLP can not only effectively reduce the computational complexity, but also meet the requirements of general visual tasks. The calculation process is shown in formula (3):
[0019]
[0020] in, is based on the output of the DW convolution module, GN(·) is group normalization, DWConv(·) is a DW convolution with a kernel size of 1×1, and CMLP(·) is a channel MLP.
[0021] Furthermore, LVC works in parallel with lightweight MLP, enabling the model to better understand and distinguish the background and edge detail features of the optic cup and optic disc, and improving the model's ability to acquire local detail features;
[0022] LVC is an encoder with an inherent dictionary, which has two components: 1. Intrinsic codebook B = {b1, b2, ..., b k}; 2. A set of scaling factors S for LVC = {s1, s2, ..., s k}. LVC first passes through the convolution layer to X in The encoded features are processed by the CBR module, which consists of 3×3 convolution, BN layer and ReLU activation function. After the above steps, the encoded features are Input codebook, use scaling factor S to use and b k Mapping the corresponding position information, the information of the entire image about the kth codeword can be calculated by the following formula:
[0023]
[0024] is the i-th pixel; b k is the kth learnable visual codeword; s k is the kth scaling factor; is the information of each pixel position corresponding to the codeword; K is the total number of visual centers.
[0025] Then use φ(·) to fuse all e k , calculate the complete information of the entire image relative to k codewords, the calculation process is as follows:
[0026]
[0027] e is the output of the codebook, and φ(·) includes ReLU and BN layers.
[0028] e is input into the fully connected layer and the 1×1 convolutional layer to predict the features of the highlighted key categories. in The channel multiplication between and the scaling factor coefficient δ(·). The above process is expressed as:
[0029]
[0030] Conv(·) represents 1×1 convolution; δ(·) is the Sigmoid function; is channel multiplication.
[0031] X in And Z are channel added, the process is:
[0032]
[0033] where ⊕ is channel addition.
[0034] Another object of the present invention is to provide an improved SegFormer optic cup and optic disc synchronous segmentation system, comprising:
[0035] A feature output module, used for extracting features of the optic cup and optic disc in the fundus image using an encoder, and outputting high-resolution shallow features and low-resolution fine features;
[0036] Edge feature learning module, the EVC module learns the edge features of the optic cup and optic disc through low-resolution fine features;
[0037] The image segmentation module uses the decoder to fuse and refine feature maps to complete the segmentation task.
[0038] Another object of the present invention is to provide a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the improved SegFormer method for synchronously segmenting the optic cup and optic disc.
[0039] Another object of the present invention is to provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of the improved SegFormer method for synchronously segmenting the optic cup and optic disc.
[0040] Another object of the present invention is to provide an information data processing terminal, which includes the improved SegFormer optic cup and optic disc synchronous segmentation system.
[0041] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:
[0042] First, the beneficial effects of the present invention:
[0043] (1) In terms of segmentation accuracy: By adding the EVC module, SegFormer can perform more accurate synchronous segmentation of the optic cup and optic disc.
[0044] Tables 1 to 3 respectively give the comparison of the evaluation indicators of optic cup and optic disc segmentation of the SegFormer model and the improved EVC-SegFormer model on the three datasets DRISHTI-GS1, ORIGA and REFUGE2.
[0045] As can be seen from Tables 1 to 3, the EVC-SegFormer model proposed in this paper shows the best segmentation results, improving the values of Cup IoU, Disc IoU, MIoU, Cup Dice, Disc Dice, MPA, and ACC. The results on the DRISHTI-GS1 dataset are 81.61% (2.01↑), 89.83% (1.29↑), 90.21% (1.11↑), 89.12% (1.48↑), 94.29% (0.8↑), 94.83% (0.73↑), and 98.52% (0.16↑), respectively. On the ORIGA dataset, the results are 81.61% (1.06↑), 79.30% (0.25↑), 86.59% (0.44↑), 89.41% (0.59↑), 88.13% (0.28↑), 92.93% (1.11↑), and 97.81% (0.04↑). On the REFUGE2 dataset, the results are 85.21% (1.05↑), 80.66% (0.88↑), 88.37% (0.67↑), 91.44% (0.53↑), 88.19% (0.74↑), 93.48% (0.36↑), and 98.81% (0.09↑).
[0046] Table 1. Comparison of evaluation indicators of optic cup and optic disc segmentation between EVC-SegFormer and SegFormer models in the DRISHTI-GS1 dataset
[0047]
[0048] Table 2. Comparison of evaluation indicators of optic cup and optic disc segmentation between EVC-SegFormer and SegFormer models in ORIGA dataset
[0049]
[0050] Table 3. Comparison of evaluation indicators of optic cup and optic disc segmentation between EVC-SegFormer and SegFormer models in REFUGE2 dataset
[0051]
[0052] (2) In terms of generalization, the Dice coefficient, mean pixel accuracy (MPA), intersection over union (IoU), mean intersection over union (MIoU), and accuracy were used as evaluation indicators, and the ORIGA and Drishti-GS datasets were used as experimental data to analyze and compare the proposed model with other models. The results show that the proposed model achieves the best results in all the above indicators.
[0053] The invention proposes an improved SegFormer optic cup and optic disc synchronous segmentation model, and adds an EVC (Explicit Visual Center) module between the encoder and decoder of SegFormer. The model synchronously segments the optic cup and optic disc in the image as a whole. The EVC module distinguishes the detailed features of the optic cup and optic disc area, and improves the model's ability to obtain global features and local features of the optic cup and optic disc edge.
[0054] The lightweight multilayer perceptron (MLP) in EVC can capture the global long-range dependency in the image, which can help the model understand the relationship between the optic cup, optic disc, and the background that are far apart in the image. At the same time, the traditional optic cup and optic disc segmentation model also ignores the importance of the optic cup and optic disc edge area. The optic cup and optic disc synchronous segmentation model of SegFormer is improved, and the EVC module is added, which can learn the visual center to focus on and aggregate the characteristics of the optic cup and optic disc edge area, and improve the model's ability to obtain the characteristics of the optic cup and the optic cup and optic disc edge.
[0055] Second, the technical solution of the present invention solves a technical problem that people have been eager to solve but have never succeeded in solving: the EVC-SegFormer segmentation model can achieve high-precision synchronous segmentation of the optic cup and optic disc while maintaining a relatively good inference speed, which is significantly better than the existing semantic segmentation model and is of great value in assisting clinicians in diagnosing glaucoma.
[0056] Third, the present invention relates to an improved SegFormer method for synchronous segmentation of the optic cup and optic disc, which solves several technical problems and achieves significant technical progress compared to the prior art.
[0057] 1. Insufficient segmentation accuracy:
[0058] The existing optic cup and optic disc segmentation methods have the problem of unclear segmentation boundaries and loss of details when processing fundus images. This is because the traditional methods cannot effectively capture the subtle features of the optic cup and optic disc area, especially at the edge.
[0059] 2. Insufficient feature fusion:
[0060] In the prior art, the fusion of feature maps often lacks full utilization of features of different scales, resulting in poor fusion effect between high-resolution and low-resolution features, thereby affecting the accuracy of the segmentation results.
[0061] 3. Insufficient edge information:
[0062] Traditional segmentation algorithms do not fully consider the edge features of the optic cup and optic disc, which may lead to blur or discontinuity at the segmentation edge of the optic cup and optic disc, thus affecting the accuracy of diagnosis.
[0063] Significant technological advancements:
[0064] 1. Edge feature enhancement:
[0065] The present invention adopts the EVC (Edge-Vision-Context) module, and significantly improves the ability to capture edge information of the optic cup and optic disc by learning edge features of low-resolution fine features. This improvement makes the segmentation result clearer and more accurate at the edge, solving the problem of edge blur in the prior art.
[0066] 2. Efficient feature fusion:
[0067] The improved SegFormer decoder combines high-resolution shallow features with low-resolution fine features to optimize the feature fusion process. The upsampling and multi-layer perceptron in the decoder structure effectively integrate multi-scale information and improve the accuracy and detail expression of the optic cup and optic disc segmentation.
[0068] 3. Improve segmentation accuracy and detail performance:
[0069] The present invention uses advanced edge detection technology and context perception modules in the process of feature extraction and feature fusion, making the synchronous segmentation results of the optic cup and optic disc more accurate. This technological advancement improves the accuracy of medical image analysis and has better clinical application value.
[0070] In summary, the present invention solves the problems of insufficient segmentation accuracy and insufficient edge information in the prior art by improving feature extraction and edge feature learning technology, and has achieved significant technical progress in the task of synchronous segmentation of optic cup and optic disc. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 is a normal fundus image provided by an embodiment of the present invention;
[0072] Figure 2 is a glaucoma fundus image provided by an embodiment of the present invention;
[0073] Figure 3 is a flow chart of a method for synchronously segmenting the optic cup and optic disc using an improved SegFormer provided by an embodiment of the present invention;
[0074] Figure 4 is a structural diagram of SegFormer provided by an embodiment of the present invention;
[0075] Figure 5 is a TransformerBlock structure diagram provided by an embodiment of the present invention;
[0076] Figure 6 is a structural diagram of an EVC provided by an embodiment of the present invention;
[0077] Figure 7 It is a structural diagram of the MLP Layer provided by an embodiment of the present invention;
[0078] Figure 8 It is a structural diagram of the optic cup and optic disc synchronous segmentation system of the improved SegFormer provided by an embodiment of the present invention;
[0079] Fig. 9 This is a comparative experiment of the Drishti-GS data set provided by an embodiment of the present invention;
[0080] Fig.10 This is an ORIGA data set comparison experiment provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0081] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0082] Example 1: Improved SegFormer segmentation method based on optic cup and optic disc edge features
[0083] 1. Step S1: Feature extraction
[0084] The encoder of SegFormer is used to extract features from the fundus image. The four-layer Transformer block of the encoder generates high-resolution shallow feature maps and low-resolution fine feature maps, which capture different levels of information in the optic cup and optic disc area.
[0085] 2. Step S2: Edge feature learning
[0086] The low-resolution fine features are further processed by the EVC (Edge-Vision-Context) module. The EVC module focuses on learning the edge features of the optic cup and optic disc, extracting the edge information of the optic cup and optic disc through the edge detection algorithm to improve the boundary accuracy of the segmentation.
[0087] 3. Step S3: Feature fusion and segmentation
[0088] The decoder of SegFormer is used to fuse high-resolution shallow features with low-resolution fine features processed by the EVC module. The decoder is lightweight and consists of upsampling and multi-layer perceptrons to complete the simultaneous segmentation of the optic cup and optic disc. The decoder refines and optimizes the feature map to generate the final optic cup and optic disc segmentation result.
[0089] Example 2: Improved SegFormer segmentation method based on enhanced edge features
[0090] Step S1: Feature extraction
[0091] The improved SegFormer encoder is used to process the fundus images. The encoder's multi-layer Transformer block outputs two types of feature maps: high-resolution shallow feature maps are used to capture the rough shape of the optic cup and optic disc, and low-resolution fine feature maps are used to extract more detailed structural information.
[0092] Step S2: Edge Enhancement Feature Learning
[0093] In the EVC module, low-resolution fine features are processed by an enhanced edge detection algorithm to enhance the edge information of the optic cup and optic disc. This module uses improved edge detection technology and combines contextual information to enhance the saliency of the optic cup and optic disc edges and improve segmentation accuracy.
[0094] Step S3: Feature fusion and refinement
[0095] The decoder is responsible for fusing high-resolution shallow features with enhanced edge features. The improved decoder structure includes additional skip connections and context-aware modules, which help to better integrate multi-scale information from the encoder. Finally, the decoder generates an accurate optic cup and optic disc segmentation map to complete the segmentation task.
[0096] These embodiments improve the feature extraction and fusion methods of SegFormer and enhance edge features using the EVC module, thereby improving the accuracy and detail expression of the synchronous segmentation of the optic cup and optic disc.
[0097] like Figure 3 As shown, the present invention can be summarized into three steps:
[0098] Step 1: Use the encoder to extract the features of the optic cup and optic disc in the fundus image and output high-resolution shallow features and low-resolution fine features.
[0099] Step 2: The EVC module learns the edge features of the optic cup and disc through low-resolution fine features.
[0100] Step 3: Use the decoder to fuse and refine the feature maps to complete the segmentation task.
[0101] (1) Synchronous segmentation of optic cup and optic disc using SegFormer
[0102] like Figure 4 As shown in Figure 1, SegFormer consists of Encoder and Decoder. Figure 5 As shown in the figure, the encoder is a multi-layer Transformer encoder, which consists of four layers of Transformer blocks, and can generate high-resolution shallow features and low-resolution fine features. Compared with other Transformer blocks, SegFormer removes the position encoding module and uses 3×3 convolution to represent position information. The decoder is responsible for receiving multi-scale encoded features. The structure is a lightweight decoder, which only consists of upsampling and multi-layer perceptrons.
[0103] (2) Add the EVC module to improve the model's ability to obtain the features of the optic cup and the edge of the optic cup and optic disc
[0104] like Figure 6 As shown in the figure, in the present invention, an EVC module is added between the encoder and decoder of SegFormer. EVC consists of two parallel parts: one is a lightweight MLP for capturing global information, and the other is a learnable visual center LVC for aggregating local region features within the layer. The resulting feature maps of the lightweight MLP and LVC are concatenated along the channel dimension as the output of EVC. The calculation process is shown in formula (1).
[0105] X=cat(MLP(X in );LVC(X in ))#(1)
[0106] X in It is the input of the EVC module.
[0107] like Figure 7 As shown in the figure, lightweight MLP helps the model understand the complex relationship between the optic cup and the background, thereby enhancing the understanding of the structural morphology of the optic cup and the optic disc without significantly increasing the computational burden, improving the model's understanding of the relationship between the optic cup and the optic disc background that are far apart in the image, and improving the model's ability to obtain global information. Lightweight MLP is mainly composed of two residual modules: a module based on DW convolution and a module based on channel MLP. Both modules are followed by channel scaling and DropPath operations to improve generalization and robustness.
[0108] For the DW convolution-based module, firstly transform X in Perform group normalization, then input to the DW convolution layer, and finally perform X in Compared with traditional spatial convolution, DW convolution can improve feature representation capability while reducing computational cost. The calculation process is shown in formula (2).
[0109] For the channel MLP-based module, we first convert Perform group normalization, then implement channel MLP, and finally perform Compared with spatial MLP, channel MLP can not only effectively reduce the computational complexity, but also meet the requirements of general visual tasks. The calculation process is shown in formula (3).
[0110]
[0111] in, is based on the output of the DW convolution module, GN(·) is group normalization, DWConv(·) is a DW convolution with a kernel size of 1×1, and CMLP(·) is a channel MLP.
[0112] LVC works in parallel with lightweight MLP, enabling the model to better understand and distinguish the background and the edge detail features of the optic cup and optic disc, and improve the model's ability to acquire local detail features.
[0113] LVC is an encoder with an inherent dictionary, which has two components: 1. Intrinsic codebook B = {b1, b2, ..., b k}; 2. A set of scaling factors S for LVC = {s1, s2, ..., s k}. LVC first passes through the convolution layer to Xin The encoded features are processed by the CBR module, which consists of 3×3 convolution, BN layer and ReLU activation function. Input codebook, use scaling factor S to use and b k Map the corresponding position information. The information of the entire image about the kth codeword can be calculated by the following formula:
[0114]
[0115] is the i-th pixel; b k is the kth learnable visual codeword; s k is the kth scaling factor; is the information of each pixel position corresponding to the codeword; K is the total number of visual centers.
[0116] Then use φ(·) to fuse all e k , calculate the complete information of the entire image relative to k codewords, the calculation process is as follows:
[0117]
[0118] e is the output of the codebook, and φ(·) includes ReLU and BN layers.
[0119] e is input into the fully connected layer and the 1×1 convolutional layer to predict the features that highlight the key categories. Then use X in The channel multiplication between and the scaling factor coefficient δ(·). The above process is expressed as:
[0120]
[0121] Conv(·) represents 1×1 convolution; δ(·) is the Sigmoid function; is channel multiplication.
[0122] X in And Z are channel added, the process is:
[0123]
[0124] in It is channel addition.
[0125] 2. Experimental results and precision comparison of the present invention
[0126] 1. Accuracy evaluation indicators
[0127] The performance of the present invention has been tested on actual images, using Dice coefficient, mean pixel accuracy (MPA), intersection over union (IoU), mean intersection over union (MIoU), and accuracy as evaluation indicators to accurately evaluate the proposed segmentation model.
[0128]
[0129] where p ii is the number of pixels correctly classified as the optic cup and optic disc area; p ij is the number of pixels misclassified as background area; p ji is the number of pixels in the area that are misclassified as cup and disc. k is the annotation result of different categories, where k=0 represents the background class, k=1 represents the cup class, k=2 represents the disc class, and n is the total number of pixels. The Dice coefficient represents the degree of overlap between the predicted result and the true label of the target object. MPA is the average pixel accuracy of the background, cup, and disc. IoU is an indicator to measure the similarity between the cup and disc labels and the predicted results. MIoU is an indicator to measure the overall similarity between the three category labels and the predicted results of the cup, disc, and background. Accuracy is used to measure the proportion of samples correctly classified by the model on the test dataset.
[0130] 2. Experimental results and comparative experiments of the present invention
[0131] To evaluate the segmentation performance of Glaucoma-SegFormer, two sets of experiments were conducted. Experiment 1 is an ablation experiment that compares Glaucoma-SegFormer with SegFormer under different settings. Experiment 2 is a comparative experiment that compares Glaucoma-SegFormer with other end-to-end models, including U-Net, DeepLabv3+, and PSPNet.
[0132] (1) Ablation experiment
[0133] Table 1 Ablation experiments on the Drishti-GS dataset
[0134]
[0135] Table 1 shows the ablation experiment of SegFormer on the Drishti-GS dataset. Glaucoma-SegFormer has some improvements in Cup IoU, MIoU, Cup Dice and Accuracy, with Cup IoU increased by 1.50%, MIoU increased by 0.43%, Cup Dice increased by 1.07%, and Accuracy increased by 0.03%, reaching 87.22%, 88.09%, 93.09%, and 98.31% respectively. However, in terms of Disc IoU, Disc Dice, and MPA, the performance of Glaucoma-SegFormer has slightly decreased, reaching 77.84%, 87.30%, and 93.60%, respectively, down 0.24%, 0.24%, and 0.46%, respectively.
[0136] Table 2 Ablation experiments on ORIGA dataset
[0137]
[0138] Table 2 shows the ablation experiment of SegFormer on the ORIGA dataset: Glaucoma-SegFormer has improvements in most indicators, including Cup IoU by 0.56%, MIoU by 0.13%, Cup Dice by 0.39%, MPA by 0.95%, and Accuracy by 0.01%, which are 80.58%, 85.91%, 88.78%, 92.91%, and 97.69% respectively. However, Disc IoU and Disc Dice have slightly decreased, which are 78.34% and 87.45%, respectively, down 0.18% and 0.05%, respectively.
[0139] (2) Comparative experiment
[0140] like Fig. 9 and Fig.10 As shown in the figure, U-Net and DeepLabv3+ are prone to mis-segmentation and missed segmentation. PSPNet shows better segmentation performance and reduces mis-segmentation, but the error rate is still high. Glaucoma-SegFormer has better segmentation performance, especially in the boundary area of the optic cup and optic disc. The segmentation results of Glaucoma-SegFormer are more consistent with the labels.
[0141] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of improving the SegFormer method for synchronously segmenting the optic cup and optic disc.
[0142] The application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of improving the method for synchronous segmentation of optic cup and optic disc of SegFormer.
[0143] The application embodiment of the present invention provides an information data processing terminal, which includes an improved SegFormer optic cup and optic disc synchronous segmentation system.
[0144] In order to evaluate the performance of the EVC-SegFormer model in segmenting the optic cup and optic disc, ablation experiments and comparative experiments were designed. In the ablation experiment, the effectiveness of the EVC module in improving the SegFormer network for optic cup and optic disc segmentation was analyzed; in the comparative experiment, the segmentation efficiency and accuracy of the EVC-SegFormer model were compared with other models, including DeeplabV3+, PSPNet, and UNet models using different backbone feature extraction networks.
[0145] Through the analysis of the segmentation results of ablation experiments and comparative experiments on three public datasets, DRISHTI-GS1, ORIGA and REFUGE2, the proposed EVC-SegFormer model has improved the intersection over union (IoU), mean intersection over union (MIoU), Dice coefficient, mean pixel accuracy (MPA), and accuracy (Acc) of optic cup and optic disc segmentation, which are 81.61%, 89.83%, 90.21%, 89.12%, 94.29%, 94.83% and 98.52% respectively. It is significantly better than the existing semantic segmentation model and can assist clinicians in the accurate segmentation of optic cup and optic disc.
[0146] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.
[0147] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.
Claims
1. An improved SegFormer method for synchronous segmentation of optic cup and optic disc, characterized in that: include: S1, uses the encoder to extract the features of the optic cup and optic disc in the fundus image, and outputs high-resolution shallow features and low-resolution fine features; S2, the EVC module learns the edge features of the optic cup and disc through low-resolution fine features; S3, uses the decoder to fuse and refine feature maps to complete the segmentation task.
2. The improved SegFormer method for synchronous segmentation of optic cup and optic disc as claimed in claim 1, characterized in that: Synchronous segmentation of optic cup and optic disc using SegFormer: SegFormer consists of Encoder and Decoder. Encoder is a multi-layer Transformer encoder composed of four layers of Transformerblock, which can produce high-resolution shallow features and low-resolution fine features. Compared with other Transformerblocks, SegFormer eliminates the position encoding module and uses 3×3 convolution to represent position information. Decoder is responsible for receiving multi-scale encoding features. Its structure is a lightweight decoder consisting only of upsampling and multi-layer perceptrons.
3. The improved SegFormer method for synchronous segmentation of optic cup and optic disc as claimed in claim 1, characterized in that: Add the EVC module to improve the model's ability to obtain the features of the optic cup and the edge of the optic cup and optic disc: The EVC module is added between the encoder and decoder of SegFormer. EVC consists of two parallel parts: one is a lightweight MLP for capturing global information, and the other is a learning visual center to aggregate local area features within the layer. The resulting feature maps of the lightweight MLP and LVC are concatenated along the channel dimension as the output of EVC. The calculation process is shown in formula (1). X=cat(MLP(X in );LVC(X in ))#(1) X in It is the input of the EVC module.
4. The improved SegFormer method for synchronous segmentation of optic cup and optic disc as claimed in claim 1, characterized in that: Lightweight MLP helps the model understand the complex relationship between the optic cup, optic disc and background, thereby enhancing the understanding of the structural morphology of the optic cup and optic disc without significantly increasing the computational burden, improving the model's understanding of the relationship between the optic cup and optic disc background that are far apart in the image, and improving the model's ability to obtain global information; Lightweight MLP is mainly composed of two residual modules: a module based on DW convolution and a module based on channel MLP; both modules are followed by channel scaling and DropPath operations to improve generalization and robustness; For the DW convolution-based module, firstly transform X in Perform group normalization, then input to the DW convolution layer, and finally perform X in residual connection; Compared with traditional spatial convolution, DW convolution can improve feature representation ability and reduce computational cost. The calculation process is shown in formula (2); For the channel MLP-based module, we first convert Perform group normalization, then implement channel MLP, and finally perform Compared with spatial MLP, channel MLP can not only effectively reduce the computational complexity, but also meet the requirements of general visual tasks. The calculation process is shown in formula (3): in, is based on the output of the DW convolution module, GN(·) is group normalization, DWConv(·) is a DW convolution with a kernel size of 1×1, and CMLP(·) is a channel MLP.
5. The improved SegFormer method for synchronous segmentation of optic cup and optic disc as claimed in claim 1, characterized in that: LVC works in parallel with lightweight MLP, enabling the model to better understand and distinguish the background and edge detail features of the optic cup and optic disc, and improve the model's ability to acquire local detail features; LVC is an encoder with an inherent dictionary, which has two components:
1. Intrinsic codebook B = {b1, b2, ..., b k }; 2. A set of scaling factors S for LVC = {s1, s2, ..., s k }; LVC first passes through the convolution layer to X in The encoded features are processed by the CBR module, which consists of 3×3 convolution, BN layer and ReLU activation function. After the above steps, the encoded features are Input codebook, use scaling factor S to use and b k Mapping the corresponding position information, the information of the entire image about the kth codeword can be calculated by the following formula: is the i-th pixel; b k is the kth learnable visual codeword; s k is the kth scaling factor; is the information of each pixel position corresponding to the codeword; K is the total number of visual centers; Then use φ(·) to fuse all e k , calculate the complete information of the entire image relative to k codewords, the calculation process is as follows: e is the output of the codebook, φ(·) contains ReLU and BN layers; e is input into the fully connected layer and the 1×1 convolutional layer to predict the features of the highlighted key categories; then X is used in The channel multiplication between and the scaling factor coefficient δ(·) is expressed as: Conv(·) represents 1×1 convolution; δ(·) is the Sigmoid function; is channel multiplication; X in And Z are channel added, the process is: in It is channel addition.
6. An improved SegFormer optic cup and optic disc synchronous segmentation system for implementing the improved SegFormer optic cup and optic disc synchronous segmentation method according to any one of claims 1 to 5, characterized in that: include: A feature output module, used for extracting features of the optic cup and optic disc in the fundus image using an encoder, and outputting high-resolution shallow features and low-resolution fine features; Edge feature learning module, the EVC module learns the edge features of the optic cup and optic disc through low-resolution fine features; The image segmentation module uses the decoder to fuse and refine feature maps to complete the segmentation task.
7. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the improved SegFormer method for synchronously segmenting the optic cup and optic disc as claimed in any one of claims 1 to 5.
8. A computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to execute the steps of the method for synchronously segmenting the optic cup and optic disc using the improved SegFormer as claimed in any one of claims 1 to 5.
9. An information data processing terminal, comprising the improved SegFormer optic cup and optic disc synchronous segmentation system as claimed in claim 6.
