A method for optic cup and disc segmentation in glaucoma based on contour-injected content

Abstract

The present invention proposes a method for glaucoma optic cup and optic disc segmentation with contour injection content, the method comprising: inputting a fundus retinal color image into a coding module for encoding; using a contour decoder to decode the contours of the optic cup and optic disc in stages for multi-level features; using a content decoder to decode the content segmentation of the optic cup and optic disc for multi-level features; using a contour injection module to accept contour decoding features corresponding to different stages of the optic cup and optic disc to perform contour injection on the content decoding features of the first stage of the optic cup and optic disc, and iteratively guide the content decoding of the next stage content decoder to obtain a optic cup and optic disc content segmentation mask map. The present invention is based on an encoder and decoder with a large-size convolution kernel, which is different from the design paradigm of a traditional convolutional network based on a 3×3 window, effectively expanding the receptive field, and can obtain more global features with a computational amount less than that of global self-attention.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning, medical signal processing and image analysis, and in particular to a method for segmenting a glaucoma optic cup and optic disc with contour injection content. Background Art

[0002] Glaucoma is one of the leading causes of irreversible vision loss worldwide. Its main pathological features are optic nerve damage and increased cup-disc ratio. Segmenting the cup and disc from fundus color images can support subsequent cup-disc ratio calculations, thereby assisting in the diagnosis and disease assessment of glaucoma.

[0003] Traditional optic cup and optic disc segmentation methods usually rely on edge detection, level set, etc. Among them, edge detection relies on gradient calculation, so it is sensitive to noise and difficult to cope with the influence of complex fundus structures such as blood vessels; level set uses implicit representation, so it has a certain robustness to noise and complex edges. However, the effects of both are affected by initialization, and both rely on manual hyperparameter selection. Therefore, when the illumination conditions of fundus imaging are not good or there are blood vessels blocking, the boundary division of the optic cup, optic disc, and background is still inaccurate.

[0004] The semantic segmentation network in deep learning can learn contextual information to better distinguish the optic cup and optic disc. However, it still cannot handle the boundary blur problem well because the semantic network focuses more on the acquisition of object-level semantics and ignores the detailed contours; even if the details are extracted, they are easily weakened by strong information during feature fusion and gradient backpropagation. Summary of the invention

[0005] In view of the above situation, the main purpose of the present invention is to propose a glaucoma optic cup and optic disc segmentation method with contour injection content to solve the above technical problems.

[0006] The present invention proposes a method for segmenting the optic cup and disc of glaucoma with contour injection content, the method comprising the following steps:

[0007] Step 1: construct an encoding module based on large-size convolution, the encoding module and the contour decoder constitute a contour branch, construct a contour injection module based on the contour injection content mechanism, the contour injection module and the content decoder constitute a content branch, and the contour branch and the content branch constitute a segmentation model;

[0008] Step 2: Based on the contour branch, the fundus retinal color image is input into the encoding module for multi-layer encoding to obtain multi-level features;

[0009] Step 3, using the contour decoder to decode the contours of the optic cup and optic disc stage by stage for the multi-level features, and obtaining contour decoding features of the optic cup and optic disc at different stages and outputting contour images of the optic cup and optic disc;

[0010] Step 4: Based on the content branch, the content decoder is used to perform content segmentation decoding of the optic cup and the optic disc on the multi-level features to obtain the content decoding features of the optic cup and the optic disc at the first stage;

[0011] Step 5: using the contour injection module to accept the contour decoding features of the different stages of the optic cup and the optic disc, perform contour injection on the content decoding features of the first stage of the optic cup and the optic disc, and iteratively guide the content decoding of the next stage content decoder to obtain the optic cup and optic disc content segmentation mask map;

[0012] A contour branch loss is constructed based on the output of the optic cup and optic disc contour map, and a content branch loss is constructed based on the optic cup and optic disc content segmentation mask map. The model is optimized using the contour branch loss and the content branch loss to obtain an optimized segmentation model, and the final segmentation result is obtained using the optimized segmentation model.

[0013] Compared with the prior art, the present invention has the following beneficial effects:

[0014] 1. The present invention uses an encoder and decoder based on a large-size convolution kernel, which is different from the design paradigm of the traditional convolutional network based on a 3×3 window. It effectively expands the receptive field and can obtain more global features with a computation amount less than that of global self-attention;

[0015] 2. The present invention achieves separate learning of contour and content output by designing a dual-branch decoder for contour and content, thus avoiding the feature confusion problem of previous methods and improving the effectiveness and interpretability of learning the two features;

[0016] 3. The present invention designs a contour injection module to extract more robust components from contour features and then inject the content, thereby ensuring that the contour features are not weakened.

[0017] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description or learned through embodiments of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a flow chart of the steps of the glaucoma optic cup and optic disc segmentation method with contour injection content proposed by the present invention.

[0019] Figure 2 The basic structure of the contour decoder of the glaucoma optic cup and optic disc segmentation method with contour injection content proposed by the present invention Figure 1 .

[0020] Figure 3 The basic structure of the contour decoder of the glaucoma optic cup and optic disc segmentation method with contour injection content proposed by the present invention Figure 2 .

[0021] Figure 4 This is a structural diagram of the contour injection module of the glaucoma optic cup and optic disc segmentation method with contour injection content proposed by the present invention.

[0022] Figure 5 This is the overall architecture diagram of the glaucoma optic cup and optic disc segmentation method with contour injection proposed by the present invention.

[0023] Figure 6 The following is a qualitative comparison of the optic cup and optic disc segmentation results using different methods. DETAILED DESCRIPTION

[0024] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout are the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0025] These and other aspects of the embodiments of the present invention will be apparent with reference to the following description and accompanying drawings. In these descriptions and accompanying drawings, some specific implementations of the embodiments of the present invention are specifically disclosed to provide some ways to implement the principles of the embodiments of the present invention, but it should be understood that the scope of the embodiments of the present invention is not limited thereto.

[0026] See also Figure 1 The embodiment of the present invention provides a method for segmenting the optic cup and disc of glaucoma with contour injection content, the method comprising the following steps:

[0027] Step 1: Construct an encoding module based on large-size convolution. The encoding module and the contour decoder constitute a contour branch. Construct a contour injection module based on the contour injection content mechanism. The contour injection module and the content decoder constitute a content branch. The contour branch and the content branch constitute a segmentation model.

[0028] Step 2: Based on the contour branch, the fundus retinal color image is input into the encoding module for multi-layer encoding to obtain multi-level features.

[0029] In step 2, the fundus retinal color image comes from a public dataset. The image is a 3-channel 8-bit color image, and the annotations contain three values ​​0, 1, and 2, where 0 corresponds to the background, 1 corresponds to the optic disc, and 2 corresponds to the optic cup. After preprocessing, the image and annotations are regularized into a 224×224 square image.

[0030] Furthermore, the encoding module based on large-size convolution receives the color image and obtains multi-level features such as fundus structure distribution, texture description, and basic components. A 16-layer convolutional network is used as the backbone network, and the outputs of the first three stages of the network (i.e., the 3rd, 6th, and 15th layers) are recorded as x1, x2, and x3, respectively, where x3 is downsampled by a convolution with a window of 2×2 and a step size of 2 and point-by-point convolution as x4. The present invention refers to the above x1~x4 as multi-level features and sends them to the contour and content decoding module. The sizes of x1~x4 are 96×56×56, 92×28×28, 384×14×14, and 768×7×7, respectively, where the first dimension is the number of channels and the last two dimensions are the height and width of the image.

[0031] Step 3: Use the contour decoder to decode the contours of the optic cup and optic disc stage by stage for the multi-level features, and obtain the contour decoding features of the optic cup and optic disc at different stages and the output of the optic cup and optic disc contour maps.

[0032] See also Figure 2 and Figure 3 In step 3, the contour decoder is used to decode the contours of the optic cup and the optic disc in stages for the multi-level features, and the contour decoding features of the optic cup and the optic disc at different stages and the contour map output of the optic cup and the optic disc are obtained respectively, which specifically includes the following steps:

[0033] A101, perform deconvolution operations and channel concatenation operations on multi-level features in sequence to obtain the first-layer contour upsampling features;

[0034] A102, performing point-by-point convolution operation and nonlinear activation processing on the first-layer contour upsampling features in sequence to obtain activation processing features, performing separable convolution operation, nonlinear activation processing and point-by-point convolution operation on the first-layer contour upsampling features in sequence to obtain the first convolution operation features, performing addition operation on the activation processing features and the first convolution operation features to obtain the first-stage contour decoding features of the optic cup and optic disc;

[0035] A103, performing deconvolution operation and channel concatenation operation on the first-stage contour decoding features of the optic cup and optic disc in sequence to obtain the second-layer contour upsampling features;

[0036] A104, performing a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a second convolution operation feature, sequentially performing a separable convolution operation, a nonlinear activation process, and a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a third convolution operation feature, and performing an addition operation on the second convolution operation feature and the third convolution operation feature to obtain a second-stage contour decoding feature of the optic cup and optic disc;

[0037] A105, repeating steps A103 and A104 in an iterative manner using the second stage contour decoding features as input, to obtain contour decoding features of the optic cup and optic disc at the next stage and contour decoding features of the optic cup and optic disc at the final stage, respectively;

[0038] A106. Perform point-by-point convolution operation, bilinear interpolation operation and normalization processing on the contour decoding features of the final stage of the optic cup and optic disc in sequence to obtain the output of the optic cup and optic disc contour map.

[0039] In the step of performing deconvolution operations and channel concatenation operations on multi-level features in sequence to obtain the first-layer contour upsampling features, the corresponding process has the following relationship:

[0040] ;

[0041] in, represents the first layer contour upsampling feature, represents the splicing operation along the channel, represents the first deconvolution operation in the contour decoder, Represents the multi-level features of the last stage of the encoder;

[0042] In the steps of performing point-by-point convolution operation and nonlinear activation processing on the first-layer contour upsampling features in sequence to obtain activation processing features, performing separable convolution operation, nonlinear activation processing and point-by-point convolution operation on the first-layer contour upsampling features in sequence to obtain the first convolution operation features, and adding the activation processing features to the first convolution operation features to obtain the first-stage contour decoding features of the optic cup and optic disc, the relationship between the corresponding processes is as follows:

[0043] ;

[0044] in, Represents the first stage contour decoding features of the optic cup and optic disc, express Non-linear activation operations, and They represent two point-wise convolution operations with different parameters in the first stage of the contour decoder. Indicates a separable convolution operation with a window size of 1×1;

[0045] It should be noted that and Since there are no learnable parameters, they are directly reused.

[0046] In the step of performing deconvolution operations and channel concatenation operations on the first-stage contour decoding features of the optic cup and optic disc in sequence to obtain the second-layer contour upsampling features, the corresponding process has the following relationship:

[0047] ;

[0048] in, represents the second layer contour upsampling feature, represents the second deconvolution operation in the contour decoder, represents the output features of the third stage of the encoder;

[0049] In the step of performing a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a second convolution operation feature, sequentially performing a separable convolution operation, a nonlinear activation process, and a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a third convolution operation feature, and adding the second convolution operation feature to the third convolution operation feature to obtain the second-stage contour decoding feature of the optic cup and optic disc, the relationship between the corresponding processes is as follows:

[0050] ;

[0051] in, Represents the second stage contour decoding features of the optic cup and optic disc, and They represent two point-wise convolution operations with different parameters in the second stage of the contour decoder, Represents a separable convolution with a window size of 7×7.

[0052] The second stage contour decoding features are used as input to repeat steps A103 and A104 in an iterative manner to obtain the contour decoding features of the next stage of the optic cup and optic disc and the contour decoding features of the final stage of the optic cup and optic disc, respectively. The relationship between the corresponding processes is as follows:

[0053] ;

[0054] ;

[0055] in, Indicates The contour upsampling features of the stage, Indicates the contour decoder The deconvolution operation of the stage, represents the total number of encoder stages, Indicates The coding characteristics of the stage, Optic cup and optic disc The contour decoding features of the stage, represents the stage index of the contour decoder, Optic cup and optic disc The contour decoding features of the stage, and Respectively represent the contour decoder The two point-by-point convolution operations with different parameters in the stage, Indicates the contour decoder Separable convolution at the stage.

[0056] In the step of performing point-by-point convolution operation, bilinear interpolation operation and normalization processing on the contour decoding features of the final stage of the optic cup and optic disc to obtain the output of the optic cup and optic disc contour map, the corresponding process has the following relationship:

[0057] ;

[0058] in, Represents the output of the optic cup and optic disc contour map, It means that after normalization, represents a bilinear interpolation operation, represents the point-wise convolution operation with different shared parameters in the contour decoder, Decoding features of the contours representing the final stages of the optic cup and disc.

[0059] Step 4: Based on the content branch, use the content decoder to perform content segmentation decoding of the visor cup and the visor disc on the multi-level features to obtain the content decoding features of the visor cup and the visor disc at the first stage.

[0060] In step 4, the content decoder is used to perform content segmentation decoding of the cup and the disc on the multi-level features to obtain the content decoding features of the cup and the disc at the first stage, which specifically includes the following steps:

[0061] Deconvolution operations and channel concatenation operations are performed on multi-level features in sequence to obtain the first-stage content upsampling features;

[0062] The first-stage content upsampling features are sequentially subjected to point-by-point convolution operations and nonlinear activation operations to obtain target nonlinear activation results. The first-stage content upsampling features are sequentially subjected to separable convolution operations, nonlinear activation operations, and point-by-point convolution operations to obtain target convolution results. The target nonlinear activation results are added to the target convolution results to obtain the first-stage content decoding features of the optic cup and optic disc.

[0063] The multi-level features are sequentially deconvolved and concatenated along the channels to obtain the first-stage content upsampling features. The corresponding relationship in the process is as follows:

[0064] ;

[0065] in, represents the first stage content upsampling feature, represents the deconvolution operation in the content decoder;

[0066] In the steps of sequentially performing point-by-point convolution operations and nonlinear activation operations on the first-stage content upsampling features to obtain target nonlinear activation results, sequentially performing separable convolution operations, nonlinear activation operations, and point-by-point convolution operations on the first-stage content upsampling features to obtain target convolution results, and adding the target nonlinear activation results to the target convolution results to obtain the first-stage content decoding features of the optic cup and optic disc, the corresponding processes have the following relationship:

[0067] ;

[0068] in, Represents the content decoding features of the first stage of the optic cup and optic disc, represents the separable convolution operation in the content decoder, and They represent two point-wise convolution operations with different parameters in the first stage of the content decoder.

[0069] Step 5: using the contour injection module to accept the contour decoding features of the different stages of the optic cup and the optic disc, perform contour injection on the content decoding features of the first stage of the optic cup and the optic disc, and iteratively guide the content decoding of the next stage content decoder to obtain the optic cup and optic disc content segmentation mask map;

[0070] A contour branch loss is constructed based on the output of the optic cup and optic disc contour map, and a content branch loss is constructed based on the optic cup and optic disc content segmentation mask map. The model is optimized using the contour branch loss and the content branch loss to obtain an optimized segmentation model, and the final segmentation result is obtained using the optimized segmentation model.

[0071] See also Figure 4 and Figure 5 In step 5, the contour injection module is used to accept the contour decoding features of the different stages of the corresponding optic cup and optic disc to perform contour injection on the content decoding features of the first stage of the optic cup and optic disc, and the content decoding of the content decoder of the next stage is guided in an iterative manner to obtain the optic cup and optic disc content segmentation mask map, so as to obtain the content segmentation mask map, which specifically includes the following steps:

[0072] S101, performing 1×1 convolution processing, 3×3 convolution processing, and 5×5 convolution processing on the first-stage contour decoding features of the optic cup and optic disc, respectively, and performing splicing operations in sequence to obtain multi-scale features of the first stage;

[0073] S102, performing global average pooling operation, full connection operation and normalization processing on the multi-scale features of the first stage in sequence, and combining the multi-scale feature channel pixel values ​​to obtain the contour channel features of the first stage;

[0074] S103, performing convolution operation and normalization processing on the contour channel features of the first stage in sequence, and combining the channel direction of the contour channel features to obtain the contour space features of the first stage;

[0075] S104, injecting the contour space features of the first stage into the content decoding features of the optic cup and the optic disc of the first stage, to obtain the content features of the optic cup and the optic disc after the contours are injected into the first stage;

[0076] S105, performing content guidance on the content decoding features of the first stage of the optic cup and the optic disc and the content features after the first stage of the optic cup and the optic disc is injected with the contour, to obtain the content features of the content guidance of the first stage of the optic cup and the optic disc;

[0077] S106, performing a deconvolution operation on the content features guided by the first-stage content of the optic cup and the optic disc, and performing a channel-wise splicing operation in combination with the encoding features to obtain the content upsampling features of the current stage;

[0078] S107, sequentially performing a separable convolution operation, a nonlinear activation operation, and a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of the content upsampling feature of the current stage, performing a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of another content upsampling feature of the current stage, and adding the convolution operation result of the content upsampling feature of the current stage to the convolution operation result of another content upsampling feature of the current stage to obtain content decoding features of the optic cup and optic disc at the current stage;

[0079] S108, taking the second stage contour decoding features of the optic cup and optic disc as input and repeating the steps of S101, S102, S103, S104, S106 and S107 in an iterative manner, to obtain the content decoding features of the next stage of the optic cup and optic disc and the content decoding features of the final stage of the optic cup and optic disc respectively;

[0080] S109, performing point-by-point convolution operation, bilinear interpolation operation and normalization mechanism processing on the content decoding features of the final stage of the optic cup and optic disc in sequence to obtain a optic cup and optic disc content segmentation mask map.

[0081] The first-stage contour decoding features of the optic cup and optic disc are processed with 1×1 convolution, 3×3 convolution, and 5×5 convolution respectively, and then concatenated in sequence to obtain the multi-scale features of the first stage. The corresponding relationship is as follows:

[0082] ;

[0083] in, Indicates Multi-scale features of the stage, It means after 3×3 convolution processing, It means after 5×5 convolution processing, Indicates the stage index of the content decoder;

[0084] In the step of performing global average pooling operation, full connection operation and normalization processing on the multi-scale features of the first stage in sequence, and combining the multi-scale feature channel pixel values ​​to obtain the contour channel features of the first stage, the corresponding process has the following relationship:

[0085] ;

[0086] in, Indicates The contour channel characteristics of the stage, Indicates passing Normalization processing, represents a full connection operation, represents the global average pooling operation, Represents the coordinate index in space;

[0087] In the step of performing convolution operation and normalization processing on the contour channel features of the first stage in sequence and combining the channel direction of the contour channel features to obtain the contour space features of the first stage, the relationship between the corresponding processes is as follows:

[0088] ;

[0089] in, Indicates The contour space characteristics of the stage, represents a convolution operation with a 7×7 window and a single output channel. represents the index along the channel direction;

[0090] In the step of injecting the contour space features of the first stage into the content decoding features of the optic cup and the optic disc of the first stage to obtain the content features of the optic cup and the optic disc after the contours are injected in the first stage, the relationship between the corresponding process is as follows:

[0091] ;

[0092] in, Represents the content features of the optic cup and optic disc after the first stage of injection contour, Indicates Stage content decoding features, represents the convolution operation of a 1×1 window for channel alignment;

[0093] It should be noted that Used for channel alignment, making channels equal through convolution operations.

[0094] In the step of performing a deconvolution operation on the content features guided by the first stage content of the optic cup and the optic disc, and performing a channel-wise splicing operation in combination with the encoding features to obtain the content upsampling features of the current stage, the corresponding process has the following relationship:

[0095] ;

[0096] in, Indicates The content upsampling features of the stage, Indicates the content decoder Deconvolution operation of the stage;

[0097] In the steps of sequentially performing a separable convolution operation, a nonlinear activation operation, and a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of the content upsampling feature of the current stage, performing a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of another content upsampling feature of the current stage, and adding the convolution operation result of the content upsampling feature of the current stage to the convolution operation result of another content upsampling feature of the current stage to obtain the content decoding features of the optic cup and the optic disc at the current stage, the relationship between the corresponding processes is as follows:

[0098] ;

[0099] in, Indicates The stage content decoder shares separable convolution operations with different parameters, and Respectively represent the first Two point-wise convolution operations with different parameters in the stage;

[0100] In the step of performing point-by-point convolution operation, bilinear interpolation operation and normalization mechanism processing on the content decoding features of the final stage of the optic cup and optic disc in order to obtain the optic cup and optic disc content segmentation mask map, the corresponding process has the following relationship:

[0101] ;

[0102] in, represents the segmentation mask of the optic cup and optic disc content, represents the point-wise convolution operation with different shared parameters in the content decoder, Represents the content decoding features at the final stage.

[0103] The contour branch loss is constructed based on the output of the optic cup and optic disc contour maps. The relationship between the corresponding process is as follows:

[0104] ;

[0105] in, represents the contour branch loss, Output of outlines of the optic cup and disc No. elements, Represents the total number of pixels output, Representation Tags and contour plot output The distance calculation function between Indicates belonging to The pixel index within the local range, represents the local range of a 5×5 window, Indicates output A Boolean mask matrix of Indicates that for the tag A Boolean mask matrix of Representation Tags Regarding the level set expression of the edge, represents calculus, Express about 's points.

[0106] It should be noted that the contour branch loss For the learning objective, the parameters of the network are updated using the gradient descent optimization method.

[0107] The content branch loss is constructed based on the optic cup and optic disc content segmentation mask map. The relationship between the corresponding process is as follows:

[0108] ;

[0109] in, represents the content branch loss, represents the 3-channel segmentation label after one-hot encoding, represents the smoothing term, Represents the spatial and channel coordinate index.

[0110] It should be noted that In the present invention, 1 is taken. The method of the present invention simultaneously outputs the optic cup and optic disc contour segmentation map and the optic cup and optic disc content segmentation mask map, and performs loss calculations respectively, and uses the calculated contour branch loss and content branch loss to jointly optimize the segmentation model.

[0111] For further information, see Figure 6,The test results are recorded in Tables 1 and 2, which respectively show the ,comparison results of the evaluation indicators Dice overlap coefficient (Dice ,similarity coefficient) and intersection and union ratio, both ,reflect the degree of overlap between the segmentation results and the annotations, and the larger ,value indicates higher accuracy.

[0112] The Dice coefficient has a high tolerance for small objects and can better reflect the overall accuracy of the segmented content. The intersection-union coefficient has a high penalty for negative samples and can better reflect the accuracy of boundary segmentation.

[0113] Table 1 Dice index of optic cup and optic disc segmentation by different methods

[0114]

[0115] Table 2 Intersection-over-union ratio of optic cup and optic disc segmentation by different methods

[0116]

[0117] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0118] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0119] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for segmenting optic cup and disc of glaucoma with contour injection content, characterized in that: The method comprises the following steps: Step 1: construct an encoding module based on large-size convolution, the encoding module and the contour decoder constitute a contour branch, construct a contour injection module based on the contour injection content mechanism, the contour injection module and the content decoder constitute a content branch, and the contour branch and the content branch constitute a segmentation model; Step 2: Based on the contour branch, the fundus retinal color image is input into the encoding module for multi-layer encoding to obtain multi-level features; Step 3, using the contour decoder to decode the contours of the optic cup and optic disc stage by stage for the multi-level features, and obtaining contour decoding features of the optic cup and optic disc at different stages and outputting contour images of the optic cup and optic disc; Step 4: Based on the content branch, the content decoder is used to perform content segmentation decoding of the optic cup and the optic disc on the multi-level features to obtain the content decoding features of the optic cup and the optic disc at the first stage; Step 5: using the contour injection module to accept the contour decoding features of the different stages of the optic cup and the optic disc, perform contour injection on the content decoding features of the first stage of the optic cup and the optic disc, and iteratively guide the content decoding of the next stage content decoder to obtain the optic cup and optic disc content segmentation mask map; A contour branch loss is constructed based on the output of the optic cup and optic disc contour map, and a content branch loss is constructed based on the optic cup and optic disc content segmentation mask map. The model is optimized using the contour branch loss and the content branch loss to obtain an optimized segmentation model, and the final segmentation result is obtained using the optimized segmentation model.

2. The method for segmenting the optic cup and disc of glaucoma based on the contour injection content according to claim 1, characterized in that: In step 3, the contour decoder is used to decode the contours of the optic cup and the optic disc in stages for the multi-level features, and contour decoding features of the optic cup and the optic disc at different stages and output contour maps of the optic cup and the optic disc are obtained respectively, which specifically includes the following steps: A101, perform deconvolution operations and channel concatenation operations on multi-level features in sequence to obtain the first-layer contour upsampling features; A102, performing point-by-point convolution operation and nonlinear activation processing on the first-layer contour upsampling features in sequence to obtain activation processing features, performing separable convolution operation, nonlinear activation processing and point-by-point convolution operation on the first-layer contour upsampling features in sequence to obtain the first convolution operation features, performing addition operation on the activation processing features and the first convolution operation features to obtain the first-stage contour decoding features of the optic cup and optic disc; A103, performing deconvolution operation and channel concatenation operation on the first-stage contour decoding features of the optic cup and optic disc in sequence to obtain the second-layer contour upsampling features; A104, performing a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a second convolution operation feature, sequentially performing a separable convolution operation, a nonlinear activation process, and a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a third convolution operation feature, and performing an addition operation on the second convolution operation feature and the third convolution operation feature to obtain a second-stage contour decoding feature of the optic cup and optic disc; A105, repeating steps A103 and A104 in an iterative manner using the second stage contour decoding features as input, to obtain contour decoding features of the optic cup and optic disc at the next stage and contour decoding features of the optic cup and optic disc at the final stage, respectively; A106. Perform point-by-point convolution operation, bilinear interpolation operation and normalization processing on the contour decoding features of the final stage of the optic cup and optic disc in sequence to obtain the output of the optic cup and optic disc contour map.

3. The method for segmenting the optic cup and disc of glaucoma based on the contour injection content according to claim 2, characterized in that: Deconvolution operations and channel concatenation operations are performed on the multi-level features in sequence to obtain the first-layer contour upsampling features. The corresponding relationship in the process is as follows: ； in, represents the first layer contour upsampling feature, represents the splicing operation along the channel, represents the first deconvolution operation in the contour decoder, Represents the multi-level features of the last stage of the encoder; In the steps of performing point-by-point convolution operation and nonlinear activation processing on the first-layer contour upsampling features in sequence to obtain activation processing features, performing separable convolution operation, nonlinear activation processing and point-by-point convolution operation on the first-layer contour upsampling features in sequence to obtain the first convolution operation features, and adding the activation processing features to the first convolution operation features to obtain the first-stage contour decoding features of the optic cup and optic disc, the relationship between the corresponding processes is as follows: ； in, Represents the first stage contour decoding features of the optic cup and optic disc, express Non-linear activation operations, and They represent two point-wise convolution operations with different parameters in the first stage of the contour decoder. Indicates a separable convolution operation with a window size of 1×1; In the step of performing deconvolution operations and channel concatenation operations on the first-stage contour decoding features of the optic cup and optic disc in sequence to obtain the second-layer contour upsampling features, the corresponding process has the following relationship: ； in, represents the second layer contour upsampling feature, represents the second deconvolution operation in the contour decoder, represents the output features of the third stage of the encoder; In the step of performing a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a second convolution operation feature, sequentially performing a separable convolution operation, a nonlinear activation process, and a point-by-point convolution operation on the second-layer contour upsampling feature to obtain a third convolution operation feature, and adding the second convolution operation feature to the third convolution operation feature to obtain the second-stage contour decoding feature of the optic cup and optic disc, the relationship between the corresponding processes is as follows: ； in, Represents the second stage contour decoding features of the optic cup and optic disc, and They represent two point-wise convolution operations with different parameters in the second stage of the contour decoder. Represents a separable convolution with a window size of 7×7.

4. The method for segmenting the optic cup and disc of glaucoma based on the contour injection content according to claim 3, characterized in that: The second stage contour decoding features are used as input to repeat steps A103 and A104 in an iterative manner to obtain the contour decoding features of the next stage of the optic cup and optic disc and the contour decoding features of the final stage of the optic cup and optic disc, respectively. The relationship between the corresponding processes is as follows: ； ； in, Indicates The contour upsampling features of the stage, Indicates the contour decoder The deconvolution operation of the stage, represents the total number of encoder stages, Indicates The coding characteristics of the stage, Optic cup and optic disc The contour decoding features of the stage, represents the stage index of the contour decoder, Optic cup and optic disc The contour decoding features of the stage, and They represent the contour decoder The two point-by-point convolution operations with different parameters in the stage, Indicates the contour decoder Separable convolution at the stage; In the step of performing point-by-point convolution operation, bilinear interpolation operation and normalization processing on the contour decoding features of the final stage of the optic cup and optic disc to obtain the output of the optic cup and optic disc contour map, the corresponding process has the following relationship: ； in, Represents the output of the optic cup and optic disc contour map, It means that after normalization, represents a bilinear interpolation operation, represents the point-wise convolution operation with different shared parameters in the contour decoder, Decoding features of the contours representing the final stages of the optic cup and disc.

5. The method for segmenting the optic cup and disc of glaucoma with contour injection content according to claim 4, characterized in that: In step 4, the content decoder is used to perform content segmentation decoding of the cup and the disc on the multi-level features to obtain the content decoding features of the cup and the disc at the first stage, which specifically includes the following steps: Deconvolution operations and channel concatenation operations are performed on multi-level features in sequence to obtain the first-stage content upsampling features; The first-stage content upsampling features are sequentially subjected to point-by-point convolution operations and nonlinear activation operations to obtain target nonlinear activation results. The first-stage content upsampling features are sequentially subjected to separable convolution operations, nonlinear activation operations, and point-by-point convolution operations to obtain target convolution results. The target nonlinear activation results are added to the target convolution results to obtain the first-stage content decoding features of the optic cup and optic disc.

6. The method for segmenting the optic cup and disc of glaucoma with contour injection content according to claim 5, characterized in that: The multi-level features are sequentially deconvolved and concatenated along the channels to obtain the first-stage content upsampling features. The corresponding relationship in the process is as follows: ； in, represents the first stage content upsampling feature, represents the deconvolution operation in the content decoder; In the steps of sequentially performing point-by-point convolution operations and nonlinear activation operations on the first-stage content upsampling features to obtain target nonlinear activation results, sequentially performing separable convolution operations, nonlinear activation operations, and point-by-point convolution operations on the first-stage content upsampling features to obtain target convolution results, and adding the target nonlinear activation results to the target convolution results to obtain the first-stage content decoding features of the optic cup and optic disc, the corresponding processes have the following relationship: ； in, Represents the content decoding features of the first stage of the optic cup and optic disc, represents the separable convolution operation in the content decoder, and They represent two point-wise convolution operations with different parameters in the first stage of the content decoder.

7. The method for segmenting the optic cup and disc of glaucoma with contour injection content according to claim 6, characterized in that: In step 5, contour injection module is used to accept contour decoding features of different stages of the optic cup and optic disc to perform contour injection on the content decoding features of the first stage of the optic cup and optic disc, and content decoding of the content decoder of the next stage is guided in an iterative manner to obtain a optic cup and optic disc content segmentation mask map, which specifically includes the following steps: S101, performing 1×1 convolution processing, 3×3 convolution processing, and 5×5 convolution processing on the first-stage contour decoding features of the optic cup and optic disc, respectively, and performing splicing operations in sequence to obtain multi-scale features of the first stage; S102, performing global average pooling operation, full connection operation and normalization processing on the multi-scale features of the first stage in sequence, and combining the multi-scale feature channel pixel values ​​to obtain the contour channel features of the first stage; S103, performing convolution operation and normalization processing on the contour channel features of the first stage in sequence, and combining the channel direction of the contour channel features to obtain the contour space features of the first stage; S104, injecting the contour space features of the first stage into the content decoding features of the optic cup and the optic disc of the first stage, to obtain the content features of the optic cup and the optic disc after the contours are injected into the first stage; S105, performing content guidance on the content decoding features of the first stage of the optic cup and the optic disc and the content features after the first stage of the optic cup and the optic disc is injected with the contour, to obtain the content features of the content guidance of the first stage of the optic cup and the optic disc; S106, performing a deconvolution operation on the content features guided by the first-stage content of the optic cup and the optic disc, and performing a channel-wise splicing operation in combination with the encoding features to obtain the content upsampling features of the current stage; S107, sequentially performing a separable convolution operation, a nonlinear activation operation, and a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of the content upsampling feature of the current stage, performing a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of another content upsampling feature of the current stage, and adding the convolution operation result of the content upsampling feature of the current stage to the convolution operation result of another content upsampling feature of the current stage to obtain content decoding features of the optic cup and optic disc at the current stage; S108, taking the second stage contour decoding features of the optic cup and optic disc as input and repeating the steps of S101, S102, S103, S104, S106 and S107 in an iterative manner, to obtain the content decoding features of the next stage of the optic cup and optic disc and the content decoding features of the final stage of the optic cup and optic disc respectively; S109, performing point-by-point convolution operation, bilinear interpolation operation and normalization mechanism processing on the content decoding features of the final stage of the optic cup and optic disc in sequence to obtain a optic cup and optic disc content segmentation mask map.

8. The method for segmenting the optic cup and disc of glaucoma with contour injection content according to claim 7, characterized in that: The first-stage contour decoding features of the optic cup and optic disc are processed with 1×1 convolution, 3×3 convolution, and 5×5 convolution respectively, and then concatenated in sequence to obtain the multi-scale features of the first stage. The corresponding relationship is as follows: ； in, Indicates Multi-scale features of the stage, It means after 3×3 convolution processing, It means after 5×5 convolution processing, Indicates the stage index of the content decoder; In the step of performing global average pooling operation, full connection operation and normalization processing on the multi-scale features of the first stage in sequence, and combining the multi-scale feature channel pixel values ​​to obtain the contour channel features of the first stage, the corresponding process has the following relationship: ； in, Indicates The contour channel characteristics of the stage, Indicates passing Normalization processing, represents a full connection operation, represents the global average pooling operation, Represents the coordinate index in space; In the step of performing convolution operation and normalization processing on the contour channel features of the first stage in sequence and combining the channel direction of the contour channel features to obtain the contour space features of the first stage, the relationship between the corresponding processes is as follows: ； in, Indicates The contour space characteristics of the stage, represents a convolution operation with a 7×7 window and a single output channel. represents the index along the channel direction; In the step of injecting the contour space features of the first stage into the content decoding features of the optic cup and the optic disc of the first stage to obtain the content features of the optic cup and the optic disc after the contours are injected in the first stage, the relationship between the corresponding processes is as follows: ； in, Represents the content features of the optic cup and optic disc after the first stage of injection contour, Indicates Stage content decoding features, represents the convolution operation of a 1×1 window for channel alignment; In the step of performing a deconvolution operation on the content features guided by the first stage content of the optic cup and the optic disc, and performing a channel-wise splicing operation in combination with the encoding features to obtain the content upsampling features of the current stage, the corresponding process has the following relationship: ； in, Indicates The content upsampling features of the stage, Indicates the content decoder The deconvolution operation of the stage; In the steps of sequentially performing a separable convolution operation, a nonlinear activation operation, and a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of the content upsampling feature of the current stage, performing a point-by-point convolution operation on the content upsampling feature of the current stage to obtain a convolution operation result of another content upsampling feature of the current stage, and adding the convolution operation result of the content upsampling feature of the current stage to the convolution operation result of another content upsampling feature of the current stage to obtain the content decoding features of the optic cup and the optic disc at the current stage, the relationship between the corresponding processes is as follows: ； in, Indicates The stage content decoder shares separable convolution operations with different parameters, and Respectively represent the first Two point-wise convolution operations with different parameters in the stage; In the step of performing point-by-point convolution operation, bilinear interpolation operation and normalization mechanism processing on the content decoding features of the final stage of the optic cup and optic disc in order to obtain the optic cup and optic disc content segmentation mask map, the corresponding process has the following relationship: ； in, represents the segmentation mask of the optic cup and optic disc content, represents the point-wise convolution operation with different shared parameters in the content decoder, Content decoding features representing the final stages of the optic cup and disc.

9. The method for segmenting the optic cup and disc of glaucoma with contour injection content according to claim 8, characterized in that: The contour branch loss is constructed based on the output of the optic cup and optic disc contour maps. The relationship between the corresponding process is as follows: ； in, represents the contour branch loss, Output of outlines of the optic cup and disc No. elements, Represents the total number of pixels output, Representation Tags and output of cup and disc contours The distance calculation function between Indicates belonging to The pixel index within the local range, represents the local range of a 5×5 window, Indicates output A Boolean mask matrix of Indicates that for the tag A Boolean mask matrix of Representation Tags Regarding the level set expression of the edge, represents calculus, Express about The integral of In the step of constructing the content branch loss based on the optic cup and optic disc content segmentation mask map, the corresponding process has the following relationship: ； in, represents the content branch loss, represents the 3-channel segmentation label after one-hot encoding, represents the smoothing term, Represents the spatial and channel coordinate index.

Images