Interactive infrared meibomian gland image segmentation method based on doodle simulation

Abstract

The invention provides an interactive infrared meibomian gland image segmentation method based on doodle simulation. According to the method, a novel multi-task network architecture is designed, various interaction information (click and graffiti) is received as input, gland segmentation is realized, and false positive and false negative areas are predicted to guide a model to perform graffiti simulation. According to the method, a two-stage network training strategy is adopted, and uncertain region simplified masks are combined to improve the accuracy of error region prediction. For each round of user interaction, the method executes prediction in two stages. In the first stage, the network outputs a rough segmentation mask and corresponding false positive and false negative prediction areas based on user clicking; and in the second stage, the model simulates graffiti according to a pseudo graffiti generation strategy based on the two error regions so as to refine the rough segmentation mask output in the first stage. Compared with an existing method, the method has the advantages that the efficiency of graffiti and the convenience of user interaction can be considered at the same time, and the labor cost of the user is further reduced.

Description

Technical Field

[0001] The invention relates to the field of infrared meibomian gland image segmentation, in particular to an interactive infrared meibomian gland image segmentation method based on graffiti simulation. Background Art

[0002] Meibomian glands are the largest sebaceous glands buried in the upper and lower tarsal plates of the human eye, with their openings located at the eyelid margins. Their number and morphology are easily affected by inflammation, blockage, changes in hormone levels, and environmental factors, which can lead to changes in secretory function, decreased tear film stability, and meibomian gland dysfunction. Currently, infrared imaging technology provides visualization images of the meibomian glands that can be saved and recorded, allowing ophthalmologists to make a rough assessment of gland atrophy and loss, and is an important clinical tool. Since doctors' purely manual diagnosis (directly looking at the film to draw conclusions) is prone to a series of problems such as strong subjectivity, poor stability, low efficiency, and inability to quantify analysis, automatic segmentation methods based on deep learning have become an important research direction and have achieved many excellent results in recent years.

[0003] Because infrared meibomian gland images contain various artifacts (such as low contrast, non-uniform illumination, out-of-focus or specular reflections of the glandular area) and high annotation costs, it is difficult for automatic segmentation methods to achieve sufficiently accurate and robust results to meet clinical needs. In actual practice, ophthalmologists often use tools such as ITKSNAP for auxiliary corrections. The interactive segmentation method is an effective and desirable alternative that is guided by user input and dynamically guides the model to segment the parts of interest and correct any misidentified results. Compared with traditional annotation tools, robust and high-quality results can be flexibly obtained with less user input and in less time.

[0004] The interactive segmentation method uses various possible simple annotations provided by the user, such as clicks, bounding boxes, or scribbles, as guidance to segment the object of interest from the target image. It has an inherent feature: interactivity, which allows users to iteratively add annotations and continuously collaborate with the model to complete the task of refining the segmented object. This human-computer collaborative approach can not only effectively give play to the doctor's subjective judgment ability, but also make full use of the computer's high-speed computing power, making the segmentation results more accurate and scientific, and easier to put into clinical application.

[0005] Click-based interactive segmentation methods are the most studied topics because clicks are simple and fast as input and have complete training and evaluation protocols. In recent years, various improved methods have made significant progress in natural image segmentation. Some of these methods focus on improving the encoding method of user annotations to enrich prior information, some methods fully integrate additional input information outside the image into the model to improve utilization, some methods propose local refinement solutions for the shortcomings of global prediction, and some methods use more effective neural network architectures. In addition, the PseudoClick method enables the model to directly predict the segmentation error area and generate pseudo-clicks to reduce the number of user clicks. The present invention deeply explores how to further reduce labor costs and focuses on refining the meibomian gland segmentation.

[0006] In the field of interactive infrared meibomian gland image segmentation, there are still many shortcomings in directly using the basic ideas of the PseudoClick method and the click-based interaction methods developed for natural images. First, for complex objects such as dense and slender meibomian glands, a rough doodle is often better than a carefully placed click, but using doodles as a user's guidance method will increase a lot of manpower costs, which is contrary to the original intention of the present invention. Secondly, the meibomian glands do not have obvious and clear object boundaries like natural images. Directly using these methods will observe that the model performs poorly in processing the edges of the glands and cannot accurately identify the object boundaries. At the same time, the collected infrared meibomian gland images contain various artifacts, making it extremely challenging for the model to accurately predict the erroneous areas of the segmentation mask. In addition, the existing meibomian gland dataset is small, and it is necessary to explore efficient backbones that are more suitable for medical image segmentation.

[0007] In summary, the existing methods not only have difficulty in balancing the efficiency of graffiti and the ease of user interaction, but also have poor gland boundary processing. At the same time, there are also challenges such as difficulty in predicting error areas and exploring more efficient backbones. Summary of the invention

[0008] The purpose of the present invention is to propose an interactive infrared meibomian gland image segmentation method based on graffiti simulation, which can provide additional, beneficial and "free" graffiti by allowing the model to accurately predict some easily identifiable error areas without changing the user input form. These graffiti focus on refining the gland margins and segmenting unidentified glands to further reduce labor costs.

[0009] To achieve the above object, the technical solution of the present invention is: an interactive infrared meibomian gland image segmentation method based on graffiti simulation, comprising the following steps:

[0010] Step S1: construct a multi-task neural network architecture, including a shared encoder, a segmentation decoder and two error prediction decoders; the shared encoder is responsible for extracting common, high-level features from the input infrared meibomian gland image, the segmentation decoder is used to segment the meibomian gland image, and the two error prediction decoders are a false positive decoder and a false negative decoder, respectively, for predicting the false positive and false negative areas in the segmentation mask output by the segmentation decoder.

[0011] Step S2: Use a two-stage training strategy to train the multi-task neural network architecture of step S1 and obtain a trained model; in the first stage, freeze the error prediction decoder, train the segmentation backbone, and optimize the task of gland segmentation, wherein the segmentation backbone includes a shared encoder and a segmentation decoder; in the second stage, freeze the shared encoder and the segmentation decoder, fine-tune the false positive decoder and the false negative decoder to optimize the task of false positive and false negative region prediction in the segmentation mask;

[0012] Step S3: Use the trained model for click-based interactive infrared meibomian gland image segmentation task; for each click interaction of the user, the model will perform prediction in two stages;

[0013] The first stage is the user interaction stage; in this stage, the user clicks to correct according to the input image and the segmentation mask previously predicted by the model; the model takes the image, the previous segmentation mask and the user's click as cascade input, performs forward propagation in the multi-task neural network, and finally outputs a rough segmentation mask of the gland and the false positive and false negative predicted areas of the mask;

[0014] The second stage is the model simulation stage; in this stage, the model simulates the real user's graffiti on the input image according to the pseudo-graffiti generation strategy based on the predicted false positive and false negative areas to refine the segmentation mask of the gland as the previously predicted segmentation mask for the next click, thereby achieving the purpose of reducing the user's labor cost.

[0015] Preferably, the segmentation backbone in the multi-task neural network architecture described in step S1 adopts a segmentation network with a two-level nested U-shaped structure.

[0016] Preferably, the two error prediction decoders in step S1 adopt different structures; the false negative decoder adopts the same structure as the segmentation decoder to predict the unidentified glands; the false positive decoder combines the edge generation module and The decoder module constructs a hybrid structure, which can learn the edge features of the gland more effectively, thereby predicting the false positive area of ​​the gland edge as accurately as possible.

[0017] Preferably, the false positive decoder is combined with an edge generation module and The decoder module constructs a hybrid structure, which is to apply the edge generation module to the convolutional features of the RSU output at the same level of the segmentation decoder to generate edge features at that level, and continuously splice and fuse these multi-scale edge features level by level to identify the false positive area of ​​the gland edge; the details are as follows:

[0018] The output features of the shared encoder and the output features of the segmentation decoder RSU-4F module are respectively concatenated after generating edge features through the edge generation module as the input of the false positive decoder RSU-4F block;

[0019] The output features of the segmentation decoder RSU-4 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4F module and used as the input of the false positive decoder RSU-4 module;

[0020] The output features of the segmentation decoder RSU-5 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4 module and used as the input of the false positive decoder RSU-5 module;

[0021] The output features of the segmentation decoder RSU-6 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-5 module and used as the input of the false positive decoder RSU-6 module;

[0022] After the output features of the segmentation decoder RSU-7 module are generated into edge features by the edge generation module, they are concatenated with the output features of the false positive decoder RSU-6 module and used as the input of the false positive decoder RSU-7 module.

[0023] Preferably, the edge generation module first performs sobel convolution operations with different step sizes on the input original features to realize multi-scale edge information, then resamples them to a resolution of the same size as the original features and concatenates them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution, instance batch normalization and leaky ReLU activation.

[0024] Preferably, the edge generation module performs a sobel convolution operation with a step size of 1, 2, and 4 on the input original features, then resamples to a resolution of the same size as the original features and concatenates them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution, instance batch normalization, and leaky ReLU activation. The specific calculation is as follows:

[0025]

[0026] Among them, F edgerepresents the output of the edge generation module, and denotes Sobel convolution operations with step sizes of 1, 2, and 4, respectively; U denotes upsampling by bilinear interpolation; [] denotes concatenation, Represents a composite layer consisting of a 1x1 convolutional layer, an instance batch normalization layer, and a Leaky ReLU activation function.

[0027] Preferably, in the first stage of step S2, the manual annotation result of the doctor is used as the true mask, and the segmentation backbone is trained in a deep supervision manner; the loss function uses the Normalized Focal Loss (NFL):

[0028]

[0029] where NFL(i,j) represents the normalized focal loss of the segmentation decoder output at (i,j), and p i,j It represents the confidence of the segmentation decoder output at (i, j), and γ is used to adjust the weight ratio of easy-to-classify samples.

[0030] Preferably, in the second stage of step S2, a simplified mask combined with uncertainty is used for deep supervision training; the simplified true mask M of the false positive region fp and the true mask M of the false negative region fn for:

[0031]

[0032] M fn =remove((M∩(P<τ))∩(τ a ≤P<τ),m) (4)

[0033] Where m is the true mask of the gland; P is the probability map output by the segmentation decoder; τ is the prediction threshold; τ a and τ b Represents the lower and upper bounds of the model prediction uncertainty region; remove(x,m) means removing the area in mask x that is less than m;

[0034] The loss function uses Intersection over Union Loss (IoU Loss):

[0035]

[0036] Where X represents the simplified true mask, and Y represents the predicted mask output by the error prediction decoder; for the false positive decoder, X is M fp ; For the false negative decoder, X is M fn .

[0037] Preferably, the cascade input described in step S3 is specifically as follows: before forward propagation in the network, the interactive information and the previous mask are first subjected to a convolution operation so that the shape of the output tensor is exactly the same as the output tensor of the first convolutional layer in the network; and then the two tensors are summed element by element to fuse all the information of the cascade input without making any modifications to the existing network architecture.

[0038] Preferably, the pseudo-graffiti generation strategy described in step S3 generates a pseudo-graffiti based on the maximum error area in the outputs of the two error prediction decoders, and the specific strategy is as follows:

[0039] First, the output of the error prediction decoder is converted into a binary mask by a predefined threshold (set to 0.5), and all non-0 pixels in the mask are incorrectly labeled pixels in the predicted segmentation mask;

[0040] Then, the connected components of the OpenCV library are used to divide adjacent pixels into the same region and non-adjacent pixels into different regions;

[0041] Next, the error region with the largest area is selected as the target region;

[0042] Finally, it is determined whether the target area comes from the prediction mask output by the false positive decoder or the prediction mask output by the false negative decoder. If it is the prediction mask output by the false positive decoder, the model performs negative graffiti annotation (representing the background), otherwise it performs positive graffiti annotation (representing the foreground).

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] The present invention integrates user click guidance, model simulation graffiti and edge refinement at the same time. Using click as user input is user-friendly, while using graffiti as model simulation user guidance input can provide rich prior information and does not require additional labor costs.

[0045] The model used in the present invention can directly generate pseudo graffiti based on the predicted segmentation error area after the user clicks the guidance to further refine the segmentation mask, which can effectively reduce the labor cost of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a network architecture of a preferred embodiment of the present invention;

[0047] Figure 2 A schematic diagram of a model training process in a preferred embodiment of the present invention;

[0048] Figure 3 It is a schematic diagram of the principle of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention is described in detail below in conjunction with the accompanying drawings.

[0050] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0051] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0052] This embodiment provides an interactive infrared meibomian gland image segmentation method based on graffiti simulation, comprising the following steps:

[0053] Step S1: Figure 1 As shown in the figure, a multi-task neural network architecture is constructed, including a shared encoder, a segmentation decoder and two error prediction decoders. The shared encoder is responsible for extracting common, high-level features from the input infrared meibomian gland image, while the segmentation decoder is used to segment the meibomian gland image. The two error prediction decoders are the false positive decoder and the false negative decoder, which are used to predict the false positive and false negative regions in the segmentation mask output by the segmentation decoder.

[0054] Step S2: Figure 2 As shown, a specialized two-stage training strategy is used to train the multi-task network architecture of step S1. The first stage freezes the error prediction decoder and focuses on training the segmentation backbone (i.e., the shared encoder and the segmentation decoder) to optimize the task of gland segmentation. The second stage freezes the shared encoder and the segmentation decoder and focuses on fine-tuning the false positive and false negative decoders to optimize the task of false positive and false negative region prediction in the segmentation mask.

[0055] Step S3: Figure 3 As shown in Figure 1, the trained model is used for the interactive click-based infrared meibomian gland image segmentation task. For each click interaction of the user, the model performs prediction in two stages.

[0056] The first stage is the user interaction stage. In this stage, the user can click to correct according to the input image and the segmentation mask previously predicted by the model. The model takes the image, the previous segmentation mask and the user's click as cascade input, performs forward propagation in the multi-task neural network, and finally outputs a rough segmentation mask of the gland and the false positive and false negative predicted areas of the mask. The second stage is the model simulation stage. In this stage, the model will simulate the real user's graffiti on the input image based on the predicted false positive and false negative areas according to the pseudo-graffiti generation strategy to further refine the segmentation mask of the gland, thereby reducing the user's labor cost.

[0057] In this embodiment, the segmentation backbone in the multi-task neural network architecture described in step S1 is The network is a two-level nested U-shaped structure. The top layer is a large U-shaped structure similar to U-Net, consisting of 11 stages. Each stage is filled with a well-configured residual U-block (RSU).

[0058] In this embodiment, the two error prediction decoders in step S1 have different structures. The false negative decoder uses the same structure as the segmentation decoder to predict the unrecognized glands; the false positive decoder combines the edge generation module and A new hybrid structure constructed with a decoder module is proposed to more effectively learn the edge features of the gland, thereby predicting the false positive area of ​​the gland edge as accurately as possible.

[0059] In this embodiment, if Figure 1 The edge generation module shown is applied to the convolutional features of the RSU output at the same level of the segmentation decoder to generate edge features at that level, and these multi-scale edge features are continuously concatenated and fused level by level to focus on identifying false positive areas at the edges of glands; specifically as follows:

[0060] The output features of the shared encoder and the output features of the segmentation decoder RSU-4F module are respectively concatenated after generating edge features through the edge generation module as the input of the false positive decoder RSU-4F block;

[0061] The output features of the segmentation decoder RSU-4 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4F module and used as the input of the false positive decoder RSU-4 module;

[0062] The output features of the segmentation decoder RSU-5 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4 module and used as the input of the false positive decoder RSU-5 module;

[0063] The output features of the segmentation decoder RSU-6 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-5 module and used as the input of the false positive decoder RSU-6 module;

[0064] After the output features of the segmentation decoder RSU-7 module are generated into edge features by the edge generation module, they are concatenated with the output features of the false positive decoder RSU-6 module and used as the input of the false positive decoder RSU-7 module.

[0065] In this embodiment, the edge generation module first performs sobel convolution operations with different step sizes on the input original features to realize multi-scale edge information, then resamples to the same resolution as the original features and concatenates them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution layer, instance batch normalization, and leaky ReLU activation. The process can be formulated as:

[0066]

[0067] Among them, F edge represents the output of the edge generation module, and denotes Sobel convolution operations with step sizes of 1, 2, and 4, respectively. U denotes upsampling by bilinear interpolation. [] denotes concatenation. Represents a composite layer consisting of a 1x1 convolution, an instance batch normalization layer, and a Leaky ReLU activation function.

[0068] In this embodiment, different loss functions are used in different stages of the two-stage training strategy described in step S2 to train a specific decoder.

[0069] The first stage task is to optimize the gland segmentation. Therefore, the doctor's manual annotation results are used as the real mask, and the segmentation backbone is trained in a deep supervision way. The loss function uses the Normalized Focal Loss (NFL):

[0070]

[0071] where NFL(i,j) represents the normalized focal loss of the segmentation decoder output at (i,j), and p i,j represents the confidence of the segmentation decoder output at (i, j), and γ is used to adjust the weight ratio of easy-to-classify samples, which is set to 2 in this embodiment.

[0072] The second stage task is to fine-tune the two error prediction decoders to predict the false positive and false negative regions in the segmentation mask output by the segmentation decoder. Since it is very challenging to directly use the false positive and false negative regions in the segmentation mask as the true mask for training, in order to improve the accuracy of the prediction as much as possible to ensure that the pseudo-graffiti generated in step S3 is annotated to the appropriate position, a simplified mask combined with uncertainty is used for deep supervision training. The simplified false positive region true mask M fp and the true mask M of the false negative region fn for:

[0073]

[0074] M fn =remove((M∩(P<τ))∩(τ a ≤P<τ),m) (4)

[0075] Where M is the true mask of the gland; P is the probability map output by the segmentation decoder; τ is the prediction threshold (set to 0.5 in this embodiment); τ a and τ b Represents the lower and upper bounds of the model prediction uncertainty region, which are set to 0.46 and 0.52 respectively in this embodiment; remove(x,m) means removing the region with an area smaller than m in the mask x, and m is set to 5 in this embodiment. This is a simplified strategy made in full consideration of the fact that the model uncertainty region is more easily identified as an error region and the small target region contains a lot of noise. The loss function uses the intersection over union loss (IoULoss):

[0076]

[0077] Where X represents the simplified true mask, and Y represents the predicted mask output by the error prediction decoder; for the false positive decoder, X is M fp ; For the false negative decoder, X is M fn .

[0078] In this embodiment, the cascade input described in step S3 is specifically as follows: before forward propagation in the network, the mutual information and the previous mask are first convolved to make the shape of the output tensor exactly the same as the output tensor of the first convolutional layer in the network. Then the two tensors are summed element by element to fuse all the information of the cascade input without making any modifications to the existing network architecture.

[0079] In this embodiment, the pseudo-graffiti generation strategy described in step S3 generates a pseudo-graffiti based on the maximum error area in the outputs of the two error prediction decoders. The specific strategy is as follows:

[0080] First, the output of the error prediction decoder is converted into a binary mask by a predefined threshold (set to 0.5), and all non-0 pixels in the mask are incorrectly labeled pixels in the predicted segmentation mask;

[0081] Then, the connected components of the OpenCV library are used to divide adjacent pixels into the same region and non-adjacent pixels into different regions;

[0082] Next, the error region with the largest area is selected as the target region;

[0083] Finally, it is determined whether the target area comes from the prediction mask output by the false positive decoder or the prediction mask output by the false negative decoder. If it is the prediction mask output by the false positive decoder, the model performs negative graffiti annotation (representing the background), otherwise it performs positive graffiti annotation (representing the foreground).

[0084] The pseudo-graffiti is generated by the model based on the false positive and false negative areas predicted by the wrong prediction decoder according to the pseudo-graffiti generation strategy to simulate real users, and is not provided by users. Therefore, these pseudo-graffiti are completely "free", additional and beneficial, not only can they automatically correct the wrong areas in the gland segmentation mask predicted by the segmentation decoder, but also do not require any labor burden from users.

[0085] In summary, the present invention uses As an efficient backbone for meibomian gland image segmentation; through edge generation module and The effective combination of RSUs can identify the false positive areas of the gland edges. Compared with the existing methods, the model can directly predict the segmentation error areas and perform pseudo-graffiti annotation to refine the gland edges and segment the unidentified glands, which can further reduce labor costs. And because these graffiti are completely "free", additional and beneficial, users still only need to provide click guidance, thus taking into account both the efficiency of graffiti and the labor input of users.

[0086] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. An interactive infrared meibomian gland image segmentation method based on graffiti simulation, characterized in that: The following steps are involved: Step S1: construct a multi-task neural network architecture, including a shared encoder, a segmentation decoder and two error prediction decoders; the shared encoder is responsible for extracting common, high-level features from the input infrared meibomian gland image, the segmentation decoder is used to segment the meibomian gland image, and the two error prediction decoders are respectively a false positive decoder and a false negative decoder, which are used to predict the false positive and false negative areas in the segmentation mask output by the segmentation decoder; Step S2: Use a two-stage training strategy to train the multi-task neural network architecture of step S1 and obtain a trained model; in the first stage, freeze the error prediction decoder, train the segmentation backbone, and optimize the gland segmentation task, wherein the segmentation backbone includes a shared encoder and a segmentation decoder; The second stage freezes the shared encoder and segmentation decoder, and fine-tunes the false positive decoder and false negative decoder to optimize the task of false positive and false negative region prediction in the segmentation mask; Step S3: Use the trained model for click-based interactive infrared meibomian gland image segmentation task; for each click interaction of the user, the model will perform prediction in two stages; The first stage is the user interaction stage; in this stage, the user clicks to correct according to the input image and the segmentation mask previously predicted by the model; the model takes the image, the previous segmentation mask and the user's click as cascade input, performs forward propagation in the multi-task neural network, and finally outputs a rough segmentation mask of the gland and the false positive and false negative predicted areas of the mask; The second stage is the model simulation stage; in this stage, the model simulates the real user's graffiti on the input image according to the pseudo-graffiti generation strategy based on the predicted false positive and false negative areas to refine the segmentation mask of the gland as the previously predicted segmentation mask for the next click.

2. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The segmentation backbone in the multi-task neural network architecture described in step S1 adopts a segmentation network with a two-level nested U-shaped structure.

3. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 2, characterized in that: The two error prediction decoders in step S1 adopt different structures; the false negative decoder adopts the same structure as the segmentation decoder to predict the unidentified glands; the false positive decoder combines the edge generation module and The decoder module constructs a hybrid structure to learn the edge features of the gland and predict the false positive area of ​​the gland edge.

4. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 3 is characterized in that: The false positive decoder combines the edge generation module and The decoder module constructs a hybrid structure, which is to apply the edge generation module to the convolutional features of the RSU output at the same level of the segmentation decoder to generate edge features at that level, and continuously splice and fuse these multi-scale edge features level by level to identify the false positive area of ​​the gland edge; the details are as follows: The output features of the shared encoder and the output features of the segmentation decoder RSU-4F module are respectively concatenated after generating edge features through the edge generation module as the input of the false positive decoder RSU-4F block; The output features of the segmentation decoder RSU-4 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4F module and used as the input of the false positive decoder RSU-4 module; The output features of the segmentation decoder RSU-5 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4 module and used as the input of the false positive decoder RSU-5 module; The output features of the segmentation decoder RSU-6 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-5 module and used as the input of the false positive decoder RSU-6 module; After the output features of the segmentation decoder RSU-7 module are generated into edge features by the edge generation module, they are concatenated with the output features of the false positive decoder RSU-6 module and used as the input of the false positive decoder RSU-7 module.

5. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 3, characterized in that: The edge generation module first performs sobel convolution operations with different step sizes on the input original features to realize multi-scale edge information, then resamples them to the same resolution as the original features and splices them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution, instance batch normalization and leaky ReLU activation.

6. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 5, characterized in that: The edge generation module performs a sobel convolution operation with a step size of 1, 2, and 4 on the input original features, then resamples them to a resolution of the same size as the original features and concatenates them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution, instance batch normalization, and leaky ReLU activation. The specific calculation is as follows: Among them, F edge represents the output of the edge generation module, and denotes Sobel convolution operations with step sizes of 1, 2, and 4, respectively; U denotes upsampling by bilinear interpolation; [] denotes concatenation, Represents a composite layer consisting of a 1x1 convolution, an instance batch normalization layer, and a Leaky ReLU activation function.

7. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: In the first stage described in step S2, the doctor's manual annotation results are used as the true mask, and the segmentation backbone is trained in a deep supervision manner; the loss function uses the normalized focal loss: where NFL(i,j) represents the normalized focal loss of the segmentation decoder output at (i,j), and p i,j It represents the confidence of the segmentation decoder output at (i, j), and γ is used to adjust the weight ratio of easy-to-classify samples.

8. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The second stage described in step S2 uses a simplified mask combined with uncertainty for deep supervision training, and the simplified false positive area true mask M fp and the true mask M of the false negative region fn for: M fn =remove((M∩(P<τ))∩(τ a ≤P<τ),m) (4) Where M is the true mask of the gland; P is the probability map output by the segmentation decoder; τ is the prediction threshold, τ a and τ b Represents the lower and upper bounds of the model prediction uncertainty region; remove(x,m) means removing the area in mask x that is less than m; The loss function uses the intersection-over-union loss: Where X represents the simplified true mask, and Y represents the predicted mask output by the error prediction decoder; for the false positive decoder, X is M fp ; For the false negative decoder, X is M fn .

9. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The cascade input described in step S3 is specifically as follows: before forward propagation in the network, the mutual information and the previous mask are first convolved to make the shape of the output tensor exactly the same as the output tensor of the first convolutional layer in the network, and then the two tensors are summed element by element to fuse all the information of the cascade input.

10. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The pseudo-graffiti generation strategy described in step S3 generates a pseudo-graffiti based on the maximum error area in the output of the two error prediction decoders. The specific strategy is as follows: First, the output of the error prediction decoder is converted into a binary mask by a predefined threshold. All non-zero pixels in the mask are incorrectly labeled pixels in the predicted segmentation mask. Then, the connected components of the OpenCV library are used to divide adjacent pixels into the same region and non-adjacent pixels into different regions; Next, the error region with the largest area is selected as the target region; Finally, it is determined whether the target area comes from the prediction mask output by the false positive decoder or the prediction mask output by the false negative decoder. If it is the prediction mask output by the false positive decoder, the model will perform negative graffiti annotation representing the background, otherwise it will perform positive graffiti annotation representing the foreground.

Images