Interactive infrared meibomian gland image segmentation method based on scribble simulation

Abstract

The application provides an interactive infrared meibomian gland image segmentation method based on graffiti simulation; the method designs a new multi-task network architecture, receives multiple interactive information (clicks and graffiti) as input, realizes gland segmentation, and predicts false positive and false negative regions to guide the model to perform graffiti simulation. The method adopts a two-stage network training strategy and combines uncertainty region simplified masks to improve the accuracy of error region prediction. For each round of user interaction, the method performs prediction in two stages. In the first stage, the network outputs a rough segmentation mask and corresponding false positive and false negative prediction regions based on user clicks; in the second stage, the model simulates graffiti based on the two error regions according to the pseudo-graffiti generation strategy to refine the rough segmentation mask output in the first stage. Compared with the existing method, the method can simultaneously consider the efficiency of graffiti and the simplicity of user interaction, and further reduce the labor cost of users.

Description

Technical Field

[0001] The present invention relates to the field of infrared meibomian gland image segmentation, in particular to an interactive infrared meibomian gland image segmentation method based on graffiti simulation. Background Art

[0002] Meibomian glands are the largest sebaceous glands buried in the upper and lower tarsal plates of the human eye, with their openings located at the eyelid margins. Their number and morphology are easily affected by inflammation, blockage, changes in hormone levels, and environmental factors, which can lead to changes in secretory function, decreased tear film stability, and meibomian gland dysfunction. Currently, infrared imaging technology provides visualization images of the meibomian glands that can be saved and recorded, allowing ophthalmologists to make rough assessments of gland atrophy and loss, and is an important clinical tool. Since purely manual diagnosis by doctors (directly looking at the film to draw conclusions) is prone to a series of problems such as strong subjectivity, poor stability, low efficiency, and inability to quantify and analyze, automatic segmentation methods based on deep learning have become an important research direction and have achieved many excellent results in recent years.

[0003] Because infrared meibomian gland images contain various artifacts (such as low contrast, non-uniform illumination, out-of-focus gland areas, or specular reflections) and high annotation costs, automatic segmentation methods are difficult to achieve sufficiently accurate and robust results to meet clinical needs. In actual practice, ophthalmologists often use tools such as ITKSNAP for auxiliary corrections. The interactive segmentation method is an effective and desirable alternative. It is guided by user input and dynamically guides the model to segment the parts of interest and correct any misidentification results. Compared with traditional annotation tools, robust and high-quality results can be obtained flexibly with less user input and in a shorter time.

[0004] Interactive segmentation methods use simple user-provided annotations, such as clicks, bounding boxes, or scribbles, as guidance to segment objects of interest from a target image. They possess an inherent characteristic: interactivity, allowing users to iteratively add annotations, continuously collaborating with the model to refine the segmented object. This human-computer collaboration not only effectively leverages the physician's subjective judgment but also fully utilizes the computer's high-speed computing power, resulting in more accurate and scientific segmentation results that are more readily applicable to clinical practice.

[0005] Click-based interactive segmentation methods are the most studied topics because they use clicks as input in a simple and fast form and have a complete training and evaluation protocol. In recent years, various improved methods have made significant progress in natural image segmentation. Some of these methods focus on improving the encoding method of user annotations to enrich prior information, some methods fully integrate additional input information outside the image into the model to improve utilization, some methods propose local refinement solutions to address the shortcomings of global prediction, and some methods use more effective neural network architectures. In addition, the PseudoClick method enables the model to directly predict the segmentation error area and generate pseudo clicks to reduce the number of user clicks. The present invention deeply explores how to further reduce labor costs and focuses on refining the meibomian gland segmentation.

[0006] In the field of interactive infrared meibomian gland image segmentation, directly using the basic ideas of the PseudoClick method and the click-based interaction methods developed for natural images still have many shortcomings. First, for dense and elongated complex objects such as the meibomian gland, a rough doodle is often more effective than a carefully placed click. However, using doodles as a user guidance method will increase a lot of manpower costs, which is contrary to the original intention of the present invention. Secondly, the meibomian glands do not have obvious and clear object boundaries like natural images. Directly using these methods will observe that the model performs poorly in processing the gland edges and cannot accurately identify the object boundaries. At the same time, the collected infrared meibomian gland images contain various artifacts, making it extremely challenging for the model to accurately predict the erroneous areas of the segmentation mask. In addition, the existing meibomian gland dataset is small, and there is a need to explore efficient backbones that are more suitable for medical image segmentation.

[0007] In summary, existing methods not only struggle to balance the efficiency of graffiti and the ease of user interaction, but also have poor gland boundary processing. Furthermore, there are challenges in predicting error regions and exploring more efficient backbones. Summary of the Invention

[0008] The present invention aims to propose an interactive infrared meibomian gland image segmentation method based on scribble simulation. This method allows the model to accurately predict some easily identifiable error regions without changing the user input format, thereby providing additional, beneficial, and "free" scribbles. These scribbles focus on refining gland margins and segmenting unidentified glands, further reducing labor costs.

[0009] To achieve the above object, the technical solution of the present invention is: an interactive infrared meibomian gland image segmentation method based on graffiti simulation, comprising the following steps:

[0010] Step S1: construct a multi-task neural network architecture, including a shared encoder, a segmentation decoder and two error prediction decoders; the shared encoder is responsible for extracting general, high-level features from the input infrared meibomian gland image, the segmentation decoder is used to segment the meibomian gland image, and the two error prediction decoders are a false positive decoder and a false negative decoder, respectively, for predicting the false positive and false negative areas in the segmentation mask output by the segmentation decoder.

[0011] Step S2: Using a two-stage training strategy to train the multi-task neural network architecture of step S1 and obtain a trained model; in the first stage, freezing the error prediction decoder, training the segmentation backbone, and optimizing the gland segmentation task, wherein the segmentation backbone includes a shared encoder and a segmentation decoder; in the second stage, freezing the shared encoder and the segmentation decoder, fine-tuning the false positive decoder and the false negative decoder to optimize the task of false positive and false negative region prediction in the segmentation mask;

[0012] Step S3: The trained model is used for the click-based interactive infrared meibomian gland image segmentation task; for each click interaction of the user, the model will perform prediction in two stages;

[0013] The first stage is the user interaction stage; in this stage, the user clicks to correct the error based on the input image and the segmentation mask previously predicted by the model. The model uses the image, the previous segmentation mask, and the user's click as cascade input, performs forward propagation in the multi-task neural network, and finally outputs a rough segmentation mask of the gland and the false positive and false negative predicted areas of the mask.

[0014] The second stage is the model simulation stage; in this stage, the model will simulate the real user's graffiti on the input image according to the pseudo-graffiti generation strategy based on the predicted false positive and false negative areas to refine the gland segmentation mask, which will serve as the previously predicted segmentation mask for the next click, thereby reducing the user's labor cost.

[0015] Preferably, the segmentation backbone in the multi-task neural network architecture described in step S1 adopts a segmentation network with a two-level nested U-shaped structure.

[0016] Preferably, the two error prediction decoders in step S1 adopt different structures; the false negative decoder adopts the same structure as the segmentation decoder to predict the unidentified glands; the false positive decoder combines the edge generation module and The decoder module constructs a hybrid structure, which can more effectively learn the edge features of the gland, thereby predicting the false positive area of ​​the gland edge as accurately as possible.

[0017] Preferably, the false positive decoder is combined with an edge generation module and The decoder module constructs a hybrid structure, which applies the edge generation module to the convolutional features output by the RSU at the same level of the segmentation decoder to generate edge features at that level, and continuously splices and fuses these multi-scale edge features level by level to identify false positive areas at the edge of the gland; the details are as follows:

[0018] The output features of the shared encoder and the output features of the segmentation decoder RSU-4F module are respectively concatenated through the edge generation module to generate edge features, which are used as the input of the false positive decoder RSU-4F block;

[0019] The output features of the segmentation decoder RSU-4 module are generated by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-4F module and used as the input of the false positive decoder RSU-4 module;

[0020] The output features of the segmentation decoder RSU-5 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4 module and used as the input of the false positive decoder RSU-5 module;

[0021] The output features of the segmentation decoder RSU-6 module are generated by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-5 module and used as the input of the false positive decoder RSU-6 module;

[0022] The output features of the segmentation decoder RSU-7 module are generated into edge features by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-6 module and used as the input of the false positive decoder RSU-7 module.

[0023] Preferably, the edge generation module first performs a Sobel convolution operation with different step sizes on the input original features to realize multi-scale edge information, then resamples them to the same resolution as the original features and splices them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution, instance batch normalization and Leaky ReLU activation.

[0024] Preferably, the edge generation module performs a Sobel convolution operation with a step size of 1, 2, and 4 on the input original features, then resamples them to the same resolution as the original features and concatenates them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution, instance batch normalization, and Leaky ReLU activation. The specific calculation is as follows:

[0025]

[0026] Among them, F edgerepresents the output of the edge generation module, and Respectively represent Sobel convolution operations with step sizes of 1, 2, and 4; U represents upsampling by bilinear interpolation; [] represents splicing, Represents a composite layer consisting of a 1x1 convolutional layer, an instance batch normalization layer, and a Leaky ReLU activation function.

[0027] Preferably, in the first stage of step S2, the doctor's manual annotation results are used as the true mask, and the segmentation backbone is trained using a deep supervision method; the loss function uses the Normalized Focal Loss (NFL):

[0028]

[0029] Where NFL(i,j) represents the normalized focal loss of the segmentation decoder output at (i,j), p i,j It represents the confidence of the segmentation decoder output at (i, j), and γ is used to adjust the weight ratio of easy-to-classify samples.

[0030] Preferably, in the second stage of step S2, a simplified mask combined with uncertainty is used for deep supervision training; the simplified true mask M of the false positive region fp And the true mask M of the false negative region fn for:

[0031]

[0032] M fn =remove((M∩(P<τ))∩(τ a ≤P<τ),m) (4)

[0033] Where m is the true mask of the gland; P is the probability map output by the segmentation decoder; τ is the prediction threshold; τ a and τ b Represents the lower and upper bounds of the model prediction uncertainty region; remove(x,m) means removing the area smaller than m in the mask x;

[0034] The loss function uses Intersection over Union Loss (IoU Loss):

[0035]

[0036] Among them, X represents the simplified true mask, Y represents the predicted mask output by the error prediction decoder; for the false positive decoder, X is M fp ; For the false negative decoder, X is M fn .

[0037] Preferably, the cascade input described in step S3 is specifically as follows: before forward propagation in the network, the interactive information and the previous mask are first subjected to a convolution operation so that the shape of the output tensor is exactly the same as the output tensor of the first convolutional layer in the network; then the two tensors are summed element by element to fuse all the information of the cascade input without making any modifications to the existing network architecture.

[0038] Preferably, the pseudo-graffiti generation strategy described in step S3 generates a pseudo-graffiti based on the maximum error region in the outputs of the two error prediction decoders. The specific strategy is as follows:

[0039] First, the output of the error prediction decoder is converted into a binary mask by a predefined threshold (set to 0.5), and all non-zero pixels in the mask are incorrectly labeled pixels in the predicted segmentation mask;

[0040] Then, we use the connected components of the OpenCV library to divide adjacent pixels into the same region and non-adjacent pixels into different regions;

[0041] Next, the error region with the largest area is selected as the target region;

[0042] Finally, it is determined whether the target area comes from the prediction mask output by the false positive decoder or the prediction mask output by the false negative decoder. If it is the prediction mask output by the false positive decoder, the model performs negative graffiti annotation (representing the background), otherwise it performs positive graffiti annotation (representing the foreground).

[0043] Compared with the prior art, the present invention has the following beneficial effects:

[0044] This invention integrates user click guidance, model simulation graffiti, and edge refinement. Using clicks as user input is user-friendly, while using graffiti as model simulation user guidance input can provide rich prior information without requiring additional human effort.

[0045] The model used in the present invention can directly generate pseudo-graffiti based on the predicted segmentation error area after the user clicks the guidance to further refine the segmentation mask, which can effectively reduce the user's labor cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of a network architecture according to a preferred embodiment of the present invention;

[0047] Figure 2 Schematic diagram of the model training process of a preferred embodiment of the present invention;

[0048] Figure 3 Schematic diagram of the principle of a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0049] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0050] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0051] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form, and it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0052] This embodiment provides an interactive infrared meibomian gland image segmentation method based on graffiti simulation, comprising the following steps:

[0053] Step S1: Figure 1 As shown in Figure 1, a multi-task neural network architecture is constructed, consisting of a shared encoder, a segmentation decoder, and two error prediction decoders. The shared encoder is responsible for extracting common, high-level features from the input infrared meibomian gland image, while the segmentation decoder is used to segment the meibomian gland image. The two error prediction decoders, namely the false positive decoder and the false negative decoder, are used to predict false positive and false negative regions in the segmentation mask output by the segmentation decoder.

[0054] Step S2: Figure 2 As shown in Figure 2, a specialized two-stage training strategy is used to train the multi-task network architecture of step S1. The first stage freezes the error prediction decoder and focuses on training the segmentation backbone (i.e., the shared encoder and segmentation decoder) to optimize the task of gland segmentation. The second stage freezes the shared encoder and segmentation decoder and focuses on fine-tuning the false positive and false negative decoders to optimize the task of false positive and false negative region prediction in the segmentation mask.

[0055] Step S3: Figure 3 As shown in Figure 2, the trained model is used for the interactive click-based infrared meibomian gland image segmentation task. For each click interaction of the user, the model performs predictions in two stages.

[0056] The first stage is the user interaction stage. During this stage, users can click to correct the segmentation mask based on the input image and the model's previously predicted segmentation mask. The model uses the image, the previous segmentation mask, and the user's clicks as a cascade input, performing forward propagation through a multi-task neural network. The model ultimately outputs a rough segmentation mask for the glands, along with the predicted false positive and false negative regions for that mask. The second stage is the model simulation stage. In this stage, based on the predicted false positive and false negative regions, the model simulates a real user scribbling on the input image using a pseudo-scribbling generation strategy to further refine the gland segmentation mask, thereby reducing user labor.

[0057] In this embodiment, the segmentation backbone in the multi-task neural network architecture described in step S1 is The network is a two-level nested U-shaped structure. The top layer is a large U-shaped structure similar to U-Net, consisting of 11 stages. Each stage is filled with a well-configured residual U-block (RSU).

[0058] In this embodiment, the two error prediction decoders in step S1 have different structures. The false negative decoder uses the same structure as the segmentation decoder to predict the unrecognized glands; the false positive decoder is a combination of the edge generation module and A new hybrid structure is constructed by the decoder module to more effectively learn the edge features of the gland, so as to predict the false positive area of ​​the gland edge as accurately as possible.

[0059] In this embodiment, if Figure 1 The edge generation module shown is applied to the convolutional features of the RSU output at the same level of the segmentation decoder to generate edge features at that level, and continuously concatenates and fuses these multi-scale edge features level by level to focus on identifying false positive areas at the edges of glands; specifically as follows:

[0060] The output features of the shared encoder and the output features of the segmentation decoder RSU-4F module are respectively concatenated through the edge generation module to generate edge features, which are used as the input of the false positive decoder RSU-4F block;

[0061] The output features of the segmentation decoder RSU-4 module are generated by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-4F module and used as the input of the false positive decoder RSU-4 module;

[0062] The output features of the segmentation decoder RSU-5 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4 module and used as the input of the false positive decoder RSU-5 module;

[0063] The output features of the segmentation decoder RSU-6 module are generated by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-5 module and used as the input of the false positive decoder RSU-6 module;

[0064] The output features of the segmentation decoder RSU-7 module are generated into edge features by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-6 module and used as the input of the false positive decoder RSU-7 module.

[0065] In this embodiment, the edge generation module first performs Sobel convolution operations with different step sizes on the input raw features to obtain multi-scale edge information, then resamples to the same resolution as the original features and concatenates them in the channel dimension. Finally, a 1x1 convolution layer, instance batch normalization, and Leaky ReLU activation are performed to obtain fine edge features. This process can be formulated as:

[0066]

[0067] Among them, F edge represents the output of the edge generation module, and Represents Sobel convolution operations with step sizes of 1, 2, and 4, respectively. U represents upsampling by bilinear interpolation, and [] represents splicing. Represents a composite layer consisting of a 1x1 convolution, an instance batch normalization layer, and a Leaky ReLU activation function.

[0068] In this embodiment, in the two-stage training strategy described in step S2, different loss functions are used in different stages to train a specific decoder.

[0069] The first stage is to optimize gland segmentation. Therefore, we use the doctor's manual annotation results as the true mask and train the segmentation backbone in a deep supervision manner. The loss function uses the Normalized Focal Loss (NFL):

[0070]

[0071] Where NFL(i,j) represents the normalized focal loss of the segmentation decoder output at (i,j), p i,j represents the confidence of the segmentation decoder output at (i, j), and γ is used to adjust the weight ratio of easy-to-classify samples, which is set to 2 in this embodiment.

[0072] The second stage task is to fine-tune the two error prediction decoders to predict the false positive and false negative regions in the segmentation mask output by the segmentation decoder. Since it is very challenging to directly use the false positive and false negative regions in the segmentation mask as the true mask for training, in order to improve the accuracy of the prediction as much as possible to ensure that the pseudo-graffiti generated in step S3 is annotated to the appropriate location, a simplified mask combined with uncertainty is used for deep supervision training. The simplified false positive region true mask M fp And the true mask M of the false negative region fn for:

[0073]

[0074] M fn =remove((M∩(P<τ))∩(τ a ≤P<τ),m) (4)

[0075] Where M is the true mask of the gland; P is the probability map output by the segmentation decoder; τ is the prediction threshold (set to 0.5 in this embodiment); τ a and τ b Represent the lower and upper bounds of the model prediction uncertainty region, and in this example are set to 0.46 and 0.52 respectively. remove(x,m) removes regions in mask x with an area smaller than m. In this example, m is set to 5. This is a simplified strategy that takes into account that model uncertainty regions are more likely to be identified as erroneous regions and that small target regions contain a lot of noise. The loss function uses the intersection over union loss (IoULoss):

[0076]

[0077] Among them, X represents the simplified true mask, Y represents the predicted mask output by the error prediction decoder; for the false positive decoder, X is M fp ; For the false negative decoder, X is M fn .

[0078] In this embodiment, the concatenated input described in step S3 is specifically as follows: before forward propagation through the network, the mutual information and the previous mask are first convolved to make the output tensor shape identical to the output tensor of the first convolutional layer in the network. These two tensors are then element-wise summed to incorporate all the information of the concatenated input without making any modifications to the existing network architecture.

[0079] In this embodiment, the pseudo-graffiti generation strategy described in step S3 generates a pseudo-graffiti based on the maximum error region in the outputs of the two error prediction decoders. The specific strategy is as follows:

[0080] First, the output of the error prediction decoder is converted into a binary mask by a predefined threshold (set to 0.5), and all non-zero pixels in the mask are incorrectly labeled pixels in the predicted segmentation mask;

[0081] Then, we use the connected components of the OpenCV library to divide adjacent pixels into the same region and non-adjacent pixels into different regions;

[0082] Next, the error region with the largest area is selected as the target region;

[0083] Finally, it is determined whether the target area comes from the prediction mask output by the false positive decoder or the prediction mask output by the false negative decoder. If it is the prediction mask output by the false positive decoder, the model performs negative graffiti annotation (representing the background), otherwise it performs positive graffiti annotation (representing the foreground).

[0084] Pseudo-graffiti are generated by the model based on the false positive and false negative regions predicted by the erroneous prediction decoder according to the pseudo-graffiti generation strategy, simulating real user generation. Therefore, these pseudo-graffiti are completely "free," additional, and beneficial, not only automatically correcting erroneous regions in the gland segmentation mask predicted by the segmentation decoder, but also requiring no user effort.

[0085] In summary, the present invention uses As an efficient backbone for meibomian gland image segmentation; through edge generation module and The effective combination of RSUs can identify false positive areas at the edges of glands. Compared with existing methods, the model can directly predict the segmentation error areas and perform pseudo-graffiti annotation to refine the gland edges and segment unidentified glands, which can further reduce labor costs. Moreover, since these graffiti are completely "free", additional and beneficial, users only need to provide click guidance, thus taking into account both the efficiency of graffiti and the labor input of users.

[0086] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. An interactive infrared meibomian gland image segmentation method based on graffiti simulation, characterized by: The following steps are involved: Step S1: constructing a multi-task neural network architecture, including a shared encoder, a segmentation decoder, and two error prediction decoders; the shared encoder is responsible for extracting common, high-level features from the input infrared meibomian gland image, the segmentation decoder is used to segment the meibomian gland image, and the two error prediction decoders are respectively a false positive decoder and a false negative decoder, which are used to predict false positive and false negative regions in the segmentation mask output by the segmentation decoder; Step S2: Use a two-stage training strategy to train the multi-task neural network architecture of step S1 and obtain a trained model; in the first stage, freeze the error prediction decoder and train the segmentation backbone to optimize the gland segmentation task, wherein the segmentation backbone includes a shared encoder and a segmentation decoder; The second stage freezes the shared encoder and segmentation decoder, and fine-tunes the false positive decoder and false negative decoder to optimize the task of false positive and false negative region prediction in the segmentation mask; Step S3: The trained model is used for the click-based interactive infrared meibomian gland image segmentation task; for each click interaction of the user, the model will perform prediction in two stages; The first stage is the user interaction stage; in this stage, the user clicks to correct the error based on the input image and the segmentation mask previously predicted by the model. The model uses the image, the previous segmentation mask, and the user's click as cascade input, performs forward propagation in the multi-task neural network, and finally outputs a rough segmentation mask of the gland and the false positive and false negative predicted areas of the mask. The second stage is the model simulation stage; in this stage, the model simulates real users scribbling on the input image according to the pseudo-graffiti generation strategy based on the predicted false positive and false negative areas to refine the segmentation mask of the gland as the previously predicted segmentation mask for the next click.

2. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The segmentation backbone in the multi-task neural network architecture described in step S1 adopts a two-level nested U-shaped segmentation network 3. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 2, characterized in that: The two error prediction decoders in step S1 adopt different structures; the false negative decoder adopts the same structure as the segmentation decoder to predict the unidentified glands; the false positive decoder combines the edge generation module and The decoder module constructs a hybrid structure to learn the edge features of the gland and predict the false positive areas of the gland edge.

4. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 3, characterized in that: The false positive decoder combines the edge generation module and The decoder module constructs a hybrid structure, which applies the edge generation module to the convolutional features output by the RSU at the same level of the segmentation decoder to generate edge features at the same level, and continuously splices and fuses these multi-scale edge features level by level to identify false positive areas at the edge of the gland; the details are as follows: The output features of the shared encoder and the output features of the segmentation decoder RSU-4F module are respectively concatenated through the edge generation module to generate edge features, which are used as the input of the false positive decoder RSU-4F block; The output features of the segmentation decoder RSU-4 module are generated by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-4F module and used as the input of the false positive decoder RSU-4 module; The output features of the segmentation decoder RSU-5 module are generated by the edge generation module, and then concatenated with the output features of the false positive decoder RSU-4 module and used as the input of the false positive decoder RSU-5 module; The output features of the segmentation decoder RSU-6 module are generated by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-5 module and used as the input of the false positive decoder RSU-6 module; The output features of the segmentation decoder RSU-7 module are generated into edge features by the edge generation module, and then spliced ​​with the output features of the false positive decoder RSU-6 module and used as the input of the false positive decoder RSU-7 module.

5. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 3, characterized in that: The edge generation module first performs Sobel convolution operations with different step sizes on the input original features to realize multi-scale edge information, then resamples them to the same resolution as the original features and splices them in the channel dimension, and finally obtains fine edge features through a 1x1 convolution, instance batch normalization and Leaky ReLU activation.

6. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 5, characterized in that: The edge generation module performs a Sobel convolution operation with strides of 1, 2, and 4 on the input raw features, then resamples them to the same resolution as the original features and concatenates them in the channel dimension. Finally, a 1x1 convolution, instance batch normalization, and Leaky ReLU activation are performed to obtain fine edge features. The specific calculation is as follows: Among them, F edge represents the output of the edge generation module, and Respectively represent Sobel convolution operations with step sizes of 1, 2, and 4; U represents upsampling by bilinear interpolation; [] represents splicing, Represents a composite layer consisting of a 1x1 convolution, an instance batch normalization layer, and a Leaky ReLU activation function.

7. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: In the first stage described in step S2, the doctor's manual annotation results are used as the true mask, and the segmentation backbone is trained using deep supervision; the loss function uses the normalized focal loss: Where NFL(i,j) represents the normalized focal loss of the segmentation decoder output at (i,j), p i,j It represents the confidence of the segmentation decoder output at (i, j), and γ is used to adjust the weight ratio of easy-to-classify samples.

8. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The second stage described in step S2 uses a simplified mask combined with uncertainty for deep supervision training, and the simplified false positive area true mask M fp And the true mask M of the false negative region fn for: M fn =remove((M∩(P<τ))∩(τ a ≤P<τ),m) (4) Where M is the true mask of the gland; P is the probability map output by the segmentation decoder; τ is the prediction threshold, τ a and τ b Represents the lower and upper bounds of the model prediction uncertainty region; remove(x,m) means removing the area smaller than m in the mask x; The loss function uses the intersection-over-union loss: Among them, X represents the simplified true mask, Y represents the predicted mask output by the error prediction decoder; for the false positive decoder, X is M fp ; For the false negative decoder, X is M fn .

9. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The cascade input described in step S3 is specifically as follows: before forward propagation in the network, the interaction information and the previous mask are first convolved to make the shape of the output tensor exactly the same as the output tensor of the first convolutional layer in the network, and then the two tensors are summed element by element to fuse all the information of the cascade input.

10. The interactive infrared meibomian gland image segmentation method based on graffiti simulation according to claim 1, characterized in that: The pseudo-graffiti generation strategy described in step S3 generates a pseudo-graffiti based on the maximum error region in the outputs of the two error prediction decoders. The specific strategy is as follows: First, the output of the error prediction decoder is converted into a binary mask by a predefined threshold. All non-zero pixels in the mask are incorrectly labeled pixels in the predicted segmentation mask. Then, we use the connected components of the OpenCV library to divide adjacent pixels into the same region and non-adjacent pixels into different regions; Next, the error region with the largest area is selected as the target region; Finally, it is determined whether the target area comes from the prediction mask output by the false positive decoder or the prediction mask output by the false negative decoder. If it is the prediction mask output by the false positive decoder, the model will perform negative graffiti annotation representing the background, otherwise it will perform positive graffiti annotation representing the foreground.

Images