Passive field adaptive eye fundus image segmentation method based on confidence coefficient constraint average teacher model

Abstract

The invention discloses a passive field adaptive eye fundus image segmentation method based on a confidence coefficient constraint average teacher model. The method comprises the following steps: step 1, segmenting an eye fundus optic cup and an optic disc of a student network; 2, segmenting a teacher network fundus optic cup and an optic disk; step 3, calculating a student network loss function and updating parameters; 4, adjusting and calculating dynamic EMA parameters; 5, updating teacher network parameters; and step 6, calculating a distributed alignment JS loss function. According to the method, confidence constraint is introduced into an updating mechanism of the average teacher model, and JS divergence is introduced to align the teacher model and the student model on a distribution level, so that the stability of a pseudo label is enhanced, and fundus optic disc and optic cup segmentation with higher precision is realized.

Description

Technical Field

[0001] The invention relates to a fundus image segmentation method, in particular to a passive domain adaptive fundus image segmentation method based on a confidence-constrained average teacher model. Background Art

[0002] Fundus image segmentation is a key component of medical image processing and has attracted increasing attention. Computer-aided fundus image segmentation methods, especially those based on unsupervised domain adaptation (UDA), have achieved good performance, reduced the need for large-scale data annotation, and improved the generalization ability of the model in different scenarios, making it widely used in medical images. The use of UDA methods reduces the need for labeled data for the model. However, when processing fundus retinal image data, there are issues of data security and privacy, which makes it challenging to obtain source domain data in many cases. Therefore, passive domain adaptation (SFDA) has gradually become an important method in medical image processing. SFDA methods only require access to the characteristics of the pre-trained model in the source domain, which poses a significant challenge to model training. In particular, in the field of fundus image segmentation, the proportions of the optic cup, optic disc, and background vary greatly, and there is a significant class imbalance problem, which further increases the difficulty of model optimization. In addition, due to the lack of labeled data in the target domain, existing methods usually rely on pseudo-labels generated by the source domain model for training, which means that the stability of pseudo-labels has a significant impact on the training process.

[0003] Recently, the mean teacher model provides a good solution to SFDA, however, current methods mainly rely on fixed exponential shift parameters to update the teacher model f t parameters, which means that f t Only based on a fixed ratio from the student model f s This updating mechanism hinders the network’s ability to adaptively adjust its learning process. In fact, the teacher model f t and student model f s The difference between them is not fixed, that is, at each stage of training, f t and f s There may be inconsistent learning progress. If the teacher model is based on a fixed ratio from f each time s Learning knowledge, it may hinder the model's ability to effectively extract distinguishing features from the data. This also hinders the model's ability to dynamically adjust its learning level according to the learning progress, thereby hindering the overall improvement of network performance. Summary of the invention

[0004] In order to enhance the learning ability of the average teacher model and alleviate the problems of class imbalance and insufficient pseudo-label stability in fundus image segmentation tasks, the present invention provides a passive domain adaptive fundus image segmentation method based on the confidence-constrained average teacher model. The method introduces confidence constraints into the update mechanism of the average teacher model, and introduces JS divergence to align the teacher model and the student model at the distribution level, thereby enhancing the stability of pseudo-labels to achieve higher-precision fundus optic disc and optic cup segmentation.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A passive domain adaptive fundus image segmentation method based on a confidence-constrained average teacher model comprises the following steps:

[0007] Step 1: Student network fundus optic cup and optic disc segmentation:

[0008] For the target domain fundus image to be segmented Using a data enhancement method called A, we get an enhanced image Will Input into the student network based on the encoder-decoder structure to obtain the optic cup and optic disc segmentation results

[0009] Step 2: Teacher network fundus optic cup and optic disc segmentation:

[0010] For the target domain fundus image to be segmented Using the data enhancement method named B, we get the enhanced image Will Input into the teacher network based on the encoder-decoder structure to obtain the optic cup and optic disc segmentation results and used as pseudo labels for the student network

[0011] Step 3: Student network loss function calculation and parameter update:

[0012] For the student network, using pseudo labels The segmentation results Perform supervision and calculate pixel-level loss function And perform back propagation to update the student network parameters

[0013] Step 4: Dynamic EMA parameter adjustment calculation:

[0014] For the confidence-based exponential moving average parameter (EMA) update module, the output difference between the student network and the teacher network is used to calculate the exponential moving average parameter λ;

[0015] Step 5: Update teacher network parameters:

[0016] For the teacher network, according to the exponential sliding average parameter λ, the sliding average formula is used to update the teacher network parameters

[0017] Step 6. Distribution alignment JS loss function calculation:

[0018] Output of the student network and the teacher network and The JS divergence between the two is calculated to achieve distribution-level alignment.

[0019] Compared with the prior art, the present invention has the following advantages:

[0020] 1. The present invention can adaptively adjust EMA parameters to ensure that the teacher model can more effectively acquire knowledge from the student model and enhance the overall learning ability of the system, thereby better solving the data imbalance problem in fundus images.

[0021] 2. The present invention uses JS divergence to align the outputs of the student and teacher models at the distribution level, which enhances the stability of pseudo labels and ensures that the model achieves more consistent segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 Schematic diagram of the confidence-constrained average teacher model (CCMT) framework proposed in the present invention.

[0023] Figure 2 The segmentation effect of the method of the present invention and the advanced fundus image segmentation method on the RIM-ONE-r3 dataset is compared. DETAILED DESCRIPTION

[0024] The technical solution of the present invention is further described below in conjunction with the accompanying drawings, but is not limited thereto. Any modification or equivalent replacement of the technical solution of the present invention without departing from the spirit and scope of the technical solution of the present invention should be included in the protection scope of the present invention.

[0025] The present invention provides a passive domain adaptive fundus image segmentation method based on a confidence-constrained average teacher model. The method is based on an average teacher structure and performs fundus image optic cup and optic disc segmentation under passive domain adaptive conditions. Figure 1 As shown, the specific steps include:

[0026] Step 1: Student network fundus optic cup and optic disc segmentation:

[0027] For the target domain fundus image to be segmented Using a data enhancement method called A, we get an enhanced image Will Input into the student network based on the encoder-decoder structure to obtain the optic cup and optic disc segmentation results

[0028] In this step, the input of the student network is the unlabeled target domain data, and the commonly used spatial learning method is used to extract image features and generate segmentation results. Formally, given the unlabeled data, that is, the target domain Among them, N t Indicates the number of samples in the target domain. For the convenience of calculation, the dimension of the image is scaled to 512*512.

[0029] First, the input target domain fundus image to be segmented Perform a series of strong data enhancements, including but not limited to random erasing, brightness adjustment, contrast adjustment, etc., to enhance the diversity and generalization ability of the data, which can help the student network better capture the characteristics and patterns of the target domain data and obtain enhanced images. Afterwards, Input to the encoder-decoder network. For the convenience of explanation, the present invention uses DeeplabV3+ based on MobileNetV2 as the encoder-decoder network, but is not limited to this. In this structure, the feature extraction network is not limited to MobileNetV2. Finally, the segmentation result with a dimension of 512*512 is obtained. It contains the optic disc and optic cup segmentation results.

[0030] Step 2: Teacher network fundus optic cup and optic disc segmentation:

[0031] For the target domain fundus image to be segmented Using the data enhancement method named B, we get the enhanced image Will Input into the teacher network based on the encoder-decoder structure to obtain the optic cup and optic disc segmentation results and used as pseudo labels for the student network

[0032] In this step, the input of the teacher network is the unlabeled target domain data, and the commonly used spatial learning method is used to extract image features and generate segmentation results. Formally, given the unlabeled data, that is, the target domain For the convenience of calculation, the dimension of the image is scaled to 512*512. The output of the teacher network is used as a pseudo label to supervise the training of the student network.

[0033] First, the input target domain fundus image to be segmented Perform a series of weaker data augmentations, including but not limited to slight color distortion, image flipping, etc., to help mitigate the risk of overfitting and provide more consistent pseudo labels for the student network to obtain enhanced images Afterwards, Input into the encoder-decoder network. In particular, the teacher network structure in this step is the same as the student network structure. For the convenience of explanation, the present invention still uses DeeplabV3+ based on the MobileNetV2 backbone as the encoder-decoder network, but is not limited to this, and in this structure, the feature extraction network is not limited to MobileNetV2. Finally, a segmentation result map with a dimension of 512*512 is obtained. It contains the optic disc and optic cup segmentation results as pseudo labels and is recorded as Supervise the training of student networks.

[0034] Step 3: Student network loss function calculation and parameter update:

[0035] For the student network, using pseudo labels The segmentation results Perform supervision and calculate pixel-level loss function And perform back propagation to update the student network parameters

[0036] In this step, the pixel-level loss function The specific calculation method is as follows:

[0037]

[0038] in, is the binary cross entropy loss function, express Each pixel in .

[0039] Step 4: Dynamic EMA parameter adjustment calculation:

[0040] For the confidence-based exponential moving average parameter (EMA) update module, the output difference between the student network and the teacher network is used to calculate the exponential moving average parameter λ.

[0041] In this step, the student model f is defined after sigmoid activation s and the teacher model f t The outputs are represented as and f s and f t The difference between the outputs can be measured in a variety of forms, including but not limited to difference, quotient, etc. Following an intuitive approach, the present invention uses f s and ft The distance between the outputs is measured as follows:

[0042]

[0043] According to the above formula, when λ is relatively small, Larger, resulting in f t The update of f is more aggressive. On the contrary, when λ is larger, f t The update of becomes more negative. This is because in the early stage of training, f s and f t There are often significant differences between the outputs of s It is impossible to effectively fit pseudo labels. In passive domain adaptation, due to f t It is mainly used to provide pseudo labels. The teacher model needs to adopt a more aggressive parameter adjustment strategy to avoid falling into the local optimum. As the training is nearing completion, f s and f t have become more stable, f s can fit the pseudo-label more accurately, so f t A more passive updating strategy should be adopted to prevent model oscillation.

[0044] Step 5: Update teacher network parameters:

[0045] For the teacher network, according to the exponential sliding average parameter λ, the sliding average formula is used to update the teacher network parameters

[0046] In this step, the update formula of the teacher network parameters is:

[0047]

[0048] in, Denotes the teacher model f t The parameters at step t are, Denotes the teacher model f t The parameters at step t-1, Represents the student model f s Parameters at step t.

[0049] Step 6. Distribution alignment JS loss function calculation:

[0050] Output of the student network and the teacher network and The JS divergence between the two is calculated to achieve distribution-level alignment.

[0051] In this step, the JS divergence is introduced to constrain the output of the student network and the teacher network to ensure that f s and ft The distribution consistency of the output is calculated as follows:

[0052]

[0053] in, is the distribution level loss function, D KL represents the KL divergence,

[0054] f t The output of is used as training f s pseudo-label, which requires f t Generate stable and reliable pseudo labels to ensure f s JS divergence can measure the similarity of two distributions. When f t and f s When there is a significant difference between the output distributions of m With f t and f s This helps minimize the gap between the two distributions, ensuring that f t and f s The present invention uses the loss function and Distribution alignment and pixel alignment are implemented respectively, aiming to align the outputs of the student and teacher models at both the distribution level and the pixel level.

[0055] Experimental Results

[0056] The model of the present invention is evaluated on three publicly available fundus image datasets, which provide a wide range of retinal images captured from various clinical settings, ensuring the robustness and generalization of the performance of the model of the present invention. For preprocessing, the input image is resized to 512×512 as the region of interest (ROI). In the testing phase, the present invention sets REFUGE as the source domain; the Drishti-GS and RIM-ONE-r3 datasets are set as the target domain. In the passive domain adaptation phase, the present invention sets f s and f t All are initialized to the pre-trained source domain model.

[0057] As part of the evaluation process, the present invention uses two commonly used evaluation metrics. The first is the Dice coefficient, which reflects the pixel-level accuracy of the segmentation result and the true value. The higher the Dice coefficient, the better the performance. The second is the Average Symmetric Surface Distance (ASSD), which measures the coherence of the segmentation result boundaries. The lower the ASSD, the better the performance. By considering these two metrics, the present invention can better evaluate the performance of the segmentation algorithm.

[0058] Table 1 shows the experimental results comparison of the present invention and other methods on the RIM-ONE-r3 dataset. The best results in the passive setting are highlighted in bold. - indicates the results not reported by the method, ± indicates the standard deviation between samples, and SF indicates passive.

[0059] Table 1 Performance comparison on the RIM-ONE-r3 dataset

[0060]

[0061] Table 2 shows the experimental results comparison of the present invention and other methods on the Drishti-GS dataset, and the best results in the passive setting are highlighted in bold. - indicates the results not reported by the method, ± indicates the standard deviation between samples, and SF indicates passive.

[0062] Table 2 Performance comparison on the Drishti-GS dataset

[0063]

[0064] As shown in Tables 1 and 2, on the RIM-ONE-r3 and Drishti-GS datasets, the present invention achieves the best results in both the optic disc and optic cup segmentation indicators, proving the effectiveness of the method proposed in the present invention.

[0065] Figure 2 The segmentation visualization effects of the present invention and other methods on the RIM-ONE-r3 dataset are shown. (a) shows the original retinal image, (b) shows the true value, and (c) presents the segmentation results of the optical disc and optic cup without adaptation. Columns (d) to (i) show the segmentation results using other methods, while (j) shows the segmentation results of the present invention. Compared with other methods, the present invention achieves better segmentation boundaries and provides excellent segmentation results for the optical disc and optic cup. The results in the last row also show that when the image quality is poor, the model is more likely to produce incorrect segmentation results. Even so, the method proposed in the present invention can still achieve high segmentation accuracy.

Claims

1. A passive domain adaptive fundus image segmentation method based on confidence constrained average teacher model, characterized in that The method comprises the following steps: Step 1: Student network fundus optic cup and optic disc segmentation: For the target domain fundus image to be segmented Using a data enhancement method called A, we get an enhanced image Will Input into the student network based on the encoder-decoder structure to obtain the optic cup and optic disc segmentation results Step 2: Teacher network fundus optic cup and optic disc segmentation: For the target domain fundus image to be segmented Using the data enhancement method named B, we get the enhanced image Will Input into the teacher network based on the encoder-decoder structure to obtain the optic cup and optic disc segmentation results and used as pseudo labels for the student network Step 3: Student network loss function calculation and parameter update: For the student network, using pseudo labels The segmentation results Perform supervision and calculate pixel-level loss function And perform back propagation to update the student network parameters Step 4: Dynamic EMA parameter adjustment calculation: For the confidence-based exponential moving average parameter update module, the output difference between the student network and the teacher network is used to calculate the exponential moving average parameter λ; Step 5: Update teacher network parameters: For the teacher network, according to the exponential sliding average parameter λ, the sliding average formula is used to update the teacher network parameters Step 6. Distribution alignment JS loss function calculation: Output of the student network and the teacher network and The JS divergence between the two is calculated to achieve distribution-level alignment.

2. The passive domain adaptive fundus image segmentation method based on the confidence constrained average teacher model according to claim 1 is characterized in that In the step 1, a data enhancement method of random erasing, brightness adjustment, and contrast adjustment is adopted.

3. The passive domain adaptive fundus image segmentation method based on the confidence constrained average teacher model according to claim 1 is characterized in that In the step 2, a data enhancement method of color distortion and image flipping is adopted.

4. The passive domain adaptive fundus image segmentation method based on the confidence constrained average teacher model according to claim 1 is characterized in that In step 3, the pixel-level loss function The specific calculation method is as follows: in, is the binary cross entropy loss function, express Each pixel in .

5. The passive domain adaptive fundus image segmentation method based on the confidence constrained average teacher model according to claim 1 is characterized in that In step 4, the student model f after sigmoid activation is defined s and the teacher model f t The outputs are represented as and The specific calculation method of the exponential moving average parameter λ is as follows:

6. The passive domain adaptive fundus image segmentation method based on the confidence constrained average teacher model according to claim 1 is characterized in that In step 5, the updating formula of the teacher network parameters is: in, Denotes the teacher model f t The parameters at step t are, Denotes the teacher model f t The parameters at step t-1, Represents the student model f s Parameters at step t.

7. The passive domain adaptive fundus image segmentation method based on the confidence constrained average teacher model according to claim 1 is characterized in that In step 6, the specific calculation method of JS divergence is as follows: in, is the distribution level loss function, D KL represents the KL divergence, 8. The passive domain adaptive fundus image segmentation method based on the confidence-constrained average teacher model according to claim 1 is characterized in that Both the student network and the teacher network use DeeplabV3+ based on the MobileNetV2 backbone as the encoder-decoder network.

Images