Disc and cup segmentation method based on unsupervised domain adaptation and imaging method thereof

Abstract

The application discloses a disc and cup segmentation method based on unsupervised domain adaptation, which comprises the following steps: obtaining a source domain fundus image dataset and a target domain fundus image dataset and processing; constructing a disc and cup segmentation original model and training to obtain a source domain disc and cup segmentation result and a disc and cup segmentation model; using the disc and cup segmentation model to perform disc and cup segmentation on the target domain fundus image dataset to obtain a target domain disc and cup segmentation result; constructing a disc and cup region discriminator and a disc and cup boundary discriminator; using the disc and cup region discriminator and the disc and cup boundary discriminator to discriminate the segmentation result; optimizing and verifying the disc and cup segmentation model to obtain a disc and cup segmentation application model; and using the disc and cup segmentation application model to perform actual disc and cup segmentation. The application also discloses an imaging method comprising the disc and cup segmentation method based on unsupervised domain adaptation. The application has high reliability, good accuracy and wide application range.

Description

Technical Field

[0001] This invention belongs to the field of image processing, specifically relating to an unsupervised, adaptive visual disc / cup segmentation method and its imaging method. Background Technology

[0002] With economic and technological development and the improvement of people's living standards, people are paying more and more attention to health. The segmentation of the optic disc and optic cup in the eye has significant clinical and theoretical research value in the medical field. Therefore, the segmentation of the optic disc and optic cup has become one of the key research focuses for researchers.

[0003] Currently, deep learning methods have achieved good segmentation results for the optic disc and optic cup. However, deep learning methods are based on the assumption that the source and target domain data have the same data distribution; in practical applications, due to factors such as different acquisition devices and acquisition environments, there are significant differences in the distribution of fundus image data in different domains. This domain bias directly leads to poor segmentation accuracy when deep learning methods are applied to fundus images of unknown target domains. Furthermore, deep learning methods require a large amount of labeled data for learning; however, labeling fundus images requires significant manpower, time, and cost, which undoubtedly hinders the widespread application of deep learning methods.

[0004] Unsupervised domain adaptation methods utilize models trained on labeled source domain data to adapt to unlabeled target domain data, achieving good generalization results. However, minimizing the differences between the source and target domains and learning common knowledge across domains remains a challenge for unsupervised domain adaptation. Currently, feature alignment-based unsupervised domain adaptation methods map source and target domain data into a high-dimensional feature space; however, the feature space contains numerous complex features, making it difficult to eliminate inter-domain differences. Compared to minimizing inter-domain differences in high-dimensional features, output alignment-based unsupervised domain adaptation, in a low-dimensional space, aligns the segmentation results predicted by the model through adversarial learning, thereby reducing inter-domain differences. However, the lack of a connection mechanism between the segmentation task and the domain discrimination task means that the domain discrimination task cannot learn semantic knowledge related to the segmentation task, resulting in poor discriminative ability. Consequently, the segmentation model predicts poor segmentation results on the target domain fundus image, especially in optic cup segmentation, where the predicted segmentation results are blurry. Summary of the Invention

[0005] One of the objectives of this invention is to provide a highly reliable, accurate, and widely applicable unsupervised domain adaptive spectral disc / cup segmentation method.

[0006] The second objective of this invention is to provide an imaging method that includes the aforementioned unsupervised domain adaptive optic disc / optic cup segmentation method.

[0007] The spectral disc / spectral cup segmentation method based on unsupervised domain adaptation provided by this invention includes the following steps:

[0008] S1. Obtain the source domain fundus image dataset and the target domain fundus image dataset, and process the obtained fundus images;

[0009] S2. Based on the U-net network, construct the original model for visual disc / cup segmentation;

[0010] S3. Use the processed source domain fundus image dataset obtained in step S1 to train the original optic disc-optic cup segmentation model constructed in step S2, and obtain the corresponding source domain optic disc-optic cup segmentation result and the corresponding optic disc-optic cup segmentation model.

[0011] S4. Using the optic disc-optic cup segmentation model obtained in step S3, segment the image data in the processed target domain fundus image dataset obtained in step S1 to obtain the target domain optic disc-optic cup segmentation result.

[0012] S5. Construct a discriminator for the region of the visual disk and the visual disk / cup, and a discriminator for the boundary of the visual disk and the visual disk;

[0013] S6. The disc-cup region discriminator constructed in step S5 is used to perform region discrimination on the segmentation results obtained in steps S3 and S4, and the disc-cup boundary discriminator constructed in step S5 is used to perform boundary discrimination on the segmentation results obtained in steps S3 and S4.

[0014] S7. Meta-learning is adopted to enhance the connection between the disc-cup segmentation model and the disc-cup region discriminator, and between the disc-cup segmentation model and the disc-cup boundary discriminator, thereby optimizing the disc-cup segmentation model;

[0015] S8. Validate the optimized disc-cup segmentation model obtained in step S7 to obtain the final disc-cup segmentation application model;

[0016] S9. Using the disc-cup segmentation application model obtained in step S8, perform actual disc-cup segmentation.

[0017] Step S1, which involves acquiring the source domain fundus image dataset and the target domain fundus image dataset and processing the acquired fundus images, specifically includes the following steps:

[0018] The fundus images in the source domain fundus image dataset and the target domain fundus image dataset are randomly cropped to a uniform size, then image enhancement is performed, and finally image normalization is performed. The image enhancement includes random flipping, random rotation, elastic transformation, brightness adjustment, and salt-and-pepper noise addition.

[0019] The optic disc and optic cup labels corresponding to the fundus images in the source domain fundus image dataset are randomly cropped to a uniform size, then image enhancement is performed, and finally image normalization is performed. The image enhancement includes random flipping, random rotation, elastic transformation, brightness adjustment, and salt-and-pepper noise addition.

[0020] Step S2, which involves constructing the original model for visual disc / visual cup segmentation based on the U-net network, specifically includes the following steps:

[0021] The original model for visual disk-visual cup segmentation includes a shared encoder, a visual disk-visual cup region decoder, and a visual disk-visual cup boundary decoder.

[0022] The shared encoder includes a downsampling layer, intermediate connections, and an upsampling layer. The downsampling layer comprises three convolutional blocks, each including a meta-learning convolutional layer, an `InstacnceNorm2d` normalization layer, and a `Leaky_relu` activation function. The meta-learning convolutional layer updates the gradient weights of the convolutional layer based on the meta-learning loss and meta-learning stride of the input model, and the kernel size is 3×3. The intermediate connection layer connects the low-level features learned by the downsampling layer with the high-level features learned by the upsampling layer. The upsampling layer also comprises three convolutional blocks, each including a meta-learning convolutional layer, an `InstacnceNorm2d` normalization layer, and a `Leaky_relu` activation function. The meta-learning convolutional layer updates the gradient weights of the convolutional layer based on the meta-learning loss and meta-learning stride of the input model, and the kernel size is 3×3. The upsampling layer further forms a feature map with contextual features from the features output by the intermediate connection layer.

[0023] The aforementioned visual disc / cup boundary decoder comprises two concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional layer, a batch normalization layer, a ReLU activation function, and a Dropout operation, and the kernel size in each convolutional layer is 3×3; the kernel size in the last convolutional layer is 1×1; the visual disc / cup boundary decoder is used to decode the shared features output by the shared encoder, thereby obtaining the visual disc / cup boundary segmentation result;

[0024] The aforementioned visual disc / visual cup region decoder includes a batch normalization layer, a ReLU activation function, a Dropout operation, and a convolutional layer connected in series, wherein the convolutional kernel size of the convolutional layer is 1×1. The visual disc / visual cup region decoder is used to concatenate the shared features output by the shared encoder and the visual disc / visual cup boundary segmentation results output by the visual disc / visual cup boundary decoder, and then decode the concatenated features to obtain the visual disc / visual cup region segmentation results.

[0025] Step S3 involves training the original optic disc-optic cup segmentation model constructed in step S2 using the processed source domain fundus image dataset obtained in step S1 to obtain the corresponding source domain optic disc-optic cup segmentation result and the corresponding optic disc-optic cup segmentation model. Specifically, this includes the following steps:

[0026] The fundus images x from the processed source domain fundus image dataset s The features learned by the shared encoder are input into the original disc-cup segmentation model constructed in step S2, thereby outputting the source domain disc-cup boundary prediction result. Then, the mean square loss L of the source domain disc-cup boundary prediction result and the corresponding label is calculated using the following formula. e :

[0027]

[0028] In the formula, N is the number of fundus images in the processed source domain fundus image dataset; X s This is the processed source domain fundus image dataset; Labels for the boundaries of the source domain view disk and view cup; The result is the prediction of the boundary of the source domain's visual disk / cup.

[0029] The features output by the visual disk / visual cup boundary decoder and the features output by the shared encoder are concatenated and then input into the visual disk / visual cup region decoder to obtain the source domain visual disk / visual cup region prediction result. Then, the binary classification cross-entropy loss L between the source domain visual disk / visual cup region prediction result and the corresponding label is calculated using the following formula. m :

[0030]

[0031] In the formula Labels for the source domain view disk and view cup area; The prediction results for the source domain view disk and view cup regions;

[0032] The overall loss L is calculated as L = L e +L m The comprehensive loss function was used to train the disc-cup segmentation model, and the corresponding source domain disc-cup segmentation results and the corresponding disc-cup segmentation model were obtained.

[0033] Step S4 involves using the optic disc-optic cup segmentation model obtained in step S3 to segment the optic disc and optic cup in the processed target domain fundus image dataset obtained in step S1, thereby obtaining the target domain optic disc-optic cup segmentation result. Specifically, this involves using the optic disc-optic cup segmentation model obtained in step S3 to segment the optic disc and optic cup in the processed target domain fundus image dataset obtained in step S1, thereby obtaining the target domain optic disc-optic cup boundary prediction result. and the corresponding target domain visual disc visual cup region prediction results Where x t This represents fundus images in the target domain fundus image dataset.

[0034] Step S5, which involves constructing the optic disc-optic cup region discriminator and the optic disc-optic cup boundary discriminator, specifically includes the following steps:

[0035] Constructing a discriminator for the visual disk and cup: The discriminator for the visual disk and cup consists of four concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional layer with a kernel size of 4×4 and a Leaky_relu activation function; the kernel size of the last convolutional layer is 4×4, and the number of output channels is 1; the input channels of the convolutional layer in the first convolutional block are 1, the input channels of the convolutional layer in the second convolutional block are 64, the input channels of the convolutional layer in the third convolutional block are 128, the input channels of the convolutional layer in the fourth convolutional block are 256, and the input channels of the last convolutional layer are 512.

[0036] Constructing a discriminator for the visual disk and visual cup region: The discriminator for the visual disk and visual cup region consists of four concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional layer with a kernel size of 4×4 and a Leaky_relu activation function; the kernel size of the last convolutional layer is 4×4, and the number of output channels is 1; the input channels of the convolutional layer in the first convolutional block are 2, the input channels of the convolutional layer in the second convolutional block are 64, the input channels of the convolutional layer in the third convolutional block are 128, the input channels of the convolutional layer in the fourth convolutional block are 256, and the input channels of the last convolutional layer are 512.

[0037] Step S6 involves using the disc-cup region discriminator constructed in step S5 to perform region discrimination on the segmentation results obtained in steps S3 and S4, and using the disc-cup boundary discriminator constructed in step S5 to perform boundary discrimination on the segmentation results obtained in steps S3 and S4. Specifically, this includes the following steps:

[0038] The disc-cup boundary discriminator constructed in step S5 performs boundary discrimination on the segmentation results obtained in steps S3 and S4: the disc-cup boundary discriminator classifies the source domain disc-cup boundary segmentation result as source neighborhood label 1, and the target domain disc-cup boundary segmentation result as target neighborhood label 0; the disc-cup boundary discriminator tries its best to correctly distinguish the input boundary segmentation results, and uses the following formula to calculate the cross-entropy loss between the input boundary prediction result and the neighborhood label.

[0039]

[0040] In the formula, N is the number of fundus images in the processed source domain fundus image dataset; X s This is the processed source domain fundus image dataset; The prediction results for the source domain view disk / view cup boundary; x s represents the fundus images in the processed source domain fundus image dataset; M represents the number of fundus images in the processed target domain fundus image dataset; x t X represents fundus images from the processed target domain fundus image dataset; t This is the processed target domain fundus image dataset; For the target domain, the prediction results of the view disk / view cup boundary; L D () represents the cross-entropy loss function;

[0041] The disc-cup region discriminator constructed in step S5 is used to perform region discrimination on the segmentation results obtained in steps S3 and S4: For the source domain disc-cup region segmentation results, the following formula is used... Perform pixel-based entropy processing, where E(x) s The result is the source domain entropy processing result. The source domain view disk / view cup region prediction results are used; the target domain view disk / view cup region segmentation results are obtained using the following formula. Perform pixel-based entropy processing, where E(x) t The result is the target domain entropy processing result. The target domain is the view disk / view cup region prediction result; the view disk / view cup region discriminator classifies the source domain view disk / view cup region segmentation result as source neighborhood label 1, and the target domain view disk / view cup region segmentation result as target neighborhood label 0; the view disk / view cup region discriminator tries its best to correctly distinguish the input region segmentation results, and uses the following formula to calculate the cross-entropy loss of the input region prediction result and neighborhood label.

[0042]

[0043] In the formula L D () represents the cross-entropy loss function;

[0044] The view disk / view cup segmentation model aims to minimize the difference between the predicted boundary results of the view disk / view cup in the source domain and the target domain, and calculates the boundary adversarial loss using the following formula.

[0045]

[0046] In the formula, 1 represents the source domain label;

[0047] The view-disk / view-cup segmentation model aims to minimize the difference between the predicted view-disk / view-cup regions in the source and target domains, and calculates the region adversarial loss using the following formula.

[0048]

[0049] In the formula, 1 represents the source domain label;

[0050] The boundary adversarial loss and region adversarial loss are used to constrain the disc-cup segmentation model, prompting the disc-cup segmentation model to continuously learn the common spatial structure information between the source domain and the target domain, thereby generating more accurate disc-cup region segmentation results and disc-cup boundary segmentation results.

[0051] Step S7 describes employing meta-learning to enhance the connections between the visual disc / glass segmentation model and the visual disc / glass region discriminator, and between the visual disc / glass segmentation model and the visual disc / glass boundary discriminator, thereby optimizing the visual disc / glass segmentation model. This specifically includes the following steps:

[0052] The boundary discrimination task corresponding to the disc / cup boundary discriminator and the region discrimination task corresponding to the disc / cup region discriminator are used as meta-training tasks, and the following formula is used as the loss function L for the meta-training tasks. dom Together, update the visual disc / visual cup segmentation model:

[0053]

[0054] In the formula The cross-entropy loss is calculated using the input boundary prediction results and the neighborhood labels. The cross-entropy loss is calculated between the input region prediction results and the neighborhood labels: θ represents the parameters of the shared encoder in the visual disc / visual cup segmentation model; φ d These are the parameters in the disc-cup region discriminator and the disc-cup boundary discriminator;

[0055] The image disc / image cup segmentation task and the image disc / image cup region segmentation task corresponding to the image disc / image cup segmentation model are used as meta-test tasks, and the following formula is used as the loss function L for the meta-test task. seg Together, update the visual disc / visual cup segmentation model:

[0056]

[0057] In the formula, λ is the countermeasure loss balance coefficient; Losses due to border confrontation; For regional combat losses; L e The mean square loss is the prediction result of the source domain view disk / view cup boundary and the corresponding label; L m The binary classification cross-entropy loss is calculated using the predicted source-domain visual disk / visual cup region and its corresponding label; φ s These are parameters in the disc / cup boundary decoder and the disc / cup region decoder.

[0058] The meta-training task loss function L dom The loss function L for the summative test task seg By combining these methods, the overall loss function in the meta-learning process is obtained as follows:

[0059] The obtained overall loss function is then optimized to obtain the optimized overall loss function as follows:

[0060]

[0061] In the formula, α is the meta-learning step size; To perform a derivative operation on the task loss;

[0062] Minimizing the optimized overall loss function using a first-order Taylor expansion yields the final overall loss function as follows:

[0063] This invention also discloses an imaging method that includes the aforementioned unsupervised domain adaptive optic disc-optic cup segmentation method, specifically comprising the following steps:

[0064] (1) Obtain the original fundus image;

[0065] (2) The optic disc-optic cup segmentation method based on unsupervised domain adaptation is used to segment the fundus image obtained in step (1);

[0066] (3) Mark the optic disc and optic cup on the original fundus image using the optic disc and optic cup segmentation results obtained in step (2);

[0067] (4) Perform secondary imaging and output fundus images with optic disc and optic cup annotations.

[0068] The present invention provides an unsupervised domain adaptive optic disc-optic cup segmentation method and its imaging method. It proposes a meta-learning approach to connect the optic disc-optic cup segmentation task with the optic disc-optic cup region and boundary discrimination task, and provides a corresponding technical solution, thereby enabling the optic disc-optic cup segmentation model to learn more semantic knowledge. Therefore, the present invention improves the segmentation accuracy of the optic disc-optic cup in the target domain fundus image, and has high reliability, good accuracy and wide applicability. Attached Figure Description

[0069] Figure 1 This is a schematic diagram of the segmentation method of the present invention.

[0070] Figure 2 This is a schematic diagram of the model structure of the viewing disc / view cup segmentation model of the segmentation method of the present invention.

[0071] Figure 3 This is a schematic diagram of the imaging method of the invention. Detailed Implementation

[0072] like Figure 1 The diagram shown is a flowchart of the segmentation method of the present invention: The unsupervised domain adaptive viewing disc / viewcup segmentation method provided by the present invention includes the following steps:

[0073] S1. Obtain the source domain fundus image dataset and the target domain fundus image dataset, and process the obtained fundus images; specifically, this includes the following steps:

[0074] The fundus images in the source domain fundus image dataset and the target domain fundus image dataset are randomly cropped to a uniform size (preferably 512*512), then image enhancement is performed, and finally image normalization is performed; the image enhancement includes random flipping, random rotation, elastic transformation, brightness adjustment, and salt and pepper noise addition.

[0075] The optic disc and optic cup labels corresponding to the fundus images in the source domain fundus image dataset are randomly cropped to a uniform size, then image enhancement is performed, and finally image normalization is performed. The image enhancement includes random flipping, random rotation, elastic transformation, brightness adjustment, and salt-and-pepper noise addition.

[0076] S2. Based on the U-net network, construct the original model for visual disc / visual cup segmentation; specifically including the following steps:

[0077] The original model for visual disk-visual cup segmentation includes a shared encoder, a visual disk-visual cup region decoder, and a visual disk-visual cup boundary decoder, as detailed below. Figure 2 As shown;

[0078] The shared encoder consists of a meta-learning-based U-net, including downsampling layers, intermediate connections, and upsampling layers. The downsampling layer comprises three convolutional blocks, each including a meta-learning convolutional layer, an `InstacnceNorm2d` normalization layer, and a `Leaky_relu` activation function. The meta-learning convolutional layer updates the gradient weights of the convolutional layer based on the meta-learning loss and meta-learning stride of the input model, with a kernel size of 3×3. The intermediate connection layer connects the low-level features learned by the downsampling layer with the high-level features learned by the upsampling layer. The upsampling layer also comprises three convolutional blocks, each including a meta-learning convolutional layer, an `InstacnceNorm2d` normalization layer, and a `Leaky_relu` activation function. The meta-learning convolutional layer updates the gradient weights of the convolutional layer based on the meta-learning loss and meta-learning stride of the input model, with a kernel size of 3×3. The upsampling layer further forms a feature map with contextual features from the features output by the intermediate connection layer.

[0079] The aforementioned visual disc / cup boundary decoder comprises two concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional layer, a batch normalization layer, a ReLU activation function, and a Dropout operation, and the kernel size in each convolutional layer is 3×3; the kernel size in the last convolutional layer is 1×1; the visual disc / cup boundary decoder is used to decode the shared features output by the shared encoder, thereby obtaining the visual disc / cup boundary segmentation result;

[0080] The disc / cup region decoder includes a batch normalization layer, a ReLU activation function, a Dropout operation, and a convolutional layer connected in sequence, wherein the convolutional kernel size of the convolutional layer is 1×1; the disc / cup region decoder is used to concatenate the shared features output by the shared encoder and the disc / cup boundary segmentation results output by the disc / cup boundary decoder, and then decode the concatenated features to obtain the disc / cup region segmentation results;

[0081] S3. Using the processed source domain fundus image dataset obtained in step S1, train the original optic disc-optic cup segmentation model constructed in step S2 to obtain the corresponding source domain optic disc-optic cup segmentation result and the corresponding optic disc-optic cup segmentation model; specifically including the following steps:

[0082] The fundus images x from the processed source domain fundus image dataset s The features learned by the shared encoder are input into the original disc-cup segmentation model constructed in step S2, thereby outputting the source domain disc-cup boundary prediction result. Then, the mean square loss L of the source domain disc-cup boundary prediction result and the corresponding label is calculated using the following formula. e :

[0083]

[0084] In the formula, N is the number of fundus images in the processed source domain fundus image dataset; X s This is the processed source domain fundus image dataset; Labels for the boundaries of the source domain's view disk and view cup; The result is the prediction of the boundary of the source domain's visual disk / visual cup.

[0085] The features output by the visual disk / visual cup boundary decoder and the features output by the shared encoder are concatenated and then input into the visual disk / visual cup region decoder to obtain the source domain visual disk / visual cup region prediction result. Then, the binary classification cross-entropy loss L between the source domain visual disk / visual cup region prediction result and the corresponding label is calculated using the following formula. m :

[0086]

[0087] In the formula Labels for the source domain view disk and view cup area; The prediction results for the source domain visual disc / visual cup region;

[0088] The overall loss L is calculated as L = L e +L m The comprehensive loss is used to train the disc-cup segmentation model to obtain the corresponding source domain disc-cup segmentation results and the corresponding disc-cup segmentation model.

[0089] S4. Using the optic disc-optic cup segmentation model obtained in step S3, segment the optic disc and optic cup in the image data of the processed target domain fundus image dataset obtained in step S1 to obtain the target domain optic disc-optic cup segmentation result; since the target domain fundus image does not have corresponding optic disc-optic cup region and boundary prediction values, the optic disc-optic cup segmentation model obtained in step S3 is used to segment the optic disc and optic cup in the image data of the processed target domain fundus image dataset obtained in step S1 to obtain the target domain optic disc-optic cup boundary prediction result. and the corresponding target domain visual disc visual cup region prediction results Where x t Represents fundus images in the target domain fundus image dataset;

[0090] S5. Construct the optic disc / optic cup region discriminator and the optic disc / optic cup boundary discriminator; specifically including the following steps:

[0091] Constructing a discriminator for the visual disk and cup: The discriminator for the visual disk and cup consists of four concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional layer with a kernel size of 4×4 and a Leaky_relu activation function; the kernel size of the last convolutional layer is 4×4, and the number of output channels is 1; the input channels of the convolutional layer in the first convolutional block are 1, the input channels of the convolutional layer in the second convolutional block are 64, the input channels of the convolutional layer in the third convolutional block are 128, the input channels of the convolutional layer in the fourth convolutional block are 256, and the input channels of the last convolutional layer are 512.

[0092] Constructing a discriminator for the visual disk and visual cup region: The discriminator for the visual disk and visual cup region consists of four concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional layer with a kernel size of 4×4 and a Leaky_relu activation function; the kernel size of the last convolutional layer is 4×4, and the number of output channels is 1; the input channels of the convolutional layer in the first convolutional block are 2, the input channels of the convolutional layer in the second convolutional block are 64, the input channels of the convolutional layer in the third convolutional block are 128, the input channels of the convolutional layer in the fourth convolutional block are 256, and the input channels of the last convolutional layer are 512.

[0093] S6. The disc-cup region discriminator constructed in step S5 is used to perform region discrimination on the segmentation results obtained in steps S3 and S4, and the disc-cup boundary discriminator constructed in step S5 is used to perform boundary discrimination on the segmentation results obtained in steps S3 and S4; specifically, the following steps are included:

[0094] The disc-cup boundary discriminator constructed in step S5 performs boundary discrimination on the segmentation results obtained in steps S3 and S4: the disc-cup boundary discriminator classifies the source domain disc-cup boundary segmentation result as source neighborhood label 1, and the target domain disc-cup boundary segmentation result as target neighborhood label 0; the disc-cup boundary discriminator tries its best to correctly distinguish the input boundary segmentation results, and uses the following formula to calculate the cross-entropy loss between the input boundary prediction result and the neighborhood label.

[0095]

[0096] In the formula, N is the number of fundus images in the processed source domain fundus image dataset; X s This is the processed source domain fundus image dataset; The prediction results for the source domain view disk / view cup boundary; x s represents the fundus images in the processed source domain fundus image dataset; M represents the number of fundus images in the processed target domain fundus image dataset; x tX represents fundus images from the processed target domain fundus image dataset; t This is the processed target domain fundus image dataset; For the target domain, the prediction results of the view disk / view cup boundary; L D () represents the cross-entropy loss function;

[0097] The disc-cup region discriminator constructed in step S5 is used to perform region discrimination on the segmentation results obtained in steps S3 and S4: For the source domain disc-cup region segmentation results, the following formula is used... Perform pixel-based entropy processing, where E(x) s The result is the source domain entropy processing result. The source domain view disk / view cup region prediction results are used; the target domain view disk / view cup region segmentation results are obtained using the following formula. Perform pixel-based entropy processing, where E(x) t The result is the target domain entropy processing result. The target domain is the view disk / view cup region prediction result; the view disk / view cup region discriminator classifies the source domain view disk / view cup region segmentation result as source neighborhood label 1, and the target domain view disk / view cup region segmentation result as target neighborhood label 0; the view disk / view cup region discriminator tries its best to correctly distinguish the input region segmentation results, and uses the following formula to calculate the cross-entropy loss of the input region prediction result and neighborhood label.

[0098]

[0099] In the formula L D () represents the cross-entropy loss function;

[0100] While the region discriminator and boundary discriminator discriminate the segmentation results of the view disk and view cup region and boundary in the source domain and the view disk and view cup region and boundary in the target domain, the view disk and view cup segmentation network tries its best to reduce the difference in spatial layout between the segmentation results of the view disk and view cup region and boundary in the target domain and the segmentation results of the view disk and view cup region and boundary in the source domain. This makes the segmentation results of the view disk and view cup region and boundary in the target domain closer to the segmentation results of the view disk and view cup region and boundary in the source domain, making the two more similar. This deceives the discriminator, making it unable to distinguish whether the input segmentation result comes from the source domain or the target domain.

[0101] The disc-cup segmentation model aims to minimize the difference between the predicted disc-cup boundaries in the source and target domains, thereby preventing the discriminator from misclassifying the target domain disc-cup segmentation result as the source domain disc-cup segmentation result. The boundary adversarial loss is calculated using the following formula.

[0102]

[0103] In the formula, 1 represents the source domain label;

[0104] The disc-cup segmentation model aims to minimize the difference between the predicted disc-cup regions in the source and target domains, thereby preventing the discriminator from misclassifying the target domain disc-cup region segmentation result as the source domain segmentation result. The region adversarial loss is calculated using the following formula.

[0105]

[0106] In the formula, 1 represents the source domain label;

[0107] The boundary adversarial loss and region adversarial loss are used to constrain the disc-cup segmentation model, prompting the disc-cup segmentation model to continuously learn the common spatial structure information between the source domain and the target domain, thereby generating more accurate disc-cup region segmentation results and disc-cup boundary segmentation results.

[0108] S7. Meta-learning is employed to enhance the connections between the visual disc / glass segmentation model and the visual disc / glass region discriminator, and between the visual disc / glass segmentation model and the visual disc / glass boundary discriminator, thereby optimizing the visual disc / glass segmentation model. Meta-learning is used to align the gradient optimization directions of the visual disc / glass region and boundary segmentation task and the visual disc / glass region and boundary discrimination task, connecting the visual disc / glass region and boundary segmentation task and the region and boundary discrimination task, improving the discriminative ability of the region discriminator and boundary discriminator, and thus helping the visual disc / glass segmentation network learn more semantic knowledge, helping the visual disc / glass segmentation model achieve accurate visual disc / glass region and boundary segmentation results in the target domain. Specific implementation includes the following steps:

[0109] The boundary discrimination task corresponding to the disc / cup boundary discriminator and the region discrimination task corresponding to the disc / cup region discriminator are used as meta-training tasks, and the following formula is used as the loss function L for the meta-training tasks. dom Together, update the visual disc / visual cup segmentation model:

[0110]

[0111] In the formula The cross-entropy loss is calculated using the input boundary prediction results and the neighborhood labels. The cross-entropy loss is calculated between the input region prediction results and the neighborhood labels: θ represents the parameters of the shared encoder in the visual disc / visual cup segmentation model; φ d These are the parameters in the disc-cup region discriminator and the disc-cup boundary discriminator;

[0112] The image disc / image cup segmentation task and the image disc / image cup region segmentation task corresponding to the image disc / image cup segmentation model are used as meta-test tasks, and the following formula is used as the loss function L for the meta-test task. seg Together, update the visual disc / visual cup segmentation model:

[0113]

[0114] In the formula, λ is the countermeasure loss balance coefficient; Losses due to border confrontation; For regional combat losses; L e The mean square loss is the prediction result of the source domain view disk / view cup boundary and the corresponding label; L m The binary classification cross-entropy loss is calculated by combining the predicted source region (view disk / view cup) with the corresponding label; φ s These are parameters in the disc / cup boundary decoder and the disc / cup region decoder.

[0115] The meta-training task loss function L dom The loss function L for the summative test task seg By combining these methods, the overall loss function in the meta-learning process is obtained as follows:

[0116] The obtained overall loss function is then optimized to obtain the optimized overall loss function:

[0117]

[0118] In the formula, α is the meta-learning step size; To perform a derivative operation on the task loss;

[0119] Minimizing the optimized overall loss function using a first-order Taylor expansion requires not only the meta-training loss L... dom (θ,φ d The loss L of the meta-test seg (θ,φ s Minimize the gradient inner product between the two meta-tasks. Maximizing; from the perspective of the vector cosine theorem, the inner product of two vectors is maximized when their directions are similar; therefore, aligning the gradient directions of the two tasks to connect the two different tasks minimizes the overall meta-learning objective loss. Finally, the final overall loss function is obtained as follows:

[0120] S8. Validate the optimized disc-cup segmentation model obtained in step S7 to obtain the final disc-cup segmentation application model;

[0121] S9. Using the disc-cup segmentation application model obtained in step S8, perform actual disc-cup segmentation.

[0122] like Figure 3 The diagram shown is a schematic representation of the optic disc-optic cup segmentation model of the segmentation method of the present invention. The imaging method disclosed in this invention, which includes the aforementioned unsupervised domain adaptive optic disc-optic cup segmentation method, specifically includes the following steps:

[0123] (1) Obtain the original fundus image;

[0124] (2) The optic disc-optic cup segmentation method based on unsupervised domain adaptation is used to segment the fundus image obtained in step (1).

[0125] (3) Mark the optic disc and optic cup on the original fundus image using the optic disc and optic cup segmentation results obtained in step (2);

[0126] (4) Perform secondary imaging and output fundus images with optic disc and optic cup annotations.

[0127] The imaging method provided by this invention is applicable to existing fundus image acquisition devices. After acquiring the fundus image of the examinee, the existing fundus image acquisition device can use the imaging method provided by this invention to perform secondary imaging on the fundus image of the examinee, and then directly output a fundus image with optic disc and optic cup annotation results, which greatly facilitates examinees, researchers and medical personnel.

Claims

1. A spectral disc / spectral cup segmentation method based on unsupervised domain adaptation, comprising the following steps: S1. Obtain the source domain fundus image dataset and the target domain fundus image dataset, and process the obtained fundus images; S2. Construct the original model for visual disc / cup segmentation based on the U-net network; S3. The original optic disc-optic cup segmentation model constructed in step S2 is trained using the processed source domain fundus image dataset obtained in step S1 to obtain the corresponding source domain optic disc-optic cup segmentation result and the corresponding optic disc-optic cup segmentation model. S4. Using the optic disc-optic cup segmentation model obtained in step S3, segment the image data in the processed target domain fundus image dataset obtained in step S1 to obtain the target domain optic disc-optic cup segmentation result; S5. Construct a discriminator for the visual disk and the visual cup region, and a discriminator for the visual disk and the visual cup boundary; S6. The disc-cup region discriminator constructed in step S5 is used to perform region discrimination on the segmentation results obtained in steps S3 and S4, and the disc-cup boundary discriminator constructed in step S5 is used to perform boundary discrimination on the segmentation results obtained in steps S3 and S4. S7. Employ meta-learning to enhance the connections between the spectral disc / cup segmentation model and the spectral disc / cup region discriminator, and between the spectral disc / cup segmentation model and the spectral disc / cup boundary discriminator, thereby optimizing the spectral disc / cup segmentation model; specifically including the following steps: The boundary discrimination task corresponding to the disc / cup boundary discriminator and the region discrimination task corresponding to the disc / cup region discriminator are used as meta-training tasks, and the following formula is used as the loss function for the meta-training tasks. Together, update the visual disc / visual cup segmentation model: In the formula The cross-entropy loss is calculated using the input boundary prediction results and the neighborhood labels. The cross-entropy loss is calculated between the input region prediction result and the neighborhood label. These are the parameters of the shared encoder in the visual disc / visual cup segmentation model; These are the parameters in the disc-cup region discriminator and the disc-cup boundary discriminator; The image disc / image cup segmentation task and the image disc / image cup region segmentation task corresponding to the image disc / image cup segmentation model are used as meta-test tasks, and the following formula is used as the loss function of the meta-test tasks. Together, update the visual disc / visual cup segmentation model: In the formula To counteract the loss balance coefficient; Losses due to border confrontation; Losses due to regional confrontation; The mean square loss is the prediction result of the source domain visual disk / visual cup boundary and the corresponding label. The binary classification cross-entropy loss is used to calculate the prediction results of the source domain's visual disk / cup region and the corresponding labels. These are parameters in the disc / cup boundary decoder and the disc / cup region decoder. The meta-training task loss function The loss function of the sum-to-subject test task By combining these methods, the overall loss function in the meta-learning process is obtained as follows: ; The obtained overall loss function is then optimized to obtain the optimized overall loss function: In the formula The learning step size; To perform a derivative operation on the task loss; Minimizing the optimized overall loss function using a first-order Taylor expansion yields the final overall loss function as follows: ; S8. Validate the optimized disc-cup segmentation model obtained in step S7 to obtain the final disc-cup segmentation application model; S9. Using the disc-cup segmentation application model obtained in step S8, perform actual disc-cup segmentation.

2. The unsupervised domain adaptive spectral disc / cup segmentation method according to claim 1, characterized in that... Step S1, which involves acquiring the source domain fundus image dataset and the target domain fundus image dataset and processing the acquired fundus images, specifically includes the following steps: The fundus images in the source domain fundus image dataset and the target domain fundus image dataset are randomly cropped to a uniform size, then image enhancement is performed, and finally image normalization is performed. The image enhancement includes random flipping, random rotation, elastic transformation, brightness adjustment, and salt-and-pepper noise addition. The optic disc and optic cup labels corresponding to the fundus images in the source domain fundus image dataset are randomly cropped to a uniform size, then image enhancement is performed, and finally image normalization is performed. The image enhancement includes random flipping, random rotation, elastic transformation, brightness adjustment, and salt-and-pepper noise addition.

3. The unsupervised domain adaptive spectral disc / cup segmentation method according to claim 2, characterized in that... Step S2, which involves constructing the original model for visual disc / visual cup segmentation based on the U-net network, specifically includes the following steps: The original model for visual disk-visual cup segmentation includes a shared encoder, a visual disk-visual cup region decoder, and a visual disk-visual cup boundary decoder. The shared encoder includes a downsampling layer, an intermediate connection layer, and an upsampling layer. The downsampling layer comprises three convolutional blocks, each including a meta-learning convolutional layer, an `InstacnceNorm2d` normalization layer, and a `Leaky_relu` activation function. The meta-learning convolutional layer updates its gradient weights based on the meta-learning loss and meta-learning stride of the input model. The kernel size is [missing value]. ; The intermediate connection layer is used to connect the low-level features learned by the downsampling layer and the high-level features learned by the upsampling layer; The upsampling layer consists of three convolutional blocks, each including a meta-learning convolutional layer, an `InstacnceNorm2d` normalization layer, and a `Leaky_relu` activation function. The meta-learning convolutional layer updates its gradient weights based on the meta-learning loss and meta-learning stride of the input model. The kernel size is [size missing]. The upsampling layer is also used to form a feature map with contextual features from the features output by the intermediate connection layer; The aforementioned video disc / cup boundary decoder comprises two concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional layer, a batch normalization layer, a ReLU activation function, and a Dropout operation, and the kernel size in each convolutional layer is [missing information]. The kernel size in the last convolutional layer is The disc-cup boundary decoder is used to decode the shared features output by the shared encoder, thereby obtaining the disc-cup boundary segmentation result. The aforementioned visual disc / visual cup region decoder includes a batch normalization layer, a ReLU activation function, a Dropout operation, and a convolutional layer connected in series, wherein the kernel size of the convolutional layer is [missing information]. The disc-cup region decoder is used to concatenate the shared features output by the shared encoder and the disc-cup boundary segmentation results output by the disc-cup boundary decoder, and then decodes the concatenated features to obtain the disc-cup region segmentation results.

4. The unsupervised domain adaptive spectral disc / cup segmentation method according to claim 3, characterized in that... Step S3 involves training the original optic disc-optic cup segmentation model constructed in step S2 using the processed source domain fundus image dataset obtained in step S1 to obtain the corresponding source domain optic disc-optic cup segmentation result and the corresponding optic disc-optic cup segmentation model. Specifically, this includes the following steps: Fundus images from the processed source domain fundus image dataset The features learned by the shared encoder are input into the original disc-cup segmentation model constructed in step S2, thereby outputting the source domain disc-cup boundary prediction result. Then, the mean square loss of the source domain disc-cup boundary prediction result and the corresponding label is calculated using the following formula. : In the formula The number of fundus images in the processed source domain fundus image dataset; This is the processed source domain fundus image dataset; Labels for the boundaries of the source domain's view disk and view cup; The result is the prediction of the boundary of the source domain's visual disk / visual cup. The features output by the visual disk / visual cup boundary decoder and the features output by the shared encoder are concatenated and then input into the visual disk / visual cup region decoder to obtain the source domain visual disk / visual cup region prediction result. Then, the binary classification cross-entropy loss of the source domain visual disk / visual cup region prediction result and the corresponding label is calculated using the following formula. : In the formula Labels for the source domain view disk and view cup area; The prediction results for the source domain view disk and view cup regions; Calculate the overall loss for The comprehensive loss function was used to train the disc-cup segmentation model, and the corresponding source domain disc-cup segmentation results and the corresponding disc-cup segmentation model were obtained.

5. The unsupervised domain adaptive spectral disc / cup segmentation method according to claim 4, characterized in that... Step S4 involves using the optic disc-optic cup segmentation model obtained in step S3 to segment the optic disc and optic cup in the processed target domain fundus image dataset obtained in step S1, thereby obtaining the target domain optic disc-optic cup segmentation result. Specifically, this involves using the optic disc-optic cup segmentation model obtained in step S3 to segment the optic disc and optic cup in the processed target domain fundus image dataset obtained in step S1, thereby obtaining the target domain optic disc-optic cup boundary prediction result. and the corresponding target domain visual disc visual cup region prediction results ,in This represents fundus images in the target domain fundus image dataset.

6. The unsupervised domain adaptive spectral disc / cup segmentation method according to claim 5, characterized in that... Step S5, which involves constructing the optic disc-optic cup region discriminator and the optic disc-optic cup boundary discriminator, specifically includes the following steps: Constructing an optic disc / optic cup boundary discriminator: The optic disc / optic cup boundary discriminator consists of four concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional kernel of size [missing information]. The convolutional layers and Leaky_relu activation function; the kernel size of the last convolutional layer is... The number of output channels is 1; the number of input channels of the convolutional layer in the first convolutional block is 1, the number of input channels of the convolutional layer in the second convolutional block is 64, the number of input channels of the convolutional layer in the third convolutional block is 128, the number of input channels of the convolutional layer in the fourth convolutional block is 256, and the number of input channels of the last convolutional layer is 512. Constructing an optical disc / optical cup region discriminator: The optical disc / optical cup region discriminator consists of four concatenated convolutional blocks and one convolutional layer; each convolutional block includes a concatenated convolutional kernel of size [missing information]. The convolutional layers and Leaky_relu activation function; the kernel size of the last convolutional layer is... The number of output channels is 1; the number of input channels of the convolutional layer in the first convolutional block is 2, the number of input channels of the convolutional layer in the second convolutional block is 64, the number of input channels of the convolutional layer in the third convolutional block is 128, the number of input channels of the convolutional layer in the fourth convolutional block is 256, and the number of input channels of the last convolutional layer is 512.

7. The unsupervised domain adaptive spectral disc / cup segmentation method according to claim 6, characterized in that... Step S6 involves using the disc-cup region discriminator constructed in step S5 to perform region discrimination on the segmentation results obtained in steps S3 and S4, and using the disc-cup boundary discriminator constructed in step S5 to perform boundary discrimination on the segmentation results obtained in steps S3 and S4. Specifically, this includes the following steps: The disc-cup boundary discriminator constructed in step S5 performs boundary discrimination on the segmentation results obtained in steps S3 and S4: the disc-cup boundary discriminator classifies the source domain disc-cup boundary segmentation result as source neighborhood label 1, and the target domain disc-cup boundary segmentation result as target neighborhood label 0; the disc-cup boundary discriminator tries its best to correctly distinguish the input boundary segmentation results, and uses the following formula to calculate the cross-entropy loss between the input boundary prediction result and the neighborhood label. : In the formula The number of fundus images in the processed source domain fundus image dataset; This is the processed source domain fundus image dataset; The result is the prediction of the boundary of the source domain's visual disk / visual cup. These are fundus images from the processed source domain fundus image dataset; The number of fundus images in the processed target domain fundus image dataset; The fundus images in the processed target domain fundus image dataset; This is the processed target domain fundus image dataset; The target domain's visual disk / cup boundary prediction results; The cross-entropy loss function; The disc-cup region discriminator constructed in step S5 is used to perform region discrimination on the segmentation results obtained in steps S3 and S4: For the source domain disc-cup region segmentation results, the following formula is used... Perform pixel-based entropy processing, where This is the result of source domain entropy processing. The source domain view disk / view cup region prediction results are used; the target domain view disk / view cup region segmentation results are obtained using the following formula. Perform pixel-based entropy processing, where The result of processing the target domain entropy. The target domain is the view disk / view cup region prediction result; the view disk / view cup region discriminator classifies the source domain view disk / view cup region segmentation result as source neighborhood label 1, and the target domain view disk / view cup region segmentation result as target neighborhood label 0; the view disk / view cup region discriminator tries its best to correctly distinguish the input region segmentation results, and uses the following formula to calculate the cross-entropy loss of the input region prediction result and neighborhood label. : In the formula The cross-entropy loss function; The view disk / view cup segmentation model aims to minimize the difference between the predicted boundary results of the view disk / view cup in the source domain and the target domain, and calculates the boundary adversarial loss using the following formula. : In the formula For source domain tags; The view-disk / view-cup segmentation model aims to minimize the difference between the predicted view-disk / view-cup regions in the source and target domains, and calculates the region adversarial loss using the following formula. : In the formula For source domain tags; The boundary adversarial loss and region adversarial loss are used to constrain the disc-cup segmentation model, prompting the disc-cup segmentation model to continuously learn the common spatial structure information between the source domain and the target domain, thereby generating more accurate disc-cup region segmentation results and disc-cup boundary segmentation results.

8. An imaging method comprising the unsupervised domain adaptive optic disc-optic cup segmentation method according to any one of claims 1 to 7, characterized in that... Specifically, the steps include the following: (1) Obtain the original fundus image; (2) The optic disc-optic cup segmentation method based on unsupervised domain adaptation as described in any one of claims 1 to 7 is used to perform optic disc-optic cup segmentation on the fundus image obtained in step (1); (3) Mark the optic disc and optic cup on the original fundus image using the optic disc and optic cup segmentation results obtained in step (2); (4) Perform secondary imaging and output fundus images with optic disc and optic cup annotations.

Images