Multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method

Abstract

The invention provides a multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method. The method comprises the steps of 1, data acquisition and preprocessing; step 2, performing Laplacian pyramid decomposition; performing Gaussian decomposition; performing Laplace decomposition; performing two-layer Laplace dispersion on different images from the source domain and the target domain; 3, constructing a domain alignment network; constructing a global structure encoder and a detail enhancement encoder; respectively capturing global upper and lower information from a source domain and a target domain, and mapping the global upper and lower information to a shared vector space; a multi-scale feature fusion mechanism is introduced, and fusion output features are mapped to a shared vector space; the inter-domain distribution difference is minimized; step 4, constructing a U-Net automatic segmentation model; building a segmentation network; building a segmentation network model by adopting a U-Net model; model training; and step 5, carrying out cross-device automatic segmentation on the fundus color photo. According to the method, the segmentation precision of diversified eye fundus images can be remarkably improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of image and visual intelligent processing, and in particular relates to a multi-scale unified modeling method for intelligent segmentation of fluorescein fundus angiography images. Background Art

[0002] As the global population ages, ophthalmic diseases, especially retinopathy, have become an important factor affecting the health of the elderly. Ophthalmic diseases such as diabetic retinopathy (DR), retinal vein occlusion (RVO) and macular degeneration (AMD) usually manifest as structural changes in retinal blood vessels in the early stages, and these changes are crucial for early diagnosis and therapeutic intervention of the disease. As an important ophthalmic imaging technology, fluorescein fundus angiography (FFA) can clearly display the morphology and pathological conditions of retinal blood vessels through the blood flow path of sodium fluorescein. Therefore, it is widely used in clinical practice to monitor and evaluate these ophthalmic diseases.

[0003] In FFA images, accurate segmentation of retinal vessels is a key step for clinicians to analyze vascular structure, measure blood flow changes, and evaluate retinal lesions. Compared with traditional color fundus images, segmentation of FFA images is more difficult, especially in the lesion area, where problems such as blurred vascular boundaries, uneven brightness, and contrast agent leakage make the segmentation task more complicated. How to achieve robust cross-domain intelligent segmentation in diverse FFA images from different devices has become an important research topic in the field of fundus image analysis.

[0004] At present, most automatic segmentation methods for FFA images are based on deep learning technology, but their performance usually depends on high-quality, identically distributed annotated datasets. Traditional segmentation methods have achieved good results on high-quality images, but the performance is unstable in diverse FFA images across devices and scenarios. The main reasons are significant differences in image characteristics between devices, inconsistent image quality, unclear boundaries, and scarce annotated data. The diversity of sources of fluorescein angiography images brings about changes in image resolution, contrast, lighting conditions, and noise levels. For example, images from portable devices usually have lower resolution and poorer contrast, while high-end fixed devices (such as laser scanning fundus cameras) provide images with higher resolution, stronger contrast, and less image noise. This difference in the characteristics of multi-device images leads to inconsistent distribution of segmentation models in training data and actual use data, which in turn leads to unstable segmentation results in cross-device scenarios and reduced generalization performance of the model.

[0005] In actual clinical scenarios, FFA images may have motion blur, uneven lighting, background noise, overexposure, and underexposure due to the patient's eye movement, the diffusion rate of the contrast agent, and inaccurate focusing of the imaging device. The degradation of image quality will significantly affect the segmentation accuracy of the segmentation model, especially at the boundaries of retinal blood vessels. Blurred boundaries lead to misclassification and omission of blood vessels, especially in lesion areas or retinal lesions. The segmentation performance usually drops significantly. Traditional segmentation methods usually rely on high-quality images and clear boundary features, and it is difficult to cope with the segmentation challenges of low-quality images. In particular, in images with low light, noise interference, and irregular lesion areas, the performance of the model is easily affected, and it is impossible to achieve robust segmentation of complex scenes.

[0006] Pixel-level annotation of fundus images usually requires manual work by professional ophthalmologists, and pixel-level vascular annotation is a time-consuming, laborious and expensive task. Although there are some public FFA segmentation datasets, the annotation data of these datasets mostly come from a single device and a single imaging condition, which cannot fully cover the diverse data distribution in cross-device scenarios. When traditional segmentation models are applied to FFA images of new devices or unseen imaging conditions, the segmentation performance of the model tends to drop significantly. At the same time, the scarcity of cross-device annotation data further exacerbates the difficulty of training segmentation models, especially when it is necessary to process multiple image sources with different distributions. The lack of annotation data leads to a significant reduction in the training effect and generalization performance of the model. Even if a certain amount of annotation data is provided in some public datasets, these annotation data are usually from specific imaging devices and cannot fully cover all devices and scenarios, which leads to a shift in data distribution. When images collected by new devices are used for segmentation, the model has poor adaptability, resulting in a decrease in segmentation accuracy, especially the unclear boundaries of blood vessels, the misclassification of blood vessel branches and the misdetection of lesion locations.

[0007] Most of the existing FFA image segmentation methods use deep learning architectures based on convolutional neural networks such as U-Net and ResNet. Patent CN202110066944.9 proposes a fundus vascular segmentation method based on spatial and channel dual attention mechanisms. Although data amplification is achieved through image enhancement, random rotation, and size transformation, it relies on training from a single data source and cannot cope with the distribution offset problem between devices, and its generalization ability in multi-device scenarios is insufficient. Patent CN201910357697.0 generates fundus images through a deep convolutional generative adversarial network (GAN) to expand the data set on unseen devices, but the authenticity and diversity of the generated images are insufficient, and it is impossible to fully cover the feature distribution of the target domain, especially in scenarios with complex data distribution or variable device imaging conditions. Its segmentation performance is still limited. Patent CN202311805255.0 proposes a low-quality fundus image segmentation method based on adversarial domain adaptation. Although the adversarial domain adaptation method can realize feature migration from the source domain to the target domain, when the target domain data is scarce, the convergence and stability of the model are not high, and the segmentation performance is still significantly affected by the data distribution between devices.

[0008] In response to the above challenges, the present invention proposes a cross-domain intelligent segmentation method for fluorescein fundus angiography images based on multi-scale unified modeling to address the problems of multi-device imaging characteristics, inconsistent image quality, blurred boundaries, and scarce annotation data in FFA images. In order to solve the problem of inconsistent distribution of multi-source data, the present invention proposes a cross-domain segmentation strategy of multi-scale unified modeling. Through Laplace pyramid decomposition and multi-scale feature fusion mechanism, the high-frequency features (such as blood vessel edges) and low-frequency features (such as background brightness distribution) of FFA images are multi-scale fused in a unified modeling framework to ensure the uniformity of the model in high-resolution and low-resolution features. Multi-scale unified modeling not only enhances the model's ability to capture image details and overall structure, but also significantly improves the segmentation robustness of the model under various imaging conditions and different device data.

[0009] In order to enhance the adaptability to multi-device data, the present invention further proposes a cross-domain feature alignment strategy, adopts an adversarial training mechanism in the feature space of multi-source data, aligns the feature distribution of the source domain and the target domain through a feature aligner, and reduces the feature distribution differences between devices. In addition, in order to address the problem of scarce labeled data, the present invention adopts a semi-supervised and unsupervised cross-domain segmentation strategy, generates pseudo labels through unlabeled data, and combines consistency loss for unsupervised training, thereby significantly improving the segmentation performance of the target device in the absence of large-scale labeled data. Unlike existing adversarial domain adaptation methods, the present invention achieves more refined cross-domain alignment in the feature space by realizing multi-scale adversarial alignment on high-frequency details and low-frequency global features, thereby reducing the feature distribution differences between target domain devices and source domain devices.

[0010] The invention aims to solve the problem of multi-source, multi-device, low-quality FFA image segmentation through an innovative combination of multi-scale feature modeling, cross-domain feature alignment, and unsupervised / semi-supervised segmentation technology. The implementation of this method not only provides technical support for the efficient segmentation of multi-device FFA images, but also provides a more efficient and economical solution for the early screening and diagnosis of ophthalmic diseases, further promoting the popularization and application of FFA technology in clinical practice. Summary of the invention

[0011] In view of the problems existing in the above-mentioned prior art, the present invention provides a multi-scale unified modeling intelligent segmentation method for fluorescein fundus angiography images, which aims to improve the segmentation performance of the model for FFA images from different sources and under diverse imaging conditions, especially when processing cross-domain data and unseen image data. It can achieve high-precision segmentation of lesion areas with limited annotated data, thereby providing a more robust and universal segmentation solution.

[0012] In order to achieve the above-mentioned object of the invention, the present invention provides a multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method, comprising the following steps:

[0013] Step 1: Data collection and preprocessing;

[0014] S11: Collect fluorescein fundus angiography image data;

[0015] Two types of fluorescein fundus angiography image data are collected, the two types of fluorescein fundus angiography image data are from different environments and different devices, wherein one type of high-quality fluorescein fundus angiography image data is used as a data source X, and another type of low-quality fluorescein fundus angiography image data is used as a data source Y;

[0016] S12: data preprocessing;

[0017] Perform expert-level segmentation and annotation processing on images from data source X; perform data preprocessing on images from data source Y to improve image quality before network training;

[0018] Step 2: Perform Laplace pyramid decomposition;

[0019] S21: Gaussian decomposition; given an input image I, a series of images with decreasing resolution d are generated by applying Gaussian filtering and downsampling layer by layer to form a Gaussian pyramid; wherein, according to formula (1), by decomposing the previous layer G i-1 Perform Gaussian filtering and then downsample to obtain the Gaussian pyramid G of the i-th layer i ;

[0020] G i =D(F(G i-1 )) (1);

[0021] Where D represents downsampling; F(·) represents the Gaussian filter function, Among them, w(i,j) is the Gaussian kernel, (x, y) is the pixel coordinate in the image, i and j are the offsets of the Gaussian kernel, and σ is the standard deviation of the Gaussian kernel, which is used to control the degree of smoothing;

[0022] S22: Laplace decomposition; each level L of the Laplace pyramid i It is composed of the difference between two adjacent layers of the Gaussian pyramid, which is used to capture the high-frequency and low-frequency information of the image. According to formula (2), through the current layer G i And its downsampled and upsampled image G i+1 The difference between them is the i-th layer Laplacian image L i ;

[0023] L i =G i -U(G i+1 ) (2);

[0024] Where U represents upsampling;

[0025] S23: For different images I from the source domain and the target domain A ,I B , two layers of Laplace dispersion are performed according to the method in S21 and S22. The first layer is used to capture the subtle details of the image, that is, high-frequency information, and the second layer is used to capture the larger-scale structural information of the image, that is, low-frequency information. The decomposition results are recorded as in, Represents image I A ,I BThe i-th layer pyramid image, i = 1, 2;

[0026] Step 3: Build a domain alignment network;

[0027] S31: construct a global structure encoder f1 and a detail enhancement encoder f2;

[0028] S32: Using output feature F A 、F B Capture the global up-down information from the source domain and the target domain respectively and map them to F A 、F B Shared vector space In the image I A The second level pyramid image As the input of the global structure encoder f1, and according to formula (3) The output feature F after passing through the global structure encoder f1 A , where F A Size Image I B The second level pyramid image As the input of the global structure encoder f1, and according to formula (4) The output feature F after passing through the global structure encoder f1 B , where F B Size

[0029]

[0030] Where f1(·) represents the mapping function of the global structure encoder f1;

[0031] S33: Introduce a multi-scale feature fusion mechanism and transform the fused output feature S A , S B Mapping to S A , S B In the shared vector space R2, use formula (5) to first transform F A and Perform feature fusion and then send it to the detail enhancement encoder f2 to obtain the fused output feature S A , whose size is H×W; using formula (6) first B and Perform feature fusion and then send it to the detail enhancement encoder f2 to obtain the fused output feature S B , whose size is H×W;

[0032]

[0033] Where f2(·) represents the mapping function of the detail enhancement encoder f2, and Up represents the transposed convolution;

[0034] S34: Minimize the distribution difference between domains; minimize the distribution difference between the two domains according to formulas (7) and (8) respectively, so that F A 、F B Closer to the same shared vector space R1, while making S A , S B Closer to the same shared vector space R2;

[0035] Then, the structural similarities and differences between the source domain and the target domain are captured through mapping to achieve cross-domain feature alignment;

[0036]

[0037] Where D div (·,·) is some measure of divergence between feature distributions;

[0038] Step 4: Build a U-Net automatic segmentation model;

[0039] S41: Build a segmentation network; Use the U-Net model to build a segmentation network model;

[0040] S42: Model training; using the feature map S of the source domain A As input data, the final output corresponds to S A The segmentation prediction result is

[0041] S42-1: Construct a discriminator; construct two discriminators D1 and D2. The network structures of the two discriminators D1 and D2 are composed of multiple convolutions, pooling and jump connections; the goal of the discriminator D1 is to distinguish the input F A 、F B The features of come from the source domain or the target domain; the goal of the discriminator D2 is to distinguish the input S A , S B The feature comes from the source domain or the target domain; for the input feature x, the loss function of the discriminator D1 is obtained according to formula (9): According to formula (10), the loss function of discriminator D2 is obtained To distinguish the features of the source domain and the target domain by minimizing the loss area;

[0042]

[0043] S42-2: Construct domain adversarial loss; for the input feature x, the adversarial loss function of the global structure encoder f1 is obtained according to formula (11) According to formula (12), the adversarial loss function of the detail enhancement encoder f2 is obtained Using adversarial loss function To ensure that the discriminators D1 and D2 can correctly distinguish the sources of generated features; through joint training, in the optimization At the same time, f1 is also adjusted. At the same time, the distribution difference between the two domains is further minimized through domain adversarial loss, so that F A 、F B Closer to the same shared vector space R1, while making S A , S B Closer to the same shared vector space R2;

[0044]

[0045] S42-3: Construct feature reconstruction loss; calculate the difference between the output of the domain alignment network and the original input image, and quantify this difference through L1 loss. Specifically, the reconstruction loss is obtained according to formula (13):

[0046]

[0047] Where N represents the batch size;

[0048] S42-4: Construct segmentation loss; select cross entropy loss as the loss function of the segmentation task to effectively measure the difference between the predicted segmentation map output by the segmentation network model and the labeled true value. Specifically, the segmentation loss is obtained according to formula (14):

[0049] In the formula, The predicted value of the i-th pixel; represents the true label of the i-th pixel;

[0050] S42-5: Joint fine-tuning; the total loss of the segmentation network is obtained according to formula (15)

[0051]

[0052] In the formula, λ1, λ2, λ rec , seg are weight parameters, which satisfy λ1+λ2+λ rec +λ seg =1, used to adjust the contribution of different loss items to the total loss.

[0053] S42-6: U-Net automatic segmentation model is obtained through training;

[0054] Step 5: Automatically segment the fundus color photos across devices;

[0055] The fundus color image data collected in real time is input into the U-Net automatic segmentation model as input data, and high-precision segmented fundus image data is obtained after automatic segmentation by the U-Net automatic segmentation model.

[0056] As a preferred embodiment, in step 1 S11, one type of high-quality fluorescein fundus angiography image data is acquired by high-end professional fundus color photography equipment, and another type of low-quality fluorescein fundus angiography image data is acquired by non-professional fundus color photography equipment.

[0057] As a preferred embodiment, in step 1 S12, the process of preprocessing the image includes removing fundus reflection, histogram equalization, denoising and color standardization.

[0058] As a preference, in step three S31, a typical U-shaped network structure is selected to construct a global structure encoder f1.

[0059] As a preference, in step 1 S12, the denoising method includes median filtering, Gaussian filtering and wavelet denoising.

[0060] The core technical solution of the present invention includes multiple innovative steps, aiming to effectively improve the robustness and cross-domain generalization performance of segmentation by comprehensively utilizing data preprocessing, Laplacian pyramid decomposition, multi-scale feature fusion, adversarial domain adaptation (ADA) and deep segmentation networks (such as UNet). The specific process is as follows:

[0061] First, in terms of data acquisition and preprocessing, the present invention proposes an image preprocessing strategy for the complexity of multi-source and multi-domain FFA images. For low-quality FFA images, the image quality is improved by removing fundus reflections, histogram equalization, denoising and color standardization in the image, ensuring its effectiveness in subsequent processing. Especially in low-quality images and cross-dataset scenarios, these processing steps can significantly improve the usability of the image and lay a solid foundation for subsequent segmentation tasks. Then, the Laplace pyramid decomposition technology is used to decompose the input FFA image into details and structural information at multiple scales, which are used to extract high-frequency detail features and low-frequency structural features respectively. This process ensures that the key features in the image can be retained during image segmentation, especially under low-quality and fuzzy boundary image conditions, the boundary and texture information of retinal blood vessels can still be effectively extracted.

[0062] In terms of feature extraction and cross-domain feature alignment, the present invention introduces an Adversarial Domain Adaptation (ADA) network, which can capture the structural similarities and feature differences of FFA images from different sources by constructing a global structure encoder and a detail enhancement encoder. By aligning the features of FFA images from different sources, the domain alignment network can achieve efficient alignment of features, thereby improving the segmentation accuracy of the model in different data domains and multi-source data. The design of this adversarial network is particularly suitable for solving the problem of cross-domain distribution shift (Domain Shift), that is, the differences in image characteristics, brightness, contrast and resolution of FFA images from different sources can be aligned in feature space through adversarial training, thereby enhancing the generalization ability of the model.

[0063] In terms of segmentation network design, the present invention adopts the classic UNet segmentation model. UNet has a strong ability to capture details in medical image segmentation, especially in the segmentation tasks of complex anatomical structures such as the retina. It can accurately capture the detailed information of the image and perform well in high-precision segmentation of lesion areas. Unlike the traditional UNet model, the present invention innovatively adopts the joint optimization method of multi-scale feature fusion and domain alignment network, which significantly enhances the adaptability of UNet in multi-domain scenarios. During the training process, the joint optimization strategy of adversarial training and consistency loss is adopted to ensure that the features in different data domains can be aligned in the feature space, thereby greatly improving the segmentation accuracy in low-quality images and unseen data environments. Unlike traditional segmentation methods, the joint optimization method of adversarial loss and consistency loss can significantly reduce the data distribution differences between the source domain and the target domain, and can still achieve efficient and accurate segmentation under the conditions of unlabeled and scarce labeled data.

[0064] Compared with the prior art, the technical solution of the present invention has significant advantages. First, the present invention designs a network architecture of multi-scale feature fusion and adversarial domain adaptation, and decomposes the input image into multiple layers through Laplacian pyramid decomposition. The first layer is used to capture fine-grained image details (such as tiny blood vessels and capillary networks in the retina), and the second layer extracts larger-scale structural information (such as the main blood vessel trunks and overall anatomical structure of the retina). The combination of this multi-scale decomposition analysis method and a deep convolutional network enables the present invention to achieve accurate segmentation in complex FFA image scenes.

[0065] In addition, the present invention introduces adversarial loss and reconstruction loss. The introduction of adversarial loss enables the features of the source domain and the target domain to be mapped to the same feature space, reducing the distribution differences between image features in different domains. Through adversarial training between the source domain and the target domain, it is ensured that the segmentation network still has high stability and robustness in image segmentation of unseen data and unseen domains. The introduction of reconstruction loss ensures that the image features can still retain the key information in the original image after Laplace decomposition and adversarial training, avoiding the loss of image information, thereby improving the stability and robustness of the segmentation model.

[0066] By training and validating on multiple small sample data sets, the segmentation model of the present invention has achieved excellent performance on multiple classic retinal vascular segmentation data sets. Experimental results show that the present invention can significantly improve the generalization ability of the model when processing FFA images from different sources, especially under the conditions of unseen data, cross-domain data and low-quality data, the segmentation accuracy is greatly improved. In addition, in the scenario of unlabeled and scarce labeled data, the segmentation performance of the present invention remains stable, showing efficient cross-domain segmentation adaptability.

[0067] In summary, the cross-domain intelligent segmentation method of fluorescein fundus angiography images based on multi-scale feature fusion proposed in the present invention can effectively solve the segmentation problems of cross-domain data distribution differences, data scarcity and unseen data. This method can not only achieve high-precision segmentation in portable devices, desktop devices and multi-source data acquisition environments, but also maintain high robustness and generalization in unseen data and cross-domain data scenarios. Its technical solution has broad application prospects in the field of medical image segmentation, especially in the cross-domain segmentation task of FFA images.

[0068] Compared with the prior art, the present invention has the following advantages:

[0069] 1) Cross-domain segmentation of unlabeled data: This method designs a segmentation network structure with multi-scale feature fusion. It decomposes the image through the Laplacian pyramid to separate the image details from the structural information. The first layer processes fine-grained image details, and the second layer extracts larger-scale structural information. This multi-scale feature extraction method can improve the segmentation accuracy of low-quality images, especially in the case of unlabeled data, and can still achieve efficient image segmentation.

[0070] 2) Adversarial training and feature alignment: Through adversarial training, the image features of the source domain and the target domain are mapped to the same feature space, reducing the distribution shift between domains. Compared with traditional methods, this method effectively improves the accuracy and robustness of cross-domain segmentation and avoids the performance degradation caused by data distribution shift. In addition, the reconstruction loss ensures the stability of feature retention and information reconstruction, further enhancing the robustness of the features.

[0071] 3) Efficient segmentation performance in unlabeled small sample environment: After verification on multiple small sample retinal segmentation datasets, the experimental results show that the present invention exhibits excellent generalization ability and robustness when processing unseen data, cross-domain data and diversified datasets, especially in the unlabeled data and cross-domain data segmentation tasks, the performance is excellent.

[0072] These advantages make the present invention have a wide range of application potentials, especially in the medical image segmentation tasks of multi-source data, unseen data and low-labeled data. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] Figure 1 It is a network architecture diagram in the present invention;

[0074] Figure 2 It is a data preprocessing flow chart of the present invention;

[0075] Figure 3 It is a structural diagram of the U-Net automatic segmentation model in the present invention. DETAILED DESCRIPTION

[0076] The purpose of the present invention is to provide a technology for automatically segmenting the region of interest of fluorescein fundus angiography images in a cross-domain environment by artificial intelligence, and to make the accuracy of segmentation as high as possible, the present invention will be explained by taking the segmentation of fundus blood vessels as an example. The present invention provides a cross-domain intelligent segmentation method for fluorescein fundus angiography images based on multi-scale feature fusion. First, data acquisition and preprocessing are performed, including removing fundus reflections, histogram equalization, denoising and color standardization to improve image quality. Then, the image is decomposed into high and low resolutions by Laplacian pyramid and sent to the global structure encoder and detail enhancement encoder. Then, the extracted features are sent to the U-Net automatic segmentation model, and joint training is performed to finally achieve high-precision fundus blood vessel segmentation. This method has important application value in the field of medical image analysis.

[0077] In order to more clearly illustrate the objectives, technical solutions and advantages of the present invention, the present invention will be further described comprehensively and clearly with reference to the accompanying drawings and implementation steps.

[0078] like Figures 1 to 3 As shown, the present invention provides a multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method, comprising the following steps:

[0079] Step 1: Data collection and preprocessing;

[0080] S11: Collect fluorescein fundus angiography image data;

[0081] Two types of fluorescein fundus angiography image data are collected, the two types of fluorescein fundus angiography image data are from different environments and different devices, wherein one type of high-quality fluorescein fundus angiography image data is used as a data source X, and another type of low-quality fluorescein fundus angiography image data is used as a data source Y;

[0082] S12: data preprocessing;

[0083] Perform expert-level segmentation and annotation processing on images from data source X; perform data preprocessing on images from data source Y to improve image quality before network training;

[0084] Step 2: Perform Laplace pyramid decomposition;

[0085] The main reason for performing Laplacian pyramid decomposition on both the data from data source X and the data from data source Y is to effectively capture the multi-scale features of the image, so that details and structural information can be processed at different resolutions. This decomposition method effectively separates high-frequency details (such as edges and textures) from low-frequency structures (such as overall shape), ensuring that the model can learn local subtle changes and global morphological features at the same time. This is particularly important in complex segmentation tasks, especially when dealing with images with exposure errors, loss of details, or cross-domain differences. The multi-layer feature representation obtained from the decomposition enhances the network's ability to restore image details and maintain structural integrity.

[0086] S21: Gaussian decomposition; given an input image I, a series of images with decreasing resolution d are generated by applying Gaussian filtering and downsampling layer by layer to form a Gaussian pyramid; wherein, according to formula (1), by decomposing the previous layer G i-1 Perform Gaussian filtering and then downsample to obtain the Gaussian pyramid G of the i-th layer i ;

[0087] G i =D(F(G i-1 )) (1);

[0088] Where D represents downsampling, which usually involves sampling every other pixel to reduce the resolution; F(·) represents the Gaussian filter function, Among them, w(i,j) is the Gaussian kernel,

[0089] (x, y) is the pixel coordinate in the image, i and j are the offsets of the Gaussian kernel, and σ is the standard deviation of the Gaussian kernel, which is used to control the degree of smoothing;

[0090] S22: Laplace decomposition; each level L of the Laplace pyramid iIt is composed of the difference between two adjacent layers of the Gaussian pyramid, which is used to capture the high-frequency and low-frequency information of the image. According to formula (2), through the current layer G i And its downsampled and upsampled image G i+1 The difference between them is the i-th layer Laplacian image L i ;

[0091] L i =G i -U(G i+1 ) (2);

[0092] Where U represents upsampling, which is usually achieved by nearest neighbor interpolation or bilinear interpolation;

[0093] S23: For different images I from the source domain and the target domain A ,I B , both perform two-layer Laplace dispersion in the same manner as in S21 and S22. The first layer is used to capture the subtle details of the image, that is, high-frequency information, such as small-scale textures and local contrast enhancement in the image. These details are crucial for fine segmentation tasks, especially when dealing with complex edge or texture areas; the second layer is used to capture the larger-scale structural information of the image, that is, low-frequency information, such as overall contours and illumination changes, to help the model better understand the global shape characteristics;

[0094] The decomposition results are recorded as in, Represents image I A ,I B The i-th layer pyramid image, i = 1, 2;

[0095] Step 3: Build a domain alignment network;

[0096] S31: construct a global structure encoder f1 and a detail enhancement encoder f2;

[0097] S32: Using output feature F A 、F B Capture the global up-down information from the source domain and the target domain respectively and map them to F A 、F B Shared vector space In the image I A The second level pyramid image As the input of the global structure encoder f1, and according to formula (3) The output feature F after passing through the global structure encoder f1 A , where F a Size Image I BThe second level pyramid image As the input of the global structure encoder f1, and according to formula (4) The output feature F after passing through the global structure encoder f1 B , where F B Size

[0098]

[0099] Where f1(·) represents the mapping function of the global structure encoder f1;

[0100] S33: Introduce a multi-scale feature fusion mechanism and transform the fused output feature S A , S B Mapping to f A , S B In the shared vector space R2, use formula (5) to first transform F A and Perform feature fusion and then send it to the detail enhancement encoder f2 to obtain the fused output feature S A , whose size is H×W; using formula (6) first B and Perform feature fusion and then send it to the detail enhancement encoder f2 to obtain the fused output feature S B , whose size is H×W;

[0101]

[0102] Where f2(·) represents the mapping function of the detail enhancement encoder f2, and Up represents the transposed convolution;

[0103] S34: Minimize the distribution difference between domains; minimize the distribution difference between the two domains according to formulas (7) and (8) respectively, so that F A 、F B Closer to the same shared vector space R1, while making S A , S B Closer to the same shared vector space R2; then the structural similarities and differences between the source domain and the target domain are captured through mapping to achieve cross-domain feature alignment;

[0104]

[0105] Where D div (·,·) is some divergence measure between feature distributions, such as the Jensen-Shannon divergence;

[0106] Step 4: Build a U-Net automatic segmentation model;

[0107] S41: Build a segmentation network; Use the U-Net model to build a segmentation network model;

[0108] S42: Model training; using the feature map S of the source domain A As input data, the final output corresponds to S A The segmentation prediction result is

[0109] S42-1: Construct a discriminator; construct two discriminators D1 and D2. The network structures of the two discriminators D1 and D2 are composed of multiple convolutions, pooling and jump connections; the goal of the discriminator D1 is to distinguish the input F A 、F B The features of come from the source domain or the target domain; the goal of the discriminator D2 is to distinguish the input S A , S B The feature comes from the source domain or the target domain; for the input feature x, the loss function of the discriminator D1 is obtained according to formula (9): According to formula (10), the loss function of discriminator D2 is obtained To distinguish the features of the source domain and the target domain by minimizing the loss area;

[0110]

[0111] S42-2: Construct domain adversarial loss; The goal of the global structure encoder f1 and detail enhancement encoder f2 of the domain alignment network is to confuse the discriminator D1, D2 so that it cannot correctly distinguish the source of the generated features. Therefore, for the input feature x, the adversarial loss function of the global structure encoder f1 is obtained according to formula (11): According to formula (12), the adversarial loss function of the detail enhancement encoder f2 is obtained Using adversarial loss function To ensure that the discriminators D1 and D2 can correctly distinguish the sources of generated features; through joint training, in the optimization At the same time, f1 is also adjusted. At the same time, the distribution difference between the two domains is further minimized through domain adversarial loss, so that F A 、F B Closer to the same shared vector space R1, while making S A , S B Closer to the same shared vector space R2;

[0112]

[0113] S42-3: Construct feature reconstruction loss; feature reconstruction loss is introduced to ensure that the feature extraction network can effectively retain the key information of the original image. Its main goal is to minimize the difference between the feature representation generated by the network and the original image, while ensuring cross-domain feature alignment and calibrating the accuracy of feature extraction without losing important details and structural information. The difference between the output of the domain alignment network and the original input image is calculated, and the difference is quantified by L1 loss. Specifically, the reconstruction loss is obtained according to formula (13): This loss is used to ensure that the extracted features are not only aligned between domains, but also retain the original information of the input image as much as possible;

[0114]

[0115] Where N represents the batch size;

[0116] S42-4: Construct segmentation loss; segmentation loss is used to evaluate the segmentation performance of the model on the source domain image. The cross entropy loss is selected as the loss function of the segmentation task to effectively measure the difference between the predicted segmentation map output by the segmentation network model and the labeled true value. Specifically, the segmentation loss is obtained according to formula (14): Cross entropy loss is a commonly used loss function in segmentation tasks, especially for pixel-level classification problems. It calculates the loss value by comparing the predicted category probability of each pixel with the matching degree of the actual category.

[0117]

[0118] In the formula, The predicted value of the i-th pixel; represents the true label of the i-th pixel;

[0119] S42-5: Joint fine-tuning; the total loss of the segmentation network is obtained according to formula (15)

[0120]

[0121] In the formula, λ1, λ2, λ rec , seg are weight parameters, which satisfy λ1+λ2+λ rec +λ seg =1, used to adjust the contribution of different loss items to the total loss.

[0122] S42-6: U-Net automatic segmentation model is obtained through training;

[0123] Step 5: Automatically segment the fundus color photos across devices;

[0124] The fundus color image data collected in real time is input into the U-Net automatic segmentation model as input data, and high-precision segmented fundus image data is obtained after automatic segmentation by the U-Net automatic segmentation model.

[0125] As a preferred embodiment, in step 1 S11, one type of high-quality fluorescein fundus angiography image data is acquired by high-end professional fundus color photography equipment, and another type of low-quality fluorescein fundus angiography image data is acquired by non-professional fundus color photography equipment.

[0126] As a preference, in step one S12, the process of preprocessing the image includes removing fundus reflections, histogram equalization, denoising and color standardization. Specifically, removing fundus reflections: the optic disc (central part of the fundus) in fluorescein fundus angiography images usually introduces large brightness changes, and sometimes it is necessary to remove or reduce the impact of this area in order to better analyze other parts of the retina. Histogram equalization: improve the contrast of the image. Apply histogram equalization (especially to the brightness channel) to improve the global contrast of the image. Denoising: fluorescein fundus angiography images may be interfered by various noises, such as light noise, artifacts, etc. Color standardization: The color and brightness of fundus images may vary due to different shooting conditions, so color standardization is required to keep the color and brightness consistent between different images.

[0127] As a preference, in step three S31, a typical U-shaped network structure is selected to construct a global structure encoder f1.

[0128] As a preference, in step 1 S12, the denoising method includes median filtering, Gaussian filtering and wavelet denoising.

[0129] In summary, the present invention proposes an artificial intelligence automatic segmentation method for fluorescein fundus angiography images in a cross-domain environment, aiming to solve the problem of fundus image segmentation methods in the absence of equipment, domain and annotation, especially when there are significant differences between different imaging devices, how to improve the accuracy and robustness of image segmentation. This method fully combines multi-scale decomposition with adversarial training technology to construct a powerful framework for handling the complexity of fundus images. Specifically, the method first uses the Laplace pyramid decomposition technology to process fluorescein fundus angiography images taken from different imaging devices. Laplace pyramid decomposition can effectively extract high-frequency and low-frequency information of the image, help capture important details and structural features in the image, and provide a solid foundation for the subsequent accurate segmentation of the image. This preprocessing process can significantly improve the quality of the image, making it more suitable for segmentation and lesion area detection.

[0130] Next, the adversarial training strategy was adopted to further enhance the performance of the method in cross-domain scenarios. Through adversarial training, the model can learn how to map images from the source domain and the target domain to the same vector space, effectively reducing the distribution differences between different image domains. Such a training process not only improves the adaptability of the model between different domains, but also enhances the accuracy and robustness of image segmentation. Adversarial training optimizes the adversarial structure of the generator and the discriminator, making the feature distribution between the source domain and the target domain tend to be consistent, thus laying the foundation for subsequent accurate image segmentation.

[0131] In the segmentation stage, the present invention further adopts the classic and effective U-Net model. U-Net has been widely used in medical image segmentation due to its unique symmetrical structure and skip connection mechanism, and is particularly suitable for fundus image segmentation tasks that require precise positioning of subtle features. The U-Net model can effectively capture the key features in the image through its encoder-decoder structure, and maintain high-resolution detail information through skip connections, thereby greatly improving the segmentation accuracy of the fundus lesion area. Through this deep learning model, images processed by adversarial training can obtain more accurate segmentation results, which can provide doctors with clearer lesion area information during the diagnosis process, which is crucial for efficient and accurate diagnosis of fundus lesions.

[0132] The significant innovation of the present invention lies in the deep learning image segmentation technology that combines multi-scale feature fusion and adversarial training, which provides an accurate and efficient solution for the intelligent segmentation of fluorescein fundus angiography images. Through multi-scale feature fusion, the present invention introduces the Laplace pyramid decomposition method to comprehensively extract the multi-level features of the fundus image, and combines the global structural information with the local detail features, thereby ensuring that the model can fully capture the key feature areas in the image in the segmentation task. At the same time, through the adversarial training mechanism, a global structure encoder and a detail enhancement encoder are constructed to coordinately optimize the feature expression ability and segmentation performance, so that the model can learn image characteristics more accurately, and significantly improve the segmentation effect and accuracy of the model for complex structures and boundary details. Compared with traditional methods, the present invention realizes the high adaptability of the model to a variety of imaging conditions through feature space alignment, without relying on additional data amplification or a simple generative method, it can maintain a highly stable segmentation accuracy in fluorescein fundus angiography images taken by different imaging devices. This design not only effectively solves the limitations of different devices and imaging conditions on segmentation performance, but also enhances the universality and robustness of the model by optimizing the training strategy. The present invention provides a reliable and efficient technical means for the diagnosis and screening of fundus diseases. While improving the efficiency of clinical applications, it also lays a solid foundation for the development of intelligent ophthalmic medical systems.

[0133] Furthermore, this segmentation method is of great significance for the application of ophthalmologists in their daily work, especially in the early diagnosis and monitoring of fundus diseases. Common fundus diseases such as diabetic retinopathy and macular degeneration, if they can be accurately identified in the early stages, will greatly improve the success rate of treatment and the quality of life of patients. Through the intelligent segmentation technology provided by the present invention, doctors can detect the lesion area more quickly and accurately, intervene in time and formulate personalized treatment plans, which has significant advantages in clinical practice.

[0134] Finally, the technical innovation of the method of the present invention has expanded its scope of application. In addition to being suitable for high-end fundus imaging equipment in large medical institutions, the present invention can also adapt to ordinary fundus imaging equipment used in primary medical and health institutions, and even portable devices. This makes the technology have broad application prospects in areas with limited resources, especially in developing areas. It can provide accurate fundus disease detection and analysis services for patients in these areas, reduce the impact of uneven distribution of medical resources, and have great social and economic benefits. Therefore, the fluorescein fundus angiography image segmentation method based on adversarial domain adaptation of the present invention is not only technically innovative, but also has important practical value in clinical applications, and can provide more efficient and accurate medical services to a wider range of patient groups.

Claims

1. A multi-scale unified modeling method for intelligent segmentation of fluorescein fundus angiography images, characterized in that: The following steps are involved: Step 1: Data collection and preprocessing; S11: Collect fluorescein fundus angiography image data; Two types of fluorescein fundus angiography image data are collected, the two types of fluorescein fundus angiography image data are from different environments and different devices, wherein one type of high-quality fluorescein fundus angiography image data is used as a data source X, and another type of low-quality fluorescein fundus angiography image data is used as a data source Y; S12: data preprocessing; Perform expert-level segmentation and annotation processing on images from data source X; perform data preprocessing on images from data source Y to improve image quality before network training; Step 2: Perform Laplace pyramid decomposition; S21: Gaussian decomposition; given an input image I, a series of images with decreasing resolution d are generated by applying Gaussian filtering and downsampling layer by layer to form a Gaussian pyramid; wherein, according to formula (1), by decomposing the previous layer G i-1 Perform Gaussian filtering and then downsample to obtain the Gaussian pyramid G of the i-th layer i ; G i =D(F(G i-1 )) (1); Where D represents downsampling; F(·) represents the Gaussian filter function, Among them, w(i,j) is the Gaussian kernel, (x, y) is the pixel coordinate in the image, i and j are the offsets of the Gaussian kernel, and σ is the standard deviation of the Gaussian kernel, which is used to control the degree of smoothing; S22: Laplace decomposition; each level L of the Laplace pyramid i It is composed of the difference between two adjacent layers of the Gaussian pyramid, which is used to capture the high-frequency and low-frequency information of the image. According to formula (2), through the current layer G i And its downsampled and upsampled image G i+1 The difference between them is the i-th layer Laplacian image L i ; L i =G i -U(G i+1 ) (2); Where U represents upsampling; S23: For different images I from the source domain and the target domain A ,I B , two layers of Laplace dispersion are performed according to the method in S21 and S22. The first layer is used to capture the subtle details of the image, that is, high-frequency information, and the second layer is used to capture the larger-scale structural information of the image, that is, low-frequency information. The decomposition results are recorded as in, Represents image I A ,I B The i-th layer pyramid image, i = 1, 2; Step 3: Build a domain alignment network; S31: construct a global structure encoder f1 and a detail enhancement encoder f2; S32: Using output feature F A 、F B Capture the global up-down information from the source domain and the target domain respectively and map them to F A 、F B In the shared vector space R1, image I A The second level pyramid image As the input of the global structure encoder f1, and according to formula (3) The output feature F after passing through the global structure encoder f1 A , where F A Size Image I B The second level pyramid image As the input of the global structure encoder f1, and according to formula (4) The output feature F after passing through the global structure encoder f1 B , where F B Size Where f1(·) represents the mapping function of the global structure encoder f1; S33: Introduce a multi-scale feature fusion mechanism and transform the fused output feature S A , S B Mapping to S A , S B In the shared vector space R2, use formula (5) to first transform F A and Perform feature fusion and then send it to the detail enhancement encoder f2 to obtain the fused output feature S A , whose size is H×W; using formula (6) first B and Perform feature fusion and then send it to the detail enhancement encoder f2 to obtain the fused output feature S B , whose size is H×W; Where f2(·) represents the mapping function of the detail enhancement encoder f2, and Up represents the transposed convolution; S34: Minimize the distribution difference between domains; minimize the distribution difference between the two domains according to formulas (7) and (8) respectively, so that F A 、F B Closer to the same shared vector space R1, while making S A , S B Closer to the same shared vector space R2; Then, the structural similarities and differences between the source domain and the target domain are captured through mapping to achieve cross-domain feature alignment; Where D div (·,·) is some measure of divergence between feature distributions; Step 4: Build a U-Net automatic segmentation model; S41: Build a segmentation network; Use the U-Net model to build a segmentation network model; S42: Model training; using the feature map S of the source domain A As input data, the final output corresponds to S A The segmentation prediction result is S42-1: Construct a discriminator; construct two discriminators D1 and D2. The network structures of the two discriminators D1 and D2 are composed of multiple convolutions, pooling and jump connections; the goal of the discriminator D1 is to distinguish the input F A 、F B The features of come from the source domain or the target domain; the goal of the discriminator D2 is to distinguish the input S A , S B The feature comes from the source domain or the target domain; for the input feature x, the loss function of the discriminator D1 is obtained according to formula (9): According to formula (10), the loss function of discriminator D2 is obtained To distinguish the features of the source domain and the target domain by minimizing the loss area; S42-2: Construct domain adversarial loss; for the input feature x, the adversarial loss function of the global structure encoder f1 is obtained according to formula (11) According to formula (12), the adversarial loss function of the detail enhancement encoder f2 is obtained Using adversarial loss function To ensure that the discriminators D1 and D2 can correctly distinguish the sources of generated features; through joint training, in the optimization At the same time, f1 is also adjusted. At the same time, the distribution difference between the two domains is further minimized through domain adversarial loss, so that F A 、F B Closer to the same shared vector space R1, while making S A , S B Closer to the same shared vector space R2; S42-3: Construct feature reconstruction loss; calculate the difference between the output of the domain alignment network and the original input image, and quantify this difference through L1 loss. Specifically, the reconstruction loss is obtained according to formula (13): Where N represents the batch size; S42-4: Construct segmentation loss; select cross entropy loss as the loss function of the segmentation task to effectively measure the difference between the predicted segmentation map output by the segmentation network model and the labeled true value. Specifically, the segmentation loss is obtained according to formula (14): In the formula, The predicted value of the i-th pixel; represents the true label of the i-th pixel; S42-5: Joint fine-tuning; the total loss of the segmentation network is obtained according to formula (15) In the formula, λ1, λ2, λ rec , seg are weight parameters, which satisfy λ1+λ2+λ rec +λ seg =1, used to adjust the contribution of different loss items to the total loss. S42-6: U-Net automatic segmentation model is obtained through training; Step 5: Automatically segment the fundus color photos across devices; The fundus color image data collected in real time is input into the U-Net automatic segmentation model as input data, and high-precision segmented fundus image data is obtained after automatic segmentation by the U-Net automatic segmentation model.

2. The multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method according to claim 1, characterized in that: In step 1 S11, one type of high-quality fluorescein fundus angiography image data is acquired by high-end professional fundus color photography equipment, and another type of low-quality fluorescein fundus angiography image data is acquired by non-professional fundus color photography equipment.

3. The multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method according to claim 1 or 2, characterized in that: In step 1 S12, the process of preprocessing the image includes removing fundus reflection, histogram equalization, denoising and color standardization.

4. The multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method according to claim 1 or 2, characterized in that: In step three S31, a typical U-shaped network structure is selected to construct a global structure encoder f1.

5. The multi-scale unified modeling fluorescein fundus angiography image intelligent segmentation method according to claim 3, characterized in that: In step 1 S12, the denoising method includes median filtering, Gaussian filtering and wavelet denoising.

Images