Fundus fluorescence angiography collaborative generation method based on feature enhancement fusion

Abstract

The invention discloses a feature enhancement fusion-based fundus fluorescence angiography collaborative generation method, which comprises the steps of constructing and training an improved cyclic generative adversarial network model, and comprises a multi-scale fusion generator, a residual network generator, a first discriminator and a second discriminator, a multi-scale fusion generator, a residual network generator and a first discriminator are sequentially connected to form a main generation network, then the residual network generator, the multi-scale fusion generator and a second discriminator are sequentially connected to form an auxiliary generation network, and the main generation network and the auxiliary generation network are trained. Completing the training of the multi-scale fusion generator in the process of generating and reconstructing the fundus fluorescent angiography image; and converting the to-be-detected color fundus structure image into a fundus fluorescent angiography image by using the trained multi-scale fusion generator.

Description

Technical Field

[0001] This invention belongs to the technical field of image recognition, specifically relating to a method for co-generating fundus fluorescence angiography based on feature enhancement fusion. Background Technology

[0002] Clinically, fundus fluorescein angiography (FFA) is an important imaging technique for assessing retinal vascular function. It involves intravenously injecting sodium fluorescein dye and recording its flow through retinal vessels, dynamically displaying blood perfusion and leakage. This technique clearly presents the structure and perfusion characteristics of retinal microvessels, playing a crucial role in the diagnosis and efficacy evaluation of various fundus diseases such as diabetic retinopathy, retinal vein occlusion, and macular degeneration. Compared to color fundus structural imaging, FFA has higher sensitivity and diagnostic value in displaying pathological features such as microaneurysms, leaking areas, non-perfused areas, and neovascularization. However, FFA requires specialized imaging equipment and intravenous injection of sodium fluorescein, a procedure that is invasive and may cause adverse reactions such as nausea, vomiting, rash, and anaphylactic shock; in severe cases, it can even lead to fatal complications. Therefore, using deep learning to generate images that can safely and non-invasively replace FFA is of great clinical significance.

[0003] Existing methods for generating fundus fluorescein angiography images use a single-scale generator to simultaneously acquire both global and capillary features. This makes it difficult to simultaneously capture the quality of both long vessels and capillaries in the generated fundus fluorescein angiography images, thus affecting subsequent studies on vascular perfusion integrity and abnormal leakage. Summary of the Invention

[0004] This invention provides a method for collaborative generation of fundus fluorescence angiography based on feature enhancement fusion, which solves technical problems such as insufficient long-distance vascular connectivity, unclear vascular edges, and poor generation quality of vascular terminals.

[0005] To achieve the above objectives, the present invention provides the following technical solution: A method for co-generating fundus fluorescence angiography based on feature enhancement fusion includes the following steps: Step 1: Construct and train the improved CycleGAN generative adversarial network model. The improved CycleGAN generative adversarial network model includes a multi-scale fusion generator, a residual network generator, and two discriminators with identical structures. The multi-scale fusion generator, residual network generator, and first discriminator are sequentially connected to form the main generative network. The multi-scale fusion generator generates fundus fluorescein angiography images based on color fundus structure images. The residual network generator reconstructs the generated fundus fluorescein angiography images into color fundus structure images. The first discriminator determines whether the generated fundus fluorescein angiography images are genuine fundus fluorescein angiography images. The residual network generator, multi-scale fusion generator, and second discriminator are then sequentially connected to form an auxiliary generation network. The residual network generator generates a color fundus structure image based on the fundus fluorescein angiography image. The multi-scale fusion generator reconstructs the generated color fundus structure image from the fundus fluorescein angiography image. The second discriminator determines whether the generated color fundus structure image is a true color fundus structure image. By training the main generation network and the auxiliary generation network, the multi-scale fusion generator is trained to generate and reconstruct fundus fluorescence angiography images. Step 2: Using a trained multi-scale fusion generator, the color fundus structure image to be examined is converted into a fundus fluorescence angiography image.

[0006] Furthermore, the multi-scale fusion generator inputs the color fundus structure image into the global module to extract multi-scale global vascular features, and simultaneously uses the vascular enhancement module to enhance the color fundus structure image. Then, the enhanced color fundus structure image and the multi-scale global vascular features are input into the fusion generation module to obtain a fundus fluorescence angiography image.

[0007] Furthermore, the global module takes a color fundus structure image as input. First, it initializes the number of channels to 64 using a convolution module. Then, it expands the receptive field to 256 and 512 channels by passing two downsampling modules in sequence. Then, it performs feature enhancement through three cascaded residual modules while keeping the number of channels unchanged. Finally, it compresses the number of channels to 256 using an upsampling module, thereby completing the extraction of multi-scale global vascular features. The multi-scale global vascular features include four scale feature maps with the number of channels being 256, 512, 512, and 256, respectively. The fusion generation module includes an encoding stage and a decoding stage. In the encoding stage, shallow features are first extracted from the enhanced color fundus structure image through a convolutional module. Then, the shallow features are sequentially expanded by three downsampling modules with 128, 256, and 512 channels respectively. The output of the second downsampling module is weighted and fused with the first-scale feature map of the multi-scale global vascular features and used as the input of the third downsampling module. The output of the third downsampling module is weighted and fused with the second-scale feature of the multi-scale global vascular features and input into the convolutional module for feature filtering. The filtered feature map is weighted and fused with the third-scale feature of the multi-scale global vascular features and input into nine cascaded residual modules for feature enhancement to obtain the fused feature map. In the decoding stage, the cascaded upsampling module and attention module are used as a unit. The cascaded three units are used to decode the fused feature map to generate a fundus fluorescence angiography image. The output of the first unit is weighted and fused with the fourth-scale feature of the multi-scale global vascular features and used as the input of the second unit. The attention module filters the feature map through two "convolution-normalization-activation function-residual connection" operations, so that the decoding and generation process is more focused on vascular features.

[0008] Furthermore, the enhancement module uses homomorphic filtering to enhance the contrast and denoise the color fundus structure image, while using a Jerman filter to classify and enhance the color fundus structure image. Then, the processing results of homomorphic filtering and Jerman filtering are weighted and fused to generate a single-channel grayscale image, which is then superimposed on the original color fundus structure image in the channel dimension to generate an enhanced color fundus structure image.

[0009] Furthermore, for the loss calculation of the main generation network, firstly, the cycle consistency loss between the real color fundus structure image and the reconstructed color fundus structure image is calculated; secondly, the perceptual loss between the generated fundus fluorescein angiography image and the real color fundus structure image is calculated, and the cycle consistency loss and perceptual loss are weighted to generate the cycle consistency perceptual loss; finally, the adversarial loss between the generated fundus fluorescein angiography image and the real fundus fluorescein angiography image is calculated, and backpropagation is performed. For loss calculation of the auxiliary generative network, a multi-scale fusion generator is used to reconstruct the generated color fundus structure image into a fundus fluorescence angiography image to calculate the cycle consistency loss; then, a second discriminator is used to distinguish the generated color fundus structure image and calculate the adversarial loss; finally, backpropagation of the loss function is performed.

[0010] Furthermore, the cycle consistency perception loss is calculated using the following formula. , Where Lcyc(G,F) represents the cycle consistency loss, Lperceptual represents the perceptual loss, α and β represent the weights corresponding to the cycle consistency loss and perceptual loss, respectively; F(G(x)) represents the color fundus structure image reconstructed by the residual network generator, and x represents the real color fundus structure image, || ||1 represents the L1 norm; Let Φl represent the number of channels, height, and width of the feature in the l-th layer of the Vgg19 pre-trained network, respectively. () represents a Vgg19 deep network pre-trained on the ImageNet dataset; Calculate the adversarial loss using the following formula. , Where G(·) represents the generator and D(·) represents the discriminator. Indicates the input image. This represents a real sample.

[0011] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention uses a multi-scale fusion generator to simultaneously extract global vascular topology features and local microvascular features for weighted fusion. Furthermore, it integrates the fusion information through SK convolution to achieve collaborative generation of multi-scale feature fusion. As a result, the generated fundus fluorescence angiography image not only improves the connectivity of long vessels but also enhances the reconstruction of microvessels.

[0012] 2. This invention uses a vascular enhancement module to enhance the slender structures in the original color fundus structure image and performs weighted fusion with the original color fundus structure image, thereby enhancing the vascular features of the original color fundus structure image and improving the clarity of the vascular structure in the generated fundus fluorescence angiography image.

[0013] 3. This invention uses cyclic consistency perceptual loss to constrain the reconstructed image and the generated image in the pixel space and the high-level semantic space respectively, so as to maintain the overall structure of blood vessels without destroying them, while further improving the generation quality of blood vessel tips and edge details. Attached Figure Description

[0014] Figure 1 This is a schematic diagram of the overall network structure of the present invention; Figure 2 This is a schematic diagram of the multi-scale fusion generator of the present invention; Figure 3This is a schematic diagram of the comparative experimental results in a specific embodiment of the present invention. Detailed Implementation

[0015] To make the technical means, creative features, objectives and effects of the present invention easier to understand, the following embodiments, in conjunction with the accompanying drawings, specifically illustrate the method for collaborative generation of fundus fluorescein angiography based on feature enhancement fusion. It should be noted that the description of these embodiments is for the purpose of helping to understand the present invention, but does not constitute a limitation of the present invention.

[0016] like Figure 1 As shown, this invention provides a collaborative generation method for fundus fluorescein angiography based on feature enhancement fusion. An improved CycleGAN model is constructed, comprising a main generation network and an auxiliary generation network. A vessel enhancement module captures and enhances slender structures in the input image to enhance vascular features, thereby improving the clarity of vessels in the generated image. Simultaneously, a multi-scale fusion generator is used to achieve collaborative generation through multi-scale feature fusion, improving the connectivity of long vessels and the reconstruction accuracy of small vessels in the generated fundus fluorescein angiography image. This achieves accurate conversion from color fundus structure images to fundus fluorescein angiography images. Furthermore, a cycle consistency-aware loss is used to constrain image consistency from pixels and the high-level semantic space, thereby improving the generation quality of vascular terminals and edge details. Experimental results show that the proposed CycleGAN model outperforms other models in the fundus fluorescein angiography generation task. The proposed model is compared with various methods to verify its generation performance. In addition, ablation experiments are conducted to understand the impact of various optimizations on model generation.

[0017] Specifically as follows: S1. Obtain the fundus image dataset and perform preprocessing: This invention utilizes the Isfahan MISP public dataset, which contains 59 pairs of retinal fluorescence angiography image pairs, covering both healthy and pathological samples. Of these, 29 pairs are from healthy individuals and 30 pairs are from patients with diabetic retinopathy. Four pairs of normal and four pairs of abnormal images were randomly selected from the dataset as the test set, and the remaining 51 pairs were used as the training set.

[0018] Preprocessing of the images in this dataset: First, adjust the resolution of all images to 512×512 pixels. Second, using the inscribed circle of the square image as a reference, crop the adjusted image to extract the effective area. Finally, to maintain image size consistency and ensure the robustness of the network to the image display style, fill the invalid area outside the cropped circular area with a zero pixel value, so that the size of the resulting image remains 512×512 pixels.

[0019] S2. Construct the main generation network, such as Figure 1 As shown in the upper part, the main generation network includes a multi-scale fusion generator, a residual network generator, a first discriminator, a blood vessel enhancement module, and a cycle consistency sensing loss. First, the original color fundus structure image is enhanced using a vessel enhancement module. Second, a multi-scale fusion generator is used to generate a fundus fluorescence angiography image through the collaborative extraction and fusion of global and local features. Specifically, the multi-scale fusion generator inputs the color fundus structure image into the global module to extract multi-scale global vessel features, while simultaneously enhancing the color fundus structure image using the vessel enhancement module. The enhanced color fundus structure image and the multi-scale global vessel features are then input into the fusion generation module to obtain the fundus fluorescence angiography image. The perceptual loss between the real color fundus structure image and the generated fundus fluorescence angiography image is calculated. Subsequently, a residual network generator is used to reconstruct the generated fundus fluorescence angiography image into a color fundus structure image to calculate the cycle consistency loss. Finally, the generated fundus fluorescence angiography image is discriminated against by a first discriminator, and the adversarial loss is calculated and backpropagated in conjunction with the perceptual loss and cycle consistency loss. This process trains the multi-scale fusion generator during the generation of the fundus fluorescence angiography image.

[0020] S201. Constructing a vascular enhancement module, the specific steps of which include: First, homomorphic filtering is used to enhance the contrast of the color fundus structure image while suppressing its noise. Then, a Jerman filter is used to classify and enhance the slender structures. Finally, the enhancement results of the homomorphic filter and the Jerman filter are weighted and fused to form a single-channel grayscale image. This image is then superimposed on the original color fundus structure image in the channel dimension and input into the fusion generation module in the multi-scale fusion generator. This allows the enhanced vascular features to be learned, thereby enhancing the clarity of the vascular structures in the generated fundus fluorescence angiography image.

[0021] S202. Constructing the global module in the multi-scale fusion generator to extract the overall structural features of blood vessels, the specific steps include: like Figure 2 As shown, the color fundus structure image with a resolution of 512×512 pixels is first scaled down to a standard size of 256×256 pixels and then input into the global module of the multi-scale fusion generator. This global module achieves feature dimension compression and receptive field expansion through a downsampling module, combines it with a residual module for efficient transfer of depth features and gradient decay suppression, and then completes feature dimension recovery through an upsampling module. Finally, it accurately extracts the overall topological structure features of blood vessels in a multi-dimensional space, generating multi-scale global vascular features.

[0022] Specifically, within the global module, a 256×256 pixel image is first input into the convolutional module to extract a 64-channel feature map from a 3-channel color fundus structure image to initialize feature extraction. Then, two downsampling modules are used sequentially to encode the feature map, successively expanding the number of channels to 256 and 512 to gradually expand the receptive field. By reducing spatial resolution and increasing the number of channels, a larger receptive field is obtained, thereby capturing global vascular structure features at different scales, such as the orientation of main blood vessels and branching frameworks. Subsequently, the feature map obtained from the second downsampling module is input into the first residual module sequence, i.e., processed through three cascaded residual modules. While maintaining the same number of channels, the feature map is further enhanced. This method enhances the representation of global vascular topology features while filtering redundant information such as image background noise, achieving feature enhancement and redundancy suppression. The core structure of the residual module comprises three feature processing links: the main link and branch one extract basic vascular features through dilated convolution, while branch two further expands the receptive field to capture global background features through dilated convolution. Then, the basic vascular features and global background features are weighted and fused to achieve efficient gradient transfer and feature enhancement while suppressing redundant information. Next, the feature map processed by the first residual module sequence is input into the upsampling module, where deconvolution operations are used to restore scale and compress channels, reducing the dimension of the 512-channel feature map to 256 channels. Finally, the output feature maps from the two downsampling modules, the first residual module sequence, and the upsampling module are sequentially used as four scale feature maps, which are input into the corresponding channels of the fusion generation module for multi-scale feature information fusion. These four scale feature maps correspond to the topological distribution features of different layers of fundus vessels, providing global structural constraints for subsequent fusion of local detail features and preventing local features from deviating from the overall vascular layout.

[0023] S203. The fusion generation module in the multi-scale fusion generator extracts and encodes local microvascular features, and fuses the extracted overall vascular structure features for decoding and generation. Specific steps include: The fusion generation module refines the extraction of local microvascular features through the synergistic effect of its built-in downsampling module and residual module. Subsequently, the multi-scale global vascular features extracted by the global module are weighted and fused with the local microvascular features. The SK convolution mechanism integrated in the residual module is introduced to adaptively allocate channel weights and integrate features to improve the effectiveness of feature representation. Finally, the integrated multi-scale fused features are decoded by the upsampling module in the fusion generation module to generate fundus fluorescence angiography images with multi-scale feature synergistic fusion, thereby improving the connectivity of long vessels and the reconstruction accuracy of microvessels in the generated image.

[0024] Specifically, in the encoding stage, for the 512×512 resolution color fundus structure image processed by the vascular enhancement module, shallow features are extracted through its built-in convolution module. Then, three downsampling modules sequentially increase the number of channels of the shallow features from 64 to 128, 256, and 512. The first-scale feature map output from the global module is weighted and fused with the output of the second downsampling module and then input into the third downsampling module to complete local feature extraction. The output of the third downsampling module is then weighted and fused with the second-scale feature map output from the global module and then input into the convolution module to filter the fused feature map. Subsequently, the filtered feature map is weighted and fused with the third-scale feature map output from the global module and then input into the second residual module sequence, i.e., the cascaded nine residual modules, to enhance the fused feature map. In this case, the residual module adds SK convolution after the two branches are fused, which adaptively calibrates the fused features. By differentially allocating feature channel weights, it enhances the global vascular topology and local microvascular details, while filtering redundant noise, thus achieving efficient fusion of multi-scale features. In the decoding stage, the cascaded upsampling module and attention module are used as a unit. Through three cascaded units, the number of channels in the 512-channel fused feature map is gradually reduced to 256 channels, 128 channels, and 64 channels. The upsampling module uses bilinear interpolation to restore the spatial resolution of the feature map step by step and compresses the channel dimension to adapt to the image generation requirements. The attention module dynamically evaluates the importance of feature channels to accurately focus on the vascular feature response area and suppress background noise and invalid information interference. Finally, after reflection filling, it is input into a 7×7 convolutional layer. The output is normalized to the [-1,1] interval through the activation function to further improve the connectivity of long blood vessels and the restoration of small blood vessels in the generated image, generating a detailed and accurate fundus fluorescence angiography image. The output of the second unit is weighted and fused with the fourth-scale feature map output by the global module and then input into the third unit. The attention module filters the feature map during the upsampling process through two "convolution-normalization-activation function-residual connection" operations, so that the model focuses more on vascular features during the decoding and generation process.

[0025] S204. Construct a first discriminator to classify and distinguish between real fundus fluorescein angiography images and the generated fundus fluorescein angiography images. The network structure of the first discriminator consists of three convolutional layers and one fully connected layer cascaded in sequence. The output of the fully connected layer is configured with a sigmoid activation function, which is used to output the final discrimination result of the discrimination network on the input image.

[0026] S205. Using the CycleGAN residual network generator, the generated fundus fluorescein angiography image is reconstructed into a color fundus structure image, and cyclic consistency perceptual loss and adversarial loss functions are constructed. First, the cyclic consistency loss between the real color fundus structure image and the reconstructed color fundus structure image is calculated. Second, the perceptual loss between the generated fundus fluorescein angiography image and the real color fundus structure image is calculated. Finally, the adversarial loss between the generated fundus fluorescein angiography image and the real fundus fluorescein angiography image is calculated, and backpropagation is performed.

[0027] The expression for the cycle consistency-aware loss is: Where Lcyc(G,F) represents the cycle consistency loss, Lperceptual represents the perception loss, and α and β represent the weights of the cycle consistency loss and perception loss, respectively.

[0028] Cyclic consistency loss calculates the error between the reconstructed color fundus structure image and the real color fundus structure image in pixel space. The expression for cyclic consistency loss is: Where F(G(x)) represents the reconstructed image generated by the residual network generator, x represents the real image, || ||1 is the L1 norm.

[0029] The perceptual loss is calculated in the high-level semantic space to measure the error between the generated fundus fluorescein angiography image and the real fundus fluorescein angiography image. The expression for the perceptual loss is: in, These represent the number of channels, height, and width of the features in the l-th layer of the Vgg19 pre-trained network, respectively. Ireal represents a real-color fundus structure image, and Igen represents a generated fundus fluorescence angiography image. This represents a Vgg19 deep network pre-trained on the ImageNet dataset.

[0030] Adversarial loss drives the generator to engage in a game with the first discriminator, thereby gradually approximating the real distribution of the generated fundus fluorescein angiography image. The expression for adversarial loss is: The generator G attempts to generate an image G that looks similar to the samples in the target domain FA. The discriminator D distinguishes the transformed image G( ), while the discriminator D distinguishes the transformed image G( ). ) and real samples .

[0031] Ultimately, the recurrent consistency perception loss and adversarial loss are used to train the multi-scale fusion generator during the generation of fundus fluorescein angiography images, thereby maintaining the overall vascular structure of the generated fundus fluorescein angiography images without damage and further improving the generation accuracy of vascular terminals and edge details.

[0032] S3. Construct an auxiliary generation network to train the multi-scale fusion generator during the reconstruction of fundus fluorescein angiography images. Its core components are similar to the main generation network. The main generation network and the auxiliary generation network achieve collaborative training of the multi-scale fusion generator through bidirectional tasks of generation and reconstruction, respectively. The specific steps include: like Figure 1 As shown in the lower part, firstly, a residual network generator is used to convert the original fundus fluorescein angiography image into a color fundus structure image; secondly, a multi-scale fusion generator is used to reconstruct the fundus fluorescein angiography image from the generated color fundus structure image to calculate the cycle consistency loss; subsequently, a second discriminator is used to distinguish the generated color fundus structure image and calculate the adversarial loss; finally, through backpropagation of the loss function, the multi-scale fusion generator is trained in the process of reconstructing the fundus fluorescein angiography image, thereby improving the generation quality of the fundus fluorescein angiography image.

[0033] S4. Input the color fundus structure image to be examined into the trained multi-scale fusion generator to obtain the corresponding fundus fluorescence angiography image.

[0034] To verify the feasibility of this invention, we conducted the following experiment: Example 1 The experimental materials and equipment, parameter settings, and evaluation indicators for this embodiment are as follows: 1) Experimental materials and equipment The experiment used the Isfahan MISP public dataset; the experiment was implemented based on the deep learning framework PyTorch, and the experimental platform configuration was as follows: using an Intel Xeon Gold 6530 computer processor with 256GB of RAM; using an Nvidia RTX4090 discrete graphics card; and using CUDA 12.1 and CUDA 8.9.7 to accelerate GPU computing.

[0035] 2) Parameter settings The model training parameters were set as follows: batch size was 2; epochs were 200; the initial learning rate was 0.001, which was reduced to 0.0001 after 100 epochs; the Adam optimizer was used to optimize the model parameters. The weight decay in the optimizer was set to 10⁻⁹.

[0036] 3) Evaluation Indicators Each model was tested 10 times, and the model generation performance was evaluated using the Structural Similarity Index (SSIM), Peak Signal-to-Noise Ratio (PSNR), and Learned Perceptual Patch Similarity (LPIPS). SSIM represents the structural similarity between the generated image and the real image, and is calculated using the following formula: Where μx and μy are the mean, σx and σy are the variances, and σxy is the covariance; PSNR represents the ratio of the generated image to the reconstruction error, and the specific calculation formula is as follows: Where x(i,j) is the gray value of the (i,j)th pixel in the real image, y(i,j) is the gray value of the (i,j)th pixel in the generated image, and M and N are the height and width of the image, respectively; LPIPS represents the perceptual consistency between two images, and the specific calculation formula is as follows: Where x and y are the real image and the generated image, respectively, F^l(h,w,c) is the feature mapping of the neural network in the l-th layer, wl(c) is the channel weighting coefficient, and Cl, Hl, and Wl are the number of channels, height, and width of the features in the l-th layer, respectively.

[0037] 4) Result Evaluation To verify the impact of the multi-scale fusion generator (MFF-Gen), the vessel enhancement module (JVEM), and the cycle consistency-aware loss (CCP Loss) on the overall model performance, ablation experiments were conducted on a publicly available dataset using CycleGAN as the baseline. As shown in Table 1, compared to CycleGAN, adding the multi-scale fusion generator improved SSIM, PNSR, and LPIPS by 7.83%, 2.29%, and 2.81%, respectively. Notably, after introducing the JVEM module, PSNR decreased by 1.45%, while LPIPS increased by 12.32%. This is because CycleGAN, when inputting the 4-channel tensor processed by the proposed JVEM module, lacks sufficient network depth and modeling ability to effectively utilize multi-channel features, resulting in noise artifacts in the generated results and reducing image quality. In contrast, the architecture of the multi-scale fusion generator and the JVEM module optimized SSIM, PSNR, and LPIPS by 12.78%, 7.68%, and 10.91%, respectively. Furthermore, after introducing Cyclic Consistency-Aware Loss (CCP Loss), SSIM, PSNR, and LPIPS improved by 4.74%, 6.22%, and 13.38%, respectively.

[0038] Table 1 shows the ablation experimental results on the Isfahan MISP public dataset. Example 2 The dataset, evaluation metrics, and model parameters used are the same as in Example 1. Example 2 provides a comparison of the generation performance of different methods.

[0039] Table 2 shows the performance comparison results of different methods on the Isfahan MISP public dataset. The method based on feature enhancement fusion for co-generation of fundus fluorescence angiography achieves SSIM, PSNR, and LPIPS scores of 0.556, 20.43, and 0.229, respectively, demonstrating significant improvement compared to other methods. Specifically, as shown in Table 2, compared to Pix2Pix and Pix2PixHD, the proposed method shows average improvements of 26.87%, 16.03%, and 20.89% in SSIM, PSNR, and LPIPS, respectively. Compared to MRGAN, which also uses CycleGAN as its base model, the proposed method optimizes SSIM, PSNR, and LPIPS by 13.24%, 4.29%, and 13.58%, respectively. The improved SSIM value indicates that the proposed method outperforms MRGAN in terms of structure generation quality. This demonstrates that, compared to MRGAN's generator optimization strategy, the combination of the proposed multi-scale fusion generator and the blood vessel enhancement module enables the model to effectively learn the enhanced blood vessel information. Through the synergistic fusion between global blood vessel topology features and local microvascular features, more accurate FA images are generated, especially improving the generation quality of long blood vessel structures and blood vessel terminals.

[0040] Table 2. Comparison of generative performance of different methods on the Isfahan MISP public dataset. Figure 3 This section presents comparative experimental results of different methods on publicly available datasets. Compared to other methods, its collaborative architecture of multi-scale fusion generator and JVEM module, combined with a strategy of using cycle consistency-aware loss for auxiliary constraints, improves the integrity of long blood vessel structures and the reconstruction accuracy of microvessels, thereby enhancing the overall angiogenesis quality of FA images.

[0041] It is important to note that the schemes and arrangements of this application shown in the exemplary embodiments are merely exemplary. Although only a few embodiments are described in detail in this disclosure, those who consult this disclosure will readily understand that many modifications are possible (e.g., variations in various parameter values ​​(temperature, power, humidity, etc.), installation arrangements, names, colors, logical orders, etc.) without substantially departing from the novel teachings and advantages of the subject matter described in this application. Therefore, all such modifications are also included within the scope of the invention, and the order or sequence of any process or method steps may be changed or rearranged according to alternative embodiments. In the claims, any "apparatus plus function" clause is intended to cover the structure described herein for performing the function, and not only structurally equivalent but also equivalent in structure. Other substitutions, modifications, alterations, and omissions may be made in the design, operation, and arrangement of the exemplary embodiments without departing from the scope of the invention. Therefore, the invention is not limited to the particular embodiments but extends to a variety of modifications that still fall within the scope of the appended claims.

[0042] Furthermore, in order to provide a concise description of exemplary embodiments, not all features of actual embodiments (i.e., those features that are not relevant to the best mode of carrying out the invention as currently considered, or those features that are not relevant to implementing the invention) may be omitted.

[0043] It should be understood that numerous specific implementation decisions can be made during the development of any practical implementation, such as in any engineering or design project. Such development efforts may be complex and time-consuming, but for those skilled in the art who benefit from this disclosure, the development effort will be a routine work of design, manufacturing, and production without requiring much experimentation.

[0044] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for co-generating fundus fluorescence angiography based on feature enhancement fusion, characterized in that... Includes the following steps: Step 1: Construct and train the improved CycleGAN generative adversarial network model. The improved CycleGAN generative adversarial network model includes a multi-scale fusion generator, a residual network generator, and two discriminators with identical structures. The multi-scale fusion generator, residual network generator, and first discriminator are sequentially connected to form the main generative network. The multi-scale fusion generator generates fundus fluorescein angiography images based on color fundus structure images. The residual network generator reconstructs the generated fundus fluorescein angiography images into color fundus structure images. The first discriminator determines whether the generated fundus fluorescein angiography images are genuine fundus fluorescein angiography images. The residual network generator, multi-scale fusion generator, and second discriminator are then sequentially connected to form an auxiliary generation network. The residual network generator generates a color fundus structure image based on the fundus fluorescein angiography image. The multi-scale fusion generator reconstructs the generated color fundus structure image from the fundus fluorescein angiography image. The second discriminator determines whether the generated color fundus structure image is a true color fundus structure image. By training the main generation network and the auxiliary generation network, the multi-scale fusion generator is trained to generate and reconstruct fundus fluorescence angiography images. Step 2: Using a trained multi-scale fusion generator, the color fundus structure image to be examined is converted into a fundus fluorescence angiography image.

2. The method for co-generating fundus fluorescence angiography based on feature enhancement fusion according to claim 1, characterized in that: The multi-scale fusion generator inputs a color fundus structure image into a global module to extract multi-scale global vascular features. At the same time, it uses a vascular enhancement module to enhance the color fundus structure image. Then, it inputs the enhanced color fundus structure image and the multi-scale global vascular features into the fusion generation module to obtain a fundus fluorescence angiography image.

3. The method for co-generating fundus fluorescence angiography based on feature enhancement fusion according to claim 2, characterized in that: The global module takes a color fundus structure image as input. First, it initializes the number of channels to 64 using a convolution module. Then, it expands the receptive field by passing two downsampling modules in sequence, expanding the number of channels to 256 and 512. Next, it performs feature enhancement by passing three cascaded residual modules, while keeping the number of channels unchanged. Finally, it compresses the number of channels to 256 using an upsampling module, thereby completing the extraction of multi-scale global vascular features. The multi-scale global vascular features include four scale feature maps with the number of channels being 256, 512, 512, and 256, respectively. The fusion generation module includes an encoding stage and a decoding stage. In the encoding stage, shallow features are first extracted from the enhanced color fundus structure image through a convolutional module. Then, the shallow features are sequentially expanded by three downsampling modules with 128, 256, and 512 channels respectively. The output of the second downsampling module is weighted and fused with the first-scale feature map of the multi-scale global vascular features and used as the input of the third downsampling module. The output of the third downsampling module is weighted and fused with the second-scale feature of the multi-scale global vascular features and input into the convolutional module for feature filtering. The filtered feature map is weighted and fused with the third-scale feature of the multi-scale global vascular features and input into nine cascaded residual modules for feature enhancement to obtain the fused feature map. In the decoding stage, the cascaded upsampling module and attention module are used as a unit. The cascaded three units are used to decode the fused feature map to generate a fundus fluorescence angiography image. The output of the first unit is weighted and fused with the fourth-scale feature of the multi-scale global vascular features and used as the input of the second unit. The attention module filters the feature map through two "convolution-normalization-activation function-residual connection" operations, so that the decoding and generation process is more focused on vascular features.

4. The method for co-generating fundus fluorescence angiography based on feature enhancement fusion according to claim 3, characterized in that: The enhancement module uses homomorphic filtering to enhance the contrast and denoise the color fundus structure image, while using a Jerman filter to classify and enhance the color fundus structure image. Then, the processing results of homomorphic filtering and Jerman filtering are weighted and fused to generate a single-channel grayscale image, which is then superimposed on the original color fundus structure image in the channel dimension to generate an enhanced color fundus structure image.

5. The method for co-generating fundus fluorescence angiography based on feature enhancement fusion according to claim 1, characterized in that: For the loss calculation of the main generator network, firstly, the cycle consistency loss between the real color fundus structure image and the reconstructed color fundus structure image is calculated. Secondly, the perceptual loss between the generated fundus fluorescein angiography image and the real color fundus structure image is calculated. The cycle consistency loss and the perceptual loss are weighted to generate the cycle consistency perceptual loss. Finally, the adversarial loss between the generated fundus fluorescein angiography image and the real fundus fluorescein angiography image is calculated and backpropagation is performed. For loss calculation of the auxiliary generative network, a multi-scale fusion generator is used to reconstruct the generated color fundus structure image into a fundus fluorescence angiography image to calculate the cycle consistency loss. Subsequently, a second discriminator is used to identify the generated color fundus structure image and calculate the adversarial loss; finally, backpropagation of the loss function is performed.

6. The method for co-generating fundus fluorescence angiography based on feature enhancement fusion according to claim 5, characterized in that: Calculate the cycle consistency-aware loss using the following formula. , Where Lcyc(G,F) represents the cycle consistency loss, Lperceptual represents the perceptual loss, α and β represent the weights corresponding to the cycle consistency loss and perceptual loss, respectively; F(G(x)) represents the color fundus structure image reconstructed by the residual network generator, and x represents the real color fundus structure image, || ||1 represents the L1 norm; Let Φl represent the number of channels, height, and width of the feature in the l-th layer of the Vgg19 pre-trained network, respectively. () represents a Vgg19 deep network pre-trained on the ImageNet dataset; Calculate the adversarial loss using the following formula. , Where G(·) represents the generator and D(·) represents the discriminator. Indicates the input image. This represents a real sample.

Images