Eye fundus image segmentation method and related equipment

Abstract

The embodiment of the invention provides a fundus image segmentation method and related equipment, and belongs to the technical field of medical image processing. The method comprises the following steps: constructing and training a segmentation model which integrates a multi-scale U-Net main network, a deblurring diffusion module and a variational auto-encoder module; wherein the multi-scale U-Net enhances detail capture by introducing frequency domain reweighted jump connection; the variational auto-encoder module provides global structure prior of the image; and the deblurring diffusion module uses the prior as condition guidance to de-noise and enhance the features. According to the method, three advanced models are innovatively and synergistically fused, the problems that in the prior art, the segmentation precision of complex blood vessels and optic disc / optic cup structures in fundus images is insufficient, the robustness of blurred images is poor, and a large amount of labeled data is relied on are effectively solved, and the segmentation accuracy and stability are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of medical image processing technology, and in particular to a method and related equipment for fundus image segmentation. Background Technology

[0002] Fundus images are crucial for diagnosing various eye diseases, such as diabetic retinopathy, glaucoma, and age-related macular degeneration. Precise segmentation of retinal vessels and the optic disc / cup is essential for early screening and quantitative assessment of these diseases.

[0003] Currently, fundus image segmentation methods are mainly divided into traditional image processing methods and deep learning methods. Traditional methods, such as thresholding, edge detection, region growing, and active contour models, often lack sufficient segmentation accuracy and robustness when processing fundus images with complex vascular details, high noise levels, and significant background interference.

[0004] With the development of deep learning, convolutional neural networks (CNNs), especially U-Net and its variants, have become the mainstream technology for fundus image segmentation. These methods have significantly improved segmentation accuracy compared to traditional methods by automatically learning features. However, they still have several drawbacks: First, they heavily rely on a large amount of high-quality labeled data, while the labeling of medical images is costly and subject to subjective differences; second, the models are very sensitive to image quality (such as blur, noise, and low contrast), and are prone to missegmentation or omission when processing images of poor quality; third, the complex network structure leads to high computational costs and slow inference speed, making it difficult to meet the real-time needs of clinical practice.

[0005] In recent years, diffusion models have demonstrated powerful capabilities in image generation and denoising, while variational autoencoders have advantages in learning compact latent representations of data. However, how to creatively combine these advanced models and construct an efficient and robust solution for specific challenges in fundus image segmentation (such as complex vascular structures and blurred boundaries) remains a pressing technical problem to be solved in this field. Summary of the Invention

[0006] The main objective of this application is to propose a fundus image segmentation method and related device that combines a diffusion model and a variational autoencoder, which can effectively improve segmentation accuracy, enhance robustness to blurred and noisy images, and reduce excessive reliance on labeled data.

[0007] To achieve the above objectives, one aspect of this application proposes a fundus image segmentation method, the method comprising: Obtain the fundus image to be segmented; The fundus image is input into the trained segmentation model, and the segmentation result is output; the segmentation model is constructed and trained in the following manner: A multi-scale U-Net main network is constructed, in which the encoder and decoder perform feature fusion through skip connections, and a frequency domain reweighting module is set in at least one of the skip connections to enhance high-frequency detail features in the image. A deblurring diffusion module is constructed, which is configured to perform forward diffusion perturbation and inverse denoising recovery on the image features extracted by the multi-scale U-Net main network to generate enhanced features; A variational autoencoder module is constructed and configured to encode the input image to obtain latent features characterizing global structural information; The multi-scale U-Net main network, the deblurring diffusion module, and the variational autoencoder module are integrated so that the deblurring diffusion module uses the latent features provided by the variational autoencoder module as conditional guidance to enhance the feature map of the multi-scale U-Net main network, and the enhanced features are used for the final segmentation prediction.

[0008] In some embodiments, the operation of the frequency domain reweighting module includes: Perform a two-dimensional discrete Fourier transform on the feature map transmitted in the skip connection to convert it to the frequency domain; In the frequency domain, a preset low-frequency window function is used to adjust the weights of the low-frequency components of the feature map; The weighted frequency domain features are then converted back to the spatial domain using an inverse Fourier transform. The feature map converted back to the spatial domain is fused with the features of the corresponding layer in the decoder.

[0009] In some embodiments, the workflow of the deblurring diffusion module includes: Forward diffusion process: Gaussian noise is gradually added to the input image features to form a series of noisy features; Inverse denoising process: Based on a denoising U-Net conditioned on the latent features of the variational autoencoder module, noise is progressively predicted and removed from the noisy features to recover clear enhanced features.

[0010] In some embodiments, during the training of the segmentation model, the fundus image is processed using a Frangi filter to extract enhanced prior information of retinal vessels; and the prior information is introduced into the input of the deblurring diffusion module in a weighted summation manner.

[0011] In some embodiments, the Frangi filter is applied only to retinal vessel segmentation tasks and not to optic disc / cup segmentation tasks.

[0012] In some embodiments, the segmentation model is trained using a two-step training method, including: Pre-training phase: Train the multi-scale U-Net main network separately until it converges; Diffusion module training phase: The parameters of the multi-scale U-Net main network are fixed, and the deblurring diffusion module is trained using a pre-trained variational autoencoder module; Joint fine-tuning stage: The parameters of the deblurring diffusion module and the variational autoencoder module are fixed, and only the parameters of the multi-scale U-Net main network are fine-tuned.

[0013] To achieve the above objectives, another aspect of this application provides a fundus image segmentation apparatus, the apparatus comprising: The image acquisition module is used to acquire fundus images to be segmented; The image segmentation module is used to input the fundus image into the trained segmentation model and output the segmentation result; the segmentation model is constructed and trained in the following manner: A multi-scale U-Net main network is constructed, in which the encoder and decoder perform feature fusion through skip connections, and a frequency domain reweighting module is set in at least one of the skip connections to enhance high-frequency detail features in the image. A deblurring diffusion module is constructed, which is configured to perform forward diffusion perturbation and inverse denoising recovery on the image features extracted by the multi-scale U-Net main network to generate enhanced features; A variational autoencoder module is constructed and configured to encode the input image to obtain latent features characterizing global structural information; The multi-scale U-Net main network, the deblurring diffusion module, and the variational autoencoder module are integrated so that the deblurring diffusion module uses the latent features provided by the variational autoencoder module as conditional guidance to enhance the feature map of the multi-scale U-Net main network, and the enhanced features are used for the final segmentation prediction.

[0014] To achieve the above objectives, another aspect of this application provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the method described above.

[0015] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0016] To achieve the above objectives, another aspect of the embodiments of this application proposes a computer program product, including a computer program that, when executed by a processor, implements the method described above.

[0017] The embodiments of this application include at least the following beneficial effects: This application provides a fundus image segmentation method, apparatus, electronic device, storage medium and program product. This solution combines a diffusion model and a variational autoencoder to enhance image structural information using generated denoising features, thereby achieving high-quality fundus image segmentation for automated and high-precision segmentation of retinal vessels and optic disc / optic cup. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the construction and training process of the segmentation model in this application embodiment; Figure 2 This is a schematic diagram of the structure of the multi-scale U-net main network in the embodiments of this application; Figure 3 This is a schematic diagram of the deblurring diffusion module in an embodiment of this application; Figure 4 This is a schematic diagram comparing the performance of the fundus image segmentation method provided in this application embodiment with existing methods; Figure 5 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit it. In the following description, when referring to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with those of this application; they are merely examples of apparatuses and methods consistent with some aspects of the embodiments of this application as detailed in the appended claims.

[0020] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0021] Before providing a detailed description of the embodiments of this application, some of the nouns and terms involved in the embodiments of this application will be explained first. The nouns and terms involved in the embodiments of this application are subject to the following interpretations.

[0022] 1) U-Net is a convolutional neural network (CNN) based architecture designed for image segmentation tasks, and it performs particularly well in medical image analysis.

[0023] 2) Deblurring Diffusion is a technique that uses diffusion models to deblur images.

[0024] 3) A variational autoencoder (VAE) is a probability-based generative model consisting of an encoder and a decoder. It generates new data samples by learning the probability distribution of the data, and supports tasks such as dimensionality reduction and feature extraction.

[0025] With the rapid development of medical imaging technology, automated segmentation techniques for fundus images (especially retinal vessels and optic disc / cup images) have gradually become a key research direction in ophthalmic diagnosis. Fundus images provide important visual clues for ophthalmic diseases, playing a crucial role in the early diagnosis of diseases such as diabetic retinopathy (DR), glaucoma, and age-related macular degeneration (AMD). Precise segmentation of the retinal vessels and optic disc / cup regions can effectively extract valuable pathological information, helping doctors identify potential disease risks in the early stages, thereby improving the accuracy and timeliness of diagnosis.

[0026] Fundus image segmentation involves more than just locating retinal vessels; it also involves the precise division of the optic disc and optic cup, whose morphology and structure in fundus images are highly complex and exhibit significant individual variability. Therefore, accurate and efficient segmentation methods are crucial for subsequent disease assessment and treatment planning.

[0027] Currently, segmentation methods for retinal vessels and the optic disc / cup mainly fall into two categories: traditional image processing algorithms and modern deep learning models. Traditional image processing algorithms, such as thresholding, edge detection, region growing, and active contour models, have achieved a certain level of processing and analysis of fundus images. However, these methods often exhibit low segmentation accuracy when dealing with complex vascular details, high noise levels, and background interference in fundus images. In particular, when the image is affected by factors such as illumination, noise, and blurring, the robustness of traditional algorithms is insufficient, leading to unsatisfactory segmentation results.

[0028] With the development of deep learning, especially convolutional neural networks (CNNs), an increasing number of deep learning-based models are being applied to the automated segmentation of retinal vessels and optic disc / cup. These methods automatically learn features by training neural networks, effectively capturing the complex structures of vessels and the optic disc / cup, significantly improving segmentation accuracy compared to traditional methods. For example, network structures such as U-Net, M_Net, and DenseNet are widely used in fundus image segmentation and have achieved good performance on multiple standard datasets. Deep learning methods can not only automatically extract key features from images but also handle more complex image content, such as the intersections and branches of blood vessels, and the blurred boundaries of the optic disc and cup.

[0029] While these deep learning methods have made some progress in segmentation accuracy, several problems remain. First, training deep learning models requires a large amount of labeled data, but labeling fundus images is complex, time-consuming, and prone to errors and inconsistencies, making it difficult to guarantee high-quality data. Second, although deep learning models can effectively extract features and achieve high segmentation accuracy, they are often sensitive to image quality and noise. Especially when processing fundus images with blurriness, noise, or low resolution, traditional deep learning models are prone to missegmentation or omissions, leading to unstable segmentation results. This application proposes a multi-scale fundus image segmentation method based on a diffusion model enhancement. By adding multi-scale detailed edge information to the neural network and using a diffusion model to generate additional features, the segmentation effect of fundus images is enhanced.

[0030] Furthermore, existing deep learning models also have certain bottlenecks in terms of computational complexity and inference speed. Due to the complexity of the network structure, many deep learning models require a large amount of computing resources for training and inference, which limits their application in real-world medical scenarios, especially in situations requiring real-time analysis, making it difficult to meet the needs of real-time diagnosis.

[0031] In summary, existing fundus image segmentation methods have made some progress in improving segmentation accuracy and reducing computational complexity, but they still face problems such as strong data dependence, difficulty in handling complex structures, insufficient robustness, and slow inference speed. In general, existing technical solutions have the following shortcomings: 1) Insufficient edge segmentation accuracy: Existing models are easily affected by noise when processing complex structures of retinal vessels and the optic disc / cup, leading to inaccurate segmentation results, especially in capturing vascular details and the edges of the optic disc / cup. 2) Reliance on manual annotation: Many methods rely on large amounts of labeled data for training, and the annotation process is often time-consuming and laborious, making it difficult to automate. 3) Difficulty in handling blurred images: Existing methods perform poorly on blurred images caused by poor image quality, patient movement, or other reasons, especially in restoring vascular details and improving the edge clarity of the optic disc / cup region. 4) High computational complexity and slow inference speed: Some deep learning models, such as U-Net and its variants, can perform segmentation well, but require a lot of computational resources during training and have a slow inference speed, which limits their real-time clinical applications.

[0032] In view of this, this application provides a fundus image segmentation method, apparatus, electronic device, storage medium, and program product, particularly in the segmentation task of retinal vessels and optic disc / cup images, which significantly improves segmentation accuracy by introducing a deblurring diffusion process. Specifically, this application adopts a multi-scale U-Net architecture, combined with a deblurring diffusion module, and uses the generated denoising features to enhance image structural information, thereby achieving high-quality fundus image segmentation. This application adopts a two-step training process: (1) Pre-training: First, the multi-scale U-Net main network is trained independently, and then the pre-trained U-Net is used as the initialization model to train the deblurring diffusion module. (2) Joint fine-tuning: The network is initialized using the pre-trained diffusion module, the diffusion module (and VAE encoder) are frozen, and only the U-Net main network is fine-tuned. This method can effectively handle the problems of noise, blurring, and missing details in fundus images, and improve the robustness and accuracy of retinal vessel and optic disc / cup segmentation.

[0033] This application provides a fundus image segmentation method, relating to the field of medical image processing technology. The fundus image segmentation method provided in this application can be applied to a terminal, a server, or software running on a terminal or server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, smartwatch, or vehicle terminal, but is not limited thereto; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The server can also be a node server in a blockchain network; the software can be an application implementing a fundus image segmentation method, but is not limited to the above forms.

[0034] This application can be used in a wide variety of general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0035] This embodiment provides a fundus image segmentation method, including the following steps: Step 1: Obtain the fundus image to be segmented; Step 2: Input the fundus image into the trained segmentation model and output the segmentation result.

[0036] See Figure 1 The segmentation model is constructed and trained in the following manner: S1: Construct a multi-scale U-Net main network, in which the encoder and decoder perform feature fusion through skip connections, and a frequency domain reweighting module is set in at least one of the skip connections to enhance high-frequency detail features in the image; S2: Construct a deblurring diffusion module, which is configured to perform forward diffusion perturbation and inverse denoising recovery on the image features extracted by the multi-scale U-Net main network to generate enhanced features; S3: Construct a variational autoencoder module, which is configured to encode the input image to obtain latent features that characterize global structural information; S4: Integrate the multi-scale U-Net main network, the deblurring diffusion module, and the variational autoencoder module, so that the deblurring diffusion module uses the latent features provided by the variational autoencoder module as conditional guidance to enhance the feature map of the multi-scale U-Net main network, and uses the enhanced features for the final segmentation prediction.

[0037] In some embodiments, during the training of the segmentation model, the fundus image is processed using a Frangi filter to extract enhanced prior information of retinal vessels; and the prior information is introduced into the input of the deblurring diffusion module in a weighted summation manner.

[0038] In some embodiments, the segmentation model is trained using a two-step training method, including: Pre-training phase: Train the multi-scale U-Net main network separately until it converges; Diffusion module training phase: The parameters of the multi-scale U-Net main network are fixed, and the deblurring diffusion module is trained using a pre-trained variational autoencoder module; Joint fine-tuning stage: The parameters of the deblurring diffusion module and the variational autoencoder module are fixed, and only the parameters of the multi-scale U-Net main network are fine-tuned.

[0039] The segmentation model of this embodiment will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0040] (1) Multi-scale U-Net architecture See Figure 2 In this embodiment, the main network adopts a multi-scale U-Net architecture, which consists of multiple convolutional layers, pooling layers, and upsampling layers, effectively extracting low-level and high-level features from fundus images. U-Net fuses low-level and high-level features through skip connections, enhancing the network's representation ability in complex image structures. The core idea of ​​U-Net is to extract multi-level features from the input image through an encoder and then gradually restore the spatial resolution of the image through a decoder. In each scale skip connection, the network adjusts low-frequency components through frequency-domain reweighting, enhancing the ability to capture small blood vessels and edge structures.

[0041] In the following embodiments, the specific implementation details of the multi-scale U-Net are as follows: 1.1) Input: The original input image with multiple channels, typically a 3-channel RGB image. In the multi-scale U-Net architecture, the image first undergoes preliminary convolution operations for feature extraction. The size of the input image is usually H×W, such as 512×512 or 1024×1024 pixels. The specific size is adjusted according to the dataset and training requirements. The following process is described using an input image size of 512×512 as an example.

[0042] 1.2) Encoder Part: This part includes multiple convolutional operations to extract low-level features from the image. The encoder mainly consists of multiple convolutional and pooling layers, with the number of input and output channels gradually increasing in each layer. The main function of the encoder is to extract low-level features from the input image. By progressively reducing the spatial resolution of the image through layer-by-layer convolutional operations, more high-level semantic information can be extracted. The specific network structure is as follows: 1.2.1) First layer encoder (Conv1): The input image is convolved to generate a 32-channel feature map. Then, max pooling is used to downsample the image and concatenate it to reduce the spatial resolution of the original image by half. After convolution, a 64-channel feature map is output. After convolution, a 96-channel feature map is obtained, and then a 64-channel feature map is output.

[0043] 1.2.2) Second Encoder (Conv2): The input is the feature map generated from the first layer, which is then subjected to convolution, resulting in an output of 128 channels. This layer also includes pooling operations, further reducing the spatial resolution by half, and then concatenating the feature map from the previous layer.

[0044] 1.2.3) Third layer encoder (Conv3): This layer further increases the number of channels of the feature map to 256. It also extracts more abstract features through convolution operations, reduces the spatial resolution through pooling operations, and stitches the feature map of the previous layer together.

[0045] 1.2.4) Fourth layer encoder (Conv4): The last layer encoder has 512 output channels and extracts the deepest semantic features of the image.

[0046] 1.3) Decoder Section: The main function of the decoder section is to progressively restore the spatial resolution of the image, restoring it to its input size through upsampling operations. Each layer of the decoder fuses the features of the corresponding encoder layer with the features of the current decoder layer through skip connections. In this way, low-level details and high-level semantic information are fully utilized, thereby enhancing the detail recovery capability.

[0047] 1.3.1) First layer decoder (Upconv1): The first layer of the decoder increases the spatial resolution of the feature map from 32×32 to 64×64 through upsampling operation, and fuses and convolves the features from the encoder in the skip connection with the features of the current decoding layer, with an output channel number of 256.

[0048] 1.3.2) Second layer decoder (Upconv2): The second layer upsamples the spatial resolution from 64×64 to 128×128, and fuses and convolves the features from the encoder in the skip connections with the features of the current decoding layer, with an output channel number of 128.

[0049] 1.3.3) Third layer decoder (Upconv3): The third layer upsamples the spatial resolution from 128×128 to 256×256, and continues to fuse features and then convolution, with an output channel number of 64.

[0050] 1.3.4) Fourth layer decoder (Upconv4): The last layer decoder restores the spatial resolution from 256×256 to the size of the original input image and generates the final segmentation output image through convolutional layers. The number of output channels is n_classes (the number of segmentation categories).

[0051] In each layer, the convolution operation uses a 3×3 convolution kernel with a stride of 1 to maintain spatial resolution; the pooling operation uses 2×2 max pooling to reduce the spatial resolution of the image, typically halving the spatial resolution with each pooling operation.

[0052] Each layer of the decoder employs upsampling and convolution operations to ensure high-precision boundary restoration and ultimately output a segmentation result of the same size as the original image. In the decoder portion of each layer, U-Net uses skip connections to fuse features of the same scale from the encoder with features from the decoder, which helps preserve image details, especially blood vessel edges and fine structures. To further improve the capture of fine blood vessels and edge structures, this embodiment also employs frequency-domain reweighting in the skip connections of each layer, called the FreeU enhancement module.

[0053] Frequency domain reweighting, by adjusting the weights of low-frequency components in the frequency domain, effectively suppresses noise and redundant information in images, thereby highlighting structural information, especially enhancing the details of blood vessel edges in fundus images. Specifically, in skip connections, the input feature map is transformed to the frequency domain via a two-dimensional discrete Fourier transform (FFT), and a low-frequency window (defined by the hyperparameter Ti) is applied to weight the low-frequency components. Then, the adjusted feature map is transformed back to the spatial domain via an inverse Fourier transform (IFFT), and finally fused with the features from the decoder. In this way, frequency domain reweighting can effectively enhance the details of vascular structures, especially the clarity at blood vessel edges, thereby improving the accuracy and precision of blood vessel segmentation in fundus images.

[0054] 1.4) Technical Effects: The multi-scale U-Net architecture, through feature extraction at different scales, enables the model to capture detailed information at different resolutions, enhancing its ability to represent complex structures. Especially in fundus images, the complex structures and edges of blood vessels and optic discs / cups often require multi-scale features for accurate identification. This architecture, by introducing skip connections between the encoder and decoder, achieves the fusion of low-level and high-level features, thereby effectively capturing detailed and edge information. Multi-scale U-Net not only improves image resolution but also enhances the model's segmentation accuracy for fine structures (such as blood vessels, optic discs, and optic cup edges) in images through cross-scale information integration.

[0055] 1.5) Technical problems to be solved: 1.5.1) Insufficient edge segmentation accuracy: By processing details and structures at multiple scales, the segmentation accuracy of small blood vessels and the edges of the optic disc / optic cup has been improved.

[0056] 1.5.2) Difficulty in handling complex structures: For complex vascular networks and optic disc / cup structures in fundus images, multi-scale U-Net can handle structures of different sizes, reducing errors caused by single-scale processing.

[0057] 1.6) Beneficial effects: Improved segmentation accuracy, especially in recognizing complex structures, small blood vessels, and the edges of the optic disc / cup. Enhanced the network's expressive power, improved the fusion of low-frequency and high-frequency features in the image, and reduced the impact of noise.

[0058] (2) Diffusion model See Figure 3The diffusion model network structure in this embodiment is based on the U-Net architecture, employing a typical encoder-decoder architecture. The encoder extracts low-level features from the image, while the decoder progressively restores high-resolution details. The diffusion process is combined with a deblurring module, applying temporal embedding in each convolutional operation to enhance the network's response to different time steps. In this embodiment, the diffusion model plays a crucial role in enhancing image structure, removing noise and blur, and generating additional features to improve segmentation performance. During the implementation of the diffusion model, the image features extracted from the highest resolution layer of the multi-scale U-Net are preprocessed with 1×1 projection and channel averaging, followed by Gaussian blurring. The image is then progressively perturbed according to a preset time step (forward diffusion process). In the backward diffusion process, a trained deblurring model is used to restore image sharpness.

[0059] In some embodiments, the specific implementation details of the Gaussian deblurring diffusion model are as follows: 2.1) Encoder Section: The encoder section contains multiple convolutional and pooling layers, responsible for extracting low-level features of the image and progressively reducing the spatial resolution of the image. The input and output channels and convolutional kernel size of each layer are shown below: 2.1.1) First Encoder Layer (Conv1): The input image enters the network through a convolution operation, and the output channel count is 32. This layer extracts the basic features of the input image through convolution operations while maintaining the spatial resolution (H×W). Then, an activation function (GELU) is applied to introduce non-linearity, enabling the network to learn more complex features. Afterward, max pooling is used to downsample the feature map, halving the spatial resolution.

[0060] 2.1.2) Second Layer Encoder (Conv2): The second layer processes features from the first layer through convolutional operations, with an output of 64 channels. The spatial resolution is further reduced after max pooling. The main function of this layer is to extract features from more layers while preserving the local spatial relationships of the input image. The activation function (GELU) is used to add non-linearity, enabling the network to learn complex image structures.

[0061] 2.1.3) Third Encoder Layer (Conv3): The third layer continues the convolution operation on the image, with an output channel count of 128. This layer extracts mid-level features of the image through convolution, and then applies max pooling to further reduce the spatial resolution. This layer is responsible for extracting more complex local features and preparing more semantically meaningful high-dimensional features for the subsequent decoder.

[0062] 2.1.4) Fourth Encoder Layer (Conv4): The fourth layer employs convolution operations, with an output channel count of 256. The spatial resolution is reduced again. This layer is primarily used to extract high-level features and capture global structural information in the image. While the spatial resolution of the image is already low, the feature maps contain rich semantic information, providing good feature support for the decoder.

[0063] 2.1.5) Center Layer: In the deepest layer of the encoder, the input feature map is further processed through convolutional operations to extract features, resulting in an output with 512 channels. This layer does not perform downsampling, preserving spatial resolution and providing high-dimensional structural information for the decoder.

[0064] 2.2) Decoder Section: The decoder restores the spatial resolution of the image through progressive upsampling and combines features from the encoder section to improve image detail recovery. The input and output channels and operations of each layer are shown below: 2.2.1) First Decoder Layer (Upconv1): The first layer of the decoder increases the spatial resolution of the feature map from 16×16 to 32×32 through upsampling, resulting in 256 output channels. Then, feature fusion is performed through convolution, and an activation function (GELU) is applied to increase non-linearity. The main function of this layer is to progressively restore the spatial resolution of the image and fuse features from the encoder with features from the current layer of the decoder.

[0065] 2.2.2) Second Layer Decoder (Upconv2): The second layer increases the resolution of the feature map from 32×32 to 64×64 through upsampling, with 128 output channels. Features are further extracted and fused through convolution and the GELU activation function. The purpose of this layer is to enhance image details and recover higher-resolution features.

[0066] 2.2.3) Third-layer decoder (Upconv3): The third layer increases the spatial resolution of the feature map from 64×64 to 128×128 through upsampling operations, with 64 output channels. Through convolution and activation function (GELU), this layer further extracts details and fuses them with features from the encoder to recover a finer structure.

[0067] 2.2.4) Fourth Decoder Layer (Upconv4): The last decoder layer restores the feature map resolution from 128×128 to 256×256 through upsampling operations, with 32 output channels. This layer continues to extract image details through convolution and fuses the details with lower-level features, ultimately restoring the spatial resolution of the input image.

[0068] In each layer, the convolution operation uses a 3×3 convolution kernel with a stride of 1 to maintain spatial resolution; the pooling operation uses 2×2 max pooling.

[0069] 2.3) Technical Effects: The key role of the diffusion model enhancement module in this invention is to enhance the structural information of the image by generating additional features, thereby improving segmentation accuracy. The diffusion process simulates noise and perturbs the image through Gaussian blurring, while the reverse diffusion uses a denoising model to gradually restore image clarity. In this process, the diffusion model not only removes noise but also generates additional features while restoring image details. These features effectively enhance the segmentation of retinal vessels and optic disc / cup, especially when processing complex and blurred images. The diffusion model can generate additional information that helps in structural recovery, enabling the model to better capture small blood vessels and edge structures.

[0070] 2.4) Technical problems to be solved: 2.4.1) Insufficient edge segmentation accuracy: The additional features generated by the diffusion model significantly improve the edge segmentation accuracy of blood vessels and optic disc / cup through detail restoration and structural enhancement.

[0071] 2.4.2) Difficulty in handling blurred images: The diffusion model effectively handles blurred images caused by poor image quality, patient movement, etc. by generating features with structural consistency, helping to restore vascular details and edge clarity of the optic disc / optic cup.

[0072] 2.5) Beneficial effects: It generates additional image features that help the network capture details more accurately, especially in complex vascular networks and optic disc / cup edges; it enhances segmentation performance, especially in cases of blurred or noisy images, providing more information to aid in structural recovery and ultimately improving segmentation accuracy; it improves the robustness of the model, enabling it to handle different types of images, especially those with missing information due to blurring or noise interference.

[0073] (3) Variational Autoencoder (VAE) module and feature fusion This embodiment also introduces a VAE module, which provides a global structural prior for the segmentation task by learning the latent spatial representation of the image. In the VAE module, the input image features are processed by an encoder to generate latent vectors representing the global features of the image. By fusing these latent features with the output of the diffusion module, the segmentation accuracy and robustness are further improved. The specific function and implementation of the VAE are as follows: 3.1) VAE Encoder: Extracts latent features from preprocessed image features. The VAE uses an encoder to map the features of the input fundus image to the latent space and learns the distribution of latent variables. These latent variables in the VAE's latent space can capture the global structure and high-level features of the image. These latent features can serve as a structural prior to guide image reconstruction during the diffusion process, such as the overall shape of the optic disc / cup region and vascular structure.

[0074] 3.2) Latent Feature Fusion: The latent vectors generated by the VAE are fused with the denoised image output by the diffusion module to provide structural prior information, thereby improving detail recovery capabilities. In latent feature fusion, the latent vectors generated by the VAE are used as guiding information, providing constraints on the global structure in the image. The denoising process of the diffusion model then recovers image details based on this latent information. In each step of the denoising process, the latent features provide a reference frame, ensuring that structural consistency is maintained during image recovery while avoiding over-denoising or loss of key details. In this way, the VAE and diffusion model can not only improve the ability to recover image details but also effectively enhance the accuracy and stability of segmentation.

[0075] 3.3) Technical Effects: The Variational Autoencoder (VAE) module provides global structural information by generating codes for the latent space, enabling the network to have stronger semantic awareness when processing images. In this invention, the VAE's role is to extract latent space features of the image, providing conditional information for the diffusion process, thereby effectively constraining and guiding the image restoration process. By reducing reliance on local annotation information, the VAE allows the model to achieve good image segmentation results even under incomplete supervision.

[0076] 3.4) Technical problems to be solved: 3.4.1) Reliance on manual annotation: VAE reduces the reliance on a large amount of labeled data and enhances the model’s generative ability through self-supervised learning.

[0077] 3.4.2) Insufficient combination of image details and global information: VAE enhances the recognition of global structure by extracting latent spatial information, especially when dealing with fuzzy or incomplete data, providing a strong structural prior.

[0078] 3.5) Beneficial effects: It reduces the dependence on labeled data, enabling the model to learn adaptively on unlabeled data and improve training efficiency; it provides global semantic information, improves the accuracy of detail recovery and structure reconstruction, and enhances the model's performance in segmentation tasks.

[0079] (4) Frangi filter In this embodiment, the Frangi filter is used to enhance the details of retinal vessels, especially their edge features. The Frangi filter is a Hessian matrix-based filtering method that enhances slender tubular structures, making it particularly suitable for enhancing and extracting blood vessels. This filter locally enhances the input image, magnifying the details of the vascular structure while suppressing larger, circular structures such as the optic disc and optic cup. The specific implementation of the Frangi filter is as follows: 4.1) As Prior Input: In the initial stage of the diffusion model's denoising process, the vascular enhancement information output by the Frangi filter is weighted and incorporated into the input features. This information is added to the features of the single-channel image with progressively decaying weights, providing prior knowledge of the vascular structure, especially in cases of poor image quality or low contrast. This approach helps the diffusion model preserve vascular details during denoising, avoiding blurring or loss of vascular structures, thereby improving the accuracy of vascular region restoration.

[0080] 4.2) Post-processing strategy: The Frangi filter can be used to further optimize the segmentation results after the diffusion model has completed denoising and restoration. By locally enhancing the vascular region, the Frangi filter can refine the vascular edges and highlight the details of the vessels, especially thin and long tubular structures. This post-processing strategy helps improve the accuracy of vascular segmentation results and avoids the loss of details that may occur during diffusion, especially for vessels with blurred boundaries.

[0081] It is important to note that Frangi filtering, a Hessian-based method for enhancing tubular structures, can amplify elongated vascular patterns while suppressing large, circular structures such as the optic disc and cup regions, and often overemphasizes vessels within capillaries. These features can interfere with accurate CDR estimation and boundary localization; therefore, this embodiment applies the Frangi filter only in retinal vessel segmentation, and not in optic disc / cup segmentation.

[0082] 4.3) Technical Effects: The Frangi filter improves the accuracy of blood vessel segmentation by enhancing elongated structures (such as blood vessels) in an image. It utilizes the Hessian matrix to detect blood vessel morphology in an image, enhancing elongated and tortuous blood vessel structures while suppressing circular or larger structures, thus making the blood vessel region more prominent. In fundus images, the Frangi filter effectively enhances the detail information of blood vessels, especially in complex vascular networks, thereby improving the segmentation accuracy.

[0083] 4.4) Technical problems to be solved: 4.4.1) Insufficient accuracy in blood vessel segmentation: The Frangi filter enhances the visibility of blood vessels, enabling the model to more accurately identify and segment small blood vessels.

[0084] 4.4.2) Noise Influence: The Frangi filter effectively reduces the interference of background noise and enhances the model's focus on vascular structure by improving vascular morphology.

[0085] 4.5) Beneficial effects: Improved the segmentation accuracy of retinal vessels, especially in complex vascular networks, enabling clear differentiation of slender vessels; enhanced the vascular structure in the image, reduced the impact of noise on vessel segmentation, and improved the model's ability to capture details.

[0086] See Figure 4 , Figure 4 The table below shows the performance of various evaluation metrics on the publicly available fundus vessel dataset CHASEDB1 and the publicly available cup / disc dataset REFUGE, comparing the method of this embodiment with existing techniques. Specifically, on the CHASEDB1 dataset, four metrics were used to evaluate the model performance of retinal vessel segmentation: specificity (Spe), sensitivity (Sen), accuracy (Acc), and area under the ROC curve (AUC). For the segmentation evaluation metrics on the cup / disc dataset REFUGE, the Dice similarity coefficient (Dice) and intersection-over-union (IOU) coefficient were used to evaluate the performance of the cup / disc segmentation. Furthermore, the results are compared with existing techniques. Figure 4 It can be seen that the evaluation indicators of the method in this embodiment far exceed those of other methods.

[0087] In summary, compared with the prior art, the method of this embodiment has at least the following advantages and beneficial effects: 1) High-precision segmentation: By combining multi-scale U-Net with frequency domain reweighting, the complex vascular network and subtle edge structure of the optic disc / optic cup in fundus images are effectively captured, significantly improving segmentation accuracy.

[0088] 2) Strong robustness: The introduced deblurring diffusion module can effectively handle image blurring and noise caused by poor imaging quality, patient movement, etc. It restores clear image structural features through the reverse denoising process, making the model more adaptable to low-quality images.

[0089] 3) Reduced dependence on annotations: The variational autoencoder module obtains the global structural prior of the image through unsupervised learning, which provides strong semantic guidance for the segmentation task, thereby reducing the dependence on a large amount of accurately labeled data and improving the practicality and generalization ability of the model.

[0090] 4) Synergistic Enhancement Effect: This embodiment is not a simple stacking of models, but rather a clever design that enables the diffusion model, VAE, and multi-scale U-Net to work together. The global prior provided by VAE guides the diffusion process to achieve more structurally consistent denoising, while the features generated by the diffusion model in turn enhance the segmentation ability of U-Net, resulting in a technical effect of "1+1>2".

[0091] 5) Highly targeted: By selectively introducing the Frangi filter as a prior for blood vessel segmentation and adopting a two-step training strategy, the performance and training stability of the model on specific tasks are further optimized.

[0092] This application embodiment also provides a fundus image segmentation device, the device comprising: The image acquisition module is used to acquire fundus images to be segmented; The image segmentation module is used to input the fundus image into the trained segmentation model and output the segmentation result; the segmentation model is constructed and trained in the following manner: A multi-scale U-Net main network is constructed, in which the encoder and decoder perform feature fusion through skip connections, and a frequency domain reweighting module is set in at least one of the skip connections to enhance high-frequency detail features in the image. A deblurring diffusion module is constructed, which is configured to perform forward diffusion perturbation and inverse denoising recovery on the image features extracted by the multi-scale U-Net main network to generate enhanced features; A variational autoencoder module is constructed and configured to encode the input image to obtain latent features characterizing global structural information; The multi-scale U-Net main network, the deblurring diffusion module, and the variational autoencoder module are integrated so that the deblurring diffusion module uses the latent features provided by the variational autoencoder module as conditional guidance to enhance the feature map of the multi-scale U-Net main network, and the enhanced features are used for the final segmentation prediction.

[0093] It is understood that the content of the above method embodiments is applicable to the present device embodiments. The specific functions implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0094] This application also provides an electronic device, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the above-described method. This electronic device can be any smart terminal, including tablet computers, servers, and mobile phones.

[0095] It is understood that the content of the above method embodiments is applicable to this device embodiment. The specific functions implemented by this device embodiment are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0096] Please see Figure 5 , Figure 5 The hardware structure of an electronic device according to another embodiment is illustrated. The electronic device includes: The processor 501 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of this application. The memory 502 can be implemented as a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 502 can store the operating system and other applications. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program code is stored in the memory 502 and is called and executed by the processor 501 using the methods described in the embodiments of this application. The input / output interface 503 is used to implement information input and output; The communication interface 504 is used to enable communication and interaction between this device and other devices. Communication can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.). Bus 505 transmits information between various components of the device (e.g., processor 501, memory 502, input / output interface 503, and communication interface 504); The processor 501, memory 502, input / output interface 503, and communication interface 504 are connected to each other within the device via bus 505.

[0097] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method.

[0098] It is understood that the content of the above method embodiments is applicable to this storage medium embodiment. The specific functions implemented in this storage medium embodiment are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments.

[0099] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0100] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0101] It is understood that the content of the above method embodiments is applicable to the embodiments of this program product. The specific functions implemented in the embodiments of this program product are the same as those in the above method embodiments, and the beneficial effects achieved are also the same as those achieved in the above method embodiments. The executable computer program code or "code" used to perform the various embodiments can be written in high-level programming languages ​​such as C, C++, Python, Smalltalk, Java, JavaScript, Visual Basic, Structured Query Language (e.g., Transact-SQL), Perl, or in various other programming languages.

[0102] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0103] Those skilled in the art will understand that the technical solutions shown in the figures do not constitute a limitation on the embodiments of this application, and may include more or fewer steps than shown, or combine certain steps, or different steps.

[0104] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0105] Those skilled in the art will understand that all or some of the steps in the methods disclosed above, as well as the functional modules / units in the systems and devices, can be implemented as software, firmware, hardware, or suitable combinations thereof.

[0106] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0107] It should be understood that in this application, "at least one (item)" means one or more, and "more than" means two or more. "And / or" is used to describe the relationship between related objects, indicating that three relationships can exist. For example, "A and / or B" can represent three cases: only A exists, only B exists, and both A and B exist simultaneously, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one (item) of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one (item) of a, b, or c can represent: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of the units described above is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0109] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes multiple instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing programs, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0112] The preferred embodiments of the present application have been described above with reference to the accompanying drawings, but this does not limit the scope of the claims of the present application. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and substance of the embodiments of the present application shall be within the scope of the claims of the present application.

Claims

1. A fundus image segmentation method characterized by, The method comprises the following steps: Obtaining an eye fundus image to be segmented; Inputting the eye fundus image into a trained segmentation model to output a segmentation result; the segmentation model is constructed and trained in the following manner: A multi-scale U-Net main network is constructed, the encoder and the decoder of which are connected through a skip connection for feature fusion, and a frequency domain reweighting module is arranged in at least one of the skip connections to enhance the high-frequency detail features in the image; A deblurring diffusion module is constructed, which is configured to forwardly diffuse and disturb the image features extracted by the multi-scale U-Net main network and reversely denoise and restore them to generate enhanced features; A variational autoencoder module is constructed, which is configured to encode the input image to obtain latent features representing global structural information; The multi-scale U-Net main network, the deblurring diffusion module and the variational autoencoder module are integrated, so that the deblurring diffusion module uses the latent features provided by the variational autoencoder module as a conditional guide to enhance the feature maps of the multi-scale U-Net main network, and uses the enhanced features for final segmentation prediction.

2. The method of claim 1, wherein, The operation of the frequency domain reweighting module comprises: Performing two-dimensional discrete Fourier transform on the feature maps transmitted in the skip connection to convert them to the frequency domain; In the frequency domain, a preset low-frequency window function is used to adjust the weight of the low-frequency components of the feature maps; The frequency domain features with adjusted weights are converted back to the spatial domain through inverse Fourier transform; The feature maps converted back to the spatial domain are fused with the features of the corresponding layers of the decoder.

3. The method of claim 1, wherein, The workflow of the deblurring diffusion module comprises: Forward diffusion process: gradually adding Gaussian noise to the input image features to form a series of noisy features; Reverse denoising process: based on a denoising U-Net conditioned on the latent features of the variational autoencoder module, gradually predicting and removing noise from the noisy features to restore clear enhanced features.

4. The method of claim 1, wherein, In the training process of the segmentation model, a Frangi filter is used to process the eye fundus image to extract enhanced prior information of retinal blood vessels; and the prior information is introduced into the input of the deblurring diffusion module in a weighted sum manner.

5. The method of claim 4, wherein, The Frangi filter is only applied to the retinal blood vessel segmentation task, but not to the optic disc / cup segmentation task.

6. The method of claim 1, wherein, The training of the segmentation model adopts a two-step training method, comprising: Pre-training phase: training the multi-scale U-Net main network alone until it converges; Diffusion module training phase: fixing the parameters of the multi-scale U-Net main network, and training the deblurring diffusion module using the pre-trained variational autoencoder module; Joint fine-tuning phase: fixing the parameters of the deblurring diffusion module and the variational autoencoder module, and fine-tuning only the parameters of the multi-scale U-Net main network.

7. An apparatus for segmenting a fundus image, characterized by comprising: The device comprises: An image acquisition module for acquiring an eye fundus image to be segmented; An image segmentation module for inputting the eye fundus image into a trained segmentation model to output a segmentation result; the segmentation model is constructed and trained in the following manner: A multi-scale U-Net main network is constructed, the encoder and the decoder of which are connected through a jump connection for feature fusion, and a frequency domain reweighting module is arranged in at least one of the jump connections to enhance the high-frequency detail features in the image; A deblurring diffusion module is constructed, which is configured to forward diffusion disturbance and reverse denoising recovery on the image features extracted by the multi-scale U-Net main network to generate enhanced features; A variational autoencoder module is constructed, which is configured to encode the input image to obtain latent features representing global structure information; The multi-scale U-Net main network, the deblurring diffusion module and the variational autoencoder module are integrated, so that the deblurring diffusion module uses the latent features provided by the variational autoencoder module as a conditional guide to enhance the feature map of the multi-scale U-Net main network, and uses the enhanced features for final segmentation prediction.

8. An electronic device, comprising: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor executes the computer program to implement the method of any one of claims 1-6.

9. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 8. The computer program is executed by the processor to implement the method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1-6.

Images