A retinal arteriovenous segmentation method based on conditional diffusion model

Abstract

The application provides a retinal arteriovenous segmentation method based on a conditional diffusion model, including: preprocessing and data enhancement of a data set, division of a training set and a test set; construction of a retinal arteriovenous segmentation model based on a conditional diffusion model; arteriovenous semantic guided diffusion process pre-training and arteriovenous segmentation task fine-tuning; pre-training and fine-tuning with the training set; model performance testing with the test set. The advantages of the application include: a semantic constraint based conditional diffusion process training framework, which introduces the noise modeling ability of the diffusion model into medical image segmentation, improves the noise robustness of the model and the ability to distinguish subtle arteriovenous features; a deformable circular attention multi-scale feature fusion module is set, which realizes accurate modeling of the morphological characteristics of blood vessels and improves the segmentation integrity and continuity of small blood vessels; a texture and arteriovenous enhancement module is set, which further refines the blood vessel boundary segmentation and improves the model segmentation accuracy.

Description

Technical Field

[0001] This invention relates to a method for retinal arteriovenous segmentation based on a conditional diffusion model. Background Technology

[0002] Retinal fundus image analysis is currently the primary method for analyzing and diagnosing eye diseases in clinical medicine. Distinguishing between arteries and veins in retinal fundus images is fundamental to disease diagnosis. Traditional manual segmentation methods suffer from subjectivity and high time costs due to human intervention, and common image segmentation methods struggle to allow networks to accurately distinguish visually similar arteries and veins, especially the small vessels at the ends of vascular structures. Therefore, developing automated retinal arteriovenous segmentation technology is a key technology for intelligent diagnosis of eye diseases and is of paramount importance for improving the efficiency of clinical ophthalmology diagnosis.

[0003] Current research largely focuses on retinal vessel segmentation techniques. Existing methods can accurately distinguish between background and blood vessels in retinal images, but retinal arteriovenous classification techniques are still in the research stage. Although deep learning has made some progress in this field, the following problems remain:

[0004] 1) Most existing methods rely on directly inputting the arterial and venous features extracted from fundus images by multi-level shallow convolutional units into the decoder to obtain the segmentation result. At the same time, the imbalance between the number of target and background pixels and the large amount of noise in fundus images often lead to overfitting and pixel confusion during the model segmentation process.

[0005] 2) Simply splicing features at different scales or applying attention to a single scale cannot achieve true feature fusion.

[0006] 3) The large number of background pixels and the small number of arterial and vein pixels also cause redundant calculations in the attention mechanism.

[0007] The above-mentioned problems all contribute to the poor performance of the model in arteriovenous segmentation of fundus retinal images. Therefore, how to achieve accurate, reliable, and universal automatic arteriovenous segmentation of the retina is an urgent problem to be solved. Summary of the Invention

[0008] To address the aforementioned technical challenges in arteriovenous segmentation, this invention provides a retinal arteriovenous segmentation method based on a conditional diffusion model. This method achieves accurate segmentation of retinal arteries and veins through five steps: data acquisition, data processing, model construction, model training, and method testing. Performance tests were conducted on the internationally available DRIVE dataset, achieving an accuracy (Accuracy, Acc) of 96.94%, a specificity (Sp) of 97.12%, and a balanced accuracy (Bacc) of 96.72%, all outperforming other existing state-of-the-art methods.

[0009] This invention provides a retinal arteriovenous segmentation method based on a conditional diffusion model. The inventors validated the method using publicly available international datasets: by preprocessing and augmenting the DRIVE dataset, the method's versatility was improved; a semantically constrained conditional diffusion process training framework was constructed, and the processed data was input into the framework for pre-training. A deformable circular attention multi-scale feature fusion module and a texture and arteriovenous enhancement module were used to efficiently fuse and enhance global information during training, improving the distinguishability of arterial pixels, vein pixels, background, and noise in the image; the trained conditional diffusion model parameters were applied to achieve retinal arteriovenous segmentation, effectively improving the segmentation accuracy.

[0010] The objective of this invention is achieved through the following technical solution:

[0011] A method for retinal arteriovenous segmentation based on a conditional diffusion model according to an embodiment of the present invention includes the following steps:

[0012] Step 1: Obtain the publicly available dataset DRIVE. Preprocess and augment the obtained dataset, and split it into training and test sets;

[0013] Step 2: Use the PyTorch deep learning framework to build a retinal arteriovenous segmentation model based on a conditional diffusion model. The training process of the model includes a pre-training stage of diffusion process guided by arteriovenous semantics and a fine-tuning stage of arteriovenous segmentation task.

[0014] Step 3: In the pre-training stage, the training set data processed in Step 1 is input into the model, and pre-training is performed through the diffusion process guided by arteriovenous semantics to generate a pre-trained model.

[0015] Step 4: In the fine-tuning stage, freeze the diffusion encoder parameters in the pre-trained model, and fine-tune the conditional encoder and deformable circular attention multi-scale feature fusion module using the training set data; train the texture and arteriovenous enhancement module and the conditional decoder to generate the final segmentation model.

[0016] Step 5: Input the test set data from Step 1 into the final segmentation model to obtain the retinal arteriovenous segmentation results.

[0017] The data preprocessing and data augmentation in step 1 include: scaling the acquired original image and the corresponding pixel-level arteriovenous annotation ground truth to a uniform size of 512×512 pixels using bilinear interpolation; performing t-step random noise addition on the scaled image, where t=1,2,...,T; and randomly applying five data augmentation methods to the scaled image with a 50% probability, including random rotation, random translation, vertical flipping, elastic deformation, and random scaling, and performing the same transformation operation on the original image, the noisy image, and the ground truth image.

[0018] In step 2, the constructed retinal arteriovenous segmentation model based on the conditional diffusion model includes: a conditional encoder, a diffusion encoder, a deformable circular attention multi-scale feature fusion module, an arteriovenous semantic guidance module, a texture and arteriovenous enhancement module, a diffusion decoder, and a conditional decoder. During the pre-training phase of the diffusion process, the conditional encoder and diffusion encoder are connected in parallel. The conditional encoder extracts the arteriovenous semantic features of the original image, while the diffusion encoder extracts noise and detail features from the noisy image. The deformable circular attention multi-scale feature fusion module fuses the multi-level features extracted by the conditional encoder and diffusion encoder, and selects key pixels conforming to the vascular topology for attention calculation by predicting the circle radius and angular offset, outputting conditional fusion features and diffusion fusion features. The arteriovenous semantic guidance module receives the conditional fusion features, extracts the class codes of arteries and veins through a cross-attention mechanism, enhances the difference in arteriovenous features through a self-attention mechanism, and performs an inner product operation using the class codes and the shallow features extracted by the encoder to obtain a coarse prediction result, thus incorporating arteriovenous semantic information as a constraint into the conditional encoder and diffusion encoder. The diffusion decoder receives the diffusion fusion features. The process involves feature decoding and reconstruction of the denoised image. During the fine-tuning stage of the arteriovenous segmentation task, a conditional encoder and a diffusion encoder are connected in parallel. The conditional encoder extracts the semantic features of arteries and veins from the original image, while the diffusion encoder extracts noise and detail features from the noisy image. A deformable circular attention multi-scale feature fusion module fuses the multi-level features extracted by the conditional encoder and the diffusion encoder, outputting conditional fusion features and diffusion fusion features. An arteriovenous semantic guidance module receives the conditional fusion features, extracts the class codes for arteries and veins through a cross-attention mechanism, enhances the difference in arteriovenous features through a self-attention mechanism, and performs an inner product operation using the class codes and the shallow features extracted by the encoder to obtain a coarse prediction result, thus incorporating arteriovenous semantic information as a constraint into the conditional encoder. A texture and arteriovenous enhancement module receives the conditional fusion features during the fine-tuning stage, drives the arteriovenous process of blurred pixels through a fuzz correction branch, enhances the discriminative power of arteriovenous features through a semantic enhancement branch, and completes vessel boundary recognition and feature enhancement through a texture convolution branch, finally outputting the enhanced features. A conditional decoder receives the enhanced features and calculates the pixel-level three-class classification results, namely the segmentation probabilities of arteries, veins, and background.

[0019] In step 3, during model training via the arteriovenous semantic guidance diffusion process, the diffusion loss function is used to calculate the error between the output of the diffusion decoder and the denoising target. Simultaneously, the category loss function is used to calculate the error between the coarse prediction result output by the arteriovenous semantic guidance module and the true value. The parameters of the conditional encoder, diffusion encoder, deformable circular attention multi-scale feature fusion module, arteriovenous semantic guidance module, and diffusion decoder are updated iteratively through error backpropagation and stochastic gradient descent. The training data batch size is set to 2, a cosine annealing learning rate scheduling strategy is adopted, and the initial model learning rate is set to 1×10⁻. 4 The minimum learning rate is 1×10. ⁻8 The Adam optimizer was used to optimize the model parameters, and the model parameters were saved after 2000 iterations of training.

[0020] In step 4, when fine-tuning the model using the arteriovenous segmentation task, the parameters of the diffusion encoder in the pre-trained model are frozen. The error between the output of the conditional decoder and the ground truth is calculated using the class loss function. The error is backpropagated to fine-tune the parameters of the conditional encoder and the deformable circular attention multi-scale feature fusion module, and to train the parameters of the texture and arteriovenous enhancement module and the conditional decoder. The training data batch size is set to 2, and the model learning rate is set to 8×10⁻. 5 The Adam optimizer was used to optimize the model parameters, and a cosine annealing learning rate scheduling strategy was adopted, with the minimum learning rate set to 1×10⁻. 6 The final segmentation model was generated after 2000 iterations of training.

[0021] In step 5, when evaluating the retinal arteriovenous segmentation results, an image segmentation evaluation strategy is adopted to automatically calculate the pixel-level comparison index between the segmentation results and the ground truth. The test set images divided in step 1 and the corresponding ground truth arteriovenous annotations are sequentially input into the trained final segmentation model to obtain the three-class segmentation probability map of arteries, veins, and background output by the model. After Softmax normalization, the class with the highest probability is selected to obtain the final segmentation result. Accuracy, specificity, and balanced accuracy are used to evaluate the segmentation performance of the model.

[0022] The main advantages of the retinal arteriovenous segmentation method based on the conditional diffusion model proposed in this invention include:

[0023] 1. This invention designs a conditional diffusion process training framework based on semantic constraints, which introduces the excellent noise modeling ability of diffusion models into medical image segmentation. By embedding category information into the diffusion encoder through mask prediction guided by arteriovenous semantics, the robustness of the model to noise and its ability to distinguish subtle arteriovenous features are significantly enhanced.

[0024] 2. The deformable circular attention multi-scale feature fusion module designed in this invention dynamically selects the pixel points most relevant to the blood vessel topology for attention calculation by predicting circular displacement, thereby achieving accurate modeling of blood vessel morphological characteristics and effectively improving the segmentation integrity and continuity of small blood vessels.

[0025] 3. The texture and arteriovenous enhancement module designed in this invention completes the arteriovenous driving of blurred pixels, the enhancement of arteriovenous feature discrimination, and the identification and feature enhancement of blood vessel boundaries through fuzzy correction branch, semantic enhancement branch, and texture convolution branch, respectively, further refining blood vessel boundary segmentation and improving the model's segmentation accuracy. On the international public dataset DRIVE, the segmentation balance accuracy of this method reaches 96.72%. Attached Figure Description

[0026] Figure 1 This is a flowchart of a retinal arteriovenous segmentation method based on a basal conditional diffusion model according to an embodiment of the present invention.

[0027] Figure 2 This is a structural diagram of a deformable circular attention-based multi-scale feature fusion module according to an embodiment of the present invention.

[0028] Figure 3 This is a structural diagram of a texture and arteriovenous enhancement module according to an embodiment of the present invention.

[0029] Figure 4 This is a diagram of the model structure during the pre-training stage according to an embodiment of the present invention.

[0030] Figure 5 This is a model structure diagram of the segmentation fine-tuning stage according to an embodiment of the present invention. Specific implementation methods

[0031] In a retinal arteriovenous segmentation method based on a conditional diffusion model according to an embodiment of the present invention, a conditional encoder, a diffusion encoder, a deformable circular attention multi-scale feature fusion module, and an arteriovenous semantic guidance module are designed to pre-train the diffusion process of arteriovenous semantic guidance; an arteriovenous segmentation task fine-tuning strategy is used to freeze the diffusion encoder parameters, fine-tune the conditional encoder and the deformable circular attention multi-scale feature fusion module, and train the texture and arteriovenous enhancement module and the conditional decoder. By combining the fuzz correction, semantic enhancement and texture convolution branch of the texture and arteriovenous enhancement module, the segmentation of blood vessel boundaries and the distinction of arteriovenous categories are automatically optimized, and finally the retinal arteriovenous segmentation result is obtained.

[0032] The following detailed description, in conjunction with the accompanying drawings and specific embodiments, illustrates a retinal arteriovenous segmentation method based on a conditional diffusion model proposed in this invention.

[0033] The overall flow of the retinal arteriovenous segmentation method based on the conditional diffusion model according to an embodiment of the present invention is as follows: Figure 1 As shown, it includes:

[0034] Step S1: Obtain the DRIVE (Digital Retinal Images for Vessel Extraction) dataset and preprocess and augment it, then divide it into training and testing sets. According to a specific embodiment, the DRIVE dataset is an internationally publicly available dataset of retinal fundus images, containing 40 color fundus images, of which 20 are used for training and 20 for testing, with an image size of 565×584 pixels.

[0035] This step specifically includes:

[0036] Step S1.1: Image scaling, including: scaling the acquired original image and the corresponding pixel-level arteriovenous annotation ground truth to a uniform size of 512×512 pixels using bilinear interpolation, and performing the same scaling operation on the ground truth image to ensure pixel-level correspondence.

[0037] Step S1.2: Image noise addition, including: adding t steps of Gaussian noise to the original image x0 to obtain a noisy image x. t The noise addition process is as follows:

[0038] (1),

[0039] in Hyperparameters added to control noise To use Gaussian noise that conforms to a normal distribution, this embodiment sets the total number of diffusion steps T=1000. From 1 to 1×10 -4 linearly increase to 1-2×10 -2 For each original image, a noisy image is generated corresponding to steps t=1, 2, ..., T, and the step number t is randomly selected as the input to the diffusion encoder. The size of the noisy image remains unchanged at 512×512 pixels.

[0040] Step S1.3: Data augmentation, including: in each training round, randomly employing the following five data augmentation methods with a 50% probability: (1) random rotation, with rotation angles randomly selected between 0° and 360°; (2) random translation, with translation distances randomly selected between 0 and 256 pixels; (3) vertical flip; (4) elastic deformation; (5) random scaling, with scaling ratios randomly selected between 0.8 and 1.2 times the original image size. The same transformation operation is performed on the original image, the noisy image, and the ground truth image to ensure pixel-level correspondence.

[0041] Step S1.4: Divide the training set and the test set, including: according to the official division, use the first 20 images of the DRIVE dataset as the training set and the last 20 images as the test set.

[0042] Step S2: Construct a retinal arteriovenous segmentation model based on a conditional diffusion model, including: using the PyTorch deep learning framework library in Python to build a retinal arteriovenous segmentation model based on a conditional diffusion model, which includes a conditional encoder, a diffusion encoder, a deformable circular attention multi-scale feature fusion module, an arteriovenous semantic guidance module, a texture and arteriovenous enhancement module, a diffusion decoder, and a conditional decoder.

[0043] In the pre-training stage, the conditional encoder and the diffusion encoder are connected in parallel, and the extracted multi-level features are input into the deformable circular attention multi-scale feature fusion module. The fused diffusion features are input into the diffusion decoder, and the conditional fusion features are input into the arteriovenous semantic guidance module.

[0044] During the fine-tuning phase, the diffusion encoder parameters are frozen, and the conditional encoder is sequentially connected to the deformable circular attention multi-scale feature fusion module, the texture and arteriovenous enhancement module, and the conditional decoder.

[0045] This step specifically includes:

[0046] (a) Construct a conditional encoder to extract arterial and venous semantic features from the original image; construct a diffusion encoder to extract noise and detail features from the noisy image.

[0047] Both the conditional encoder and the diffusion encoder adopt a U-Net-like coding structure. Each encoder contains five downsampling levels. Each downsampling level includes two 3×3 convolutional layers (stride of 1, padding of 1) and a 2×2 max pooling layer, with channel numbers of 64, 128, 256, 512, and 1024 respectively. Each 3×3 convolutional layer has a stride of 1 and padding of 1.

[0048] For each downsampling level of the input image x 0i The downsampling process of a conditional encoder is represented as follows:

[0049] (2),

[0050] in The output feature map of the i-th downsampling layer of the conditional encoder, i=1,...,5, where ReLU is the activation function and Maxpool represents a 2×2 max pooling layer. The 3×3 convolution is passed through a conditional encoder to obtain feature maps at 5 downsampling levels.

[0051] For each downsampling level of the input image xti The downsampling process of the diffusion encoder is represented as:

[0052] (3),

[0053] in Let be the output feature map of the i-th downsampling layer of the diffusion encoder, i=1,...,5, where ReLU is the activation function and Maxpool represents a 2×2 max pooling layer. The feature maps are obtained by a 3×3 convolution and a diffusion encoder, resulting in 5 downsampling levels.

[0054] In this embodiment, the input image size is 512×512, so the feature map sizes of each level are 256×256, 128×128, 64×64, 32×32, and 16×16 respectively, and the number of channels are 64, 128, 256, 512, and 1024 respectively.

[0055] (b) Construct a deformable circular attention multi-scale feature fusion module to fuse the output features of the 2nd to 5th downsampling levels extracted by the conditional encoder and the diffusion encoder, where:

[0056] The deformable circular attention multi-scale feature fusion module first aligns the deep features, aligning all features to 256 dimensions, and outputs alignment condition features and alignment diffusion features. The feature alignment process can be represented as follows:

[0057] (4),

[0058] (5),

[0059] in, For the alignment condition feature of the i-th layer, This is an alignment diffusion feature.

[0060] Bundle and Each pixel is considered a query point, and each query point q is determined by predicting the radius r of the circle and the angular offset. q Key pixels conforming to the vascular topology are selected for attention calculation, and conditional fusion features and diffusion fusion features are output. r and q Prediction is made by performing a linear layer transformation on the query point:

[0061] (6),

[0062] (7),

[0063] Linear1 and Linear2 represent different linear layer transformations.

[0064] For each query point q, its corresponding key point position P q Predict using the following formula:

[0065] (8),

[0066] Where r is the predicted radius of the circle, Z is the angular offset determined by equation (7). iq For P q The reference point at the i-th layer is obtained through relative position.

[0067] Then, the query point q and its corresponding key point are evenly divided into multiple attention heads. The features of all levels, all attention heads, and all key points are weighted and summed using deformable circular attention to complete the deformable circular attention calculation operation.

[0068] (9),

[0069] in This represents the attention computation operation for deformable circles. and Here, represents the attention head weights and keypoint weights, respectively; are randomly generated trainable parameters following a normal distribution; L is the number of scales involved in the fusion; M is the number of attention heads; and K is the number of keypoints selected for each attention head. After the deformable circular attention computation operation, The parameters are updated to the conditional fusion feature T fi , The parameters are updated to the diffusion fusion feature S fi .

[0070] Specifically, the above fusion operation is performed on the alignment conditional features and alignment diffusion features respectively, and the conditional fusion features and diffusion fusion features are output respectively, with the same size as the input alignment conditional features and alignment diffusion features.

[0071] (c) Establish an arteriovenous semantic guidance module to receive conditional fusion features, extract arterial and venous category codes through a cross-attention mechanism, enhance the difference between arteriovenous features through a self-attention mechanism, and use the category codes and the shallow features extracted by the encoder to perform an inner product operation to obtain a coarse prediction result. Thus, the arteriovenous semantic features are incorporated as constraints into the diffusion encoder, including:

[0072] First, the arterial and vein category code E is learned from the conditional fusion features through cross-attention. AV :

[0073] (10)

[0074] (11),

[0075] Where Q represents the query, K i V represents the i-th layer bond. i Represents the value of the i-th layer. Represents cross attention. For activation function, Represents the feature dimension scaling factor. AV W is a learnable semantic encoding of arteriovenous veins. Q W K W V Q and K respectively i V i The learnable projection matrix.

[0076] Then, the differential characteristics of arteriovenous features are enhanced through a self-attention mechanism to obtain the enhanced semantic encoding of arteriovenous features. :

[0077] (12),

[0078] in This indicates the self-attention mechanism.

[0079] Finally, the enhanced arteriovenous semantic encoding is performed. The first layer features extracted by the conditional encoder and the diffusion encoder, respectively, are the shallowest layer features T1. ’ With S1 ’ The sum is then subjected to an inner product operation, followed by the sigmoid activation function and upsampling to obtain the coarse prediction result M:

[0080] (13)

[0081] The size of M is 512×512×2, and the last two dimensions represent the arterial probability and vein probability of the pixel, respectively.

[0082] (d) Establish a texture and arteriovenous enhancement module to receive conditional fusion features during the fine-tuning stage. This module uses a fuzz correction branch to drive the arteriovenous flow of blurred pixels, a semantic enhancement branch to enhance the discriminative power of arteriovenous features, and a texture convolution branch to identify blood vessel boundaries and enhance features. Finally, it outputs the enhanced features. ,include:

[0083] Input the i-th layer conditional fusion feature T fi The operation of the three branches is as follows:

[0084] d1) The operation of the fuzzy correction branch includes: performing preliminary segmentation of the input features, filtering out fuzzy pixels with prediction probabilities between 0.4 and 0.6, and correcting the fuzzy pixel features by using a cross-attention mechanism to approximate the artery feature encoding or vein feature encoding. The driving formula is:

[0085] (14)

[0086] Where D represents the enhancement result of arteriovenous vessels after driving, and k is... The number of channels, fi The value of the middle pixel, for The e-th channel encoding of the enhanced arteriovenous semantic coding, j This is the encoding of the j-th channel in the enhanced arteriovenous semantic coding.

[0087] The value of the post-drive arteriovenous enhancement result D was used as... The value of F is used to obtain the fuzzy correction feature of the i-th layer. cori .

[0088] d2) The operations of the semantic enhancement branch include:

[0089] Receive enhanced arteriovenous semantic coding The input, after passing through the Softmax activation function, is used as the minuend to distinguish the degree of difference in arterial and venous features. Then, a linear layer is used to generate channel weights, which are subsequently compared with the input fuzzy correction feature F. cori The semantically enhanced feature F is obtained by performing an inner product operation along the channel dimension. sem :

[0090] (15)

[0091] d3) The operations of the texture convolution branch include: inputting the conditional fusion feature of the i-th layer as T fi After performing dilated convolutions with different void ratios and summing them, the Sobel operator is used for edge detection to extract blood vessel boundary texture information and obtain edge enhancement features.

[0092] (16)

[0093] in This indicates the edge detection using the Sobel operator. for convolution, For dilated convolutions with a dilation rate of 2, For dilated convolutions with a dilation rate of 3,

[0094] d4) The output features of the semantic enhancement branch and the texture convolution branch are fused to obtain the enhanced features. :

[0095] (17).

[0096] (e) Establish a diffusion decoder and a conditional decoder. The diffusion decoder is used to receive the diffusion fusion feature S during the pre-training phase. fi And reconstruct the denoised image The conditional decoder is used to receive the enhanced feature T during the fine-tuning phase. fi It outputs pixel-level three-class classification results, namely artery-vein-background classification results.

[0097] The diffusion decoder structure is symmetrical to the encoder, containing five upsampling layers. Each layer consists of one 2×2 upsampling convolution and two 3×3 convolutional layers, with 256 channels in each layer. The formula is as follows:

[0098] (18)

[0099] in For the output of the i-th upsampling level of the diffusion decoder, For upsampling, ReLU is the activation function. for Convolution. The output of the diffusion decoder is the reconstructed denoised image. By analyzing the last layer ( ) to perform 1 The result is obtained through convolution, with dimensions of 512×512×3. The last three dimensions represent the RGB channels.

[0100] The conditional decoder has the same structure as the diffusion decoder, and is finally connected to a 1×1 convolutional layer and a softmax activation function.

[0101] The output of the conditional decoder is a three-channel probability map P with dimensions of 512×512×3, where the last three dimensions correspond to the artery segmentation probability, vein segmentation probability, and background segmentation probability, respectively. i This represents the probability of the class of pixel i predicted by the model.

[0102] Step S3: Perform pre-training, including: inputting the training set data processed in Step S1 into the retinal arteriovenous segmentation model based on the conditional diffusion model constructed in Step S2 for pre-training. The model structure of the retinal arteriovenous segmentation model based on the conditional diffusion model during the pre-training stage is as follows: Figure 4 As shown. Pre-training is performed through a diffusion process guided by arteriovenous semantics to generate pre-trained model parameters, including:

[0103] The diffusion reconstruction error and coarse segmentation error are calculated using the training set data. The diffusion loss function and the category loss function are jointly optimized, and the calculation formula is shown in (19):

[0104] Calculate the error between the output of the diffusion decoder and the denoising target. :

[0105] (19)

[0106] in, This represents a noisy image. This represents the denoised image reconstructed by the diffusion decoder. To conform to a normally distributed Gaussian noise, This represents the hyperparameter used to control the addition of noise. It is mainly used to constrain the parameters of the diffusion decoder so that the model can reconstruct the denoised image as accurately as possible.

[0107] The category loss function, combined with cross-entropy loss and Dice loss, calculates the error between the coarse prediction result output by the arteriovenous semantic guidance module and the final segmentation result output by the conditional decoder and the ground truth. :

[0108] (20)

[0109] in, This represents the true label (artery, vein, or background) of pixel i obtained directly from the DRIVE dataset. The value is between 0 and 1. This represents the probability of the class of pixel i predicted by the model. It is mainly used to constrain model parameters so that the model can classify arteries, veins and background pixels as accurately as possible.

[0110] The total loss function during the pre-training phase is:

[0111] (twenty one).

[0112] The pre-training process specifically includes:

[0113] Step S3.1: Input the original image into the conditional encoder and the noisy image into the diffusion encoder. The multi-level features extracted by the two encoders are fed into the deformable circular attention multi-scale feature fusion module to obtain conditional fusion features and diffusion fusion features. The diffusion fusion features are input into the diffusion decoder to reconstruct the denoised image, and the conditional fusion features are input into the arteriovenous semantic guidance module to obtain the coarse prediction result.

[0114] Step S3.2: Calculate the total loss function The parameters of the conditional encoder, diffusion encoder, deformable circular attention multi-scale feature fusion module, arteriovenous semantic guidance module, and diffusion decoder are updated iteratively through error backpropagation and stochastic gradient descent. The training data batch size is set to 2, a cosine annealing learning rate scheduling strategy is adopted, and the initial model learning rate is set to 1×10⁻. 4 The minimum learning rate is 1×10. ⁻8 The Adam optimizer is used to optimize the model parameters.

[0115] Step S3.3: If the pre-training loss If the number of rounds does not decrease after 50 consecutive rounds, the pre-training ends; otherwise, repeat steps S3.1 and S3.2.

[0116] Step S4: In the fine-tuning stage, the diffusion encoder parameters in the pre-trained model are frozen, and the conditional encoder and deformable circular attention multi-scale feature fusion module are fine-tuned using the training set data; the texture and arteriovenous enhancement module and the conditional decoder are trained to generate the final segmentation model. The model structure of the retinal arteriovenous segmentation model based on the conditional diffusion model in the fine-tuning stage is as follows: Figure 5 As shown, it includes:

[0117] The fine-tuning phase uses only the original images (without noise) for training, freezing the diffusion encoder parameters obtained in the pre-training phase to preserve their sensitivity to image detail features. The error between the final segmentation result output by the conditional decoder and the ground truth is calculated using a class loss function.

[0118] (twenty two)

[0119] in C is the total number of pixels, and C is the number of categories. The true label for pixel i is obtained directly from the DRIVE dataset. for The model predicts the probability that pixel i belongs to the c-th class. The class-balanced weight hyperparameter is set to 0.25. To focus on hyperparameters and control the decay of easily classified samples, we set it to 2.0. Parameters used for fine-tuning the conditional encoder, deformable circular attention multi-scale feature fusion module, texture and arteriovenous enhancement module, and conditional decoder.

[0120] The fine-tuning training process specifically includes:

[0121] Step S4.1: Freeze the diffusion encoder parameters in the pre-trained model, input the training set data (original images without noise) into the conditional encoder, and obtain the conditional fused features after the extracted features are fused by the deformable circular attention multi-scale feature fusion module.

[0122] Step S4.2: Input the conditional fusion features into the texture and arteriovenous enhancement module, and enhance the features through the fuzzy correction branch, semantic enhancement branch and texture convolution branch. Input the enhanced features into the conditional decoder and output the pixel-level three-class prediction results (artery, vein and background).

[0123] Step S4.3: Calculate the category loss function Backpropagation error was used to fine-tune the parameters of the conditional encoder and the deformable circular attention multi-scale feature fusion module, and the parameters of the texture and arteriovenous enhancement module and the conditional decoder were trained. The training data batch size was set to 2, and the model learning rate was set to 8×10⁻⁶. -5 The Adam optimizer is used, employing a cosine annealing learning rate scheduling strategy, with a minimum learning rate set to 1×10⁻⁶. -6 .

[0124] Step S4.4: If the model’s balanced accuracy on the validation set does not improve for 50 consecutive rounds, then end the fine-tuning; otherwise, repeat steps S4.1 to S4.3.

[0125] Step S5: Input the test set data from Step 1 into the final segmentation model to obtain the retinal arteriovenous segmentation results, including:

[0126] The test set images divided in step S1 and the corresponding ground truth values ​​of arterial and vein annotations are sequentially input into the trained final segmentation model to obtain the three-channel segmentation probability map of artery, vein and background output by the model. After Softmax normalization, the class with the highest probability is taken as the final prediction result of each pixel to obtain the retinal arterial and vein segmentation map.

[0127] The segmentation performance was evaluated using four metrics: accuracy (Accuracy, Acc), recall (Sensitivity, Se), specificity (Specificity, Sp), and balanced accuracy (Balanced Accuracy, Bacc).

[0128] To illustrate the advantages of the method proposed in this invention, the performance metrics of the latest deep learning models for epilepsy prediction on the same test set are compared, as shown in Table 1. Table 1: Performance Comparison of the Method of this Invention and Existing State-of-the-Art Methods on the DRIVE Dataset:

[0129] Table 1. Performance of different state-of-the-art methods on the public dataset DRIVE

[0130] MSGANet-RAV 2022 95.48 95.61 97.27 TR-GAN 2022 95.46 95.7 96.31 MMF-Net 2022 91.89 95.68 92.34 MISSFormer 2023 88.85 - 86.67 H2Former 2023 94.92 - 92.63 RRWNet 2024 95.31 - 95.78 MIUnet 2025 96.59 - 95.43 The method proposed in this invention 2026 96.72 96.94 97.12

[0131] As shown in Table 1, the retinal arteriovenous segmentation method based on the conditional diffusion model proposed in this invention outperforms existing advanced methods in terms of Acc, Bacc, and Sp.

[0132] The retinal arteriovenous segmentation method based on the conditional diffusion model provided by this invention has been described in detail above, but it is obvious that the scope of this invention is not limited thereto. Various modifications to the above embodiments without departing from the scope of protection defined by the appended claims are within the scope of this invention.

Claims

1. A modeling method for retinal arteriovenous segmentation based on a conditional diffusion model, characterized in that... include: (a) Construct a conditional encoder to extract the semantic features of arteriovenous veins from the original image; A diffusion encoder is constructed to extract noise and detail features from the noisy image. Both the conditional encoder and the diffusion encoder adopt a U-Net-like coding structure. Each encoder contains five downsampling levels, and each downsampling level includes two 3×3 convolutional layers and one 2×2 max pooling layer. The number of channels is 64, 128, 256, 512, and 1024 respectively. Each 3×3 convolutional layer has a stride of 1 and is padded with 1. For each downsampling level of the input image x 0i The downsampling process of the conditional encoder is represented as follows: （1）， in The output feature map of the i-th downsampling layer of the conditional encoder, i=1,...,5, where ReLU is the activation function and Maxpool represents a 2×2 max pooling layer. The 3×3 convolution is passed through a conditional encoder to obtain feature maps at 5 downsampling levels. For each downsampling level of the input image x ti The downsampling process of the diffusion encoder is represented as: （2）， in Let be the output feature map of the i-th downsampling layer of the diffusion encoder, i=1,...,5, where ReLU is the activation function and Maxpool represents a 2×2 max pooling layer. The feature maps are obtained through a 3×3 convolution and a diffusion encoder, resulting in 5 downsampling levels. (b) Construct a deformable circular attention multi-scale feature fusion module to fuse the output features of the 2nd to 5th downsampling levels extracted by the conditional encoder and the diffusion encoder, wherein: The deformable circular attention multi-scale feature fusion module first aligns the deep features, aligning all features to 256 dimensions, and outputs alignment conditional features and alignment diffusion features. The feature alignment process can be represented as follows: （3）， （4）， For the alignment condition feature of the i-th layer, For alignment diffusion characteristics, Bundle and Each pixel is considered a query point, and each query point q is determined by predicting the radius r of the circle and the angular offset. q Attention is calculated by selecting key pixels that conform to the vascular topology, and the outputs conditional fusion features and diffusion fusion features, r and q Prediction is made by performing a linear layer transformation on the query point: （5）， （6）， Where Linear1 and Linear2 represent the first linear layer transformation and the second linear layer transformation, respectively. For each query point q, the location P of its corresponding key point is predicted using the following formula. q : （7）， Where r is the predicted radius of the circle, Z is the angular offset determined by equation (7). iq For P q The reference point at the i-th layer is obtained through relative position. Then, the query point q and its corresponding key point are evenly divided into multiple attention heads. The features of all levels, all attention heads, and all key points are weighted and summed using deformable circular attention to complete the deformable circular attention calculation operation. （8）， in This represents the attention computation operation for deformable circles. and Here, represents the attention head weights and keypoint weights, respectively; are randomly generated trainable parameters that conform to a normal distribution; L is the number of scales involved in the fusion; M is the number of attention heads; and K is the number of keypoints selected for each attention head. After the deformable circular attention operation, The parameters are updated to the conditional fusion feature T fi , The parameters are updated to the diffusion fusion feature S fi , Specifically, the above fusion operation is performed on the alignment conditional features and alignment diffusion features respectively, outputting conditional fusion features and diffusion fusion features with the same dimensions as the input alignment conditional features and alignment diffusion features. (c) Establish an arteriovenous semantic guidance module to receive conditional fusion features, extract arterial and venous category codes through a cross-attention mechanism, enhance the difference between arteriovenous features through a self-attention mechanism, and use the category codes and the shallow features extracted by the encoder to perform an inner product operation to obtain a coarse prediction result, thereby incorporating arteriovenous semantic features as constraints into the diffusion encoder, including: First, the arterial and vein category code E is learned from the conditional fusion features through cross-attention. AV : （9）， （10）， Where Q represents the query, K i V represents the i-th layer bond. i Represents the value of the i-th layer. Represents cross attention. For activation function, E represents the feature dimension scaling factor. AV W is a learnable semantic encoding of arteriovenous veins. Q W K W V Q and K respectively i V i The learnable projection matrix, Then, the differential characteristics of arteriovenous features are enhanced through a self-attention mechanism to obtain the enhanced semantic encoding of arteriovenous features. : （11）， in This indicates the self-attention mechanism. Finally, the enhanced arteriovenous semantic encoding is performed. The first layer features extracted by the conditional encoder and the diffusion encoder, respectively, are the shallowest layer features T1. ’ With S1 ’ The sum is then subjected to an inner product operation, followed by the sigmoid activation function and upsampling to obtain the coarse prediction result M: （12）， The size of M is 512×512×2, and the last two dimensions represent the artery probability and vein probability of the pixel, respectively. (d) Establish a texture and arteriovenous enhancement module to receive conditional fusion features during the fine-tuning stage. This module uses a fuzzy correction branch to drive the arteriovenous flow of blurred pixels, a semantic enhancement branch to enhance the discriminative power of arteriovenous features, and a texture convolution branch to identify blood vessel boundaries and enhance features. Finally, it outputs the enhanced features. ,include: Input the i-th layer conditional fusion feature T fi The operation of the three branches is as follows: d1) The operation of the fuzzy correction branch includes: performing preliminary segmentation of the input features, filtering out fuzzy pixels with prediction probabilities between 0.4 and 0.6, and correcting the fuzzy pixel features by using a cross-attention mechanism to approximate the artery feature encoding or vein feature encoding. The driving formula is: （13）， Where D represents the enhancement result of arteriovenous vessels after driving, and k is... The number of channels, For T fi The value of the middle pixel, for transpose, This is the encoding of the e-th channel in the enhanced arteriovenous semantic coding. j This is the encoding of the j-th channel in the enhanced arteriovenous semantic coding. The value of the post-drive arteriovenous enhancement result D was used as... The value of F is used to obtain the fuzzy correction feature of the i-th layer. cori , d2) The operations of the semantic enhancement branch include: Receive enhanced arteriovenous semantic coding The input, after passing through the Softmax activation function, is used as the minuend to distinguish the degree of difference in arterial and venous features. Then, a linear layer is used to generate channel weights, which are subsequently compared with the input fuzzy correction feature F. cori The semantically enhanced feature F is obtained by performing an inner product operation along the channel dimension. sem : （14）， d3) The operations of the texture convolution branch include: inputting the conditional fusion feature of the i-th layer as T fi After performing dilated convolutions with different void ratios and summing them, the Sobel operator is used for edge detection to extract blood vessel boundary texture information and obtain edge enhancement features. （15）， in This indicates the edge detection using the Sobel operator. for convolution, For dilated convolutions with a dilation rate of 2, For dilated convolutions with a dilation rate of 3, d4) The output features of the semantic enhancement branch and the texture convolution branch are fused to obtain the enhanced features. : （16）， (e) Establish a diffusion decoder and a conditional decoder. The diffusion decoder is used to receive the diffusion fusion feature S during the pre-training phase. fi And reconstruct the denoised image The conditional decoder is used to receive the enhanced feature T during the fine-tuning phase. fi It outputs pixel-level three-class classification results, namely, artery-vein-background classification results. The diffusion decoder structure is symmetrical to the encoder, containing five upsampling layers. Each layer consists of one 2×2 upsampling convolution and two 3×3 convolutional layers, with 256 channels in each layer. The formula is as follows: （17）， in For the output of the i-th upsampling level of the diffusion decoder, For upsampling, ReLU is the activation function. for The output of the convolution and diffusion decoder is the reconstructed denoised image. By analyzing the last layer ( ) to perform 1 The result is obtained through convolution, with dimensions of 512×512×3. The last three dimensions represent the RGB channels. The conditional decoder has the same structure as the diffusion decoder, and is finally connected to a 1×1 convolutional layer and a softmax activation function. The output of the conditional decoder is a three-channel probability map P with dimensions of 512×512×3, where the last three dimensions correspond to the artery segmentation probability, vein segmentation probability, and background segmentation probability, respectively. i This represents the probability of the class of pixel i predicted by the model.

2. The modeling method for retinal arteriovenous segmentation based on the conditional diffusion model according to claim 1, characterized in that: Modeling is performed using the PyTorch deep learning framework library in Python.

3. A method for retinal arteriovenous segmentation based on a conditional diffusion model, characterized in that... include: Step S1: Preprocess and augment the pre-acquired publicly available retinal fundus image dataset DRIVE, and divide it into training and test sets; Step S2: Perform the modeling method according to claim 1 or 2. Step S3: Train the retinal arteriovenous segmentation model established in step S2. The training includes a pre-training stage of the diffusion process guided by arteriovenous semantics and a fine-tuning stage of the arteriovenous segmentation task. In the pre-training stage of the diffusion process, the data of the training set processed in step S1 is input into the retinal arteriovenous segmentation model, and pre-training is performed through the diffusion process guided by arteriovenous semantics to generate a pre-trained model. Step S4: In the fine-tuning stage, freeze the diffusion encoder parameters in the pre-trained model, and fine-tune the conditional encoder and deformable circular attention multi-scale feature fusion module using the training set data; train the texture and arteriovenous enhancement module and the conditional decoder to generate the final segmentation model. Step S5: Input the test set data from Step 1 into the final segmentation model to obtain the retinal artery and vein segmentation results. in: Step S1 includes: scaling the acquired original image and the corresponding pixel-level arteriovenous annotation ground truth to a uniform size of 512×512 pixels using bilinear interpolation; performing t-step random noise addition on the scaled image, t=1,2,…,T; and randomly applying five data augmentation methods to the scaled image with a 50% probability, including random rotation, random translation, vertical flip, elastic deformation, and random scaling, and performing the same transformation operation on the original image, the noisy image, and the ground truth image. In step S3: When training the model through the diffusion process guided by arteriovenous semantics, the diffusion loss function is used to calculate the error between the output of the diffusion decoder and the denoising target. At the same time, the category loss function is used to calculate the error between the coarse prediction result output by the arteriovenous semantic guidance module and the true value. The parameters of the conditional encoder, diffusion encoder, deformable circular attention multi-scale feature fusion module, arteriovenous semantic guidance module and diffusion decoder are updated iteratively through error backpropagation and stochastic gradient descent. In step S4: when fine-tuning the model through the arteriovenous segmentation task, the parameters of the diffusion encoder in the pre-trained model are frozen, the error between the output of the conditional decoder and the ground truth is calculated using the class loss function, the error is backpropagated to fine-tune the parameters of the conditional encoder and the deformable circular attention multi-scale feature fusion module, and the parameters of the texture and arteriovenous enhancement module and the conditional decoder are trained. In step S5: when evaluating the retinal arteriovenous segmentation results, an image segmentation evaluation strategy is adopted to automatically calculate the pixel-level comparison index between the segmentation results and the ground truth; the test set images and the corresponding ground truth values ​​of arteriovenous annotations are sequentially input into the trained final segmentation model to obtain the three-class segmentation probability map of artery, vein and background output by the model, and the final segmentation result is obtained by taking the class with the highest probability after Softmax normalization; and the segmentation effect of the model is evaluated by accuracy, specificity and balanced accuracy.

4. The retinal arteriovenous segmentation method based on the conditional diffusion model as described in claim 3, characterized in that: Step S1 includes: Step S1.1: Image scaling, including: scaling the acquired original image and the corresponding pixel-level arteriovenous annotation ground value to a uniform size of 512×512 pixels through bilinear interpolation, and performing the same scaling operation on the ground value image to ensure pixel-level correspondence; Step S1.2: Image noise addition, including: adding t steps of Gaussian noise to the original image x0 to obtain a noisy image x. t The noise addition process is as follows: （18）， in Hyperparameters added to control noise To accommodate Gaussian noise that conforms to a normal distribution, the total number of diffusion steps is set to T = 1000. From 1 to 1×10 -4 linearly increase to 1-2×10 -2 ; Step S1.3: Data augmentation, including: in each round of training, with a 50% probability, randomly use the following five data augmentation methods: random rotation, random translation, vertical flip, elastic deformation, and random scaling, and perform the same transformation operation on the original image, the noisy image, and the ground truth image. Step S1.4: Divide the training set and the test set, including: According to the official classification, the dataset is divided into a training set and a test set.

5. The retinal arteriovenous segmentation method based on the conditional diffusion model as described in claim 3, characterized in that: In step S3, the training set data processed in step 1 is input into the model, and pre-training is performed through a diffusion process guided by arteriovenous semantics to generate a pre-trained model, including: The diffusion reconstruction error and coarse segmentation error are calculated using the training set data, and jointly optimized using the diffusion loss function and the category loss function. The calculation formula is as follows: The diffusion loss function calculates the error between the output of the diffusion decoder and the denoising target. : （19）， in, This represents a noisy image. This represents the denoised image predicted by the diffusion decoder. This indicates the addition of Gaussian noise. Indicates noise scheduling parameters, It is mainly used to constrain the parameters of the diffusion decoder, so that the model can reconstruct the denoised image as accurately as possible. The category loss function, combined with cross-entropy loss and Dice loss, calculates the error between the coarse prediction result output by the arteriovenous semantic guidance module and the final segmentation result output by the conditional decoder and the ground truth. : （20）， in, Represents the true label (artery, vein, or background) of pixel i. Indicates the model's predicted probability. It is mainly used to constrain model parameters so that the model can classify artery, vein, and background pixels as accurately as possible. The total loss function during the pre-training phase is: （21）， The pre-training process includes: Step S3.1: Input the original image into the conditional encoder and the noisy image into the diffusion encoder. The multi-level features extracted by the two encoders are fed into the deformable circular attention multi-scale feature fusion module to obtain conditional fusion features and diffusion fusion features. The diffusion fusion features are input into the diffusion decoder to reconstruct the denoised image, and the conditional fusion features are input into the arteriovenous semantic guidance module to obtain the coarse prediction result. Step S3.2: Calculate the total loss function The parameters of the conditional encoder, diffusion encoder, deformable circular attention multi-scale feature fusion module, arteriovenous semantic guidance module, and diffusion decoder are updated iteratively through error backpropagation and stochastic gradient descent. Step S3.3: If the pre-training loss If the number of rounds does not decrease after 50 consecutive rounds, the pre-training ends; otherwise, repeat steps S3.1 and S3.

2.

6. The retinal arteriovenous segmentation method based on the conditional diffusion model as described in claim 3, characterized in that: In step S4, the diffusion encoder parameters in the pre-trained model are frozen, and the conditional encoder and deformable circular attention multi-scale feature fusion module are fine-tuned using the training set data. The texture and arteriovenous enhancement module and the conditional decoder are trained to generate the final segmentation model, including: In the fine-tuning stage, only the original images (without noise) are used for training. The parameters of the diffusion encoder obtained in the pre-training stage are frozen to preserve their sensitivity to image detail features. The error between the final segmentation result output by the conditional decoder and the ground truth is calculated using a class loss function. （22）， in C is the total number of pixels, and C is the number of categories. Let i be the truth label for pixel i. To predict the probability that pixel i belongs to c for the model, For class-balanced weights, To focus on parameters and control the decay of easily classified samples, Parameters used for fine-tuning the conditional encoder, deformable circular attention multi-scale feature fusion module, texture and arteriovenous enhancement module, and conditional decoder. The fine-tuning training process specifically includes: Step S4.1: Freeze the diffusion encoder parameters in the pre-trained model, input the training set data (original images without noise) into the conditional encoder, and fuse the extracted features through the deformable circular attention multi-scale feature fusion module to obtain the conditional fused features. Step S4.2: Input the conditional fusion features into the texture and arteriovenous enhancement module, and enhance the features through the fuzzy correction branch, semantic enhancement branch and texture convolution branch. Input the enhanced features into the conditional decoder and output the pixel-level three-class prediction results (artery, vein and background). Step S4.3: Calculate the category loss function The backpropagation error is used to fine-tune the parameters of the conditional encoder and the deformable circular attention multi-scale feature fusion module, and to train the parameters of the texture and arteriovenous enhancement module and the conditional decoder. Step S4.4: If the model’s balanced accuracy on the validation set does not improve for 50 consecutive rounds, then end the fine-tuning; otherwise, repeat steps S4.1 to S4.

3.

7. The retinal arteriovenous segmentation method based on the conditional diffusion model as described in claim 3, characterized in that: In step S5, the model parameters saved in step S4 are loaded to obtain the trained retinal artery and vein segmentation model. The test set data from step S1 is input into the final segmentation model to obtain the retinal artery and vein segmentation results, including: The test set images and corresponding ground truth values ​​of arterial and vein annotations from step S1 are sequentially input into the trained final segmentation model to obtain the three-channel segmentation probability map of artery, vein, and background output by the model. After Softmax normalization, the class with the highest probability is taken as the final prediction result for each pixel, thus obtaining the retinal arterial and vein segmentation map. The segmentation performance was evaluated using four metrics: accuracy (Accuracy, Acc), recall (Sensitivity, Se), specificity (Specificity, Sp), and balanced accuracy (Balanced Accuracy, Bacc).

8. A computer-readable storage medium storing a computer program that enables a processor to perform the method according to any one of claims 1-7.

Images