Diabetic retinopathy detection algorithm based on diffusion model

Abstract

According to the method, the de-noising diffusion model TCG-DiffDRC is innovatively and specially applied to classification of the diabetic retinopathy. According to the model, a triple condition guidance strategy is designed, deep mining of lesion features is achieved through three independent branches, overall features of an image are extracted through an improved ResNet network in a global feature branch, a global descriptor is generated through a class activation graph CAM, a lesion detail branch is based on an interpretable neural transducer (INTR) model, and the lesion detail branch is based on an explained neural transducer (INTR) model. And fine features of the lesion are extracted through a multi-head attention mechanism of Transform. And then the extracted features are sent to a diffusion model for training, image reconstruction is refined step by step, and finally efficient classification of diabetic retinopathy is realized. The performance of experimental results on a challenging APTOS2019 data set proves the superiority of the TCG-DiffDRC model, the accuracy of the TCG-DiffDRC model reaches 86.3%, the Kappa value reaches 75.8%, the effectiveness of the TCG-DiffDRC model in a medical image classification task is proved, and the TCG-DiffDRC model is superior to the current most advanced method.

Description

Technical Field:

[0001] The present invention relates to the field of medical image processing, and particularly to a method for detecting diabetic retinopathy based on a diffusion model. Background Art:

[0002] Diabetic Retinopathy (DR) is a major cause of blindness, affecting 93 million people worldwide. DR is typically divided into five stages: asymptomatic, mild (microaneurysm (MA)), moderate (hemorrhage (HE)), severe (hard exudate (EX), soft exudate (SE)), and proliferative lesions (PDR (neovascularization, venous beading, scar formation)). In the past few decades, many researchers have been working on solving the classification of diabetic retinopathy. In the early years, researchers focused on traditional image processing methods, using color fundus images, extracting features through image processing techniques, and using Support Vector Machines (SVM) for binary classification. With the progress of technology, multi-class classification models have been applied, including Principal Component Analysis (PCA) and classification using decision trees, Naive Bayes, or k-NN algorithms. Deep learning methods, especially CNN-based models, have been widely used in automated diagnosis due to their excellent performance in image classification and object detection tasks. For example, the U-Net model, which originated from segmentation in the biomedical field, has been widely used in the segmentation research of diabetic retinopathy. In addition, deep learning models such as the Encoder-decoder architecture have also been used for the detection of diabetic retinopathy lesion areas.

[0003] Although deep learning technology has made significant progress in improving the accuracy of diabetic retinopathy classification, the lack of training data remains a challenge. To address this issue, researchers have adopted strategies such as data augmentation to expand the dataset and improve the generalization ability of the model. In addition, to improve the interpretability of the model and reduce attention bias, researchers are also exploring multi-task learning and ensemble learning methods. Nevertheless, deep learning models still face challenges in clinical applications, including how to better integrate the model with clinical practice and how to improve the applicability of the model to patients of different races and regions.

[0004] In recent years, thanks to its powerful data distribution modeling capabilities, diffusion models have made significant progress in the field of image generation. Among them, the denoising diffusion probability model is inspired by non-equilibrium thermodynamics. At present, there are abundant parameters inside the probability denoising diffusion model, which makes the performance of the model seriously dependent on the selection of model parameters. Therefore, related derivative models with the goal of optimizing various parameters are gradually proposed. In the image generation task in the medical field, for the inverse problem of reconstructing images from measured data, some scholars use the generative model of scores to reconstruct images consistent with the prior data. When performing medical defect detection tasks, the DDPM model can be used to replace the original autoencoder model to train healthy images. And during inference, abnormalities can be detected by subtracting the healthy image samples generated from the original image. It can be seen that the diffusion model has certain advantages over the traditional generative model and has great potential in the field of generation, but there are still some problems that need to be solved. Summary of the invention:

[0005] This paper innovatively applies the denoising diffusion model TCG-DiffDRC specifically to the classification of diabetic retinopathy. We designed a triple conditional guidance strategy to achieve deep mining of lesion features through three independent branches, and then send the extracted features to the diffusion model for training, gradually refine the image reconstruction, and finally achieve efficient classification of diabetic retinopathy.

[0006] A diabetic retinopathy detection algorithm based on a diffusion model, characterized by comprising the following steps:

[0007] Step 1: Combine the global, local, and lesion branches to extract and fuse different features of the image;

[0008] Step 2: Send the extracted features to the diffusion model for training, and gradually refine the image reconstruction;

[0009] Step 3: Use MMD regularization to converge the model faster in a stable way;

[0010] The implementation of step 1 includes:

[0011] Step 1.1: Use the improved ResNet network and class activation map (CAM) technology to extract the overall features of the image from the global feature branch and generate a global descriptor;

[0012] Step 1.2: Process multiple regions of the image in the local feature branch through the gated attention mechanism to integrate local information;

[0013] Step 1.3: Based on the interpretable neural transformer (INTR) model, extract the subtle features of the lesion in the lesion detail branch;

[0014] The implementation of Step 2 includes:

[0015] Step 2.1: Fuse global, local, and lesion features to form a comprehensive feature representation;

[0016] Step 2.2: During the diffusion process, achieve fine-grained reconstruction of the image by simulating the progressive process from data to noise and reversing this process;

[0017] The implementation of Step 3 includes:

[0018] Step 3.1: Integrate MMD regularization in the loss function to reduce the distribution difference between the generated data and the real data;

[0019] Step 3.2: Compare with the performance of recent advanced methods.

[0020] Beneficial effects of the present invention: In this study, the denoising diffusion model TCG-DiffDRC was innovatively applied specifically to the classification of diabetic retinopathy. We designed a triple conditional guidance strategy to deeply mine the lesion features through three independent branches. Among them, the global feature branch uses an improved ResNet network to extract the overall features of the image and generates a global descriptor through the class activation map CAM; the local feature branch processes multiple regions of the image and integrates local information through a gated attention mechanism; the lesion detail branch is based on the interpretable neural transformer (INTR) model and extracts the subtle features of the lesion through the multi-head attention mechanism of the Transformer. Subsequently, the extracted features are fed into the diffusion model for training to gradually refine the reconstruction of the image, and finally achieve efficient classification of diabetic retinopathy. The experimental results on the challenging APTOS2019 dataset demonstrate the superiority of the TCG-DiffDRC model, with an accuracy rate of 86.3% and a Kappa value of 75.8%, confirming the effectiveness of the TCG-DiffDRC model in medical image classification tasks and being superior to the current state-of-the-art methods. Description of the drawings:

[0021] Figure 1 It is the structural diagram of the TCG-DiffDRC model.

[0022] Figure 2 It is the structural diagram of the three branches of TCG.

[0023] Figure 3 It is the denoising diffusion model diagram. Detailed implementation manners:

[0024] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0025] Figure 1 is the structural diagram of the TCG-DiffDRC model implemented by the present invention, Figure 2 is the structural diagram of the three branches of TCG, Figure 3 is the diagram of the denoising diffusion model, as Figure 1 Figure 2 Figure 3 shown, the method includes:

[0026] Step 1: Extract and fuse different features of the image through three branches: global, local, and lesion;

[0027] Step 2: Send the extracted features into the diffusion model for training to gradually refine the reconstruction of the image;

[0028] Step 3: Use MMD regularization to make the model converge faster in a stable manner;

[0029] The implementation of Step 1 includes:

[0030] Step 1.1: Use the improved ResNet network to extract the overall features of the image, denoted as y global ; generate a global descriptor through the Class Activation Map (CAM) technology, and the global feature y global extract the most critical features from the CAM of each category by adopting the Top-T percentage aggregation strategy.

[0031]

[0032] Among them, is the global descriptor for category c, T c is the set of the top T percentage pixels in the CAM of category c, M c (i, j) is the pixel value of M c at position (i, j).

[0033] Step 1.2: Process multiple regions x of the image crop , integrate local information through the gated attention mechanism, and the hidden representation h crops has a shape of batch size ×num_crops×h dim . Convert it into a two-dimensional tensor h through the reshaping function and reshaped, whose shape is (batch size ×num_crops)×h dim . This operation facilitates the subsequent linear transformation. The reshaped tensor h reshaped is used to calculate the attention projection. After passing through the gated attention mechanism to calculate the weights of each feature vector, the formula is as follows:

[0034] w i =σ(W Q ·h reshaped )⊙tanh(W K ·h reshaped )

[0035] n is the number of crops in each batch, and h dim is the feature dimension of each crop. Among them, and represent the weight matrices of the query and key respectively, d is the intermediate dimension, and these two matrices are used to extract attention-related representations from the input features. σ is the Sigmoid activation function, which is used to introduce non-linearity and simulate the gating mechanism, and ⊙ is the element-wise multiplication.

[0036] Step 1.3: Extract the subtle features of the lesion based on the Interpretable Neural Transformer (INTR) model, denoted as y lesion , and use the multi-head self-attention mechanism to capture the features of the lesion area;

[0037] The implementation of Step 2 includes:

[0038] Step 2.1: Fuse the global, local, and lesion features to form the final feature representation y fusion . y fusion =αy global +βy local +γy lesion (α + β + γ = 1);

[0039] Step 2.2: Simulate the gradual transformation process from the data distribution to Gaussian noise, and then reverse this process to reconstruct or generate data:

[0040]

[0041] Among them α t =1 - β t , and use the linear noise schedule {β t} t＝1:T ∈(0, 1) T . After that;

[0042] The implementation of Step 3 includes:

[0043] Step 3.1: Integrate MMD regularization into the loss function to reduce the distribution difference between the generated data and the real data:

[0044]

[0045] where K(x, x ′ ) represents the kernel function value within the dataset x, K(x, y) represents the kernel function value between x and y, and K(y, y ′ ) represents the kernel function value within the dataset y. Conditional-specific MMD regularization is also applied to the local prior. That is

[0046] Table 1 shows the comparison of the quadratic weighted Kappa of the method of the present invention and other advanced methods in the DR classification evaluation model. The results fully demonstrate the superiority of the method of the present invention in the diabetic retinopathy classification task.

[0047] Table 1 Performance comparison of the method of the present invention and recent advanced methods (APTOS dataset)

[0048]

[0049] Table 2 shows the comparison of the ablation experiment performance of the method of the present invention. The results fully demonstrate that the method of the present invention has better performance in the diabetic retinopathy detection task.

[0050] Table 2 Comparison of the performance of replacing the basic network by the method of the present invention

[0051]

[0052] It should be understood that the parts not elaborated in detail in this specification belong to the prior art.

[0053] As described above in conjunction with the accompanying drawings, this is only the specific implementation manner and process of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art should understand that this is only an example, and various changes and substitutions can be made to this implementation manner without departing from the essence of the present invention. The scope of the present invention is only defined by the appended claims.

[0054] The embodiments described by the present invention with reference to the accompanying drawings are exemplary and are only used to explain the present invention. It cannot be understood as a limitation of the present invention. The specific scope of the embodiments of the present invention is not limited by this. On the contrary, all embodiments of the present invention include all changes and modifications that fall within the spirit and connotation of the appended claims.

Claims

1. A diabetic retinopathy detection algorithm based on a diffusion model, characterized in that: The steps include: Step 1: Extract and fuse different features of the image through three branches: global, local, and lesion; Step 2: Send the extracted features to the diffusion model for training, and gradually refine the image reconstruction; Step 3: Use MMD regularization to converge the model faster in a stable way; The implementation of step 1 includes: Step 1.1: Use the improved ResNet network to extract the overall features of the image, denoted as y global ; Generate a global descriptor through the class activation map (CAM) technology, the global feature y global By adopting the Top-T percentage aggregation strategy, the most critical features are extracted from each category of CAM; in, is the global descriptor for category c, T c is the set of pixels with the highest T percentage in the CAM of category c, M c (i, j) is M c The pixel value at position (i, j); Step 1.2: Process multiple regions x of the image crop , integrate local information through the gated attention mechanism, and hide the representation h crops The shape is batch size ×num_crops×h dim By reshaping the function Convert it to a two-dimensional tensor h reshaped , whose shape is (batch size × num_crops) × h dim This operation facilitates the subsequent linear transformation. The reshaped tensor h reshaped It is used to calculate the attention projection, and the weight of each feature vector is calculated through the gated attention mechanism. The formula is as follows: w i =σ(W Q ·h reshaped )⊙tanh(W K ·h reshaped ) n is the number of crops in each batch, h dim is the feature dimension of each crop. and Represent the weight matrices of query and key respectively, d is the intermediate dimension, and these two matrices are used to extract attention-related representations from the input features. σ is the Sigmoid activation function, which is used to introduce nonlinearity and simulate the gating mechanism, and ⊙ is the element-by-element multiplication; Step 1.3: Extract the subtle features of the lesion based on the interpretable neural transformer (INTR) model, denoted as y lesion ,Use multi-head self-attention mechanism to capture the features of the lesion area; The implementation of step 2 includes: Step 2.1: Fusion of global, local and lesion features to form the final feature representation y fusion .y fusion =αy global +βy local +γy lesion (α+β+γ=1); Step 2.2: Simulate the gradual transformation of the data distribution into Gaussian noise and then reverse the process to reconstruct or generate the data: in α t =1-β t , using a linear noise schedule {β t } t＝1：T ∈(0, 1) T .after; The implementation of step 3 includes: Step 3.1: Integrate MMD regularization into the loss function to reduce the distribution difference between generated data and real data: Among them, K(x, x′) represents the kernel function value inside the dataset x, K(x, y) represents the kernel function value between x and y, and K(y, y′) represents the kernel function value inside the dataset y. Condition-specific MMD regularization is also applied to local priors. That is,

Images