A retinal image super-resolution reconstruction method based on an attention diffusion model

Abstract

The application provides a retinal image super-resolution reconstruction method based on an attention diffusion model, comprising: based on original collected fundus image data, a multi-source fundus image dataset is constructed; based on a pre-trained model and the multi-source fundus image dataset, three-prior feature of a low-resolution image is extracted, and multi-prior fusion features are generated by fusion; based on a pathological perception-vascular mixed attention diffusion model and the multi-prior fusion features, a high-resolution fundus image is generated; based on clinical diagnosis feedback and a multi-scale perception loop, clinical consistency loss and perception similarity loss are calculated; based on a prior confidence semi-supervised learning strategy, a lightweight adversarial branch containing a time-dependent discriminator is constructed, an adversarial loss is introduced, and an end-to-end joint optimization training is performed on the PA-VHAD model through a weighted total loss function. The method realizes end-to-end pathological feature faithful reconstruction, and significantly improves the accuracy of downstream clinical auxiliary diagnosis tasks.

Description

Technical Field

[0001] This application belongs to the field of computer vision and medical image technology, specifically relating to a method for super-resolution reconstruction of retinal images based on an attention diffusion model. Background Technology

[0002] The retina, as a tissue in the human body whose microcirculatory vascular system can be directly observed non-invasively, contains rich pathophysiological information in its morphological features. In modern precision medicine, high-resolution retinal fundus images are not only the "gold standard" for screening and diagnosing ophthalmic diseases (such as diabetic retinopathy, age-related macular degeneration, and glaucoma), but also an important window for assessing the risk of systemic cardiovascular diseases such as hypertension, arteriosclerosis, and stroke. Clinical pathological studies have shown that many microvascular lesions, such as the formation of microaneurysms, the expansion of capillary non-perfusion areas, and subtle changes in the retinal arteriovenous ratio, often occur in the very early stages of disease. These sub-millimeter-level pathological features are typically only a few pixels wide, placing extremely high demands on the spatial resolution and contrast of the images. Therefore, acquiring high-quality images that clearly present the topological structure of retinal vessels and the details of terminal capillaries is crucial for the early detection, graded diagnosis, and personalized treatment planning of diseases.

[0003] Despite significant advancements in fundus imaging technology over the past few decades, acquiring ideal high-resolution images in practical clinical applications still faces insurmountable physical and economic barriers. On one hand, limitations imposed by the diffraction limit of optical imaging systems, inherent physiological tremors of the eye, and medial opacities (such as cataracts) inevitably introduce motion blur and optical aberrations during the acquisition process, leading to the loss of high-frequency details. On the other hand, while high-performance specialized fundus imaging equipment (such as the Adaptive Optics Scanning Laser Ophthalmoscopy, AO-SLO) can achieve cellular-level resolution, their extremely high cost, narrow field of view, and complex operation make them difficult to widely adopt in primary care hospitals or large-scale screening programs. The vast majority of primary healthcare institutions still rely on low-cost portable fundus cameras, whose image resolution is often insufficient to support detailed pathological analysis due to limitations in sensor size and lens quality. Furthermore, with the rise of telemedicine, fundus images are often subjected to lossy compression during cloud transmission and storage in order to save bandwidth and space. This further exacerbates image degradation, causing minute lesion features to be masked by compression artifacts.

[0004] Faced with the high cost and physical limitations of hardware upgrades, image super-resolution reconstruction techniques based on software algorithms have emerged, aiming to recover potential high-resolution details from low-resolution observation data, thus becoming an effective way to break through the bottleneck of imaging resolution. Early super-resolution methods mainly relied on interpolation algorithms or reconstruction theories based on sparse coding. However, these traditional methods are essentially based on linear or shallow nonlinear mapping of local pixels, making it difficult to handle complex image degradation processes. The reconstruction results are often accompanied by severe edge jaggedness, excessive texture smoothing, and ringing effects, and cannot effectively restore the complex geometric structure of the retinal vascular network.

[0005] In recent years, the explosive development of deep learning technology has completely reshaped the landscape of medical image enhancement. Deep models, represented by Convolutional Neural Networks (CNNs), such as SRCNN and its subsequent variants (VDSR, EDSR), have established a complex nonlinear mapping relationship between low-resolution and high-resolution image spaces through end-to-end feature learning, significantly improving the peak signal-to-noise ratio (PSNR) of reconstructed images. Subsequently, the introduction of Generative Adversarial Networks (GANs) has pushed super-resolution technology to new heights. Models such as SRGAN and ESRGAN, by introducing adversarial and perceptual losses, force the generator to output results that conform to the statistical distribution of natural images, thereby generating realistic images with rich high-frequency textures. In the field of fundus image processing, researchers have attempted to transfer and apply these general visual models to address the problem of blurred fundus images and have improved the visual quality of the images to some extent.

[0006] While existing deep learning super-resolution methods excel in natural images, their application to retinal medical imaging reveals a series of profound and dangerous limitations, severely restricting their clinical applicability. The core problem lies in the distortion of pathological features caused by the "hallucination effect." Since medical image super-resolution is essentially an pathological inverse problem—a low-resolution input may correspond to countless high-resolution solutions—GAN-type models, in pursuit of visual "realism," often "invent" non-existent texture details based on statistical regularities in the training data. In natural scenes, slight deviations in the texture of grass or hair are acceptable, but in retinal images, such generative artifacts may misrepresent background noise as microvessels, break previously continuous blood vessels into fragments, or even generate non-existent microaneurysms. This inconsistency in topology directly violates the principles of medical anatomy, potentially leading to misdiagnosis or missed diagnosis by clinicians.

[0007] Further analysis of existing technologies reveals that most models lack prior knowledge guidance regarding the specific anatomical structures and pathological features of the retina. Retinal images not only contain highly structured vascular networks but also, in diseased states, lesions of varying morphologies (such as hemorrhages, hard exudates, and microaneurysms). Existing super-resolution algorithms typically treat images as indistinguishable pixel matrices, failing to differentiate between normal physiological structures and pathological regions. When lesions appear in fundus images, due to significant differences in texture frequency and color distribution between the lesion area and surrounding normal tissue, general models often misclassify them as noise and smooth them, or incorrectly enhance their edge textures, leading to blurred or even disappeared lesion outlines. This loss or distortion of pathological information directly weakens the reference value of reconstructed images in subsequent lesion grading and diagnosis. Although some studies have attempted to introduce vascular segmentation tasks as auxiliary supervision or utilize attention mechanisms to enhance the extraction of key features, most focus only on vascular connectivity while neglecting the specificity of lesion regions, lacking a hybrid guidance mechanism that can simultaneously utilize vascular topological priors and lesion morphological priors. Furthermore, existing optimization objectives are mostly based on pixel-level mean squared error (MSE) or perceptual similarity, lacking a clinical feedback loop. High PSNR values ​​do not equate to high diagnostic efficacy. Existing algorithms cannot dynamically adjust reconstruction strategies based on feedback from downstream tasks (such as lesion detection), resulting in wasted computational resources on irrelevant background areas while key pathological details are not specifically enhanced.

[0008] Furthermore, the degradation process of retinal images is often multi-stage and multi-factor coupled (including blurring, downsampling, noise, and compression artifacts), and single-stage end-to-end networks often struggle to decouple these complex degradation factors. While existing multi-stage progressive networks alleviate this problem to some extent, they often lead to a surge in model parameters and excessive computational complexity, making them difficult to deploy on resource-constrained medical devices. How to construct lightweight, efficient, and clinically perceptive reconstruction models while maintaining the topological fidelity of pathological features is a challenge currently faced by both academia and industry. The retina, as a tissue in the human body whose microcirculatory vascular system can be directly observed non-invasively, contains rich pathophysiological information in its morphological features. In modern precision medicine systems, high-resolution retinal fundus images are not only the "gold standard" for screening and diagnosing ophthalmic diseases (such as diabetic retinopathy, age-related macular degeneration, and glaucoma), but also an important window for assessing the risk of systemic cardiovascular diseases such as hypertension, arteriosclerosis, and stroke. Clinical pathological studies have shown that many microvascular lesions, such as the formation of microaneurysms, the expansion of capillary non-perfusion areas, and subtle changes in the retinal arteriovenous ratio, often occur in the very early stages of disease. These sub-millimeter-scale pathological features are typically only a few pixels wide, placing extremely high demands on the spatial resolution and contrast of the images. Therefore, acquiring high-quality images that clearly present the topology of retinal vessels and the details of terminal capillaries is crucial for early disease detection, graded diagnosis, and the development of personalized treatment plans.

[0009] Despite significant advancements in fundus imaging technology over the past few decades, acquiring ideal high-resolution images in practical clinical applications still faces insurmountable physical and economic barriers. On one hand, limitations imposed by the diffraction limit of optical imaging systems, inherent physiological tremors of the eye, and medial opacities (such as cataracts) inevitably introduce motion blur and optical aberrations during the acquisition process, leading to the loss of high-frequency details. On the other hand, while high-performance specialized fundus imaging equipment (such as the Adaptive Optics Scanning Laser Ophthalmoscopy, AO-SLO) can achieve cellular-level resolution, their extremely high cost, narrow field of view, and complex operation make them difficult to widely adopt in primary care hospitals or large-scale screening programs. The vast majority of primary healthcare institutions still rely on low-cost portable fundus cameras, whose image resolution is often insufficient to support detailed pathological analysis due to limitations in sensor size and lens quality. Furthermore, with the rise of telemedicine, fundus images are often subjected to lossy compression during cloud transmission and storage in order to save bandwidth and space. This further exacerbates image degradation, causing minute lesion features to be masked by compression artifacts.

[0010] Faced with the high cost and physical limitations of hardware upgrades, image super-resolution reconstruction techniques based on software algorithms have emerged, aiming to recover potential high-resolution details from low-resolution observation data, thus becoming an effective way to break through the bottleneck of imaging resolution. Early super-resolution methods mainly relied on interpolation algorithms or reconstruction theories based on sparse coding. However, these traditional methods are essentially based on linear or shallow nonlinear mapping of local pixels, making it difficult to handle complex image degradation processes. The reconstruction results are often accompanied by severe edge jaggedness, excessive texture smoothing, and ringing effects, and cannot effectively restore the complex geometric structure of the retinal vascular network.

[0011] In recent years, the explosive development of deep learning technology has completely reshaped the landscape of medical image enhancement. Deep models, represented by Convolutional Neural Networks (CNNs), such as SRCNN and its subsequent variants (VDSR, EDSR), have established a complex nonlinear mapping relationship between low-resolution and high-resolution image spaces through end-to-end feature learning, significantly improving the peak signal-to-noise ratio (PSNR) of reconstructed images. Subsequently, the introduction of Generative Adversarial Networks (GANs) has pushed super-resolution technology to new heights. Models such as SRGAN and ESRGAN, by introducing adversarial and perceptual losses, force the generator to output results that conform to the statistical distribution of natural images, thereby generating realistic images with rich high-frequency textures. In the field of fundus image processing, researchers have attempted to transfer and apply these general visual models to address the problem of blurred fundus images and have improved the visual quality of the images to some extent.

[0012] While existing deep learning super-resolution methods excel in natural images, their application to retinal medical imaging reveals a series of profound and dangerous limitations, severely restricting their clinical applicability. The core problem lies in the distortion of pathological features caused by the "hallucination effect." Since medical image super-resolution is essentially an pathological inverse problem—a low-resolution input may correspond to countless high-resolution solutions—GAN-type models, in pursuit of visual "realism," often "invent" non-existent texture details based on statistical regularities in the training data. In natural scenes, slight deviations in the texture of grass or hair are acceptable, but in retinal images, such generative artifacts may misrepresent background noise as microvessels, break previously continuous blood vessels into fragments, or even generate non-existent microaneurysms. This inconsistency in topology directly violates the principles of medical anatomy, potentially leading to misdiagnosis or missed diagnosis by clinicians.

[0013] Further analysis of existing technologies reveals that most models lack prior knowledge guidance regarding the specific anatomical structures and pathological features of the retina. Retinal images not only contain highly structured vascular networks but also, in diseased states, lesions of varying morphologies (such as hemorrhages, hard exudates, and microaneurysms). Existing super-resolution algorithms typically treat images as indistinguishable pixel matrices, failing to differentiate between normal physiological structures and pathological regions. When lesions appear in fundus images, due to significant differences in texture frequency and color distribution between the lesion area and surrounding normal tissue, general models often misclassify them as noise and smooth them, or incorrectly enhance their edge textures, leading to blurred or even disappeared lesion outlines. This loss or distortion of pathological information directly weakens the reference value of reconstructed images in subsequent lesion grading and diagnosis. Although some studies have attempted to introduce vascular segmentation tasks as auxiliary supervision or utilize attention mechanisms to enhance the extraction of key features, most focus only on vascular connectivity while neglecting the specificity of lesion regions, lacking a hybrid guidance mechanism that can simultaneously utilize vascular topological priors and lesion morphological priors. Furthermore, existing optimization objectives are mostly based on pixel-level mean squared error (MSE) or perceptual similarity, lacking a clinical feedback loop. High PSNR values ​​do not equate to high diagnostic efficacy. Existing algorithms cannot dynamically adjust reconstruction strategies based on feedback from downstream tasks (such as lesion detection), resulting in wasted computational resources on irrelevant background areas while key pathological details are not specifically enhanced.

[0014] Furthermore, the degradation process of retinal images is often multi-stage and multi-factor coupled (including blurring, downsampling, noise, and compression artifacts), and single-stage end-to-end networks often struggle to decouple these complex degradation factors. While existing multi-stage progressive networks alleviate this problem to some extent, they often lead to a surge in model parameters and excessive computational complexity, making them difficult to deploy on resource-constrained medical devices. How to construct lightweight, efficient, and clinically perceptive reconstruction models while maintaining the topological fidelity of pathological features is a challenge currently faced by both academia and industry. Summary of the Invention

[0015] This application provides a method for super-resolution reconstruction of retinal images based on an attention diffusion model to solve the above-mentioned technical problems.

[0016] To address the aforementioned technical problems, this application adopts the following technical solution: a retinal image super-resolution reconstruction method based on an attention diffusion model, comprising:

[0017] S1. Based on the original acquired fundus image data, construct a multi-source fundus image dataset; the dataset includes high- and low-resolution images, supervised data annotated by human experts, and semi-supervised data lacking human expert annotations;

[0018] S2. Based on the pre-trained model and multi-source fundus image dataset, extract the ternary prior features of low-resolution images and fuse the ternary prior features to generate multi-prior fusion features;

[0019] S3. Based on the pathological perception-vascular hybrid attention diffusion model and multiple prior fusion features, high-resolution fundus images are generated;

[0020] S4. Based on clinical diagnostic feedback and multi-scale perception loop, calculate clinical consistency loss and perception similarity loss;

[0021] S5. Based on a prior confidence-based semi-supervised learning strategy, a lightweight adversarial branch containing a time-dependent discriminator is constructed, adversarial loss is introduced, and the PA-VHAD model is jointly optimized end-to-end through a weighted total loss function.

[0022] Furthermore, the method in step S2 includes:

[0023] S21. Process low-resolution images based on multi-scale Frangi filters, calculate the eigenvalues ​​of the Hessian matrix, extract tubular geometric structure maps, and obtain prior features of geometric structures.

[0024] S22. Based on the lightweight U-Net segmentation network, inference is performed on low-resolution images to output a pixel-level probability map of blood vessel presence and obtain semantic prior features of blood vessels.

[0025] S23. Based on the lesion perception network, perform pixel-level reasoning on low-resolution images to identify microaneurysms, bleeding points and exudation areas in the images, output lesion semantic probability maps, and obtain lesion attention prior features.

[0026] S24. Based on prior features of geometric structure, prior features of vascular semantics, and prior features of lesion attention, the features are concatenated in the channel dimension and encoded through convolutional layers to obtain multi-prior fusion features.

[0027] Furthermore, the method in step S3 includes:

[0028] S31. Based on the U-Net architecture, a diffusion denoising network is constructed, and a hybrid transform module is embedded in the encoder and decoder layers of the diffusion denoising network;

[0029] S32. Based on the local convolution branch and the window-based self-attention branch connected in parallel in the hybrid transformation module, local high-frequency textures are extracted by using deep convolution combined with channel attention mechanism, and global long-distance dependencies are captured by using window partitioning combined with self-attention mechanism. The outputs of the two are then fused by residual as the module output.

[0030] S33. Based on the bottleneck layer of the diffusion denoising network, a window cross-condition fusion module and a multi-prior topology gating unit are deployed sequentially;

[0031] S34. Based on the window cross-condition fusion module, multiple prior fusion features are used as condition keys, and intermediate network features are used as queries. Cross-attention calculation is performed within the divided window to inject multimodal prior information into the network features.

[0032] S35. Based on multi-prior topological gating units, spatial gating is performed on the injected features using multi-prior fusion features to enhance the saliency of pathological structural features;

[0033] S36. Based on the denoising diffusion implicit model sampling algorithm, the low-resolution image and multiple prior fusion features are used as conditional inputs, and the image is inversely recovered from Gaussian noise according to the preset time step to obtain a high-resolution fundus image.

[0034] Furthermore, the method in step S35 includes:

[0035] Based on formula (1), the gated output is obtained; where formula (1) is:

[0036] (1);

[0037] in, As input features, For multiple prior fusion features, It is the Sigmoid activation function. This is a convolution operation.

[0038] Furthermore, the method in step S4 includes:

[0039] S41. Based on a pre-trained multi-class fundus disease diagnosis network, a feature extractor is constructed, and its parameters are frozen during training;

[0040] S42. Based on the generated high-resolution fundus image and the real high-resolution image, input them into the feature extractor to extract the deep feature vector before the fully connected layer;

[0041] S43. Calculate cosine similarity based on deep feature vectors to obtain clinical consistency loss;

[0042] S44. Based on the learning of image patch similarity metrics, a pre-trained perceptual network is used to extract multi-level feature maps;

[0043] S45. Based on the distance between the generated high-resolution fundus image and the real high-resolution image in the feature space, obtain the perceptual loss.

[0044] Furthermore, the method in step S43 includes:

[0045] Based on formula (2), obtain the clinical consistency loss. ; where formula (2) is

[0046] (2);

[0047] in, For feature extractors; High-resolution fundus images; It is a true high-resolution image.

[0048] Furthermore, the method in step S45 includes:

[0049] Based on formula (3), the perceived loss is obtained. ; where formula (3) is:

[0050] (3);

[0051] in, Represents the perceptron network Feature map of the layer This refers to the hierarchical weight.

[0052] Furthermore, the method in step S5 includes:

[0053] S51. Based on a prior confidence-based semi-supervised learning strategy, construct student and teacher models with identical structures, and build a lightweight time-dependent discriminator;

[0054] S52. Based on a time-dependent discriminant, perform multi-scale authenticity discrimination between generated and real images to obtain adversarial loss;

[0055] S53. Based on supervised data, construct a spatial weight map using manually labeled lesion area masks, calculate the weighted pixel loss of lesions, and combine adversarial loss, clinical consistency loss and perceptual loss to perform strongly supervised training;

[0056] S54. Based on semi-supervised data, use the prior features generated by the pre-trained model as weak labels to perform weakly supervised training and obtain semi-supervised loss;

[0057] S55. Based on lesion-weighted pixel loss, adversarial loss, clinical consistency loss, perceptual loss, and semi-supervised loss, a total loss function is constructed, and the model is optimized by weighted summation.

[0058] Furthermore, the method in step S55 includes:

[0059] Based on formula (4), the total loss function is obtained. ; where formula (4) is:

[0060] (4);

[0061] in, Weighted pixel loss for lesions, For semi-supervised losses, Weighted pixel loss for lesions, To balance the hyperparameters.

[0062] The beneficial effects of this application are: This application uses a confidence mask to filter the input unlabeled data, and combines clinical consistency loss and perceptual loss to quickly input the calculated gradient feedback into the student model, so as to obtain accurate physical morphological information of retinal vessels and micro lesions at different scales, providing high-fidelity virtual augmented samples for subsequent downstream clinical analysis tasks. Attached Figure Description

[0063] Figure 1 This is a schematic flowchart of an embodiment of the retinal image super-resolution reconstruction method based on the attention diffusion model of this application;

[0064] Figure 2 yes Figure 1 A flowchart illustrating an embodiment of step S2;

[0065] Figure 3 yes Figure 1 A flowchart illustrating an embodiment of step S3 in the process;

[0066] Figure 4 yes Figure 1 A flowchart illustrating an embodiment of step S4 in the process;

[0067] Figure 5 yes Figure 1 A flowchart illustrating an embodiment of step S5;

[0068] Figure 6 This is a flowchart of an embodiment of the retinal image super-resolution reconstruction method based on the attention diffusion model of this application;

[0069] Figure 7This is a detailed network structure diagram of the pathological perception-vascular hybrid attention diffusion model (PA-VHAD) of an embodiment of the retinal image super-resolution reconstruction method based on the attention diffusion model of this application. Detailed Implementation

[0070] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments.

[0071] Numerous specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways than those described herein, and therefore the invention is not limited to the specific embodiments disclosed in the following specification.

[0072] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the retinal image super-resolution reconstruction method based on the attention diffusion model of this application. The method includes:

[0073] S1. Based on the original acquired fundus image data, construct a multi-source fundus image dataset; the dataset includes high- and low-resolution images, supervised data annotated by human experts, and semi-supervised data lacking human expert annotations.

[0074] Specifically, raw fundus camera images captured in a clinical setting are collected. Low-resolution images are simulated using bicubic interpolation downsampling to construct high-low resolution image pairs (HR-LR pairs). All images undergo uniform preprocessing, including region of interest (ROI) cropping based on the optic disc center and size normalization (e.g., ...). and pixel value normalization to The dataset is divided into two parts: one part is supervised data containing finely annotated data by human experts (gold standard for blood vessel segmentation, lesion location annotation); the other part is semi-supervised data lacking human expert annotation, in order to improve the generalization ability of the model by utilizing massive amounts of unannotated clinical data.

[0075] S2. Based on the pre-trained model and multi-source fundus image dataset, extract the ternary prior features of low-resolution images, and fuse the ternary prior features to generate multi-prior fusion features.

[0076] Specifically, step S2 aims to extract the degraded low-resolution image The method for step S2 involves extracting potential pathological and structural information.

[0077] S21. Process low-resolution images based on multi-scale Frangi filters, calculate the eigenvalues ​​of the Hessian matrix, extract tubular geometric structure maps, and obtain prior features of the geometric structure.

[0078] Specifically, the input low-resolution fundus image is processed based on a multi-scale Frangi filter. First, the image is calculated... Different Gaussian scales The Hessian matrix is ​​given by the image. For any pixel (x, y) in the image, its Hessian matrix is ​​given by the image. Defined as a second-order partial derivative matrix:

[0079] (5);

[0080] in, It is the convolution of the image with the second derivative of the Gaussian kernel.

[0081] calculate Two eigenvalues and and satisfy At this point, at the scale Vesselness Function Defined as:

[0082] (6);

[0083] in, Used to distinguish between spot-like structures and tubular structures For structural strength norm, and This is a constant used to control sensitivity.

[0084] Final geometric prior features The maximum response value across multiple scales:

[0085] (7);

[0086] Among them, prior features of geometric structure It can significantly enhance the continuity of microvessels and compensate for the fragmentation artifacts at low resolution.

[0087] S22. Based on the lightweight U-Net segmentation network, inference is performed on low-resolution images to output a pixel-level probability map of blood vessel presence and obtain semantic prior features of blood vessels.

[0088] Specifically, a lightweight U-Net segmentation network is constructed and pre-trained. To convert low-resolution images The network is input for forward inference. The output layer uses the Sigmoid activation function and outputs a pixel-level probability map of blood vessel presence.

[0089] (8);

[0090] in, These are the model parameters locked during pre-training. Each pixel value in the graph represents the confidence probability that the location belongs to the blood vessel category, providing global topological semantic guidance.

[0091] S23. Based on the lesion perception network, perform pixel-level reasoning on low-resolution images to identify microaneurysms, bleeding points and exudation areas in the images, output lesion semantic probability maps, and obtain lesion attention prior features.

[0092] Specifically, based on lesion sensing networks This network identifies typical signs of diabetic retinopathy in images. It employs a lightweight, fully convolutional architecture and is pre-trained to recognize minute lesions such as microaneurysms, hemorrhages, and exudates. It can handle low-resolution images... Input the data into the network for forward inference, and output a pixel-level semantic probability map of lesions: (9); in, These are the pre-trained, locked model parameters. If the network identifies multiple lesion categories, a weighted fusion is performed on the multi-channel output, compressing it into a single-channel attention map. This feature map... It accurately depicts the morphology and boundaries of lesions, effectively instructing the super-resolution model to focus on restoring high-frequency details in the lesion area.

[0093] S24. Based on prior features of geometric structure, prior features of vascular semantics, and prior features of lesion attention, the features are concatenated in the channel dimension and encoded through convolutional layers to obtain multi-prior fusion features.

[0094] Specifically, the aforementioned acquired prior features of geometric structure semantic prior features of blood vessels and lesion attention prior characteristics Concatenation (Concat) is performed along the channel dimension to obtain multiple prior concatenated features. :

[0095] (9);

[0096] Subsequently, through a convolutional layers The concatenated features are encoded and fused to align the feature distributions of different modalities, outputting the final multi-priority fused features. :

[0097] (10);

[0098] in, The SiLU activation function is used. This fusion feature... It contains complete information from the underlying geometry to the high-level semantics, which will serve as the conditional input for the subsequent diffusion model.

[0099] S3. High-resolution fundus images are generated based on the Pathology-Aware VascularHybrid Attention Diffusion model (PA-VHAD) and multiple prior fusion features.

[0100] Specifically, this embodiment employs a generative paradigm based on Denoising Diffusion Probabilistic Models (DDPM) to learn conditional probability distributions. The method in step S3 includes:

[0101] S31. Based on the U-Net architecture, a diffusion denoising network is constructed, and a hybrid transformation module is embedded in the encoder and decoder layers of the diffusion denoising network.

[0102] Specifically, a noise prediction network is built based on the U-Net architecture. To balance the local microvascular details and global pathological distribution in retinal images, a hybrid transform module is embedded in the encoder and decoder layers. Let the input features be... This module contains two parallel branches:

[0103] Convolutional branch: Utilizes depthwise convolution (DWConv) to capture local high-frequency textures, and combines it with a channel attention layer (SE-Layer) to recalibrate feature weights. The calculation formula is:

[0104] (11);

[0105] Where LN is the layer normalization, The Squeeze-and-Excitation operation involves the following steps: First, spatial features are compressed (squeeze) into global channel descriptors using global average pooling; then, excitation is performed using a two-layer fully connected network containing dimensionality reduction and expansion, along with a Sigmoid activation function, to generate weight coefficients for each channel; finally, the weights are multiplied back into the feature map channel by channel to complete the recalibration.

[0106] S32. Based on the parallel-connected local convolutional branch and window-based self-attention branch in the hybrid transformation module, local high-frequency textures are extracted using depth convolution combined with channel attention mechanism, and global long-distance dependencies are captured using window partitioning combined with self-attention mechanism. The outputs of the two are then fused with residuals as the module output.

[0107] Specifically, the window-based self-attention branch divides the feature map X into... There are 3 non-overlapping windows. Multi-head self-attention (MSA) is computed within each window. Let the input features of the i-th window be... Through linear projection matrix Generate queries, keys, and values: ;

[0108] Introducing relative position encoding To preserve spatial location information, self-attention is calculated as follows:

[0109] (12);

[0110] The output of this branch is ,in This is for output projection layer.

[0111] Feature fusion: The outputs of the two branches are fused using learnable parameters. Perform weighted residual fusion to obtain the module output. :

[0112] (13);

[0113] In this context, || represents channel splicing.

[0114] S33. Based on the bottleneck layer of the diffusion denoising network, a window cross-condition fusion module and a multi-prior topology gating unit are deployed sequentially.

[0115] Specifically, at the bottleneck layer of the diffusion network, multiple prior features are fused. Injected as an external condition.

[0116] S34. Based on the window cross-condition fusion module, multiple prior fusion features are used as condition keys, and intermediate network features are used as queries. Cross-attention calculation is performed within the divided window to inject multimodal prior information into the network features.

[0117] Specifically, to reduce computational complexity, a windowing mechanism is also used. Let the intermediate features of the network be... (As a query), multiple prior features (As Key / Value). First, align both to the same window partition. Within each window, perform cross-attention calculation:

[0118] ;

[0119] ;

[0120] ;

[0121] The cross-modal fusion output Z is calculated as follows, thereby explicitly injecting geometric and pathological priors into the feature space:

[0122] (14).

[0123] S35. Based on multi-prior topological gating units, spatial gating is performed on the injected features using multi-prior fusion features to enhance the saliency of pathological structural features.

[0124] Specifically, to further highlight the lesion area and suppress background noise, spatial gating units are deployed after cross-fusion. Multiple prior features are utilized. Generate a spatial attention mask. The gating mechanism is defined as:

[0125] (15);

[0126] in, For the Sigmoid activation function, The SiLU activation function is used. Hollow convolution is used to expand the receptive field.

[0127] Final gated output Through the Hadamard Product Modulate the features:

[0128] (16);

[0129] in, This is a learnable scaling factor, initialized to 0. This formula achieves adaptive enhancement for regions containing pathological features.

[0130] Based on formula (1), the gated output is obtained; where formula (1) is:

[0131] (1);

[0132] in, As input features, For multiple prior fusion features, It is the Sigmoid activation function. This is a convolution operation.

[0133] S36. Based on the denoising diffusion implicit model sampling algorithm, the low-resolution image and multiple prior fusion features are used as conditional inputs, and the image is inversely recovered from Gaussian noise according to the preset time step to obtain a high-resolution fundus image.

[0134] Specifically, during the model inference phase or semi-supervised training, when generating pseudo-labels, a denoising diffusion implicit model (DDIM) is used to accelerate sampling. The model uses low-resolution images. and prior As a condition, from standard Gaussian noise Begin iteration. At any time step t, first use the network to predict noise. Next, the potential sharp image at time t=0 is estimated. :

[0135] (17);

[0136] Finally, the previous time step is calculated according to the non-Markov chain update rule of DDIM. :

[0137] (18);

[0138] in, This is the cumulative product of the noise scheduling parameters. Hyperparameters for controlling sampling randomness (when (This is a deterministic sampling process). Through T iterations, a high-resolution fundus image is finally obtained. .

[0139] S4. Based on clinical diagnostic feedback and multi-scale perception loops, calculate clinical consistency loss and perception similarity loss.

[0140] Specifically, this step aims to ensure the fidelity of pathological features and the realism of high-frequency details by introducing high-level semantic constraints and low-level texture constraints.

[0141] Prediction noise based on the current time step t The Tweedie formula is used to estimate the potential denoised image. (i.e., as described below) ):

[0142] (19);

[0143] Subsequently, based on this estimated image With real images Calculate the following losses.

[0144] The method in step S4 includes:

[0145] S41. Based on a pre-trained multi-class fundus disease diagnosis network, a feature extractor is constructed, and its parameters are frozen during training.

[0146] Specifically, to ensure the effectiveness of the reconstructed images in clinical diagnosis, a pre-trained multi-class fundus disease diagnosis network was used as a feature extractor. During the training phase, freeze. parameters .

[0147] S42. Based on the generated high-resolution fundus image and the real high-resolution image, input them into the feature extractor to extract the deep feature vector before the fully connected layer.

[0148] Specifically, the generated super-resolution image and true high-resolution images Enter them separately Extracting deep semantic feature vectors before the global average pooling layer and :

[0149] ;

[0150] S43. Calculate cosine similarity based on deep feature vectors to obtain clinical consistency loss.

[0151] Specifically, to eliminate the influence of feature amplitude, focusing only on feature direction (i.e., consistency of pathological semantics), a loss function is constructed using cosine similarity distance. :

[0152] (2);

[0153] in, To prevent the use of tiny constants with a denominator of zero (such as 1e-8), this loss forces... In higher-dimensional manifold space The pathological categories are aligned, effectively preventing the loss of lesions or the generation of artifacts.

[0154] S44. Based on the learning of image patch similarity metrics, a pre-trained perceptual network is used to extract multi-level feature maps.

[0155] Specifically, to align with the characteristics of human visual perception, the Learned Perceptual Image Patch Similarity (LPIPS) metric is employed. A pre-trained VGG-16 network is used to extract L layers of multi-scale feature maps. For the l-th layer features, channel-dimensional normalization is first performed, followed by multiplication by the pre-trained channel weight vector. The calculation formula is as follows:

[0156] ;

[0157] in, The Hadamard product representing the channel dimension. Let be the spatial dimensions of the feature map at layer l. Compared to the traditional Mean Squared Error (MSE) loss, LPIPS can more effectively drive the model to recover the sharp edges of microvessels.

[0158] S45. Based on the distance between the generated high-resolution fundus image and the real high-resolution image in the feature space, obtain the perceptual loss.

[0159] Specifically, based on formula (3), the perceptual loss is obtained. ; where formula (3) is:

[0160] (3);

[0161] in, Represents the perceptron network Feature map of the layer This refers to the hierarchical weight.

[0162] S5. Based on a prior confidence-based semi-supervised learning strategy, a lightweight adversarial branch containing a time-dependent discriminator is constructed, adversarial loss is introduced, and the PA-VHAD model is jointly optimized end-to-end through a weighted total loss function.

[0163] For details, please refer to Figure 5This embodiment constructs a lightweight adversarial branch including a time-dependent discriminator and performs end-to-end joint optimization under a mixed supervised and unsupervised data setting. Specifically, step S5 includes:

[0164] S51. A semi-supervised learning strategy based on prior confidence is used to construct a student model and a teacher model with identical structures, and a lightweight time-dependent discriminator is built. The teacher model does not participate in gradient backpropagation, and its parameters are dynamically updated based on the exponential moving average of the student model's parameters.

[0165] Specifically, construct student models with identical structures. (parameter ) and teacher model (parameter In training iteration k, the student model is updated via gradient descent, while the teacher model does not participate in gradient backpropagation and is updated via the exponential moving average (EMA) of the student model parameters.

[0166] ;

[0167] in, This is the momentum decay factor. The EMA strategy can smooth out noise fluctuations during training, making the pseudo-labels generated by the teacher model more stable.

[0168] To improve the realism of the generated images, a discriminator is constructed. Considering the temporal nature of the diffusion process, the discriminator simultaneously receives the image x and the embedding vector at time step t as input. An adversarial loss of the Hinge Loss form is employed to stabilize training.

[0169] S52. Based on a time-dependent discriminant, perform multi-scale authenticity discrimination between generated and real images, and obtain adversarial loss. ).

[0170] Specifically, the optimization goal of the discriminator To maximize the distinction between real images and generating images Abilities:

[0171] ;

[0172] Adversarial loss of student generation model This is a deception detector:

[0173] ;

[0174] S53. Based on supervised data, construct a spatial weight map using manually annotated lesion region masks, and calculate the weighted pixel loss of the lesions. In addition, strong supervised training is performed by combining the adversarial loss, the clinical consistency loss, and the perceptual loss.

[0175] Specifically, in order to ensure reconstruction accuracy at the pixel level while focusing on the recovery of lesion areas, a spatial weight map is constructed. Lesion mask based on manual annotation (Lesion area is 1, background is 0), define the weight map:

[0176] ;

[0177] in, Weighting coefficients for lesions (e.g.) =10). The weighted L1 loss is calculated as follows:

[0178] .

[0179] S54. Based on semi-supervised data, weakly supervised training is performed using prior features generated by the pre-trained model as weak labels. The weakly supervised training includes: inputting the same sample into the teacher model and the student model respectively, using the pseudo-high-resolution image generated by the teacher model as the pseudo-true value, and calculating the consistency loss between the student model output and the pseudo-true value; and constructing a confidence mask based on the vascular semantic prior features and lesion attention prior features generated by the pre-trained model as reference weak labels, and applying structural constraints to the student model's generation results in regions with high confidence to obtain the semi-supervised loss.

[0180] Specifically, when the input data is unlabeled semi-supervised data... At that time, the teacher-student consistency constraint path is activated.

[0181] First, the sampling process described in step S34 is performed using the teacher model to generate a pseudo-high-resolution image. As a pseudo-true value (Pseudo-GT).

[0182] Secondly, to avoid the teacher model's erroneous predictions misleading the student model, a confidence mask is constructed based on the prior features extracted in step S2. Using blood vessel probability maps semantic probability map of lesions Calculate pixel-level confidence:

[0183] ;

[0184] Set threshold ,when At this point, the region is considered to have reliable prior knowledge. The confidence mask is then used. .

[0185] Semi-supervised consistency loss is calculated as the difference between student output and teacher output within the confidence region:

[0186] ;

[0187] S55. Based on lesion-weighted pixel loss, adversarial loss, clinical consistency loss, perceptual loss, and semi-supervised loss, a total loss function is constructed, and the model is optimized by weighted summation.

[0188] Specifically, strong supervision loss based on supervised data ( Using the semi-supervised loss from both unsupervised and unsupervised data, construct the total loss function:

[0189] ;

[0190] ;

[0191] in, The series represents balanced hyperparameters. This is achieved by minimizing... While learning known pathological features, the model improves its generalization ability to unseen samples by utilizing massive amounts of unlabeled data.

[0192] See Figure 6 and Figure 7 The above steps enable super-resolution reconstruction of retinal images. This method integrates a teacher-student semi-supervised framework, a time-dependent discriminator, and a multi-scale perceptual feedback loop. It jointly inputs supervised / semi-supervised fundus images from multi-source clinical environments, extracted ternary prior features, and time-step embedding vectors into the model. The teacher model is updated in real-time based on an exponential moving average strategy using the student model parameters, responsible for generating stable pseudo-label signals for unlabeled data. The PA-VHAD model deploys hybrid transformation modules at key levels of the encoder and decoder to capture local high-frequency textures and global long-distance dependencies in parallel. Simultaneously, a priori-guided interaction unit is integrated at the bottleneck layer. This unit injects multimodal priors into the feature space through a window cross-conditional fusion (WCCF) module and cascades a multi-prior topology gating unit (MPTG) to perform pixel-level topology filtering and enhancement, thereby achieving deep fusion of pathological features.

[0193] This method uses a confidence mask to filter the input unlabeled data, and combines clinical consistency loss and perceptual loss to quickly input the calculated gradient feedback into the student model to obtain accurate physical morphological information of retinal vessels and micro lesions at different scales, providing high-fidelity virtual augmented samples for subsequent downstream clinical analysis tasks.

[0194] This application enables the creation of high-precision pathological mapping models by reconstructing and digitally enhancing low-quality, blurred, or compressed fundus images from real-world clinical scenarios, thus facilitating refined auxiliary diagnosis. This method effectively addresses the scarcity of high-quality labeled data in the field of medical imaging. While simulating and restoring the microvascular topology, it rigorously suppresses generative artifacts, significantly improving the detection rate of subtle pathological features such as microaneurysms and hemorrhages. This provides doctors with a reference for developing treatment plans, such as early screening for diabetic retinopathy, assessment of macular lesions, and remote medical image quality control.

[0195] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A method for super-resolution reconstruction of retinal images based on an attention diffusion model, characterized in that, include: S1. Construct a multi-source fundus image dataset based on the original acquired fundus image data; wherein, the dataset includes high- and low-resolution images, supervised data annotated by human experts, and semi-supervised data lacking human expert annotations; S2. Based on the pre-trained model and the multi-source fundus image dataset, extract the ternary prior features of the low-resolution image, and fuse the ternary prior features to generate multi-prior fusion features; S3. Based on the pathological perception-vascular hybrid attention diffusion model and the aforementioned multi-prior fusion features, generate high-resolution fundus images; S4. Based on clinical diagnostic feedback and multi-scale perception loop, calculate clinical consistency loss and perception similarity loss; S5. Based on a semi-supervised learning strategy using prior confidence, a lightweight adversarial branch containing a time-dependent discriminator is constructed, adversarial loss is introduced, and the PA-VHAD model is jointly optimized end-to-end using a weighted total loss function.

2. The method according to claim 1, characterized in that, The method of step S2 includes: S21. Process low-resolution images based on multi-scale Frangi filters, calculate the eigenvalues ​​of the Hessian matrix, extract tubular geometric structure maps, and obtain prior features of geometric structures. S22. Based on the lightweight U-Net segmentation network, inference is performed on low-resolution images to output a pixel-level probability map of blood vessel presence and obtain semantic prior features of blood vessels. S23. Based on the lesion perception network, perform pixel-level reasoning on low-resolution images to identify microaneurysms, bleeding points and exudation areas in the images, output lesion semantic probability maps, and obtain lesion attention prior features. S24. Based on the prior features of the geometric structure, the prior features of the blood vessel semantics, and the prior features of the lesion attention, the features are concatenated in the channel dimension and encoded through a convolutional layer to obtain multi-prior fusion features.

3. The method according to claim 1, characterized in that, The method of step S3 includes: S31. Based on the U-Net architecture, a diffusion denoising network is constructed, and a hybrid transformation module is embedded in the encoder and decoder layers of the diffusion denoising network; S32. Based on the parallel-connected local convolutional branch and window-based self-attention branch in the hybrid transformation module, local high-frequency textures are extracted using deep convolution combined with channel attention mechanism, and global long-distance dependencies are captured using window partitioning combined with self-attention mechanism, and the outputs of the two are fused by residual as the module output; S33. Based on the bottleneck layer of the diffusion denoising network, a window cross-condition fusion module and a multi-prior topology gating unit are deployed sequentially; S34. Based on the window cross-condition fusion module, the multi-prior fusion features are used as condition keys, the intermediate network features are used as queries, cross-attention calculation is performed within the divided window, and multimodal prior information is injected into the network features. S35. Based on the multi-prior topological gating unit, spatial gating is performed on the injected features using the multi-prior fusion features to enhance the saliency of pathological structural features; S36. Based on the denoising diffusion implicit model sampling algorithm, the low-resolution image and the multi-prior fusion features are used as conditional inputs, and the image is inversely recovered from Gaussian noise according to a preset time step to obtain a high-resolution fundus image.

4. The method according to claim 3, characterized in that, The method of step S35 includes: Based on formula (1), the gated output is obtained; wherein, formula (1) is: （1）； in, As input features, For multiple prior fusion features, It is the Sigmoid activation function. This is a convolution operation.

5. The method according to claim 1, characterized in that, The method of step S4 includes: S41. Based on a pre-trained multi-class fundus disease diagnosis network, a feature extractor is constructed, and its parameters are frozen during training; S42. Based on the generated high-resolution fundus image and the real high-resolution image, respectively input them into the feature extractor to extract the deep feature vector before the fully connected layer; S43. Calculate the cosine similarity based on the deep feature vector to obtain the clinical consistency loss; S44. Based on the learning of image patch similarity metrics, a pre-trained perceptual network is used to extract multi-level feature maps; S45. Based on the distance between the generated high-resolution fundus image and the real high-resolution image in the feature space, obtain the perceptual loss.

6. The method according to claim 5, characterized in that, The method of step S43 includes: Based on formula (2), the clinical consistency loss is obtained. ; where formula (2) is （2）； in, For feature extractors; High-resolution fundus images; It is a true high-resolution image.

7. The method according to claim 1, characterized in that, The method of step S45 includes: Based on formula (3), the perceived loss is obtained. ; where formula (3) is: （3）； in, Represents the perceptron network Feature map of the layer This refers to the hierarchical weight.

8. The method according to claim 1, characterized in that, The method of step S5 includes: S51. Based on a prior confidence-based semi-supervised learning strategy, construct student and teacher models with identical structures, and build a lightweight time-dependent discriminator; S52. Based on the time-dependent discriminant, perform multi-scale authenticity discrimination between the generated image and the real image to obtain adversarial loss; S53. Based on the supervised data, construct a spatial weight map using manually labeled lesion area masks, calculate the lesion weighted pixel loss, and combine the adversarial loss, the clinical consistency loss, and the perceptual loss to perform strongly supervised training; S54. Based on the semi-supervised data, use the prior features generated by the pre-trained model as weak labels to perform weak-supervised training and obtain the semi-supervised loss; S55. Based on the lesion-weighted pixel loss, the adversarial loss, the clinical consistency loss, the perceptual loss, and the semi-supervised loss, a total loss function is constructed, and the model is optimized by weighted summation.

9. The method according to claim 8, characterized in that, The method of step S55 includes: Based on formula (4), the total loss function is obtained. ; where formula (4) is: （4）； in, Weighted pixel loss for lesions, For semi-supervised losses, Weighted pixel loss for lesions, To balance the hyperparameters.

Images