Retina image quality evaluation method based on double-path frequency domain cross attention fusion

Abstract

The invention relates to the technical field of retina image quality evaluation, and particularly discloses a retina image quality evaluation method based on double-path frequency domain cross attention fusion, comprising the following steps: step S1, acquiring an RGB image; s2, preprocessing the quality of the BGR image through a contrast limit adaptive histogram equalization algorithm to obtain a CLAHE image; s3, inputting the RGB image and the CLAHE image into a dual-path frequency domain DenseNet121 network, and extracting features of the RGB image and the CLAHE image through a frequency domain attention mechanism algorithm; s4, refining the extracted features through a double cross attention algorithm, and obtaining integration of the BGR image and the CLAHE image; and S5, obtaining a quality evaluation result through a linear classifier. According to the method, important retina image structure details are reserved, and the adaptability of the model to different illumination conditions is also improved.

Description

Technical Field

[0001] The present application relates to the technical field of retinal image quality assessment, and specifically discloses a retinal image quality assessment method based on dual-path frequency domain cross-attention fusion. Background Art

[0002] The process of determining whether the quality of retinal fundus images is sufficient for medical diagnosis is called retinal image quality assessment. This process is an important prerequisite for diagnosing eye diseases such as diabetic retinopathy, macular degeneration, and glaucoma. Currently, retinal images acquired non-invasively by fundus cameras are the only way to directly and centrally observe arteries, veins, and capillaries with the naked eye. Therefore, proper illumination and clarity of fundus structures in these images are critical. However, with the increasing prevalence of fundus diseases and the relative shortage of ophthalmologists worldwide, the acquisition and subsequent diagnosis of high-quality fundus images face great challenges. In areas with poor medical conditions and scarce medical resources, due to the uneven experience level of fundus image acquisition personnel, many of the acquired fundus images have varying degrees of quality problems, such as Figure 1 Problems such as underexposure or overexposure are shown.

[0003] For ophthalmologists, the discontinuity between the acquisition time and the diagnostic evaluation time can be problematic. Specifically, if image quality issues are discovered during the review process, the patient must return for another fundus scan, requiring further financial and time expenditures. More importantly, poor image quality may lead to misdiagnosis and delayed treatment. Studies involving a large number of retinal image databases have shown that more than 25% of retinal images are not suitable for medical diagnosis due to poor quality, rendering them unusable in computer-aided diagnosis systems (CADS). Therefore, the assessment of retinal image quality before disease diagnosis is particularly important. However, the traditional manual quality assessment method (inspection of digital retinal images by trained ophthalmologists or optometrists) is very cumbersome and time-consuming, and the limited number of professionals exacerbates this challenge. Therefore, automatically assessing the quality of retinal images to exclude low-quality entries before further analysis is essential to improve diagnostic efficiency and accuracy.

[0004] Significant progress has been made in the field of image analysis through the application of deep learning, especially deep convolutional neural networks (CNNs). CNNs have gradually replaced traditional manual feature extraction methods by achieving end-to-end feature extraction, which has shown great potential in automated retinal image quality assessment. These networks autonomously learn complex features from images, thereby improving the efficiency and accuracy of assessment.

[0005] Although deep learning provides significant advantages for advanced image quality assessment, it also encounters significant challenges, such as: relying solely on RGB images may cause the model to overemphasize lighting factors and ignore important underlying structural information; when performing three-category quality assessment (good, usable, rejected), traditional deep learning models often have difficulty distinguishing adjacent categories, often misclassifying "usable" images as "good" or "rejected", thereby reducing the accuracy of the assessment of the graphics. Therefore, in view of this, the inventors provide a retinal image quality assessment method based on dual-path frequency domain cross-attention fusion to solve the above problems. Summary of the invention

[0006] The purpose of the present invention is to solve the problem that traditional deep learning models have difficulty distinguishing adjacent categories in retinal image quality assessment, thereby reducing the accuracy of image assessment.

[0007] In order to achieve the above object, the basic scheme of the present invention provides a retinal image quality assessment method based on dual-path frequency domain cross-attention fusion, comprising the following steps:

[0008] Step S1, obtaining an RGB image;

[0009] Step S2, preprocessing the BGR image quality by using a contrast-limited adaptive histogram equalization algorithm to obtain a preprocessed CLAHE image;

[0010] Step S3, inputting the RGB image and the CLAHE image into the dual-path frequency domain DenseNet121 network, and extracting the features of the BGR image and the LAB image through the frequency domain attention mechanism algorithm;

[0011] Step S4, refining the extracted features through a double cross attention algorithm, obtaining the integration of the BGR image and the LAB image, and obtaining a merged feature set;

[0012] Step S5: input the combined feature set into a linear classifier, and obtain a quality assessment result through the linear classifier.

[0013] Furthermore, the dual-path frequency-domain DenseNet121 network includes two parallel DenseNet121 networks, each DenseNet121 network operates independently and a frequency-domain attention mechanism algorithm module is provided after the operation module of each DenseNet121 network.

[0014] Further, the specific steps of step S2 are as follows:

[0015] Step S21, converting the BGR image in BGR image form into LAB image form, the expression is as follows:

[0016]

[0017] Where Transform represents the color space conversion matrix from BGR image to LAB image;

[0018] Step S22, separating the luminance channel L, and performing a CLAHE operation on the L channel alone;

[0019] In step S23, the three LAB channels are combined and finally converted into a LAB image.

[0020] Further, the specific steps of step S22 are as follows:

[0021] Step S221, divide the image I into M×N small blocks I m,n , the grayscale histogram of each small block is calculated by the following formula:

[0022]

[0023] In the formula, H m,n (K) is the grayscale histogram of each small block, the x and y spatial coordinates of each small block, reflecting the size of each small block;

[0024] Step S222, clip_limit is defined as the average number of pixels per small block divided by the number of histogram bins, adjusted to a specific contrast enhancement limit, expressed as follows:

[0025]

[0026] Where C is the contrast limit hyperparameter, and P is the gray level;

[0027] In step S223, the redundant part of each bin k is removed and evenly distributed to meet the contrast constraint, as shown in the following expression:

[0028]

[0029] Adjusted histogram H' m,n (k) is:

[0030]

[0031] Step S224, using the adjusted histogram H' m,n (k) Calculate the cumulative distribution function, which is expressed as follows:

[0032]

[0033] Where CDF is the cumulative distribution function;

[0034] Step S225, using CDF m,n(k) The original patch I m,n The gray value r in the channel is mapped to the new gray value L', the expression is as follows:

[0035] L'=round((P-1)·CDF m,n (r))

[0036] Step S226 , bilinear interpolation is applied between patches to smooth patch edges and ensure overall uniformity of the image.

[0037] Furthermore, in step S226, for the boundary pixels of four adjacent blocks, the final pixel value is calculated by bilinear interpolation using the mapping results of two or four adjacent blocks.

[0038] Further, the specific steps of step S3 are as follows:

[0039] Step S31, transform the input data from the spatial domain to the frequency domain through 2D fast Fourier transform, the expression is as follows:

[0040]

[0041] In the formula, X is the input data, and X∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width respectively, x and y are the coordinates in the spatial domain, u and v are the coordinates in the frequency domain, and F(u,v) is the complex number representation in the frequency domain, representing the amplitude and phase of frequency u and v;

[0042] Step S32, using fftshift operation to move the low-frequency part of the FFT result to the center, thereby obtaining the center frequency domain data X shifted ;

[0043] For even dimensions H and W:

[0044]

[0045] For odd dimensions H and W:

[0046]

[0047] Step S33: X shifted Decompose it into real part and imaginary part and process them separately. The expression is as follows:

[0048]

[0049] In step S34, a feature vector representing the global information of each channel is obtained by global average pooling, and the domain attention weight is updated by passing through two fully connected layers:

[0050] Xatt,real =σ(FC2(ReLU(FC1(GAP(X real )))))

[0051] X att,imag =σ(FC2(ReLU(FC1(GAP(X imag )))))

[0052] Step S35, converting Xatt back to space by two-dimensional inverse fast Fourier transform to obtain a spatial representation s(x, y);

[0053] Step S36, evaluate f(x,y) through the inverse fftshift operation and restore its original spatial domain configuration, the expression is as follows:

[0054] For even dimensions H and W:

[0055]

[0056] For odd dimensions H and W, the adjustment is slightly different to accommodate the asymmetric distribution of the zero-frequency components:

[0057]

[0058] Step S37, taking the real part of g(x,y) as the final output.

[0059] 8. Further, the specific steps of step S4 are as follows:

[0060] Step S41, convert the two batches of input data x and y into Q and K forms through independent convolutional layers, as shown below:

[0061] Q=Conv query (x)

[0062] K=Conv key (y)

[0063] In the formula, x and y are input data, x, y∈R B×C×H×W , where B represents the batch size, C represents the number of channels, H and W represent the height and width respectively, and Q and K represent the feature representation of query and key respectively;

[0064] Step S42, calculate the energy score E by performing a dot product operation bmm() between the query and the key, and the expression is as follows:

[0065] E=bmm(Q,K T )

[0066] Step S43, the calculation result of the energy score E is sent to the Softmax function to obtain the final attention weight, and ensure that the sum of all weights is 1, the expression is as follows:

[0067] A=Softmax(E)

[0068] Step S44, the final output feature is generated by calculating the weighted sum between the value feature and the previously obtained attention weight, and the expression is as follows:

[0069] V=Conv value (y)

[0070] In the formula, V represents the value feature;

[0071] Step S45, use the attention weight A to weight these value features to generate a weighted output, the expression is as follows:

[0072] O=bmm(A,V)

[0073] In the formula, O represents the weighted output feature;

[0074] Step S46, combining the weighted output feature with the original query input as the output result, the expression is as follows:

[0075] Output=O+x

[0076] Where x is the original query input.

[0077] The principle and effect of this scheme are:

[0078] 1. The implementation of the dual-path DenseNet121 architecture in this invention utilizes raw RGB and CLAHE preprocessed images to significantly improve retinal image quality assessment (RIQA). Traditional methods that rely solely on RGB images often fail due to changes in lighting and contrast, which adversely affects performance. By combining CLAHE, which can enhance local contrast and balance brightness without changing the intrinsic content of the image, the DFC-Net model introduces a powerful preprocessing layer that can significantly reduce the variability of the input. This approach not only preserves important structural details, but also improves the model's adaptability to different lighting conditions, which is critical for the accuracy of high-risk medical imaging tasks.

[0079] 2. This invention alleviates the typical limitation of traditional deep learning models: poor discrimination between adjacent quality categories. FDAM targets specific frequency components associated with quality degradation, such as blur and uneven lighting, thereby increasing the model's sensitivity to subtle but critical defects. This approach effectively distinguishes between "usable" and "rejected" categories by highlighting the loss of high-frequency details that are prevalent in low-quality images. This enhanced sensitivity greatly improves the model's ability to classify subtle differences between quality levels, demonstrating the value of integrating advanced attention mechanisms into deep learning frameworks.

[0080] 3. We introduce a cross-attention module that can synergize features extracted from the RGB and CLAHE preprocessing paths. This strategy not only retains essential information of each input type, but also dynamically adjusts feature contributions based on their relevance to the quality assessment task. This adjustment is critical to managing the inherent variability of retinal images, where small differences in quality can have significant diagnostic consequences. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0082] Figure 1 An example diagram of a low-quality fundus image is shown;

[0083] Figure 2 The dual-path frequency domain cross-attention fusion network architecture diagram is shown, where (a) is the DFC-Net network framework and (b) is the Dense block of the DFC-Net network;

[0084] Figure 3 A comparison diagram of the original RGB image and the preprocessed CLAHE image is shown, where (a) is the original RGB image and (b) is the preprocessed CLAHE image;

[0085] Figure 4 A schematic diagram of the frequency domain attention mechanism (FDAM) structure is shown;

[0086] Figure 5 A schematic diagram of the cross-attention mechanism structure is shown;

[0087] Figure 6 A process diagram showing the heat map generation process;

[0088] Figure 7Distribution diagrams of EyeQ and RIQA-RFMiD datasets are shown, where (a) is the distribution diagram of EyeQ training dataset, (b) is the distribution diagram of EyeQ valid dataset, and (c) is the distribution diagram of RIQA-RFMiD valid dataset;

[0089] Figure 8 Schematic diagrams of representative fundus images of each category are shown, where (a) is a “good” grade, (b) is a “usable” grade, and (c) is a “reject” grade;

[0090] Fig. 9 A schematic diagram showing the comparison of the impact of different modules in DFC-Net on the EyeQ dataset;

[0091] Fig.10 A schematic diagram showing the comparison of the impact of different modules in DFC-Net on the RIQA RFMiD dataset;

[0092] Fig.11 The heat maps of different levels in different modules based on Grad CAM are shown, where (a) is the RGB module, (b) is the CLAHE module, (c) is the DenseNet121 module, (d) is the DFC-DenseNet121(RGB) module, and (e) is the DFC-DenseNet121(CLAHE) module. DETAILED DESCRIPTION

[0093] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.

[0094] A retinal image quality assessment method using a dual-path frequency domain cross-attention fusion network, comprising the following steps:

[0095] Step S1, obtaining an RGB image;

[0096] Step S2, preprocessing the BGR image quality by using a contrast-limited adaptive histogram equalization algorithm to obtain a preprocessed CLAHE image;

[0097] Step S3, inputting the RGB image and the CLAHE image into the dual-path frequency domain DenseNet121 network, and extracting the features of the RGB image and the CLAHE image through the frequency domain attention mechanism algorithm;

[0098] Step S4, refining the extracted features through a double cross attention algorithm, obtaining the integration of the BGR image and the LAB image, and obtaining a merged feature set;

[0099] Step S5: input the combined feature set into a linear classifier, and obtain a quality assessment result through the linear classifier.

[0100] The architecture of the dual-path frequency domain cross-attention fusion network for retinal image quality assessment, namely the dual-path frequency domain DenseNet121 network, is shown in Figure 2. Figure 2 As shown in the figure, it contains two parallel DenseNet121 networks, one for processing the original RGB image and the other for processing the image enhanced by contrast limited adaptive histogram equalization (CLAHE). This setting allows the use of complementary features through different preprocessing methods. In the feature extraction stage, each DenseNet121 network operates independently, and the frequency domain attention module (FDAM) is implemented after each block. This setting helps the model emphasize specific frequency features that are usually ignored by traditional processing, especially in images affected by problems such as blur or low contrast. The effect is more obvious. Before the avgpool layer, the extracted features are refined by the dual cross attention module, which uses the generated attention map to emphasize key features and reduce less relevant features, thereby promoting effective feature integration. Subsequently, the features are merged through the global average pooling (avgpool) layer, and the merged feature set is input into the linear classifier. The classifier combines the characteristics of dual-channel network processing and outputs three quality levels of the image, thereby achieving accurate evaluation of the quality of fundus retinal images.

[0101] In retinal image quality assessment, image quality may vary greatly due to different shooting conditions, different equipment used, etc. In this regard, contrast-limited adaptive histogram equalization (CLAHE) is used as a preprocessing step to standardize the quality of input images to a certain extent and reduce the uncertainty in the model processing process. The dual-path input structure enables the model to not only enhance the quality of a single image, but also integrate information from two different image representations, thereby improving the model's adaptability and robustness to various image qualities.

[0102] like Figure 3As shown, by performing a preprocessing step with CLAHE, the model becomes more sensitive to subtle changes in image quality. This is particularly useful when evaluating images with blurred boundaries, where CLAHE highlights these boundaries, helping the model make more accurate quality assessments. In high-quality fundus images showing clear retinal structures, optimal lighting, and contrast, CLAHE processing can further improve detail visibility, making key anatomical features such as retinal vessels and macular areas clearer and easier to identify. These enhancements not only help clinicians make more accurate diagnoses, but also improve the ability of automated image quality assessment systems to identify and prioritize high-quality images by providing more detailed and more important information at the feature extraction stage. For poor quality images characterized by blur, uneven lighting, or occlusion, CLAHE can enhance local contrast to some extent, but may also exacerbate noise and non-ideal features, such as less distinct boundaries of blurred areas. In these cases, while details may become more distinct, the overall quality assessment may still classify the image as low quality if basic diagnostic features (such as clear vascular networks) are blurred or blurred. Nonetheless, the improved contrast helps automated systems to more accurately portray image defects, such as by highlighting contrast changes between blurred and normal structures. The specific steps of step S2 are as follows:

[0103] Step S21: In order to obtain a color CLAHE enhanced fundus retinal image, the BGR image needs to be converted into a LAB image. The specific formula is as follows:

[0104]

[0105] Among them, Transform represents the color space conversion matrix from BGR to LAB.

[0106] Step S22, then it is necessary to separate the brightness channel L, and perform CLAHE operation on the L channel alone. The specific process of CLAHE operation is as follows:

[0107] Step S221, first divide the image I into M×N small blocks I m,n (where m = 1, 2, ..., M and n = 1, 2, ..., N), the size of each small block is X × Y. For each small block I m,n , calculate its grayscale histogram H by the following formula m,n (K), where k represents a specific pixel value and K represents a set of pixel values:

[0108]

[0109] Step S222, then define clip_limit as the average number of pixels per small block divided by the number of histogram bins, adjusted to a specific contrast enhancement limit, the specific expression is as follows:

[0110]

[0111] Wherein, C is a contrast limiting hyperparameter. In this embodiment, C=2 is set. P represents the grayscale. Usually, P is 256.

[0112] The redundant parts of each bin k are removed and evenly distributed to satisfy the contrast constraint:

[0113]

[0114] Adjusted histogram H' m,n (k) is:

[0115]

[0116] Where L represents the range of pixel values ​​in the image.

[0117] Step S224, using the adjusted histogram H' m,n (k) Calculate the cumulative distribution function (CDF):

[0118]

[0119] Step S225, using CDF m,n (k) The original patch I m,n The gray value r in the channel is mapped to the new gray value L':

[0120] L'=round((P-1)·CDF m,n (r))

[0121] Step S226, bilinear interpolation is then applied between patches to smooth the patch edges and ensure the overall uniformity of the image. For the boundary pixels of four adjacent blocks, the final pixel value is calculated by bilinear interpolation using the mapping results of two or four adjacent blocks.

[0122] Step S23, combining the three LAB channels and finally converting them into a BGR image.

[0123] In this embodiment, the frequency domain attention mechanism (FDAM) is used to enhance the model's sensitivity to key quality indicators such as blur, lighting, and contrast. This mechanism significantly enhances the model's ability to identify problems related to these indicators by shifting attention. The focus is shifted from traditional spatial domain methods to frequency domain analysis. By emphasizing specific frequency components, FDAM enables the model to more accurately detect and evaluate image quality deterioration caused by blur, uneven lighting, or low contrast. Specifically, Figure 4 As shown, the entire frequency domain attention mechanism process in this embodiment is expanded into the following consecutive steps, that is, the specific steps of step S3 are as follows:

[0124] Step S31, given a batch of input data X∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width respectively, and the data X is transformed from the spatial domain to the frequency domain by 2D fast Fourier transform (FFT2), where the FFT2 operation is defined as:

[0125]

[0126] Where x and y are coordinates in the spatial domain, u and v are coordinates in the frequency domain, and F(u,v) is a complex number representation in the frequency domain, representing the amplitude and phase of frequencies u and v.

[0127] Step S32, use the fftshift operation to move the low-frequency part of the FFT result to the center, thereby obtaining the center frequency domain data Xshifted. Specifically, for each element in F(u,v), the fftshift operation modifies the array to G(u,v), and the elements are recalculated as follows:

[0128] For even dimensions H and W:

[0129]

[0130] For odd dimensions H and W:

[0131]

[0132] Step S33, then X shifted Decompose it into real and imaginary parts and process them separately as follows:

[0133]

[0134] Step S34: For each channel’s feature map, firstly, global average pooling is performed to obtain a feature vector representing the global information of each channel, and then the feature vector is passed through two fully connected layers (including ReLU activation function and S-type function). att,real ,X att,img :

[0135] X att,real =σ(FC2(ReLU(FC1(GAP(X real )))))

[0136] X att,imag =σ(FC2(ReLU(FC1(GAP(X imag )))))

[0137] Among them, σ() is the Sigmoid activation function, FC1 and FC2 represent the first and second fully connected layers respectively. Next, we recombined the real and imaginary parts of the attention weight into a complex frequency domain signal X att :

[0138] X att =C(u,v)=Complex(X att,real ,X att,imag )

[0139]

[0140] Step S35, complete the two-dimensional inverse fast Fourier transform (IFFT2) to convert X att After converting back to space, we can get the spatial representation s(x,y).

[0141] Step S36, the frequency is adjusted by using the inverse fftshift (ifftshift) operation to perform high-frequency domain FFT processing. Specifically, the inverse fftshift operation evaluates f(x, y) and restores its original spatial domain configuration. The detailed formula is as follows:

[0142] For even dimensions H and W:

[0143]

[0144] For odd dimensions H and W, the adjustment is slightly different to accommodate the asymmetric distribution of the zero-frequency components:

[0145]

[0146] Then, on this basis, the real part of g(x,y) is directly used as the final output, completing the entire process from the frequency domain attention mechanism to the spatial domain enhanced feature representation. Substituting this step into the following formula, the final output X out It can be expressed as:

[0147]

[0148] In the formula, In this way, the real part is extracted from g(x,y) to obtain the final output X out, effectively completing the conversion of attention from the frequency domain back to the spatial domain and enhancing feature representation.

[0149] In order to more effectively utilize the illumination characteristics of the original retinal fundus images as well as the enhanced contrast features provided by contrast-limited adaptive histogram equalization (CLAHE), a cross-attention mechanism is integrated in the dual-pathway model, strategically placed before the average pooling (avgpool) layer for optimal feature fusion. This approach enables the model to effectively synthesize information from the two processing paths, dynamically adjusting the contribution of each feature to maximize the utility of the convolutional features derived from different input images. Therefore, the final feature representation not only retains the essential information from the two inputs, but also retains their unique and complementary characteristics. This feature is crucial to enhancing the model's sensitivity to subtle changes in image quality. The training process of the model is as follows Figure 5 As shown, Figure 5 Here Q represents query, K represents key, V represents value, · represents dot product, A represents attention, O represents output feature, and + represents adding feature.

[0150] Specifically, the specific steps of step S4 are as follows:

[0151] Step S41, the starting point of the module processing is to receive two batches of input data x and y, both of which have the shape of x, y∈RB×C×H×W, where B represents the batch size, C represents the number of channels, and H and W represent the height and width, respectively. This approach makes the module compatible with the four-dimensional tensor input in the standard convolutional neural network architecture. In addition, the module converts x and y into Q and K forms through independent convolutional layers to reduce the computational complexity of subsequent processing steps while retaining key information. Please note that only the formula of the upper path is written here. The inverse of x and y is the formula of the lower path. The specific expression is:

[0152] Q=Conv query (x)

[0153] K=Conv key (y)

[0154] Among them, Q and K represent the feature representations of query and key respectively.

[0155] Step S42, calculate the energy score E by performing a dot product operation bmm() between the query and the key. This score measures the strength of the interaction between the two input features, and the calculation formula is as follows:

[0156] E=bmm(Q,K T )

[0157] Step S43, then send the calculated result of the energy score E to the Softmax function to obtain the final attention weights. This step ensures that the sum of all weights is 1. The equation is as follows:

[0158] A=Softmax(E)

[0159] Step S44, generate the final output feature by calculating the weighted sum between the value feature and the attention weight obtained previously. This step aims to incorporate the focus information directly into the output. The value feature is obtained by applying another convolution transformation to the input y, expressed as:

[0160] V=Conv value (y)

[0161] Where V represents the value feature.

[0162] In step S45, these value features are weighted using the previously calculated attention weights A to produce a weighted output. This process can be represented as the following matrix multiplication, where the attention weights are applied to the value features to focus on the most important information:

[0163] O=bmm(A,V)

[0164] Where O represents the weighted output feature.

[0165] Step S46, in order to enhance the model's ability to retain input features and enhance feature representation, the weighted output is combined with the original query input x. This step is achieved by simple element-by-element addition, ensuring that the model enhances key information while retaining the original context of the input data:

[0166] Output=O+x

[0167] Through these attention weights, the module dynamically adjusts its focus on the input data, prioritizing information that is more relevant to the query. This mechanism not only improves the model's sensitivity to key features, but also enhances its ability to handle complex data interactions.

[0168] In the DFC-Net model, gradient-weighted class activation maps are used to enhance its interpretability, which visualizes the regions within the fundus image that are important for determining the quality grade assigned by the model. This visualization is achieved through heat maps representing class activations, providing a clear depiction of the impact region of the predicted class.

[0169] The heat map output process is as follows Figure 6As shown, the DFC-Net model is used to process the input image and extract feature maps. These feature maps are derived from the cross-attention mechanism (CAM) layer of the DFC-Net model. The process can be described mathematically as follows:

[0170] Given an input image I, the DFC-Net model is used to extract the feature map F(I). Subsequently, we focus on the generation of the CAM layer heat map, considering the output features A of this layer. Based on these features, we first predict the score y for each category c. c :

[0171] y c = prediction(F(I))

[0172] where y c is the predicted class of image I, and prediction is a function of applying a threshold to the regression output to determine the class.

[0173] To generate the heatmap of the CAM layer, we focus on the output feature A of this layer. It represents the importance of channel k in feature layer A for identifying category c. The calculation formula is as follows:

[0174]

[0175] here, represents the value of the CAM layer feature at channel k and position (i, j), and Z is the product of the width and height of the CAM layer feature. Finally, a heat map of category c is generated on the CAM layer:

[0176]

[0177] In this formula, ReLU is the rectified linear unit function, which ensures that only positive values ​​are included in the heatmap.

[0178] This approach enables the implementation to generate heat maps for specific layers (such as the CAM layer in the model), which can help to gain a deeper understanding and explanation of the role of this layer in the model's decision-making process.

[0179] In this embodiment, experimental analysis is performed on two representative RIQA datasets: EyeQ and RIQA-RFMiD datasets. The data distribution of the two datasets is as follows: Figure 7 Representative fundus images of each category are shown in Figure 8 shown.

[0180] The EyeQ dataset makes a significant contribution to the field of retinal image quality assessment (RIQA) and utilizes approximately 30,000 newly annotated images from the EyePACS dataset. The dataset is specifically designed to improve the accuracy and applicability of RIQA systems under a variety of imaging conditions and camera types. It employs a three-level quality grading system to classify images according to four popular quality metrics: blur, uneven illumination, low contrast, and artifacts including shadow assessment. The details of these quality levels are as follows:

[0181] (1) “Good” level:

[0182] Definition: Images that are free of any low-quality elements and display all retinal features in high definition.

[0183] Features: These images show no signs of blur, uneven lighting, low contrast, or distracting artifacts. The clarity of these images makes them ideal for both automated and manual medical analysis.

[0184] (2) “Available” level:

[0185] Definition: These images contain minor quality defects that do not obscure the complete observation of the image but may interfere with some automated analysis methods.

[0186] Features: Minor problems such as slight blurring or shrinkage exist; major structures such as the optic disc and macular area, as well as any lesions, are clear enough for ophthalmologists to identify; for images affected by uneven lighting, at least 80% of the fundus is clearly visible, ensuring that most retinal areas are readable.

[0187] (3) “Rejection” level:

[0188] Definition: Images at this level have serious quality issues that severely hamper their usefulness for reliable analysis.

[0189] Characteristics: Such defects may include noticeable blur, extensive lighting issues, or major artifacts, all of which can compromise the integrity and readability of the image.

[0190] To ensure the accuracy of the EyeQ dataset, the images in the EyePACS dataset were re-annotated, and 28,792 retinal images that met the quality standards were finally selected. These images were distributed between the training set and the test set. Specifically, the training set included 8,347, 1,876, and 2,320 images of three different severity levels, respectively, while the test set included 8,470, 4,559, and 3,220 images of these levels, respectively.

[0191] The RIQA-RFMiD dataset annotates fundus image quality categories based on the RFMiD dataset, which helps develop a general model for automatic classification of 46 common diseases. It includes 1,920 images for training and 640 images for validation, with resolutions of 4288×2848, 2144×1424, or 2048×1536. Among them, 2,239 images were classified as "good", 194 images were classified as "usable", and the remaining 127 images were classified as "rejected".

[0192] The model framework of this embodiment is based on the DenseNet architecture, which uses pre-training on the extended ImageNet-1k dataset:

[0193] First, the field of view (FoV) is cropped using the Hough circle transform and the size is adjusted by padding the short sides with zeros to balance the width and height of the cropped area. Subsequently, the image size is resized to 224×224 pixels. In order to effectively optimize the network, training is started with an initial learning rate of 0.01 and the training is set to 60 epochs. MultiStepLR is used to dynamically adjust the learning rate and further refine the training process. The configuration uses the SGD optimizer and the cross entropy loss function with a batch size of 32. All implementations are performed using the PyTorch framework on a high-performance NVIDIA GeForce RTX 3090 GPU with 24GB of memory.

[0194] To evaluate the performance of this example, a confusion matrix is ​​used to visualize the differences between the various methods, using four key evaluation metrics: precision (P), recall (R), F1 score, and the important quadratic weighted Kappa index, which is preferred over accuracy when dealing with imbalanced datasets. The Kappa coefficient is particularly suitable for evaluating classification tasks in such contexts. These metrics are calculated on the test set to measure the effectiveness of the model. The formulas for these performance metrics are as follows:

[0195]

[0196]

[0197]

[0198] Among them, TPi represents True Positives, the number of instances accurately identified as category i; FP i represents a false positive, i.e., an instance belonging to a different category is incorrectly classified as category i; FN i Represents False Negatives, i.e., instances of class i are incorrectly labeled as not belonging to class i.

[0199] The formula for the quadratic weighted kappa coefficient measure is as follows:

[0200]

[0201] Where C represents the number of categories. The subscripts i and j represent the row index and column index in the confusion matrix, respectively. W represents the weight matrix, where the weight W ij Defined as O represents the observation matrix, where O ij Represents the element at position (i,j) in the confusion matrix. E represents the expectation matrix, which is calculated as the outer product of the predicted histogram and the actual histogram (the ground truth vector).

[0202] The classification capability of the DFC-Net model was fully evaluated, and a wide range of three-class classification tasks were performed using the EyeQ and RIQA-RFMiD datasets. In pursuit of excellence, the results were benchmarked against the results of the commonly used fundus image quality assessment (RIQA) classification. And the quantitative analysis involved a detailed comparison of the results of this embodiment with the results of advanced peers. The specific results of this evaluation are shown in Tables 1 and 2, which illustrate the superior performance of DFC-Net in this embodiment in the RIQA task.

[0203] Table 1. Comparison with state-of-the-art methods on the EyeQ dataset.

[0204]

[0205] Table 2. Comparison of different methods on the Private RIQA RFMiD dataset. (* represents the results of Xu et al.

[21] )

[0206]

[0207]

[0208] To evaluate the effectiveness of using paired RGB and CLAHE enhanced images as input for the fundus retinal image quality assessment task, alternative inputs including LAB images and images processed by U-net based fundus vessel segmentation were tried during the experiments for RIQA. The results show that the best results were achieved by the original pairing of RGB and CLAHE images, as shown in Table 3. Using only U-net segmented images (binary images) ignores key features such as color and thus proves to be insufficient for quality assessment. Similarly, since deep learning based segmentation accuracy is low compared to manual labeling, the input of combining RGB images with vessel segmentation images introduces noise, resulting in frequent classification errors. In contrast, CLAHE processing only enhances image contrast and structural clarity without changing the inherent lighting conditions, significantly improving the performance of the model in assessing image quality. In contrast, LAB images significantly change the illumination, which may change the characteristics of the original RGB image and reduce the classification effect. Therefore, CLAHE processing is confirmed to be a beneficial method to improve image quality while retaining essential features, making it an ideal choice for the fundus retinal image quality assessment task.

[0209] Table 3. Comparison of results for different input images on the EyeQ dataset.

[0210]

[0211] During training, using a continuously high learning rate may cause the model to oscillate or become unstable, which may lead to performance degradation. Conversely, a low learning rate may lead to slow convergence, continuous oscillations around the local optimum, and ineffective weight parameter updates. To address these issues, a multi-stage learning rate descent strategy was implemented, which reduced the learning rate by a factor of 10 at the 10th, 15th, and 25th epochs. Implementing this strategy with an initial learning rate of 0.01 increased the Kappa coefficient by 0.72% compared to the initial learning rate of 0.001. This increase can be attributed to the higher starting rate, which enables the model to quickly reach a better parameter space early in training, thereby avoiding local optimality. In addition, the Kappa coefficient obtained using the multi-stage learning rate descent strategy is 0.25% higher than that obtained using the cosine gradient descent strategy. The multi-stage learning rate descent strategy method adjusts the learning rate at predetermined milestones, which is well aligned with the variable requirements of the problem and promotes robust convergence to the optimal parameters. In contrast, the cosine gradient descent strategy periodically adjusts the rate according to the cosine function, which can cause the learning rate to change too fast or too slow during training, which can have a certain impact on the convergence performance of the model. These results are detailed in Table 4.

[0212] Table 4. Comparison of results under different learning rates and decay strategies.

[0213]

[0214] In order to explore the effectiveness of FDAM on the RIQA task, the SE and CBAM attention mechanisms are compared in the experiment. The experimental results are shown in Table 5:

[0215] Table 5. Comparison of the effects of different attention mechanisms in dual-path DenseNet121.

[0216]

[0217] The frequency domain attention mechanism (FDAM) is generally better than channel attention (SE) and channel space attention (CBAM). For example, in terms of the Kappa coefficient. The Kappa coefficient of the frequency domain attention mechanism is 0.18% higher than that of the channel attention (SE) and 0.07% higher than that of the channel space attention (CBAM). This shows that the frequency domain attention mechanism has better adaptability to the imbalance between different categories. From the F1 index, the frequency domain attention mechanism (FDAM) is 0.14% higher than the channel attention (SE), and the Kappa coefficient is 0.07% higher than that of the channel space attention (CBAM). This shows that the frequency domain attention mechanism (FDAM) has achieved a better balance in model prediction than the channel attention (SE) and the channel space attention (CBAM), that is, the model can take into account both accuracy and recall, and has strong predictive ability for both positive and negative examples. The reason why the attention mechanism in this embodiment performs better than the channel attention (SE) and channel space attention (CBAM) attention mechanisms is that it is effectively designed for the characteristics of the fundus retinal image quality assessment task. The channel attention (SE) and channel-wise spatial attention (CBAM) attention mechanisms mainly rely on the channel and spatial attention mechanisms in the spatial domain. The frequency domain attention mechanism (FDAM) first converts the image to the frequency domain, and by analyzing and adjusting the model's emphasis on specific frequency components, it can capture the quality degradation caused by factors such as blur, uneven lighting, or low contrast more sensitively. In contrast, the channel attention (SE) and channel-wise spatial attention (CBAM) only focus on the channel and spatial attention mechanisms in the spatial domain and cannot directly capture the frequency domain information. Therefore, they have certain limitations in dealing with quality issues related to specific frequency components.

[0218] A series of experiments were conducted using the EyeQ and RIQA-RFMiD datasets to investigate the impact of individual modules in DFC-Net. Initially, classification using only CLAHE images showed a slight drop in performance metrics, which could be due to the altered lighting conditions caused by CLAHE enhancement. However, inputting paired RGB and CLAHE images significantly improved the F1 and Kappa coefficients. Specifically, on the EyeQ dataset, the F1 score was improved by 0.22% and the Kappa coefficient was improved by 0.27%. On the RIQA-RFMiD dataset, these scores increased by 0.34% and 0.27%, respectively. To enhance the model’s sensitivity to illumination and feature discrimination across classes, we integrated a frequency domain attention mechanism (FDAM) after each DenseNet block. This modification resulted in a 0.31% improvement in the F1 score and a 0.27% improvement in the Kappa coefficient on the EyeQ dataset. Similarly, on the RIQA-RFMiD dataset, the F1 and Kappa coefficients were improved by 0.39% and 0.29%, respectively. In addition, to better integrate features from different sources, we introduced a contextual attention module (CAM) before the average pooling layer. This adjustment further improves the F1 and Kappa coefficient on the EyeQ dataset by 0.39% and 0.38%, respectively, and improves both indicators on the RIQA-RFMiD dataset by 0.40%. After fusing all these modules, the DFC-Net model shows significant improvements: on the EyeQ dataset, the precision is improved by 0.89%, the recall is improved by 1.26%, the F1 score is improved by 1.08%, and the Kappa coefficient is improved by 0.90%. On the RIQA-RFMiD dataset, the precision is improved by 1.32%, the recall is improved by 1.16%, the F1 score is improved by 1.20%, and the Kappa coefficient is improved by 0.79%. These comprehensive results are shown in Figure 2. Fig. 9 and Fig.10 shown.

[0219] There are two main purposes for generating attention maps using Grad-CAM: (1) to clarify the complex decision process of the model classification and enhance its interpretability, and (2) to evaluate the impact of the attention regions in the DFC-Net model compared to the standard DenseNet121. It is important to note that Grad-CAM does not participate in the training phase; it is applied after training and generates heat maps directly on the output of the relevant modules. The visualization includes five columns: the first column shows the original RGB image, the second column shows the image preprocessed with CLAHE, the third column contains the heatmaps generated by DenseNet121 alone for RGB images, the fourth column illustrates the heatmaps from the DFC-Net model on RGB images, and the fifth shows the heatmaps of DFC-Net on CLAHE enhanced images, as shown in Figure 2. Fig.11 As shown, the white arrow indicates the correct area and the red arrow indicates the incorrect area.

[0220] First row (“good” level): The DFC-Net model emphasizes the optic disc in CLAHE images more than in RGB images, highlighting the enhancement of structural details by CLAHE.

[0221] Second row (“usable” level): DenseNet121 incorrectly focuses on the optic disc (marked with a red arrow) and ignores tiny spots of light. In contrast, the DFC-Net model can identify these small spots of light in both RGB and CLAHE images.

[0222] Third row (“usable” level): While the DenseNet121 model focuses primarily on overly dark regions, it erroneously emphasizes the optic disc and ignores overly bright regions located above the optic disc. In contrast, the DFC-Net model accurately focuses on these darker regions in both RGB and CLAHE images. In addition, it is adept at identifying subtle, brighter regions just above the optic disc, especially in CLAHE-enhanced images.

[0223] Fourth row (“Reject” level): DenseNet121 incorrectly prioritizes the outer black border, possibly mistaking it for an extremely dark region. The DFC-Net model correctly focuses on the relevant light spots in both RGB and CLAHE images without being distracted by the black border.

[0224] Fifth row (“rejection” level): While DenseNet121 generally focuses on the correct bright spot region, it misses the key points on the left and below. In contrast, the DFC-Net model fully covers the bright spots on the left in the RGB image and detects the small bright spot region below in the CLAHE image.

[0225] As can be seen, the implementation of the dual-path DenseNet121 architecture utilizes both raw RGB and CLAHE preprocessed images, significantly improving retinal image quality assessment (RIQA). Traditional methods that rely solely on RGB images often fail due to changes in lighting and contrast, which adversely affects performance. By incorporating CLAHE, which enhances local contrast and balances brightness without changing the intrinsic content of the image, the DFC-Net model introduces a powerful preprocessing layer that significantly reduces the variability of the input. This approach not only preserves important structural details, but also improves the model's adaptability to different lighting conditions, which is critical for accuracy in high-risk medical imaging tasks.

[0226] The inclusion of a frequency-domain attention mechanism (FDAM) in the DFC-Net model mitigates a typical limitation of traditional deep learning models: poor discrimination between adjacent quality categories. FDAM targets specific frequency components associated with quality degradation, such as blur and uneven lighting, thereby increasing the model's sensitivity to subtle but critical defects. This approach effectively distinguishes between "usable" and "rejected" categories by highlighting the loss of high-frequency details that are prevalent in low-quality images. This enhanced sensitivity significantly improves the model's ability to classify subtle differences between quality levels, demonstrating the value of integrating advanced attention mechanisms into deep learning frameworks.

[0227] The DFC-Net model also introduces a cross-attention module that synergizes features extracted from the RGB and CLAHE preprocessing paths. This strategy not only retains essential information of each input type, but also dynamically adjusts feature contributions based on their relevance to the quality assessment task. This adjustment is critical to managing the inherent variability of retinal images, where small differences in quality can have significant diagnostic consequences.

[0228] The quantitative enhancements achieved by the DFC-Net model on the large-scale public EyeQ dataset and the RIQA-RFMiD private dataset demonstrate its improved capabilities. The improvements in precision, recall, F1 score, and Kappa coefficient reflect not only superior image processing performance, but also more reliable retinal image quality assessment, which is critical for accurate diagnosis. These advances suggest that the DFC-Net model can significantly reduce misdiagnosis rates and improve the efficiency of retinal screening programs, especially in settings where expert human evaluation is limited.

[0229] While the DFC-Net model shows considerable progress in RIQA, it faces challenges that need to be addressed by future research. The reliance on high-quality preprocessing and substantial computational resources to train the dual-pathway network may limit its deployment in resource-limited settings. Furthermore, its efficacy in extremely low-quality images is another area of ​​potential enhancement, where key features may be too degraded to be effectively extracted. Future research could explore incorporating sophisticated image restoration techniques prior to the quality assessment stage and employing semi-supervised learning models to improve generalizability and robustness. Another promising direction is to improve the image quality of classified low-quality images through advanced image processing methods, such as super-resolution reconstruction, to aid in the diagnosis of fundus diseases.

[0230] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.

Claims

1. A retinal image quality assessment method based on dual-path frequency domain cross-attention fusion, characterized in that: The following steps are involved: Step S1, obtaining an RGB image; Step S2, preprocessing the BGR image quality by using a contrast-limited adaptive histogram equalization algorithm to obtain a preprocessed CLAHE image; Step S3, inputting the RGB image and the CLAHE image into the dual-path frequency domain DenseNet121 network, and extracting the features of the RGB image and the CLAHE image through the frequency domain attention mechanism algorithm; Step S4, refining the extracted features through a double cross attention algorithm, obtaining the integration of the BGR image and the LAB image, and obtaining a merged feature set; Step S5: input the combined feature set into a linear classifier, and obtain a quality assessment result through the linear classifier.

2. According to claim 1, a retinal image quality assessment method based on dual-path frequency domain cross-attention fusion is characterized in that: The dual-path frequency-domain DenseNet121 network includes two parallel DenseNet121 networks, each of which operates independently and a frequency-domain attention mechanism algorithm module is provided after the operation module of each DenseNet121 network.

3. The retinal image quality assessment method based on dual-path frequency domain cross-attention fusion according to claim 2 is characterized in that: The specific steps of step S2 are as follows: Step S21, converting the BGR image in BGR image form into LAB image form, the expression is as follows: Where Transform represents the color space conversion matrix from BGR image to LAB image; Step S22, separating the luminance channel L, and performing a CLAHE operation on the L channel alone; In step S23, the three LAB channels are combined and finally converted into a LAB image.

4. The retinal image quality assessment method based on dual-path frequency domain cross-attention fusion according to claim 3 is characterized in that: The specific steps of step S22 are as follows: Step S221, divide the image I into M×N small blocks I m,n , the grayscale histogram of each small block is calculated by the following formula: In the formula, H m,n (K) is the grayscale histogram of each small block, x and y are the spatial coordinates of each small block, reflecting the size of each small block; Step S222, clip_limit is defined as the average number of pixels per small block divided by the number of histogram bins, adjusted to a specific contrast enhancement limit, expressed as follows: Where C is the contrast limit hyperparameter, and P is the gray level; In step S223, the redundant part of each bin k is removed and evenly distributed to meet the contrast constraint, as shown in the following expression: The adjusted histogram H'm,n(k) is: Step S224, using the adjusted histogram H'm,n(k) to calculate the cumulative distribution function, the expression is as follows: Where CDF is the cumulative distribution function; Step S225, using CDF m,n (k) The original patch I m,n The gray value r in the channel is mapped to the new gray value L', the expression is as follows: L'=round((P-1)·CDF m,n (r)) Step S226 , bilinear interpolation is applied between patches to smooth patch edges and ensure overall uniformity of the image.

5. The retinal image quality assessment method based on dual-path frequency domain cross-attention fusion according to claim 4 is characterized in that: In step S226, for the boundary pixels of four adjacent blocks, the final pixel value is calculated by bilinear interpolation using the mapping results of two or four adjacent blocks.

6. A retinal image quality assessment method based on dual-path frequency domain cross-attention fusion according to any one of claims 1 to 4, characterized in that: The specific steps of step S3 are as follows: Step S31, transform the input data from the spatial domain to the frequency domain through 2D fast Fourier transform, the expression is as follows: In the formula, X is the input data, and X∈R B×C×H×W , where B is the batch size, C is the number of channels, H and W are the height and width respectively, x and y are the coordinates in the spatial domain, u and v are the coordinates in the frequency domain, and F(u,v) is the complex number representation in the frequency domain, representing the amplitude and phase of frequency u and v; Step S32, using fftshift operation to move the low-frequency part of the FFT result to the center, thereby obtaining the center frequency domain data X shifted ; For even dimensions H and W: For odd dimensions H and W: Step S33: X shifted Decompose it into real part and imaginary part and process them separately. The expression is as follows: Step S34, obtain the feature vector representing the global information of each channel through global average pooling, and update the domain attention weight through two fully connected layers: X att,real σ(FC2(ReLU(FC1(GAP(X real ))))) X att,imag σ(FC2(ReLU(FC1(GAP(X imag ))))) Step S35, transforming X att Convert back to space and obtain the spatial representation s(x,y); Step S36, evaluate f(x,y) through the inverse fftshift operation and restore its original spatial domain configuration, the expression is as follows: For even dimensions H and W: For odd dimensions H and W, the adjustment is slightly different to accommodate the asymmetric distribution of the zero-frequency components: Step S37, taking the real part of g(x,y) as the final output.

7. The retinal image quality assessment method based on dual-path frequency domain cross-attention fusion according to claim 6 is characterized in that: The specific steps of step S4 are as follows: Step S41, convert the two batches of input data x and y into Q and K forms through independent convolutional layers, as shown below: Q=Conv query (x) K=Conv key (y) In the formula, x and y are input data, x, y∈R B×C×H×W , where B represents the batch size, C represents the number of channels, H and W represent the height and width respectively, and Q and K represent the feature representation of query and key respectively; Step S42, calculate the energy score E by performing a dot product operation bmm() between the query and the key, and the expression is as follows: E=bmm(Q,K T ) Step S43, the calculation result of the energy score E is sent to the Softmax function to obtain the final attention weight, and ensure that the sum of all weights is 1, the expression is as follows: A=Softmax(E) Step S44, the final output feature is generated by calculating the weighted sum between the value feature and the previously obtained attention weight, and the expression is as follows: V=Conv value (y) In the formula, V represents the value feature; Step S45, use the attention weight A to weight these value features to generate a weighted output, the expression is as follows: O=bmm(A,V) In the formula, O represents the weighted output feature; Step S46, combining the weighted output feature with the original query input as the output result, the expression is as follows: Output=O+x Where x is the original query input.

Images