An enhanced method for retinopathy grading detection

Abstract

The application relates to an enhanced method for retinopathy grading detection. The sensitivity of a CNN network to global features is enhanced by using a Transformer module; image interaction is established by using a cross-attention module; and information related to various lesion sizes is effectively obtained and integrated by using a multi-level feature map. The application has a positive promoting effect on retinopathy grading detection based on fundus images.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology and is used to classify the severity level of retinal lesions based on fundus images. Specifically, it relates to an enhancement method for retinal lesion grading detection. Background Technology

[0002] In recent years, an increasing number of CNN-based methods have been introduced to help ophthalmologists detect diabetic retinopathy (DR). CNN-based DR detection methods are mainly divided into two categories based on the type of annotation required: pixel-level methods and image-level methods.

[0003] Pixel-level methods leverage information from minute features (e.g., MA, HE, and EX) within lesions using pixel-level labels. Van et al. proposed an enhanced convolutional network for hemorrhage detection in retinal images, dynamically selecting misclassified negative samples during the training phase. Dai et al. developed an interleaved depth mining technique for detecting microaneurysms by combining retinal images and expert reports. Lin et al. introduced a novel framework combining raw image and lesion features for the grading diagnosis of diabetic retinopathy (DR). They also designed a lesion detection model to mitigate the impact of missing sample annotations. Zhou et al. introduced a collaborative learning approach combining segmentation and grading, improving the performance of disease grading and lesion segmentation through attention-based semi-supervised learning. Huang et al. presented a self-supervised deep learning method based on contrastive learning. They employed a supervised Faster R-CNN network to extract pathological patches. After data augmentation of these patches, they were used as positive examples in the context of contrastive learning, while different patches were used as negative examples. Chen et al. designed a two-stage classification model that integrates multi-lesion segmentation and classification.

[0004] Image-level methods directly train retinal image classification models using image-level labels. Gargeya and Leng developed a reliable CNN-based automatic detection algorithm for diabetic retinopathy (DR). Wang et al. simulated the clinical examination process of retinal images by magnifying retinal images and highlighting high-resolution suspicious patches to help predict the severity of diabetic retinopathy (DR). Bodapati et al. employed pooling and fusion techniques to combine feature maps from various stages of VGG-16. Shaik and Cherukuri used a channel attention cascaded neural network with a spatial attention autoencoder to classify diabetic retinopathy (DR). Luo et al. introduced a nonlocal means algorithm to capture long-range dependencies between patches.

[0005] Most CNN-based methods do not consider the potential effectiveness of using long-range dependencies to improve grading accuracy. However, in the context of DR detection, valuable information can be extracted not only from local features but also from the global relationships between lesion patches (such as microaneurysms, hemorrhages, and exudates) scattered across retinal images. Luo et al. proposed a method to address the problem of capturing long-range dependencies between patches. Another noteworthy point is that existing CNN methods give little consideration to establishing interactions between images, whereas in reality, individuals can visually compare and distinguish differences and similarities between images. Furthermore, very few CNN-based methods consider utilizing feature maps from different stages. Due to the varying downsampling levels of feature maps at different stages, they always provide more information for the corresponding field of view of lesions of different sizes, as small lesions are easily missed in a large field of view, and large lesions cannot be fully observed in a small field of view. Summary of the Invention

[0006] The purpose of this invention is to provide an enhanced method for grading retinal lesion detection. First, a transformer-based structure is used to capture long-range dependencies between pixels to provide a finer-grained consideration of minute lesions. Second, a cross-attention structure is introduced to establish interactions between images to capture fine-grained differences between different images. Furthermore, an attention-based method is employed to effectively distinguish lesion size and merge feature maps from different levels, thereby enhancing our adaptability to different images with varying lesion sizes.

[0007] To achieve the above objectives, the technical solution of the present invention is: an enhancement method for retinal lesion grading detection, which utilizes the Transformer module to enhance the sensitivity of the CNN network to global features; establishes image interaction through the cross-attention module; and effectively acquires and integrates information related to the size of various lesions through multi-level feature maps, thereby constructing a network architecture for retinal lesion grading detection.

[0008] In one embodiment of the present invention, the network architecture includes an improved CMT and Fusion_SK.

[0009] In one embodiment of the present invention, the improved CMT includes a Layer Norm layer (layer normalization, used for feature map normalization within the same channel), a Lightweight MHSA layer, a LayerNorm layer, and an InvertedResidualFFN layer connected in sequence; wherein,

[0010] Lightweight MHSA layer:

[0011] Input Linear to query key value Symbols d, d k and d v These represent the dimensions of input, key or query, and value, respectively; H and W denote the height and width of the input feature map; to reduce computational overhead, a k×k depthwise separable convolution (DWConv) with stride k is used to reduce the spatial size of K and V before the attention operation, i.e. and Furthermore, the learnable relative positional bias B is integrated into each self-attention module, replacing the original absolute positional bias in the Transformer module, aiming to give the image rotation invariance similar to convolution; the lightweight self-attention is:

[0012]

[0013] Where the normalized exponential function

[0014] Then, a cross-self-attention mechanism is proposed, which aggregates the information of all images in batch size (referring to the number of samples used in one parameter update of the model) before the attention operation. (K c (Calculated values ​​of the keys in different channels) and (V c The calculated values ​​for different channels), the corresponding cross-lightweight attention is defined as:

[0015]

[0016] Cross-lightweight attention is only used during the attention learning training phase, while lightweight self-attention is used during the attention learning testing phase.

[0017] Inverted Residual FFN Layer: This layer consists of expansion layers, depthwise convolutions, and projection layers, interspersed with GELU activation layers and a final linear layer followed by batch normalization, similar to an inverted residual block.

[0018] IRFFN(X)=Conv(F(Conv(x)))

[0019] Where IRFFN(X) is a reverse residual feedforward network used to enhance the model's ability to acquire image features, and Conv(x) is a convolutional layer used to extract image features;

[0020] F(x) = DWConv(X) + X

[0021] Where F(x) represents the residual block, which is used to avoid gradient vanishing and make the network deeper; X is the input image feature;

[0022] The location of the activation layer is omitted in the above formula.

[0023] 1. The enhancement method for grading retinal disease detection according to claim 2, characterized in that the Fusion_SK includes a Fuse module and a Select module, and the input consists of various feature maps adjusted to the same dimension. in,

[0024] Fuse module:

[0025] First, the results of multiple feature maps are merged by element-wise summation:

[0026] U=∑U i

[0027] Among them U i The feature map representing the i-th stage of the network architecture;

[0028] Subsequently, global average pooling is used to extract global features for each channel, followed by dimensionality reduction using a fully connected layer:

[0029] Z = RELU(BN(W(F) gp (U))))

[0030] Where RELU = max(0,x); BN stands for Batch Normalization, which can be used to speed up convergence and avoid gradient vanishing and gradient exploding.

[0031] Where F gp It is global average pooling. This represents the weight matrix of the fully connected layer, with its values ​​controlled by deceleration ratios r=2 and L=32:

[0032] d = max(C / r, L)

[0033] Where d is the number of features output from the fully connected layer, C is the number of channels in the input feature map, and max is the maximum of the two numbers.

[0034] Finally, multiple fully connected layers are used to map the score vector for each feature map:

[0035] S i =RELU(BN(W i Z))

[0036] in

[0037] Select module:

[0038] By leveraging cross-channel soft attention, different weights are adaptively assigned to various feature maps to control the size of their output streams. Specifically, multiple score vectors are subjected to a softmax operator at the channel level to obtain weight vectors, which are then used to activate or suppress different feature maps.

[0039]

[0040] Where A ik The weights of different channels are calculated using the Select module; e is the base of the logarithmic function, and S... ik These are scores from different channels.

[0041] U i =U i A i

[0042] Where i is the feature map index and k is the channel index.

[0043] Compared to existing technologies, this invention offers the following advantages: By integrating the transformer module into the CNN architecture, this invention simultaneously preserves features related to both local and global dependencies, thereby enhancing representational capabilities. The introduction of cross-attention within the transformer architecture promotes contrastive learning between different images and also serves a regularization function. Furthermore, this invention proposes an attention fusion method to comprehensively utilize feature maps of different sizes, thereby enhancing the accurate identification of lesions of varying sizes. Attached Figure Description

[0044] Figure 1 This is a flowchart of the method of the present invention.

[0045] Figure 2 This is a model structure diagram of the method of the present invention.

[0046] Figure 3 This is the confusion matrix for the five-class classification accuracy on the APTOS dataset.

[0047] Figure 4 This is the confusion matrix for normal / abnormal classification accuracy on the Messidor dataset.

[0048] Figure 5 Visualization of Grad-CAM on the Messidor dataset; from left to right, the original image, the results of the InceptionV3 model, and the results of our proposed method are shown.

[0049] Figure 6Comparison of receiver operating characteristic (ROC) curves for normal / abnormal grading of different methods on the Messidor dataset. Detailed Implementation

[0050] The technical solution of the present invention will now be described in detail with reference to the accompanying drawings.

[0051] like Figure 1 As shown, the proposed method of this invention has three objectives: to enhance the sensitivity of CNN networks to global features using the Transformer module; to establish inter-image interaction through the cross-attention module; and to effectively acquire and integrate information related to the size of various lesions through multi-level feature maps. Figure 2 As shown, the proposed network architecture mainly includes CMT and Fusion_SK structures, which will be described in detail in the following two sections.

[0052] Traditional vision transformer architectures directly divide the input image into non-overlapping blocks and serialize each block. This structure requires significant computational resources when processing high-resolution images and is insufficient in modeling the structural information within each block, as it can only capture it through linear projection. Guo et al. designed a hybrid architecture combining CNNs and transformers (CMT) to leverage the advantages of CNNs and compensate for some of the shortcomings of using pure transformers. This structure uses scalable self-attention modules and learnable relative position encoding to make it compatible with convolutional networks. The scalable self-attention structure effectively addresses the problem of excessive memory consumption when the feature maps are large. Figure 2 As shown, before the self-attention operation, depthwise separable convolutions are used to downsample K and V in the self-attention process by a factor of k, allowing for flexible adjustment of memory usage based on the value of k. On the other hand, relative position encoding is similar to absolute position encoding commonly found in transformer structures. Its learnability ensures invariance to image rotation and translation, while absolute position encoding breaks this invariance. We employ this structure to address the limitations of convolutional networks in capturing long-range dependencies. In this process, we remove the convolutional part of the original CMT structure because it duplicates the functionality of the CNN network itself.

[0053] Intuitively, a highly effective approach when considering images with minimal contrast differences is to repeatedly and carefully observe and compare them. This is particularly important in the context of DR grading, where the distinction between closely related categories (especially normal and mild DR) is very subtle. Furthermore, the number of datasets available for training is relatively limited due to the inherent challenges of acquiring datasets for medical images. With relatively small datasets, increasing the number of neural network parameters and model depth can lead to a higher risk of overfitting. Therefore, structures capable of establishing interactions between images and facilitating regularization are highly beneficial for DR classification. Zhu et al. introduced a pairwise cross-attention mechanism based on a visual transformer to establish interactions between image pairs, thereby enabling the differentiation of fine-grained differences between image pairs. In this approach, attention scores are diffused to another image, increasing the complexity of attention learning for the current image. This is a form of regularization. Inspired by this approach, we integrate the cross-attention mechanism into the aforementioned transformer architecture to better capture variations caused by subtle lesions in images and provide regularization. To maintain consistency with the batch normalization structure commonly used in traditional CNNs, we introduce multiple images as perturbations for regularization, rather than normalizing attention learning in a single image. To enhance regularization capabilities and enable non-lesion images to acquire lesion-related information for better differentiation, we employed direct fusion instead of concatenation. These modifications led to our final improved CMT structure, and experimental results confirmed the benefits of these changes.

[0054] The Fusion_SK structure primarily involves the utilization of multi-level feature maps, where feature maps from different stages are resized to the same size and fused with different weights. Different lesions (even those of the same type) may have different sizes. Accordingly, feature maps from different stages have different downsampling rates, resulting in different receptive fields. An appropriate receptive field contains more lesion-related information because a suitable field of view will not miss lesions due to their small or large size, thus preventing incomplete observation. Therefore, since the grading of diabetic retinopathy is strongly correlated with the lesion, we introduce this structure to acquire lesion information under different receptive fields. The attention structure employed can adaptively adjust the weights to accommodate the differences in lesions across different images.

[0055] 1. CMT

[0056] Layer Norm, or layer normalization, is used to normalize feature maps within the same channel.

[0057] Lightweight Multi-head Self-attention Layer: In the original lightweight multi-head self-attention module, the input... Linear to query key value Symbol d,d k and d v These represent the dimensions of the input, key (query), and value, respectively. The symbols H and W denote the height and width of the input feature map. To reduce computational overhead, a k×k depthwise separable convolution with a stride of k is used to reduce the spatial size of K and V before the attention operation. and Furthermore, it integrates learnable relative positional biases (denoted as B) into each self-attention module, replacing the original absolute positional biases in the Transformer module. This integration aims to give the image rotation invariance similar to convolution. The lightweight self-attention module is then applied as follows:

[0058]

[0059] The grading of diabetic retinopathy is closely related to the presence of lesions. Normal images often lack lesions, and the type and number of lesions typically vary across different images. While self-attention can capture global information and model the interrelationships of each pixel in an image, it cannot capture information about lesions that are not present. Mixing features from multiple images provides a method for obtaining rich lesion information. During training, this allows images without lesions and those with fewer lesions to develop stronger lesion recognition and discrimination abilities, thus affecting the overall network's ability to acquire lesion information. Furthermore, this process can be viewed as introducing noise from other images during self-attention, where information from other images diverts the self-attention of the image, resulting in a regularization effect. During the testing phase, a standard self-attention structure is employed to ensure that the self-attention of an image is not affected by other images and does not introduce any additional computational overhead. Therefore, we propose a cross-self-attention mechanism that aggregates information from all images in a batch size before the attention operation. and The corresponding cross-lightweight attention definition is:

[0060]

[0061] It can be viewed as a regularization technique applied only during attention learning training and removed during inference without incurring any additional computational cost.

[0062] Inverted Residual Feed-forward Network: Figure 2 The architecture is visualized, consisting of an extension layer, followed by depthwise convolutional and projection layers, interspersed with GELU activation layers and a final linear layer before batch normalization, similar to an inverse residual block:

[0063] IRFFN(X)=Conv(F(Conv(x)))

[0064] F(x) = DWConv(X) + X

[0065] The location of the activation layer is omitted in the formula.

[0066] 2. Fusion_SK

[0067] To dynamically adjust the weights of different feature maps, the Fusion_SK module is mainly divided into two parts: Fuse and Select. The input consists of various feature maps that have been adjusted to the same dimension. Its structure is as follows Figure 2 As shown.

[0068] Fuse: To achieve dynamic adjustment of the weights of different feature maps, the basic concept is to control the amount of information from different feature maps, which can then flow through gates to the neurons in the next layer. To achieve this, these gates need to integrate information from all feature maps. We first combine the results of multiple feature maps by element-wise summation:

[0069] U=∑U i

[0070] Among them U i This represents the feature map of the i-th stage of the network.

[0071] Subsequently, we employ global average pooling to extract global features for each channel, and then use a fully connected (fc) layer for dimensionality reduction:

[0072] Z = RELU(BN(W(F) gp (U))))

[0073] Where F gp It is global average pooling. To represent the weight matrix of the fully connected layer, we use deceleration ratios r=2 and L=32 to control its values:

[0074] d = max(C / r, L)

[0075] Finally, multiple fully connected layers are used to map the score vector for each feature map:

[0076] Si =RELU(BN(W i Z))

[0077] in

[0078] Selection: Utilizing cross-channel soft attention, different weights are adaptively assigned to various feature maps to control the size of their output streams. Specifically, multiple score vectors undergo a softmax operator at the channel level to obtain weight vectors. These weight vectors are then used to activate or suppress different feature maps.

[0079]

[0080] U i =U i A i

[0081] Where i is the feature map index and k is the channel index.

[0082] 3. Experimental Data and Evaluation

[0083] 3.1 Dataset

[0084] The APTOS2019 dataset was used for the APTOS2019 Blindness Detection Challenge, a collaboration between the Asia Pacific Teleophthalmology Society (APTOS) and Kaggle. The dataset contains 5,990 high-resolution retinal scan images, with 3,662 retinal images allocated to the training set and 1,928 images allocated to the test set. Only 3,662 images in the training set are publicly accessible. The retinal images provided in the dataset are categorized into five classes: normal (grade 0), mild DR (grade 1), moderate DR (grade 2), severe DR (grade 3), and proliferative DR (grade 4), depending on the level of lesion presence associated with vascular abnormalities caused by diabetic retinopathy (DR). We resized all images to 512×512, striking a balance between largely preserving pathological information and avoiding excessive hardware requirements.

[0085] The Messidor dataset: This dataset is part of the TECHNO-VISION project sponsored by the French Ministry of Defense in 2004. It includes a total of 1200 retinal images from three different ophthalmology institutions. We randomly selected three-quarters of the images as the training set and reserved the remaining images for the test set. In the Messidor dataset, diabetic retinopathy (DR) severity is divided into four stages from 0 to 3. In this study, we employed a binary classification method to divide DR into two categories: normal and abnormal. Specifically, stage 0 is generally considered normal, while the other stages (1, 2, and 3) are considered abnormal. This method effectively divides DR into two major categories, enabling the model to predict whether an eye is affected by DR.

[0086] 3.2 Evaluation Indicators

[0087] In our experiments, we used the most widely used metrics to evaluate DR classification performance, including accuracy (ACC), sensitivity (SN), specificity (SP), precision (Pre), and F1 score, calculated as follows:

[0088]

[0089]

[0090]

[0091]

[0092]

[0093] In this equation, C i C represents the number of images belonging to class i in the dataset, and C represents the total number of images in the dataset, C = ∑ i C i TP, TN, FP, and FN represent true positive, true negative, false positive, and false negative, respectively.

[0094] In the context of disease prediction, assuming a patient has no disease, one doctor predicts the condition to be highly severe, while another doctor predicts it to be moderately severe. Clearly, the first doctor's prediction is less acceptable. Therefore, we further utilize quadratic weighted kappa (QWK) to evaluate the model's performance:

[0095]

[0096] Among them O i,j w represents the number of classes i predicted as class j. i,j E represents the square of the weighted distance. i,j These are elements in the expectation matrix.

[0097] 3.3 Experiment Details

[0098] 3.3.1 Experimental Results on the APTOS Dataset

[0099] To evaluate the performance of the proposed method on the APTOS dataset, we first visualize the prediction results using a confusion matrix, such as... Figure 3 As shown in the diagram, in this confusion matrix, each row represents the actual class of the data, and each column represents the predicted class. For ease of analysis, we use a normalized confusion matrix. It can be seen that the prediction accuracy for the "normal" class is 98.7%, while the accuracy for the "mild," "moderate," and "PDR" classes is 72.6%, 75.8%, and 78.5%, respectively. The prediction accuracy for the "severe" level is only 55%, but when errors occur, they are mainly concentrated in two closely related lesion types, making these errors more acceptable.

[0100] To obtain more reliable results, we employed k-fold cross-validation to evaluate the model's performance. Considering the dataset size, we chose 5-fold cross-validation, with 80% of the data used for training and 20% for validation. When splitting the training and validation sets, we ensured consistency in class distribution between each subset and the original dataset. After five rounds of training, we calculated the average of all results.

[0101] In the experimental comparison phase, we used accuracy as the evaluation metric and compared our model with some state-of-the-art methods, as shown in Table 1. The results show that our method outperforms the comparison methods by 0.42% to 3.96%, demonstrating its superiority.

[0102] Table 1. Accuracy of different methods

[0103]

[0104] To further quantitatively evaluate the model's performance, we introduced additional evaluation metrics, including specificity (SP), accuracy, F1 score, and quadratic weighted kappa (QWK). We comprehensively compared our method with various open-source models, including VGG, ResNet, DenseNet, and InceptionV3. The results are shown in Table 2.

[0105] Table 2 Comparison with various baseline models

[0106]

[0107] As can be seen, our method achieves excellent results across all metrics. The highest F1 score reflects that our model optimally balances precision and recall in the classification task. The highest value of the quadratic weighted kappa indicates that our model is more likely to make fewer major errors in its predictions, such as misclassifying class 0 as class 4. Specifically, the backbone network used in our method is InceptionV3, and our final results show improved performance on all metrics. This demonstrates the effectiveness of our proposed method.

[0108] 3.3.2 Experimental Results on the Messidor Dataset

[0109] To evaluate the performance of the proposed method on the Messidor dataset, we use a confusion matrix to visualize the algorithm's performance, such as... Figure 4 As shown in the diagram, in this confusion matrix, each row represents the actual class of the data, and each column represents the predicted class. For ease of analysis, we use a normalized confusion matrix. It can be seen that the prediction accuracy for the normal class is 96.4%, while the prediction accuracy for the outlier samples is 94.5%. These results indicate that our model has positive prediction results.

[0110] Furthermore, we utilize Grad-CAM to further visualize the advantages of our proposed method. For example... Figure 5 As shown, the image on the left is the original image, the image in the middle represents the baseline network Inceptionv3 we used, and the image on the right shows the results obtained from our proposed method. These results were computed on the layer immediately preceding global average pooling. Clearly, our method effectively focuses attention on pathological regions, providing valuable interpretability for medical diagnosis. In contrast, Inceptionv3's attention appears more scattered and lacks clear interpretability. This demonstrates that our proposed method enhances the ability to identify lesions and effectively locates and focuses on lesion regions.

[0111] In our comparative experiments on the Missidor dataset, we referenced the comparison method described by Wang et al., using AUC (Area Under the Curve) and accuracy (ACC) as our evaluation metrics. AUC measures the area under the receiver operating characteristic curve and is used to evaluate the classifier's performance. We compared our method with some state-of-the-art methods, and the results are shown in Table 3. Our method achieved an AUC of 98.3%, which is 1.6% to 11.3% higher than the comparison methods. In terms of ACC, our method outperformed the comparison methods by 3.2% to 8.2%. These results clearly demonstrate that our method exhibits superior recognition ability in binary classification, surpassing some state-of-the-art methods.

[0112] Table 3. AUC of different methods for normal / abnormal classification on the Messidor dataset.

[0113]

[0114] Furthermore, we compared our method with various popular open-source models, using evaluation metrics such as sensitivity, specificity, and F1 score, to further validate the effectiveness of our approach. We also generated receiver operating characteristic (ROC) curves to visualize the model's performance. ROC curves for different methods are shown below. Figure 6 As shown, our proposed method is intuitively demonstrated to outperform all comparative models and achieves the best performance in the classification of normal / abnormal cases.

[0115] Table 4 Comparison with various baseline models

[0116]

[0117] Table 4 shows that our method performs exceptionally well in terms of precision, sensitivity, F1 score, and AUC score, improving upon the control method by 1% to 2.66%, 0.58% to 3.68%, 1.05% to 2.44%, and 0.72% to 1.63%, respectively. Furthermore, due to the limited size of the dataset, we observed a decrease in accuracy with increasing network depth in the ResNet and DenseNet family of networks. This again highlights the advantages of incorporating the cross-attention mechanism into our CMT. This further underscores the advantages of incorporating the cross-attention mechanism into our CMT. Due to its inherent regularization effect, it effectively mitigates overfitting when dealing with small datasets, a common limitation in the medical field, especially in the detection of rare cases. Moreover, when the dataset is sufficiently extensive, it can effectively supplement lesion information from different images, thereby enhancing the network's sensitivity to lesions.

[0118] The above are preferred embodiments of the present invention. Any changes made to the technical solution of the present invention that do not exceed the scope of the technical solution of the present invention shall fall within the protection scope of the present invention.

Claims

1. An enhancement method for grading retinal diseases, characterized in that, This paper utilizes a Transformer module to enhance the sensitivity of CNN networks to global features; establishes image-to-image interaction through a cross-attention module; and effectively acquires and integrates information related to various lesion sizes through multi-level feature maps, thereby constructing a network architecture for retinal lesion grading detection. The network architecture includes an improved CMT and Fusion_SK; the improved CMT comprises a LayerNorm layer, a Lightweight MHSA layer, a Layer Norm layer, and an Inverted Residual FFN layer connected in sequence. Layer Norm is a layer normalization layer used for feature map normalization within the same channel; Lightweight MHSA layer: Input Linear transformation to query key value Symbols d, d k and d v These represent the dimensions of the input (key or query) and value, respectively; H and W denote the height and width of the input feature map; to reduce computational overhead, a k×k depthwise separable convolution DWConv with stride k is used to reduce the spatial size of K and V before the attention operation, i.e. and Furthermore, the learnable relative positional bias B is integrated into each self-attention module, replacing the original absolute positional bias in the Transformer module, aiming to give the image rotation invariance similar to convolution; the lightweight self-attention is: Where Softmax is the normalization exponential function; Then, a cross-self-attention mechanism is proposed, which aggregates the information of all images in the batch size before the attention operation. and K c V represents the calculated value of the bond in different channels. c The calculated values ​​for the median of different channels are given, and the corresponding cross-lightweight attention is defined as follows: Cross-lightweight attention is only used during the attention learning training phase, while lightweight self-attention is used during the attention learning testing phase. The Fusion_SK includes a Fuse module and a Select module, and its input consists of various feature maps that have been adjusted to the same dimension. in, Fuse module: First, the results of multiple feature maps are merged by element-wise summation: U=∑U i Among them U i The feature map representing the i-th stage of the network architecture; Subsequently, global average pooling is used to extract global features for each channel, followed by dimensionality reduction using a fully connected layer: Z=RELU(BN(W(F gp (AT)))) Where RELU = max(0,x); BN is batch normalization, used to speed up convergence and avoid gradient vanishing and gradient exploding; Where F gp It is global average pooling. This represents the weight matrix of the fully connected layer, with its values ​​controlled by deceleration ratios r=2 and L=32: d = max(C / r, L) Where d is the number of features output by the fully connected layer, C is the number of channels in the input feature map, and max is the maximum of the two numbers; Finally, multiple fully connected layers are used to map the score vector for each feature map: S i =RELU(BN(W i WITH)) in Select module: By leveraging cross-channel soft attention, different weights are adaptively assigned to various feature maps to control the size of their output streams. Specifically, multiple score vectors are subjected to a softmax operator at the channel level to obtain weight vectors, which are then used to activate or suppress different feature maps. Where A ik The weights of different channels are calculated using the Select module; e is the base of the logarithmic function, and S... ik These are scores from different channels; U i =U i A i Where i is the feature map index and k is the channel index.

2. The enhancement method for grading retinal diseases according to claim 1, characterized in that, Inverted Residual FFN Layer: This layer consists of expansion layers, depthwise convolutions, and projection layers, interspersed with GELU activation layers and a final linear layer followed by batch normalization, similar to an inverted residual block. IRFFN(X)=Conv(F(Conv(x))) IRFFN(X) is a reverse residual feedforward fully connected layer used to enhance the model's ability to acquire image features, and Conv(x) is a convolutional layer used to extract image features. F(x) = DWConv(X) + X Where F(x) represents the residual block; X is the input image feature.

Images