Diabetic retinopathy intelligent grading method based on enhanced efficientNetB0 and MobileNetV1 fusion
By improving the fusion model of EfficientNetB0 and MobileNetV1 and combining RLGA and GLAM modules, the problems of low efficiency, misdiagnosis and data imbalance in the diagnosis of diabetic retinopathy were solved, and the classification accuracy and model performance were improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- NANJING FORESTRY UNIV
- Filing Date
- 2025-07-01
- Publication Date
- 2026-05-01
AI Technical Summary
Existing computer-aided diagnostic systems for diabetic retinopathy suffer from low efficiency, high cost, long processing time, and a high risk of misdiagnosis. Furthermore, CNN technology requires a large amount of labeled data and suffers from multi-source heterogeneity and class imbalance.
A novel fusion model is designed by improving the EfficientNetB0 model and combining it with MobileNetV1. The RLGA and GLAM modules are introduced, and transfer learning and deep convolution are utilized. By combining the compound scaling strategy of EfficientNetB0 and the depthwise separable convolution of MobileNetV1, the feature extraction capability and classification accuracy are improved.
It significantly improves the classification accuracy and generalization ability of diabetic retinopathy, solves the problems of multi-source heterogeneity and class imbalance, and enhances the model's feature extraction ability and classification performance.
Smart Images
Figure CN121963280A_ABST
Abstract
Description
A Smart Grading Method for Diabetic Retinopathy Based on the Fusion of Enhanced EfficientNetB0 and MobileNetV1 Technical Field
[0001] This invention belongs to the field of medical image processing technology, specifically a smart classification method for diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1. The core of this invention lies in designing and implementing a novel fusion model after image preprocessing and image augmentation. By improving the EfficientNetB0 model and fusing it with the MobileNetV1 model, multi-scale feature complementarity is achieved, significantly improving classification accuracy. The improvements to the EfficientNetB0 model include introducing an RLGA module after the top-level feature map to enhance feature extraction capabilities, and combining a Local Perception Module (LPM) and a Global Local Attention Module (GLAM) to refine local features and enhance global feature perception. Simultaneously, transfer learning from EfficientNetB0 is used to improve classification performance, and a 3×3 depthwise convolution is added to the residual shortcut branch to avoid gradient propagation and strengthen learned features, thereby further improving evaluation metrics. Finally, the improved EfficientNetB0 is fused with MobileNetV1, combining the composite scaling strategy of EfficientNetB0 and the depthwise separable convolution advantages of MobileNetV1 to improve the model's classification accuracy. Background Technology
[0002] Long-term complications of diabetes can lead to damage to multiple organ functions, particularly the vascular, nervous, visual, and renal systems, such as diabetic retinopathy (DR), myopia, glaucoma, cataracts, and hypertension. DR is a microvascular complication specific to diabetes, pathologically manifested as retinal vascular disease. The pathological mechanism involves changes in blood components leading to abnormal function of retinal microvascular endothelial cells, resulting in microvascular leakage and occlusion. Many studies have shown that DR not only seriously threatens the visual system but may also induce cardiovascular disease. While lifestyle changes such as a healthy diet, moderate exercise, and weight management can prevent this damage to some extent, early diagnosis and continuous monitoring are crucial for preventing further vision impairment. Traditional DR diagnosis relies on ophthalmologists manually screening fundus images, which is inefficient, costly, time-consuming, and prone to misdiagnosis. With the development of computer and artificial intelligence (AI) technologies, computer-aided diagnostic (CAD) systems have greatly improved diagnostic efficiency and accuracy. In recent years, convolutional neural networks (CNNs) have made significant progress in the field of computer vision, becoming an important tool in this field. Benefiting from the powerful feature extraction capabilities of CNNs, they have been widely applied to DR classification tasks. However, despite the good results achieved by CNN-based DR classification methods, challenges remain in practical clinical applications, mainly due to the following reasons: CNN technology requires a large amount of labeled data; traditional datasets suffer from multi-source heterogeneity and class imbalance; and CNN technology suffers from imbalance between local and global features in feature extraction. Summary of the Invention
[0003] In view of this, the present invention aims to propose a novel hierarchical diagnostic model, RLGA-EfficientNetB0, and based on this, fuses the improved EfficientNetB0 with the MobiletNetV1 model. The improvement and fusion of the models aim to significantly enhance the accuracy and generalization ability of DR classification.
[0004] To achieve the above objectives, the technical solution of this invention is implemented as follows: a DR classification method based on the fusion of enhanced EfficientNetB0 and MobileNetV1. Further, image preprocessing and image augmentation are performed on the DR images. Further, based on transfer learning, the EfficientNetB0 structure is improved, and a new hierarchical diagnostic model RLGA-EfficientNetB0 is designed, including an LPM module, a GLAM module, and a 3×3 shortcut residual branch module. Further, RLGA-EfficientNetB0 is fused with the MobileNetV1 module, combining the compound scaling strategy of EfficientNetB0 and the depthwise separable convolution advantage of MobileNetV1 to improve the model's classification accuracy. Further, the training after module fusion includes: image preprocessing and image augmentation on the Kaggle open-source dataset, dividing the augmented images into training, validation, and test sets. The training set is input into the fused model for training, the validation set is used to evaluate the classification performance of the fused model, and the test set is used to demonstrate the classification effect of the fused model.
[0005] Furthermore, the image preprocessing and image augmentation operations of this fusion model include the following steps: S1, cropping the black redundant borders of the DR image; S2, using the Resize method to unify the DR image to a size of 224×224; S3, performing Gamma correction on the DR image; S4, using CLAHE equalization to enhance the details of the DR image; S5, using Laplacian sharpening filtering to make the edges and blood vessels of the DR image clearer; S6, randomly rotating the preprocessed DR image within the range of [-180°, 180°]; S7, randomly shifting the preprocessed DR image horizontally; S8, adding 100 blank pixels to the preprocessed DR image for padding.
[0006] Furthermore, the ratio of the training set, validation set, and test set is 7:2:1.
[0007] Furthermore, adjusting the training weights of the improved fusion model includes the following steps: T1, Iterative Training: Initialize the improved fusion model and repeat the training process multiple times to gradually optimize the model parameters. T2, Validation and Evaluation: Record the training results of each step in T1 and systematically validate the training results using a confusion matrix. Calculate the evaluation metrics of the improved fusion model based on the training results for each training session. T3, Weight Update: Based on the evaluation metrics, dynamically adjust the model's learning rate, weight decay, and other metrics after each training session.
[0008] Furthermore, the evaluation metrics in T2 include quadratic weighted Kappa value, accuracy (Acc), precision (Pre), recall (Rec), specificity (Spe), and loss value. T2 includes the following sub-steps: T21, Loss value: Calculate the cross-entropy loss function of the improved fusion model in this training: In the formula, M represents the number of categories, and y ic The sign function (0 or 1) is used; it takes the value 1 if the true class of sample i equals c, and 0 otherwise, where c represents the predicted probability that observed sample i belongs to class c; T22, double-weighted Kappa: In the formula, P0 represents the overall classification accuracy, which is the sum of the number of correctly classified samples in each class divided by the total number of samples. e This is the chance-match rate, calculated as the sum of the products of the actual and predicted quantities for each category, divided by the square of the total sample size. T23, Acc, Pre, Rec, Spe: In the formula, TP, TN, FN, and FP represent the number of true positive, true negative, false negative, and false positive diabetic retinopathy retina, respectively.
[0009] Furthermore, the weight update method in T3 is stochastic gradient descent (SGD), and the SGD update formulas are as follows: In the formula, θ represents the model parameters, t represents the current iteration number, η is the learning rate, and f i It is the loss function, which is applied to the i-th data point or data batch.
[0010] Furthermore, the Efficient NetB0 structure was improved by adding an RLGA module after the top-level feature map, which enhanced the model's feature extraction capability.
[0011] Furthermore, the main function of the RLGA block is to introduce two key components into the feature extraction process: the LPM module and the GLAM module. Specifically, LPM aims to extract image features more efficiently, while GLAM enhances the model's ability to understand complex features by expanding the receptive field to capture long-range dependencies of semantic information.
[0012] Furthermore, to ensure that network performance does not degrade with increasing network layers, skip connections from ResNet are added to the RLGA block. To prevent direct gradient transfer caused by skip connections, a 3×3 depthwise convolution is introduced in the residual shortcut branch. This design not only helps alleviate the vanishing gradient problem but also further improves the model's learning ability.
[0013] Furthermore, a transfer learning strategy, which initializes parameters using weights pre-trained on the ImageNet dataset, can quickly adapt to the DR classification task while significantly improving the accuracy of DR classification.
[0014] Furthermore, an ablation experiment method was proposed, which involved adding LPM modules, GLAM modules, and 3×3 shortcut branches to the EfficientNetB0 model and fusing it with MobileNetV1 to test the improvement effects of different improvement measures on the model.
[0015] Furthermore, a visualization strategy is proposed to enhance the interpretability of the model by utilizing the Grad-CAM visualization method, thereby better demonstrating the performance changes of the EfficientNetB0 model before and after the improvement.
[0016] Furthermore, a computer-readable storage medium includes a stored computer program, wherein the computer program, when executed by a processor, implements a DR classification method. This method utilizes a graphics processing unit (GPU) for acceleration to improve the performance of the computer in processing DR images, reduce DR classification time, and ensure that the program can be responded to quickly.
[0017] Furthermore, an integrated development environment (IDE) is provided. This system uses the PyCharm IDE, employs the Python programming language to build models, configures the CUDA environment to enable efficient GPU operation, and quickly responds to the processing needs of DR images, providing strong support for the early diagnosis of DR lesions.
[0018] Compared with existing technologies, the deep learning-based method for classifying diabetic retinopathy images described in this invention has the following beneficial effects: (1) The image preprocessing and image augmentation described in this invention solve the problems of multi-source heterogeneity and class imbalance in DR fundus images, effectively improving image quality and sample distribution. (2) The transfer learning framework of the EfficientNetB0 model described in this invention constructs a new feature extraction architecture, which introduces the LPM module and GLAM module. LPM is used to capture local lesion details, and GLAM is used to extract global language information, overcoming the problem of imbalance between local and global features in feature extraction based on traditional CNNs. (3) The optimized ResNet skip connection structure described in this invention avoids the direct transmission of information gradients, enhances the model's feature extraction capability, and thus improves the model's classification performance. The accompanying drawings, which constitute a part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention. In the accompanying drawings: Figure 1 is a preprocessed image of the deep learning model for grading diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 according to an embodiment of the present invention; Figure 2 is a feature extraction network diagram of the deep learning model for grading diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 according to an embodiment of the present invention; Figure 3 is an RLGA structure diagram of the deep learning model for grading diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 according to an embodiment of the present invention; Figure 4 is an LPM structure diagram of the deep learning model for grading diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 according to an embodiment of the present invention; Figure 5 is a GLAM structure diagram of the deep learning model for grading diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 according to an embodiment of the present invention; Figure 6 is a Grad-CAM visualization structure diagram of the deep learning model for grading diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 according to an embodiment of the present invention.
[0020] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the present invention will now be described in detail with reference to the accompanying drawings and embodiments.
[0021] Specifically, the preprocessed images are shown in Figure 1: (a) is the cropped image, (b) is the grayscale image, (c) is the gamma-corrected image, (d) is the CLAHE-equalized image, and (e) is the Laplacian sharpening filtered image. First, the DR images are preprocessed, including cropping the redundant black borders of the DR images. Then, the images are resized to a uniform size. Next, grayscale conversion, gamma correction, CLAHE equalization, and Laplacian sharpening filtering are used to make the lesion features of the DR images clearer. Then, image augmentation is performed, using various strategies to increase the number of samples. The augmented samples are then fed into the improved fusion model for training, specifically into the model's five-class classification. Finally, the model is evaluated using evaluation metrics.
[0022] Furthermore, the model feature structure diagram is shown in Figure 2. The preprocessed images are fed into the EfficientNetB0 model and the MobileNetV1 model after transfer learning, respectively. An RLGA module is added after the top feature map of EfficientNetB0, and then the improved EfficientNetB0 model and the MobileNetV1 model are concatted together.
[0023] Furthermore, the RLGA architecture, as shown in Figure 3, first passes through two LPM modules and then one GLAM module, improving the model's feature extraction capability. Simultaneously, the addition of improved skip connections, specifically 3×3 depthwise convolutions in the residual shortcut branch, helps alleviate the gradient vanishing problem and further enhances the model's learning ability.
[0024] Furthermore, the LPM structure diagram is shown in Figure 4. The LPM structure starts with a 1×1 convolution to reduce computational cost and burden. Then it employs parallel 3×3 and 1×1 depthwise convolutions, where the 1×1 depthwise convolution allows for more detailed extraction of local features. Finally, another 1×1 convolution is performed to restore the original number of channels. The ReLU activation function is introduced to further enhance the network's non-linear fitting capability.
[0025] Furthermore, the GLAM architecture diagram is shown in Figure 5. By increasing the spacing between convolutional kernels, a 5×5 depthwise convolution is used to further expand the receptive field. However, this global feature extraction method also has challenges. Specifically, these convolutional layers may capture irrelevant features, potentially degrading model performance. To address this issue, a CBAM attention mechanism is introduced after the depthwise convolution. CBAM extracts important features and suppresses irrelevant features through channel attention modules and spatial attention modules, respectively.
[0026] Furthermore, the visualization is shown in Figure 6. The first column shows two preprocessed lesion images, the second column shows the Grad-CAM images corresponding to the two preprocessed lesion images, and the third column shows the Grad-CAM images of the two model-enhanced lesion images.
[0027] Furthermore, the deep learning network is trained as follows:
[0028] We acquired a dataset of fundus images, performed data augmentation and image enhancement on the Kaggle dataset, and divided the dataset into training, validation, and test sets in a 7:2:1 ratio.
[0029] Specifically, the data augmentation operations include: cropping the redundant black borders, selecting the G channel with the strongest representational capability for grayscale processing, and using bilinear interpolation to uniformly adjust the image to 224×224 pixels. To address the uneven exposure caused by device differences and operational factors during image acquisition, a Gamma correction mechanism is introduced for brightness standardization to achieve optimal brightness equalization. To enhance image detail features, Contrast-Limited Adaptive Histogram Equalization (CLAHE) is used for image enhancement. CLAHE overcomes the over-enhancement problem of traditional HE by limiting contrast gain. Its core conversion formula is as follows: g(i,j)=[CDF(f(i,j)×(L-1))], where: g(i,j) is the grayscale value of the output image at position (i,j), CDF is the cumulative distribution function, f(i,j) is the grayscale value of the input image at position (i,j), and L is the number of grayscale levels (usually 256). The enhanced image is then subjected to Laplacian sharpening filtering to highlight edges and details, making lesion features such as blood vessels clearer.
[0030] Furthermore, EfficientNetB0 and MobileNetV1 were chosen as the models because EfficientNetB0 balances computational efficiency and accuracy based on a composite scaling method and is easy to extend later; MobileNetV1 achieves extreme lightweighting through depthwise separable convolutions, making it suitable for resource-constrained scenarios with high real-time requirements. CrossEntropy Loss was used as the loss function, and SGD optimizer was used as the optimizer for subsequent iterations and model training.
[0031] Those skilled in the art will understand that the units and method steps in the examples disclosed herein can be flexibly adjusted. In particular, the LPM module, GLAM module, and CBAM attention mechanism are interchangeable. Although this document has provided a general description of the composition and operation steps of each module from a functional perspective, the specific convolution method or attention mechanism used for replacement needs to be determined based on the application scenario and design constraints of the specific technical solution. Those skilled in the art can choose different methods to implement the corresponding functions according to specific needs, but such implementations should all be considered within the scope of this invention.
[0032] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments or equivalent substitutions can be made to some or all of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be covered within the scope of the claims and specification of the present invention.
[0033] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A novel intelligent grading method for diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 networks is proposed. After image preprocessing and image augmentation, a new fusion model is designed and implemented. By fusing the improved EfficientNetB0 and MobileNetV1 networks, multi-scale feature complementarity is achieved, significantly improving grading accuracy. Improvements to the EfficientNetB0 model include introducing an RLGA module after the top-level feature map to enhance feature extraction capabilities, and combining a Local Perception Module (LPM) and a Global Local Attention Module (GLAM) to refine local features and enhance global feature perception. Meanwhile, EfficientNetB0 transfer learning is used to improve classification performance, and a 3×3 depthwise convolution is added to the residual shortcut branch to avoid gradient propagation and strengthen the learned features, thereby further improving the evaluation metrics. Finally, the improved EfficientNetB0 is fused with MobileNetV1, combining the advantages of EfficientNetB0's compound scaling strategy and MobileNetV1's depthwise separable convolution to improve the model's classification accuracy. The specific steps are as follows: Step 1: Intelligent classification method for diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1. The training after module fusion includes: Image preprocessing and augmentation were performed on the Kaggle open-source dataset. The augmented images were divided into training, validation, and test sets. The training set was used to train the fused model, the validation set was used to evaluate the classification performance of the fused model, and the test set was used to demonstrate the classification effect of the fused model. Step 2: The image preprocessing and augmentation operations for the intelligent classification method for diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 are as follows: S1, crop the black redundant borders of the DR images; S2, use the resize method to unify the DR images to a size of 224×224; S3, perform Gamma correction on the DR images; S4. Enhance DR image details using CLAHE equalization; S5. Make DR image edges and blood vessels clearer using Laplacian sharpening filter; S6. Randomly rotate the preprocessed DR image within the range of [-180°, 180°]; S7. Randomly offset the preprocessed DR image horizontally; S8. Add 100 blank pixels to the preprocessed DR image for padding. Step 3: Based on the intelligent grading method for diabetic retinopathy fused with enhanced EfficientNetB0 and MobileNetV1, adjust the training weights of each sub-model as follows: T1. Iterative training: Initialize the improved fusion model and repeat the training process multiple times to gradually optimize the model parameters. T2. Validation and evaluation: Record the training results of each T1 step and use the confusion matrix to systematically validate the training results. Calculate the evaluation index of the improved fusion model in each training session based on the training results; T3. Update Weights: Based on the evaluation metrics, dynamically adjust the model's learning rate and decay weights after each training iteration. Step 4: Intelligent grading method for diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1. The evaluation metrics in T2 include quadratic weighted Kappa value, accuracy (Acc), precision (Pre), recall (Rec), specificity (Spe), and loss value. T2 is shown below: T21. Loss Value: Calculate the five-class cross-entropy loss function (CrossEntropyLoss) for each sub-model in this training iteration: In the formula, M represents the number of categories, and y ic The sign function (0 or 1) is set to 1 if the true class of sample i equals c, and 0 otherwise, c represents the predicted probability that observed sample i belongs to class c; T22, double-weighted Kappa: In the formula, P0 represents the overall classification accuracy, which is the sum of the number of correctly classified samples in each class divided by the total number of samples. e This is the chance-match rate, calculated as the sum of the products of the actual and predicted quantities for each category, divided by the square of the total sample size. T23, Acc, Pre, Rec, Spe: In the formula, TP, TN, FN, and FP represent the number of true positive, true negative, false negative, and false positive diabetic retinopathy cases, respectively. Step 5: The intelligent grading method for diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1, wherein the weight update method in T3 is stochastic gradient descent (SGD), and the SGD update formulas are as follows: In the formula, θ represents the model parameters, t represents the current iteration number, η is the learning rate, and f i It is the loss function, which is applied to the i-th data point or data batch.
2. The intelligent grading method for diabetic retinopathy based on the fusion of enhanced EfficientNetB0 and MobileNetV1 as described in claim 1, characterized in that: The EfficientNetB0 architecture is improved by adding an RLGA module after the top-level feature map, enhancing the model's feature extraction capability. The main function of the RLGA block is to introduce two key components during feature extraction: the LPM module and the GLAM module. Specifically, LPM aims to extract image features more efficiently, while GLAM captures long-range dependencies of semantic information by expanding the receptive field, thereby enhancing the model's ability to understand complex features. To ensure that network performance does not degrade with increasing network layers, ResNet skip connections are added to the RLGA block. To prevent direct gradient transfer caused by skip connections, a 3×3 depthwise convolution is introduced in the residual shortcut branch. This design not only helps alleviate the gradient vanishing problem but also further improves the model's learning ability. Using a transfer learning strategy, parameter initialization is performed using weights pre-trained on the ImageNet dataset, enabling rapid adaptation to the DR classification task while significantly improving DR classification accuracy. Ablation experiments are conducted to test the performance improvement of the EfficientNetB0 model by adding the LPM module, GLAM module, and 3×3 shortcut branch, respectively, and fusing MobileNetV1. The performance changes after the improved fusion are illustrated below. A visualization strategy is used to enhance the interpretability of the model by leveraging the Grad-CAM visualization method.
3. A computer-readable storage medium and an integrated development environment (IDE), wherein the storage medium stores a computer program that, when executed by a processor, implements a classification method for diabetic retinopathy. This method utilizes a graphics processing unit (GPU) for acceleration to improve the efficiency of model training and inference, ensuring rapid response when processing large amounts of fundus image data. The system uses the PyCharm IDE, employs the Python programming language for model building, and configures a CUDA environment to achieve efficient GPU operation, rapidly responding to the processing needs of DR images and providing strong support for the early diagnosis of DR lesions.