Diabetic retinopathy image classification system and method based on deep learning

Abstract

The invention belongs to the field of image data processing, and relates to a diabetic retinopathy image classification system and method based on deep learning, the system is obtained by improving a VMama network model, and a VSS module of the VMama network model is replaced by a VSS-SL module; the VSS-SL module is of a multi-branch structure and comprises a VSS module, an SE attention module and a Local local attention module, one path of input data is input into the VSS module, the other path of input data is input into the Local local attention module and the SE attention module in sequence, the VSS module comprises a first branch path and a second branch path which are parallel, and the second branch path is connected with the first branch path. The features output by the second branch path after standardization of the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module; the fused features are input into a linear layer, and the features output by the linear layer and the original input data are subjected to feature addition and then output. The system can classify the diabetic retinopathy pictures at a higher speed, and has a higher accuracy rate.

Description

Technical Field

[0001] The present invention belongs to the technical field of image data processing, and particularly relates to a diabetic retinopathy image classification system and method based on deep learning. Background Art

[0002] The incidence of visual impairment is high, which is often due to delayed referrals, highlighting the importance of timely screening and diagnosis. Most visual impairments and blindness can be avoided through early prevention.

[0003] Diabetic retinopathy (DR), as a complication of diabetes, is caused by pathological changes in the fundus microvasculature, which in turn leads to retinal hemorrhage, edema, ischemia, formation of retinal proliferative membranes, retinal detachment, etc., ultimately resulting in blindness of the patient. Fundus images provide a graphical record of the current ophthalmic morphology of an individual's retina. In diabetic fundus screening, early methods require professional ophthalmologists to physically analyze fundus images, manually segment data, and determine the presence of this disease and classify (grade) the degree of retinal lesions in patients based on vascular characteristics combined with other lesion sites, facilitating the adoption of corresponding measures to reduce the risk of blindness. However, this is a cumbersome and time-consuming process, and due to the tiny size of retinal blood vessels and lesion areas, misdiagnosis and missed diagnosis may occur during the diagnosis process; in addition, in many countries, the number of trained professionals may not be proportional to the number of patients, which may prevent timely treatment for many patients. Therefore, having a computer system that can assist in the diagnosis of DR is highly beneficial.

[0004] Currently, the research status of scholars around the world is that they are developing automated systems for DR diagnosis, mainly by identifying the presence of certain DR lesions in fundus images. DR lesions include microaneurysms, hard exudates, cotton wool spots, neovascularization, vitreous proliferation, macular edema, and even retinal detachment. The DR grade is also determined by fundus lesions. For example, the initial stage of DR is marked by the growth of new blood vessels, the weakening of the new blood vessel walls, and tiny protrusions. Later, when DR develops into proliferative DR, other lesions such as macular edema and vitreous proliferation may occur. Macular edema is a well-defined yellowish-white deposit, while cotton wool spots are pale yellow or white areas with unclear edges. Although optical coherence tomography (OCT) has become a reliable method for effective detection, due to the high cost of OCT equipment, in many developing countries, only large hospitals or specialized ophthalmology clinics in urban areas are equipped with such equipment, making it difficult to promote. Therefore, it is best to choose retinal images as input data and use deep learning techniques to achieve automatic classification of diabetic retinopathy images. The biggest advantage of this method compared to traditional machine learning-based methods is that it does not require the extraction of artificial features, and the deep model will autonomously learn the internal relationships between features, avoiding the influence of subjective human factors on the result determination.

[0005] Chinese Patent CN108172291 B discloses a "diabetic retinopathy recognition system based on fundus images", which uses deep learning methods to automatically determine fundus images and output judgment results, and can assist doctors in diagnosing fundus image lesions, greatly improving the recognition efficiency. However, this system highly depends on the quality and quantity of training data. If the training data is insufficient or there are biases, it may lead to poor generalization ability of the model. In addition, the accuracy of this system needs to be investigated. Summary of the Invention

[0006] In view of the above technical problems and defects, the purpose of the present invention is to provide a diabetic retinopathy image classification system based on deep learning. This system uses retinal images (fundus images) as input data, and based on the VMamba network model (visual state space model), replaces the VSS module of the VMamba network model with the VSS-SL module. The improved VMamba-m model can classify diabetic retinopathy pictures at a faster speed, and at the same time has a high accuracy, can meet the needs of real-time detection, and is applicable to low-computing-power platforms.

[0007] To achieve the above purpose, the present invention adopts the following technical solutions: A deep learning-based diabetic retinopathy image classification system, wherein the system adopts a trained VMamba-m model, wherein the VMamba-m model is obtained by improving the VMamba network model, and the VSS module of the VMamba network model is replaced by a VSS-SL module; The VSS-SL module is a multi-branch structure, including a VSS module, a SE attention module, and a Local attention module. The input data is input into the VSS module in one way, and the other way is input into the Local attention module and the SE attention module in turn. The VSS module includes an LN normalization layer and a parallel first branch path and a second branch path. The first branch path uses a linear layer and an activation function for feature conversion. The second branch path passes through a linear layer, a depth-separable convolution and an activation function in turn, and then enters the SS2D module for in-depth feature extraction. After passing through the SS2D module, the LN normalization layer is used to standardize the output features. The features standardized by the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module. The fused features are input into the linear layer, and the features output by the linear layer are output after feature addition with the original input data. The Local attention module includes an initial 2D convolution layer, which performs a batch normalization operation; followed by a 2D convolution layer, which uses a 3×3 convolution kernel, a step size set to 1, a padding of 1, and then a ReLU activation function; then a second convolution is performed, the second convolution uses a 2D convolution layer, which uses a 3×3 convolution kernel, a step size set to 1, a padding of 1, and again passes through batch normalization and ReLU activation function; then a 2D convolution layer is input, which uses a 1×1 convolution kernel, a step size of 1, followed by a ReLU function to continue to maintain nonlinearity, gradually extract high-order features, and finally output.

[0008] As a preferred embodiment of the present invention, the data processing flow of the SE attention module includes compression, excitation and Scale weighting; the Local attention module outputs features Figure X ,feature Figure X First, the feature map U is generated through the Transformer operation, and then the input feature map U is compressed into a vector through the global average pooling operation; the compressed vector is processed by the fully connected network to obtain the weights of different channels; finally, the Scale weighting operation combines the weights obtained by the excitation operation with the original feature map Multiply channel by channel to get the weighted feature map .

[0009] As a preferred embodiment of the present invention, the expression of the Local attention module is: ; Among them, is the output feature, and are two positions in the input feature map, is a function for calculating the correlation between two positions, is a function for transforming the features at position ; is the mathematical symbol for all , that is, traversing all relevant ; is the normalization factor.

[0010] The present invention also provides a deep learning-based diabetic retinopathy image classification method, which includes the following steps: Step 1. Construct the above-mentioned VMamba-m model; Step 2. Train the VMamba-m model to obtain a trained VMamba-m model; Step 3. Input the retinal image (fundus image) data into the trained VMamba-m model for classification.

[0011] As a preference of the present invention, the process of training the VMamba-m model is as follows: Step a. Construct a data set, download the publicly available diabetic retinopathy classification data set Aptos2019, preprocess the images to obtain a data set for training the model; then divide the preprocessed data set into a training set and a test set according to a ratio; Step b. Configure the training parameters, and use a loss function during training to solve the problem of extreme imbalance in the number of positive and negative samples; Step c. Input the constructed training set into the constructed VMamba-m model for testing and training to obtain the optimal diabetic retinopathy image classification model after training; Step d. After training, put the test set into the obtained optimal diabetic retinopathy image classification model for detection to obtain the classification level of the fundus images of diabetic retinopathy to be detected, and finally obtain the recognition result.

[0012] As a preference of the present invention, the method for preprocessing the images in Step a includes: performing size normalization processing on the images and performing data augmentation processing on the images; wherein, the data augmentation processing includes random rotation, random cropping, random flipping, brightness adjustment, and contrast adjustment.

[0013] As a preference of the present invention, the loss function used during training is the Focal loss function.

[0014] Advantages and beneficial effects of the present invention: (1) The VSS module in the original VMamba model can only extract global features. After adding local attention and channel attention to the VSS module in the present invention, different positions and weights can be assigned to the image from the perspective of the channel domain through the weight matrix, so as to obtain more important feature information. The purpose of adding dual-branch attention is to extract local features, and at the same time, it has a faster training speed and higher accuracy than the previous single-branch attention.

[0015] (2) In the training process of the present invention, the Focal loss function is used to adjust the sample weights. This method enhances the model's attention to rare categories, solves the extreme imbalance of the number of positive and negative samples, makes the model pay more attention to difficult-to-classify samples, effectively improves the accuracy and automation level of diabetic retinopathy recognition, and thus improves the classification performance.

[0016] (3) For the VMamba-m model proposed in the present invention, the loss function is the Focal loss function, and the SE attention mechanism and the self-designed Local local attention mechanism are introduced. The accuracy is significantly improved (the accuracy is as high as 79.1%, and the AUC value is 89.1%). Moreover, the time for each round of iteration is only 34.027 s. Different from traditional classification models, the model proposed in the present invention has a faster calculation speed and higher accuracy, and solves the problem of high computational complexity when traditional models process long sequences.

[0017] (4) The VMamba-m proposed in the present invention can classify diabetic retinopathy pictures at a faster speed, and at the same time has a higher accuracy, can meet the needs of real-time detection, and is applicable to low-computing power platforms, such as adapting to real-time detection tasks in low-computing power scenarios in areas with backward medical conditions. Brief Description of the Drawings

[0018] Through the following description with reference to the drawings, and with a more comprehensive understanding of the present invention, other objects and results of the present invention will become more obvious and easier to understand. In the drawings: Figure 1 It is a schematic diagram of the overall structure of the VMamba-m model constructed in the present invention; Figure 2 It is a schematic diagram of the VSS-SL module in the present invention; Figure 3 It is a schematic diagram of the Local local attention module in the present invention; Figure 4 It is a schematic diagram of the SE attention module in the present invention; Figure 5 It is a schematic diagram of a representative sample of the Aptos2019 dataset; Figure 6Schematic diagram of representative samples after preprocessing of the Aptos 2019 dataset; Figure 7 For the first training result (loss per round) on the Aptos 2019 dataset; Figure 8 For the second training result (accuracy per round) on the Aptos 2019 dataset; Figure 9 For the ROC training result on the Aptos 2019 dataset; Figure 10 For the confusion matrix on the Aptos 2019 dataset. Detailed implementation manner

[0019] To enable those skilled in the art to better understand the technical solutions and advantages of the present invention, the present application will be described in detail below with reference to the accompanying drawings, but it is not intended to limit the protection scope of the present invention.

[0020] Example 1:

[0021] This example provides a diabetic retinopathy image classification system based on deep learning. Next, with reference to the accompanying drawings, the technical solutions of the present invention will be described in detail.

[0022] As shown in Figure 1 、 Figure 2 The diabetic retinopathy image classification system based on deep learning provided in this example uses a trained VMamba-m model. The VMamba-m model is obtained by improving the VMamba network model. The VSS module of the VMamba network model is replaced with a VSS-SL module, changing from the original global feature extraction method to a local feature extraction method to enhance the network's ability to extract image edge information and spatial information, thereby improving the detection accuracy; Among them, the VSS-SL module is an efficient front-end module in image recognition tasks, aiming to improve the model's ability to parse images by combining visual information and semantic information. Especially in complex scene understanding tasks, this module explicitly models the dependencies between channels through the SE attention module and the Local local attention module, dynamically adjusting the weights of each channel in the feature map, thereby enhancing the model's representation ability; The VSS-SL module has a multi-branch structure, that is, a double-branch path is added on the basis of the VSS module to further extract features and provide global modeling ability with linear complexity. Specifically, the VSS-SL module includes a VSS module, an SE attention module, and a Local local attention module. The input data (feature map) is input into the VSS module in one way, and into the Local local attention module and the SE attention module in turn in the other way. Among them, the VSS module includes an LN normalization layer and parallel first and second branch paths. The first branch path uses a linear layer and an activation function for feature transformation. The second branch path passes through a linear layer, a depthwise separable convolution, and an activation function in turn, and then enters the SS2D module for in-depth feature extraction. After passing through the SS2D module, an LN normalization layer is used to standardize the output features. The features standardized by the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module. The fused features are input into a linear layer, and the features output by the linear layer are added to the original input data for feature addition and then output.

[0023] Furthermore, in this embodiment, the purpose of setting the Local local attention module is to capture the global dependence relationship between all positions in the input feature map. Different from traditional convolution operations, the Local local attention module can capture long-range dependence relationships by calculating the correlation between any two positions, thereby enhancing the global modeling ability of the model. The expression is: ; Among them, is the output feature, usually representing the new representation or attention aggregation result of node , and are two positions in the input feature map (which can be spatial positions, temporal positions, or spatio-temporal positions), is a function for calculating the correlation between two positions, is a function for transforming the features of position , similar to the Value mapping; is the mathematical symbol for all , that is, traversing all relevant ; is the normalization factor, usually the sum of all correlations.

[0024] In this embodiment, a function is used to calculate the correlation between two positions, and a standardization operation is performed on the mini-batch input data of each layer to reduce the internal covariate shift, accelerate training, and alleviate the problem of gradient disappearance or explosion, improving the training speed and stability. Common correlation functions include the Gaussian function: ; dot product: ; Embedded Gaussian function: , where and are embedding functions.

[0025] Specifically, as Figure 3 shown, in this embodiment, the Local local attention module includes an initial 2D convolutional layer, and batch normalization is performed through the initial 2D convolutional layer; immediately followed by a 2D convolutional layer, which uses a 3×3 convolutional kernel, the stride is set to 1, and the padding is 1. The purpose is to use the 3×3 convolutional kernel to extract spatial features. 3×3 is a small-size convolutional kernel that can capture local details and has fewer parameters; then through the ReLU activation function, the ReLU activation function is a non-linear activation function, which enhances the expression ability of the model and at the same time alleviates the problem of gradient disappearance; then a second convolution is performed. The second convolution uses a 2D convolutional layer. The 2D convolutional layer of the second convolution is the same as that of the first convolution, using a 3×3 convolutional kernel, the stride is set to 1, and the padding is 1, which is used for feature extraction to further extract higher-order shape and complex texture features. Again, through batch normalization (batch normalization is performed through the initial 2D convolutional layer) and the ReLU activation function, the batch normalization operation is used to standardize the output of the convolutional layer to ensure the stability and non-linearity of the feature distribution; then input into a 2D convolutional layer, which uses a 1×1 convolutional kernel, the stride is 1. The introduction of the 1×1 convolutional layer can reduce the computational amount, which is used for cross-channel feature fusion or adjusting the channel dimension, and is followed by the ReLU function to continue to maintain non-linearity, gradually extract high-order features, and finally output.

[0026] Furthermore, as Figure 4 shown, in this embodiment, the data processing flow of the SE attention module includes compression (Squeeze), excitation (Excitation), and Scale weighting; among them, the purpose of compression is to compress the global spatial information into a channel descriptor; specifically, a global average pooling (Global AveragePooling, GAP) operation is performed on the input feature map of each channel to compress the two-dimensional feature map of each channel into a scalar value.

[0027] In this embodiment, the Local local attention module outputs a feature Figure X , whose shape is H’×W’×C’ (representing height, width, and number of channels respectively), and this feature Figure X goes through the Transformer operation (F tr)Generate the feature map U; in the compression step, the feature U aggregates the feature maps across the spatial dimensions H×W through a compression operation to generate channel descriptors; H×W×C → 1×1×C, compressing the global spatial information into a single channel descriptor so that these channel descriptors can be utilized by their input layers; the input feature map is compressed into a vector through global average pooling operation and then mapped to a smaller vector through a fully connected layer, as shown in the following formula: ; where, is the result of global average pooling for the th channel, compressing the spatial information into a scalar value, is the th channel of the input feature map, with a size of , is the height, is the width, represents the FeatureSqueeze operation; The purpose of excitation is to learn the importance weights of each channel. It learns the dependencies between channels through a simple fully connected network (usually consisting of two fully connected layers) and outputs the weights for each channel. In the excitation step, each element in this vector is compressed to a value between 0 and 1 using a sigmoid function and then multiplied by the original input feature map to obtain a weighted feature map; the vector z obtained in the previous step is processed through two fully connected layers to obtain the desired channel weight value s; after passing through two fully connected layers, different values in s represent the weight information of different channels, assigning different weights to the channels, as shown in the following formula: ; where, is the vector output by the compression operation, is the weight matrix of the first fully connected layer for dimensionality reduction; is the weight matrix of the second fully connected layer for dimensionality restoration; is the ReLU activation function introducing non-linearity; is the Sigmoid activation function mapping the output to the [0,1] interval, representing the importance weights of each channel , represents the excitation operation, represents outputting an unnormalized weight vector, which is subsequently compressed to the [0,1] interval through the Sigmoid activation function.

[0028] The purpose of Scale weighting is to apply the learned weights to the original feature map, multiplying the weight vector with the original feature map Multiply channel by channel to obtain the weighted feature map Specifically, the weights of the excitation output are added to the original input features through channel convolution; the SE attention module can make the model more focused on key information through its ingenious attention mechanism, thus effectively promoting the application of deep learning in many fields. The formula is as follows: ; Among them, is the weight of the c th channel, is the feature map of the original c th channel input, is the feature map of the c th channel after weighting, and the feature importance is adjusted through channel-by-channel multiplication.

[0029] Furthermore, in this embodiment, the shape of the feature map U is H×W×C. The feature map U is compressed (F sq ) through global average pooling operation to generate a vector, and each channel is represented by a numerical value; then, the excitation operation (F ex ) is implemented through two fully connected layers to generate weight information for different channels; finally, through the Scale weighting operation (F scale ), the weight vector is used to assign weights to the original feature map to obtain the required feature map , whose size is exactly the same as that of the feature map .

[0030] Embodiment 2: This embodiment provides a method for classifying diabetic retinopathy images based on deep learning, which includes the following steps: Step 1. Construct the VMamba-m model described in Embodiment 1; Step 2. Train the VMamba-m model to obtain a trained VMamba-m model; Step 3. Input the retinal image data into the trained VMamba-m model for classification.

[0031] In this embodiment, the process of training the VMamba-m model is as follows: Step a. Construct a dataset, download the publicly available diabetic retinopathy classification dataset Aptos2019, preprocess the images, and the images before and after preprocessing are as shown in Figure 5 and 6 . Obtain the dataset for training the model through preprocessing; then divide the preprocessed dataset into a training set and a test set according to a certain proportion; Step b. Configure the training parameters and use a loss function during the training process to address the extreme imbalance in the number of positive and negative samples. Step c. Input the constructed training set into the constructed VMamba-m model for testing and training to obtain the optimal diabetic retinopathy image classification model after training. Step d. After the training is completed, put the test set into the obtained optimal model for detection to obtain the classification level of the fundus images of diabetic retinopathy to be detected, and finally obtain the recognition result.

[0032] Furthermore, in this embodiment, the method for preprocessing the image in step a includes: performing size normalization processing on the image and performing data augmentation processing on the image; wherein, the data augmentation processing includes random rotation, random cropping, random flipping, brightness adjustment, and contrast adjustment; finally, divide the test set and the training set according to a ratio of 2:8, where the image specification is 1050*1050 pixels.

[0033] Furthermore, in this embodiment, the loss function used during the training process is the Focal loss function. The Focal Loss function is an effective method for dealing with class imbalance problems. By introducing a focal factor and adjusting the sample weights, the model pays more attention to difficult-to-classify samples, thereby improving the classification performance. This function is particularly suitable for class imbalance tasks such as object detection.

[0034] The formula of the Focal loss function is: ; where is the predicted probability of the model for the target class; is the balance factor, and its value range is (0, +∞), which is used to adjust the influence between positive and negative samples; γ is the focal factor, and its value range is [0, +∞), which is used to adjust the weight of difficult-to-classify samples; the modulation factor is used to reduce the loss rate of easily distinguishable samples. Whether it is the foreground class or the background class, the larger it is, the easier it is to distinguish the sample, and the smaller the modulation factor is.

[0035] To verify the effectiveness of the improved module in the present invention, the VMamba-m model is selected to verify the comparison of training loss and accuracy on the Aptos2019 dataset. Through Figure 7 and Figure 8 it can be seen that the accuracy of training and verification gradually increases with the number of iterations, and the loss rate of the verification data gradually decreases with the number of iterations.

[0036] In the selection of the convolutional model, the purpose of the present invention is to lighten the weight on the basis of ensuring a certain detection efficiency to adapt to the real-time detection tasks in low-computing power scenarios such as underdeveloped medical regions.

[0037] To this end, on the premise of data preprocessing, after selecting the model, the present invention conducts model training, testing and ablation experiments on the data set corresponding to the task of the present invention. The final test results are shown in Tables 1 and 2.

[0038] After multiple rounds of experiments, the experimental results of each model on the Aptos2019 data set are shown in Table 1. From the data in the table, it can be seen that the Mamba series of models perform well in the diabetic retinopathy image classification task. In the five-classification task, the VMamba-m model proposed by the present invention performs the best, followed by VMamba, with classification accuracies of 0.791 and 0.714 respectively, and the number of iterations per round being 34.027 and 76.322 respectively, both of which are better than other models.

[0039] The ROC curve diagram of the VMamba-m model of the present invention is shown in Figure 9. The ROC curve and AUC value represent the closeness of the prediction to the perfect classification. The perfect classification is shown in the upper left corner of the ROC coordinates; the AUC value shows the area under the ROC curve; the closer the value is to 1, the better the model performance; the figure shows that the AUC values are 99.1%, 85.5%, 87.50%, 87% and 86.8% respectively, representing the categories of no DR, mild, moderate, severe and proliferative DR; due to the morphological changes in the fundus images of moderate DR affecting the recognition of pathological structures, the AUC of moderate DR is lower than that of other DRs, at 85.5%.

[0040] Figure 10 The confusion matrix is described, which shows how to evaluate the performance of each category in the model. The diagonal cells show the proportion of examples correctly classified for each category. In the model designed by the present invention, 354 images were correctly predicted as category 0 (healthy); 43 images were predicted as category 1 (mild); 126 images were predicted as category 2 (moderate); 20 images were predicted as category 3 (severe); 34 images were predicted as category 4 (proliferative).

[0041] In addition, through ablation experiments, the impact of these mechanisms on the model's performance can be determined to judge whether the above problems have been effectively alleviated. The ablation experiments using three different mechanisms in the original modeling task are shown in Table 2. Taking the VMamba architecture as the baseline, after introducing the SE attention module, the accuracy has improved, and the time per iteration has been reduced by 52.7%; after introducing the Local local attention module, the accuracy has improved, and the time per iteration has been reduced by 52.4%; when using the focal function mechanism, the accuracy has improved, and the calculation time has basically remained unchanged.

[0042] After replacing the loss function in the training of the present invention, the ablation experiments of different mechanisms are shown in Table 2. Based on the VMamba architecture, after introducing the SE attention module, the accuracy has increased, and the time per iteration has been reduced by 54.9%; after introducing the Local local attention module, the accuracy has increased, and the time per iteration has been reduced by 53.7%. For the VMamba-m model proposed in the present invention, the loss function is replaced by the Focal loss function and the SE attention mechanism and the Local local attention mechanism are introduced, and the accuracy has been significantly improved, and the time per iteration has been reduced by 55.4%.

[0043] In this embodiment, the effectiveness of each proposed module is verified through the verification model and ablation experiments; hereinafter, the advantages of the model designed in the present invention are demonstrated by comparing various indicators with classical algorithms and advanced algorithms. By comparing the transfer learning strategy and the mixed dataset training strategy, it is verified that the mixed dataset training strategy has better effects in the case of scarce samples.

[0044] Table 1 shows the classification results of different models Model Accuracy Recall Precision AUC F1 Time InceptionV3 0.701 0.684 0.618 0.813 0.650 106.345 VGG16 0.686 0.795 0.653 0.805 0.701 108.012 ResNet50 0.698 0.542 0.701 0.847 0.660 107.453 ResNet34 0.688 0.540 0.700 0.839 0.681 108.753 VGGNet 0.702 0.550 0.655 0.832 0.651 105.424 DensNet121 0.684 0.574 0.640 0.814 0.606 111.652 ResNet108 0.675 0.702 0.690 0.821 0.686 107.379 NASNetMobile 0.795 0.648 0.712 0.871 0.732 152.613 MobileNet 0.769 0.655 0.722 0.874 0.751 125.410 MobileNetV2 0.759 0.647 0.724 0.863 0.756 130.031 MobileNetV3Small 0.692 0.643 0.694 0.812 0.701 127.721 MobileNetV3Large 0.665 0.613 0.684 0.846 0.717 129.285 VMamba 0.714 0.500 0.718 0.881 0.500 76.322 Mamba 0.703 0.512 0.706 0.850 0.501 77.034 VMamba-m 0.791 0.537 0.766 0.891 0.527 34.027 Table 2 shows the results of the ablation experiments Model Accuracy Recall Precision AUC F1 Time VMamba 0.714 0.500 0.718 0.871 0.500 76.322 VMamba+Focal 0.722 0.504 0.724 0.874 0.510 75.122 VMamba+SE 0.742 0.507 0.723 0.878 0.501 36.081 VMamba+Local 0.746 0.502 0.734 0.879 0.500 36.255 VMamba+Focall+SE 0.771 0.517 0.733 0.881 0.506 34.391 VMamba+Focall+Local 0.766 0.514 0.742 0.886 0.515 35.295 VMamba-m 0.791 0.537 0.766 0.891 0.527 34.027 From the above data, it can be seen that in the diabetic retinopathy image classification task, the VMamba-m model proposed in the present invention improves the VSS module of the original VMamba model, adds an attention mechanism, enhances the overall expressiveness of the model, improves the details of the implementation process and the architecture design, and improves the model performance and inference speed; flexibly uses the FocalLoss function, enables the model to enhance the attention to rare categories, and reduces the time complexity for different samples in the classification task. The VMamba-m model was evaluated on the Aptos2019 dataset released on the Kaggle platform. In the five-classification task of the lesion grading degree, the accuracy of the model proposed in the present invention is as high as 79.1%, the AUC value is 89.1%, and the iteration time per round is only 34.027s, which is one-third to one-fourth of the iteration time per round of most traditional CNN models. Through comparative experiments and ablation experiments, it is proved that the model proposed in the present invention performs well in the five-classification task of diabetic retinopathy.

[0045] The present invention also provides an electronic device, including: one or more processors and a memory; wherein, the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned method for classifying diabetic retinopathy images based on deep learning.

[0046] The present invention also provides a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned method for classifying diabetic retinopathy images based on deep learning is implemented.

[0047] Those skilled in the art can understand that all or part of the functions of the above-mentioned various methods / modules can be implemented in a hardware manner or in a computer program manner. When all or part of the functions in the above-mentioned embodiments are implemented in a computer program manner, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc., and the above functions are implemented by the computer executing the program. For example, the program is stored in the memory of the device, and when the program in the memory is executed by the processor, all or part of the above functions can be realized.

[0048] In addition, when all or part of the functions in the above-mentioned embodiments are implemented in a computer program manner, the program can also be stored in a storage medium such as a server, another computer, magnetic disk, optical disk, flash drive or mobile hard disk, downloaded or copied and saved to the memory of the local device, or the system of the local device is updated, and when the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be realized.

[0049] The above uses specific examples to illustrate the present invention, which is only used to help understand the present invention and is not intended to limit the present invention. For those skilled in the art to which the present invention pertains, based on the idea of the present invention, several simple deductions, deformations or substitutions can also be made. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A diabetic retinopathy image classification system based on deep learning, characterized in that, The system adopts a trained VMamba-m model, which is obtained by improving the VMamba network model, replacing the VSS module of the VMamba network model with a VSS-SL module; The VSS-SL module is a multi-branch structure, including a VSS module, a SE attention module, and a Local attention module. The input data is input into the VSS module in one way, and the other way is input into the Local attention module and the SE attention module in turn. The VSS module includes an LN normalization layer and a parallel first branch path and a second branch path. The first branch path uses a linear layer and an activation function for feature conversion. The second branch path passes through a linear layer, a depth-separable convolution and an activation function in turn, and then enters the SS2D module for in-depth feature extraction. After passing through the SS2D module, the LN normalization layer is used to standardize the output features. The features standardized by the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module. The fused features are input into the linear layer, and the features output by the linear layer are output after feature addition with the original input data. The Local attention module includes an initial 2D convolution layer, which performs a batch normalization operation; followed by a 2D convolution layer, which uses a 3×3 convolution kernel, a step size set to 1, a padding of 1, and then a ReLU activation function; then a second convolution is performed, the second convolution uses a 2D convolution layer, which uses a 3×3 convolution kernel, a step size set to 1, a padding of 1, and again passes through batch normalization and ReLU activation function; then a 2D convolution layer is input, which uses a 1×1 convolution kernel, a step size of 1, followed by a ReLU function to continue to maintain nonlinearity, gradually extract high-order features, and finally output.

2. The diabetic retinopathy image classification system based on deep learning according to claim 1, wherein, The data processing flow of the SE attention module includes compression, excitation, and Scale weighting; the Local local attention module outputs the feature map X, the feature map X first undergoes a Transformer operation to generate the feature map U, and then the input feature map U is compressed into a vector through a global average pooling operation; the vector obtained through compression is processed by a fully connected network to obtain the weights of different channels; finally, the Scale weighting operation multiplies the weights obtained through the excitation operation with the original feature map channel by channel to obtain the weighted feature map .

3. The diabetic retinopathy image classification system based on deep learning according to claim 1, characterized in that, The expression of the Local attention module is: ； Among them, is the output feature, and are two positions in the input feature map, is a function for calculating the correlation between two positions, is a function for transforming the features at position ; is the mathematical symbol for all , that is, traverse all relevant ; is the normalization factor.

4. A method for classifying diabetic retinopathy images based on deep learning, characterized in that, The method comprises the following steps: Step 1. Construct the VMamba-m model as described in any one of claims 1 to 3; Step 2. Train the VMamba-m model to obtain a trained VMamba-m model; Step 3. Input the retinal image data into the trained VMamba-m model for classification.

5. A method for classifying diabetic retinopathy images based on deep learning according to claim 4, characterized in that, The process of training the VMamba-m model is as follows: Step a. Build a dataset, download the public diabetic retinopathy classification dataset Aptos2019, preprocess the images to obtain a dataset for training the model; then divide the preprocessed dataset into a training set and a test set in proportion; Step b. Configure the training parameters and use the loss function during the training process; Step c. Inputting the constructed training set into the constructed VMamba-m model for testing and training to obtain the optimal diabetic retinopathy image classification model after training; Step d. After the training is completed, the test set is placed in the obtained optimal diabetic retinopathy image classification model for detection, the classification level of the diabetic retinopathy fundus image to be detected is obtained, and finally the recognition result is obtained.

6. The method for classifying diabetic retinopathy images based on deep learning according to claim 5, characterized in that, The method for preprocessing the image in step a includes: performing size normalization processing on the image and performing data augmentation processing on the image; wherein, the data augmentation processing includes random rotation, random cropping, random flipping, brightness adjustment, and contrast adjustment.

7. A method for classifying diabetic retinopathy images based on deep learning according to claim 5, characterized in that, The loss function used during the training process is the Focal loss function.

Images