Diabetic retinopathy image classification system and method based on deep learning

Abstract

The present invention belongs to the field of image data processing and relates to a deep learning-based diabetic retinopathy image classification system and method. The system is obtained by improving the VMamba network model by replacing the VSS module of the VMamba network model with a VSS-SL module. The VSS-SL module has a multi-branch structure, including a VSS module, an SE attention module, and a local attention module. Input data is input to the VSS module on one path and to the local attention module and the SE attention module in sequence on another path. The VSS module includes a parallel first branch path and a second branch path. The features output by the second branch path after normalization by the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module. The fused features are input to a linear layer, and the features output by the linear layer are added to the original input data and then output. The system can classify diabetic retinopathy images at a faster speed and with higher accuracy.

Description

Technical Field

[0001] The present invention belongs to the technical field of image data processing, and specifically relates to a diabetic retinopathy image classification system and method based on deep learning. Background Art

[0002] The high incidence of visual impairment, often due to delayed referral, highlights the importance of timely screening and diagnosis. Most visual impairment and blindness can be avoided through early prevention.

[0003] Diabetic retinopathy (DR), a complication of diabetes, is caused by microvascular changes in the fundus, leading to retinal hemorrhage, edema, ischemia, retinal membrane formation, retinal detachment, and ultimately blindness. Fundus images provide a detailed graphical record of the current ophthalmic morphology of an individual's retina. Early methods for diabetic fundus screening required specialized ophthalmologists to physically analyze fundus images, manually segment the data, and determine the presence of the disease and classify (grade) the patient's retinopathy based on vascular characteristics and other lesion locations, facilitating appropriate measures to reduce the risk of blindness. However, this process is tedious and time-consuming, and due to the small size of retinal vessels and lesions, misdiagnosis and missed diagnoses may occur during the diagnostic process. Furthermore, in many countries, the number of trained professionals may be disproportionate to the number of patients, which can hinder timely treatment for many patients. Therefore, having a computer system that can assist in the diagnosis of DR would be greatly beneficial.

[0004] Currently, researchers around the world are developing automated systems for DR diagnosis, primarily by identifying the presence of certain DR lesions in fundus images. DR lesions include microaneurysms, hard exudates, cotton-wool spots, neovascularization, vitreous proliferation, macular edema, and even retinal detachment. The DR grade is also determined based on fundus lesions. For example, the early stages of DR are marked by the onset of new vessel growth, with weakened walls and tiny protrusions. Later, when DR progresses to proliferative DR, other lesions such as macular edema and vitreous proliferation may appear. Macular edema is a well-defined yellowish-white deposit, while cotton-wool spots are light yellow or white areas with unclear edges. Although optical coherence tomography (OCT) has become a reliable and effective detection method, due to the high cost of OCT equipment, in many developing countries, only large hospitals or specialized ophthalmology clinics in urban areas are equipped with such equipment, making it difficult to promote. Therefore, it is best to use retinal images as input data and use deep learning technology to automatically classify diabetic retinopathy images. The biggest advantage of this method compared to traditional machine learning-based methods is that it does not require the extraction of artificial features. The deep model will autonomously learn the intrinsic connections between features, avoiding the influence of human subjective factors on the result judgment.

[0005] Chinese patent CN108172291 B discloses a "diabetic retinopathy identification system based on fundus images." This system utilizes deep learning methods to automatically assess fundus images and output the results. This system can assist doctors in diagnosing lesions from fundus images, significantly improving recognition efficiency. However, this system is highly dependent on the quality and quantity of training data. Insufficient or biased training data can lead to poor generalization of the model. Furthermore, the system's recognition accuracy remains to be determined. Summary of the Invention

[0006] In view of the above-mentioned technical problems and defects, the purpose of the present invention is to provide a diabetic retinopathy image classification system based on deep learning. The system uses retinal images (fundus images) as input data and is based on the VMamba network model (visual state space model). The VSS module of the VMamba network model is replaced by the VSS-SL module. The improved VMamba-m model can classify diabetic retinopathy images at a faster speed while having a higher accuracy rate, which can meet the needs of real-time detection and is suitable for low-computing power platforms.

[0007] To achieve the above object, the present invention adopts the following technical solutions:

[0008] A deep learning-based diabetic retinopathy image classification system uses a trained VMamba-m model, which is obtained by improving the VMamba network model by replacing the VSS module of the VMamba network model with the VSS-SL module;

[0009] Among them, the VSS-SL module is a multi-branch structure, including a VSS module, an SE attention module, and a Local attention module. The input data is input into the VSS module in one path, and the other path is input into the Local attention module and the SE attention module in sequence; wherein, the VSS module includes an LN normalization layer and a parallel first branch path and a second branch path. The first branch path uses a linear layer and an activation function for feature conversion, and the second branch path passes through a linear layer, a depth-separable convolution and an activation function in sequence, and then enters the SS2D module for in-depth feature extraction; after passing through the SS2D module, an LN normalization layer is used to standardize the output features, and the features standardized by the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module; the fused features are input into the linear layer, and the features output by the linear layer are output after feature addition with the original input data;

[0010] The Local attention module includes an initial 2D convolution layer, which performs a batch normalization operation; followed by a 2D convolution layer, which uses a 3×3 convolution kernel, a step size of 1, and a padding of 1, and then passes through a ReLU activation function; then a second convolution is performed, and the second convolution uses a 2D convolution layer, which uses a 3×3 convolution kernel, a step size of 1, and a padding of 1, and again passes through batch normalization and a ReLU activation function; then a 2D convolution layer is input, which uses a 1×1 convolution kernel and a step size of 1, followed by a ReLU function to continue to maintain nonlinearity, gradually extract high-order features, and finally output.

[0011] As a preferred embodiment of the present invention, the data processing flow of the SE attention module includes compression, excitation and Scale weighting; the local attention module outputs features Figure X ,feature Figure X First, the feature map U is generated by the Transformer operation, and then the input feature map U is compressed into a vector through the global average pooling operation; the compressed vector is processed by the fully connected network to obtain the weights of different channels; finally, the Scale weighting operation combines the weights obtained by the excitation operation with the original feature map Multiply channel by channel to get the weighted feature map .

[0012] As a preferred embodiment of the present invention, the expression of the Local attention module is:

[0013] ;

[0014] in, is the output feature, and are two locations in the input feature map, is a function that calculates the correlation between two positions, It's the position The function that transforms the characteristics of For all The mathematical symbol of ; is the normalization factor.

[0015] The present invention also provides a deep learning-based diabetic retinopathy image classification method, which includes the following steps:

[0016] Step 1. Construct the VMamba-m model described above;

[0017] Step 2. Train the VMamba-m model to obtain a trained VMamba-m model;

[0018] Step 3. Input the retinal image (fundus image) data into the trained VMamba-m model for classification.

[0019] As a preferred embodiment of the present invention, the process of training the VMamba-m model is as follows:

[0020] Step a. Build a dataset. Download the publicly available diabetic retinopathy classification dataset, Aptos2019, and preprocess the images to obtain a dataset for training the model. Then, divide the preprocessed dataset into a training set and a test set in proportion.

[0021] Step b. Configure the training parameters and use the loss function during training to solve the problem of extreme imbalance in the number of positive and negative samples;

[0022] Step c. Inputting the constructed training set into the constructed VMamba-m model for testing and training to obtain the optimal diabetic retinopathy image classification model after training;

[0023] Step d. After the training is completed, the test set is placed in the obtained optimal diabetic retinopathy image classification model for detection, the classification level of the diabetic retinopathy fundus image to be detected is obtained, and finally the recognition result is obtained.

[0024] As a preferred embodiment of the present invention, the method of preprocessing the image in step a includes: performing size normalization processing on the image and performing data enhancement processing on the image; wherein the data enhancement processing includes random rotation, random cropping, random flipping, brightness adjustment and contrast adjustment.

[0025] As a preference of the present invention, the loss function used in the training process is a Focal loss function.

[0026] Advantages and beneficial effects of the present invention:

[0027] (1) The VSS module in the original VMamba model can only extract global features. After adding local attention and channel attention to the VSS module, the present invention can assign different positions and weights to the image from the perspective of the channel domain through the weight matrix, thereby obtaining more important feature information; the purpose of adding dual-branch attention is to extract local features, and at the same time, it has faster training speed and higher accuracy than the previous single-branch attention.

[0028] (2) The present invention uses the Focal loss function to adjust the sample weights during the training process. This method enhances the model's attention to rare categories, solves the extreme imbalance in the number of positive and negative samples, and makes the model pay more attention to samples that are difficult to classify, effectively improving the accuracy and automation level of diabetic retinopathy recognition, thereby improving classification performance.

[0029] (3) The VMamba-m model proposed in this invention uses the Focal loss function as the loss function and introduces the SE attention mechanism and the self-designed Local attention mechanism. The accuracy is significantly improved (the accuracy is as high as 79.1%, and the AUC value is 89.1%), and the iteration time per round is only 34.027s. Unlike traditional classification models, the model proposed in this invention has faster calculation speed and higher accuracy, which solves the problem of high computational complexity of traditional models when processing long sequences.

[0030] (4) The VMamba-m proposed in the present invention can classify diabetic retinopathy images at a faster speed and with higher accuracy, which can meet the needs of real-time detection and is suitable for low-computing power platforms, such as real-time detection tasks in low-computing power scenarios in similar medically backward areas. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] By referring to the following description in conjunction with the accompanying drawings, and with a more complete understanding of the present invention, other objects and results of the present invention will become more clear and easy to understand. In the accompanying drawings:

[0032] Figure 1 Schematic diagram of the overall structure of the VMamba-m model constructed in the present invention;

[0033] Figure 2 Schematic diagram of the VSS-SL module in the present invention;

[0034] Figure 3 Schematic diagram of the Local attention module of the present invention;

[0035] Figure 4 Schematic diagram of the SE attention module of the present invention;

[0036] Figure 5 Schematic diagram of representative samples of the Aptos2019 dataset;

[0037] Figure 6 Schematic diagram of representative samples after preprocessing of the Aptos2019 dataset;

[0038] Figure 7 Training results for the Aptos2019 dataset (loss per round);

[0039] Figure 8 Training results 2 for the Aptos2019 dataset (accuracy per round);

[0040] Figure 9 This is the ROC training result on the Aptos2019 dataset;

[0041] Figure 10 Is the confusion matrix on the Aptos2019 dataset. DETAILED DESCRIPTION

[0042] In order to enable those skilled in the art to better understand the technical solutions and advantages of the present invention, the present application is described in detail below with reference to the accompanying drawings, but this is not intended to limit the scope of protection of the present invention.

[0043] Example 1:

[0044] This embodiment provides a deep learning-based diabetic retinopathy image classification system. The technical solution of the present invention is described in detail below with reference to the accompanying drawings.

[0045] like Figure 1 、 Figure 2As shown, the deep learning-based diabetic retinopathy image classification system provided in this embodiment uses a trained VMamba-m model. The VMamba-m model is obtained by improving the VMamba network model. The VSS module of the VMamba network model is replaced with the VSS-SL module, and the original global feature extraction method is changed to a local feature extraction method to enhance the network's ability to extract image edge information and spatial information, thereby improving detection accuracy.

[0046] The VSS-SL module is an efficient front-end module for image recognition tasks. It aims to improve the model's ability to parse images by combining visual and semantic information. Especially in complex scene understanding tasks, this module explicitly models the dependencies between channels through the SE attention module and the Local attention module, and dynamically adjusts the weight of each channel in the feature map, thereby improving the model's representation ability.

[0047] The VSS-SL module has a multi-branch structure, that is, a dual-branch path is added on the basis of the VSS module to further extract features to provide global modeling capabilities with linear complexity; specifically, the VSS-SL module includes a VSS module, an SE attention module, and a Local attention module. The input data (feature map) is input into the VSS module in one path, and the other path is input into the Local attention module and the SE attention module in turn; wherein, the VSS module includes an LN normalization layer and a parallel first branch path and a second branch path. The first branch path uses a linear layer and an activation function for feature conversion, and the second branch path passes through a linear layer, a depth-separable convolution and an activation function in turn, and then enters the SS2D module for in-depth feature extraction; after passing through the SS2D module, an LN normalization layer is used to standardize the output features, and the features standardized by the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module; the fused features are input into the linear layer, and the features output by the linear layer are output after feature addition with the original input data.

[0048] Furthermore, in this embodiment, the purpose of setting the Local attention module is to capture the global dependencies between all positions in the input feature map. Unlike traditional convolution operations, the Local attention module can capture long-range dependencies by calculating the correlation between any two positions, thereby enhancing the global modeling capability of the model. The expression is:

[0049] ;

[0050] in, is the output feature, usually representing a node The new representation or attention aggregation result of and are two positions in the input feature map (which can be spatial positions, temporal positions, or spatiotemporal positions), is a function that calculates the correlation between two positions, It's the position A function that transforms the features of , similar to Value mapping; For all The mathematical symbol of ; is a normalization factor, typically the sum of all correlations.

[0051] In this embodiment, a function is used Calculate the correlation between two positions and perform standardization on the small batch input data of each layer to reduce internal covariate shift, accelerate training, alleviate the problem of gradient disappearance or explosion, and improve training speed and stability; common correlation functions include Gaussian functions: ; Dot product: ; Embedded Gaussian function: ,in and is an embedding function.

[0052] Specifically, if Figure 3 As shown, in this embodiment, the Local attention module includes an initial 2D convolution layer, which performs batch normalization operation through the initial 2D convolution layer; followed by a 2D convolution layer, which uses a 3×3 convolution kernel, with a step size of 1 and a padding of 1. The purpose is to use a 3×3 convolution kernel to extract spatial features. 3×3 is a small-size convolution kernel that can capture local details and has a small number of parameters; then the ReLU activation function is used, which is a nonlinear activation function to enhance the expression ability of the model and alleviate the gradient disappearance problem; then a second convolution is performed, which uses a 2D convolution layer. The 2D convolution layer of the second convolution and the 2D convolution layer of the first convolution are combined. The D convolution layer is consistent with the previous one, using a 3×3 convolution kernel, a step size of 1, and a padding of 1 for feature extraction. It further extracts higher-order shape and complex texture features, and then goes through batch normalization (batch normalization operation through the initial 2D convolution layer) and ReLU activation function. The batch normalization operation is used to standardize the output of the convolution layer to ensure the stability and nonlinearity of the feature distribution; then the 2D convolution layer is input, which uses a 1×1 convolution kernel and a step size of 1. The introduction of the 1×1 convolution layer can reduce the amount of calculation and is used for cross-channel feature fusion or adjustment of channel dimension. The ReLU function is then used to continue to maintain nonlinearity, gradually extract high-order features, and finally output.

[0053] Further, if Figure 4As shown in the figure, in this embodiment, the data processing flow of the SE attention module includes compression (Squeeze), excitation (Excitation) and scale weighting; the purpose of compression is to compress the global spatial information into a channel descriptor; specifically, a global average pooling (GAP) operation is performed on the input feature map of each channel to compress the two-dimensional feature map of each channel into a scalar value.

[0054] In this embodiment, the Local attention module outputs features Figure X , whose shape is H'×W'×C' (representing height, width, and number of channels respectively). Figure X After Transformer operation (F tr ) generates a feature map U; in the compression step, the feature U aggregates the feature maps across the spatial dimensions H×W through a compression operation to generate a channel descriptor; H×W×C→1×1×C, compressing the global spatial information into a channel descriptor so that these channel descriptors can be used by its input layer; the input feature map is compressed into a vector through a global average pooling operation, and then mapped to a smaller vector through a fully connected layer, as follows:

[0055] ;

[0056] in, It is The global average pooling result of the channels compresses the spatial information into a scalar value. is the first input feature map channels with a size of , is the height, is the width, Represents the feature compression (FeatureSqueeze) operation;

[0057] The goal of excitation is to learn the importance weights of each channel. This is done through a simple fully connected network (typically consisting of two fully connected layers) to learn the dependencies between channels and output the weights for each channel. In the excitation step, a sigmoid function is used to compress each element in this vector to a value between 0 and 1. This is then multiplied by the original input feature map to produce a weighted feature map. The vector z obtained in the previous step is processed through two fully connected layers to obtain the desired channel weight s. After passing through the two fully connected layers, different values in s represent the weight information for different channels, assigning different weights to the channels. The formula is as follows:

[0058] ;

[0059] in, is the vector output by the compression operation, is the weight matrix of the first fully connected layer, used for dimensionality reduction; is the weight matrix of the second fully connected layer, used to restore the dimension; It is the ReLU activation function, which introduces nonlinearity; It is the Sigmoid activation function, which maps the output to the [0,1] interval, indicating the importance weight of each channel. , represents the incentive operation, Represents the output of an unnormalized weight vector, which is then The Sigmoid activation function is compressed to the [0,1] interval.

[0060] The purpose of scale weighting is to apply the learned weights to the original feature map and convert the weight vector With the original feature map Multiply channel by channel to get the weighted feature map Specifically, the weight of the excitation output is added to the original input features through channel convolution. The SE attention module can make the model more focused on key information through its clever attention mechanism, thereby effectively promoting the application of deep learning in many fields. The formula is as follows:

[0061] ;

[0062] in, It is c The weight of each channel, It is the original c The feature map of channel input, After weighting c The feature maps of each channel are multiplied by each channel to adjust the feature importance.

[0063] Furthermore, in this embodiment, the shape of the feature map U is H×W×C, and the feature map U is compressed by the global average pooling operation (F sq ) generates a A vector of each channel is represented by a numerical value; then the excitation operation is realized through two fully connected layers (F ex ), generate weight information of different channels; finally, through the Scale weighting operation (F scale ) The weight vector For the original feature map Assign weights to obtain the required feature maps , its size and feature map Exactly the same.

[0064] Example 2:

[0065] This embodiment provides a deep learning-based diabetic retinopathy image classification method, which includes the following steps:

[0066] Step 1. Construct the VMamba-m model described in Example 1;

[0067] Step 2. Train the VMamba-m model to obtain a trained VMamba-m model;

[0068] Step 3. Input the retinal image data into the trained VMamba-m model for classification.

[0069] In this embodiment, the process of training the VMamba-m model is as follows:

[0070] Step a. Build a dataset, download the public diabetic retinopathy classification dataset Aptos2019, and preprocess the images. The images before and after preprocessing are as follows: Figure 5 and 6 As shown in FIG, a data set for training a model is obtained by preprocessing; the preprocessed data set is then divided into a training set and a test set in proportion;

[0071] Step b. Configure the training parameters and use the loss function during training to solve the problem of extreme imbalance in the number of positive and negative samples;

[0072] Step c. Inputting the constructed training set into the constructed VMamba-m model for testing and training to obtain the optimal diabetic retinopathy image classification model after training;

[0073] Step d. After the training is completed, the test set is placed into the obtained optimal model for testing to obtain the classification level of the diabetic retinopathy fundus image to be tested, and finally the recognition result is obtained.

[0074] Furthermore, in this embodiment, the method of preprocessing the image in step a includes: normalizing the image size and performing data enhancement on the image; wherein the data enhancement includes random rotation, random cropping, random flipping, brightness adjustment, and contrast adjustment; finally, dividing the test set and the training set according to a ratio of 2:8, wherein the image size is 1050*1050 pixels.

[0075] Furthermore, in this embodiment, the loss function used in the training process is the Focal Loss function. The Focal Loss function is an effective method for dealing with the problem of class imbalance. By introducing a focal factor and adjusting the sample weights, the model pays more attention to samples that are difficult to classify, thereby improving the classification performance. This function is particularly suitable for class imbalance tasks such as target detection.

[0076] The formula of the Focal loss function is:

[0077] ;

[0078] in, is the model’s predicted probability for the target category; is the balance factor, with a value range of (0, +∞), which is used to adjust the influence between positive and negative samples; γ is the focus factor, with a value range of [0, +∞), which is used to adjust the weight of difficult-to-classify samples; the modulation factor Used to reduce the loss rate of easily distinguishable samples, regardless of foreground or background classes, The larger it is, the easier it is to distinguish the samples and the smaller the modulation factor.

[0079] In order to verify the effectiveness of the improved module in this invention, the VMamba-m model was selected to verify the training loss and accuracy comparison on the Aptos2019 dataset. Figure 7 and Figure 8 It can be seen that the accuracy of training and validation gradually increases with the number of iterations, and the loss rate of validation data gradually decreases with the number of iterations.

[0080] In the selection of convolutional models, the purpose of this invention is to make them lightweight while ensuring a certain detection efficiency, so as to adapt to real-time detection tasks in low-computing power scenarios such as those in underdeveloped medical areas.

[0081] To this end, under the premise of data preprocessing, the present invention selects a model and performs model training, testing and ablation experiments on the data set corresponding to the task of the present invention. The final test results are shown in Tables 1 and 2.

[0082] After multiple rounds of experiments, the experimental results of each model on the Aptos2019 dataset are shown in Table 1. The data in this table show that the Mamba series of models performs well in the diabetic retinopathy image classification task. In the five-category task, the proposed VMamba-m model performed best, followed by VMamba, with classification accuracies of 0.791 and 0.714, respectively, and 34.027 and 76.322 iterations per round, respectively, outperforming the other models.

[0083] The ROC curve for the VMamba-m model of the present invention is shown in Figure 9. The ROC curve and AUC value indicate how close the prediction is to perfect classification, with perfect classification displayed in the upper left corner of the ROC coordinates. The AUC value represents the area under the ROC curve; the closer the value is to 1, the better the model performance. The AUC values shown in the figure are 99.1%, 85.5%, 87.50%, 87%, and 86.8%, representing the categories of no DR, mild, moderate, severe, and proliferative DR, respectively. Because morphological changes in fundus images of moderate DR affect the identification of pathological structures, the AUC for moderate DR is lower than that for other DR categories, at 85.5%.

[0084] Figure 10 The confusion matrix shows how the model's performance for each class was evaluated, with the diagonal cells showing the proportion of examples successfully classified for each class. In our model, 354 images correctly predicted class 0 (healthy); 43 images predicted class 1 (mild); 126 images predicted class 2 (moderate); 20 images predicted class 3 (severe); and 34 images predicted class 4 (hyperplastic).

[0085] Furthermore, ablation experiments can determine the impact of these mechanisms on model performance and assess whether the aforementioned issues have been effectively mitigated. Table 2 shows ablation experiments using three different mechanisms on the original modeling task. Using the VMamba architecture as a baseline, the introduction of the SE attention module improved accuracy and reduced the time per iteration by 52.7%. The introduction of the Local attention module also improved accuracy and reduced the time per iteration by 52.4%. The use of the focus function mechanism also improved accuracy while maintaining essentially the same computation time.

[0086] After replacing the loss function during training, ablation experiments using different mechanisms are shown in Table 2. Based on the VMamba architecture, the introduction of the SE attention module improved accuracy and reduced iteration time by 54.9%. The introduction of the Local attention module also improved accuracy and reduced iteration time by 53.7%. The VMamba-m model proposed in this paper replaces the loss function with a focal loss function and introduces both the SE attention mechanism and the Local attention mechanism, significantly improving accuracy and reducing iteration time by 55.4%.

[0087] In this example, the effectiveness of each proposed module was verified through validation models and ablation experiments. The advantages of the proposed model are demonstrated by comparing various metrics with classic and advanced algorithms. A comparison of transfer learning strategies and mixed dataset training strategies demonstrates that mixed dataset training strategies are more effective when samples are scarce.

[0088] Table 1 shows the classification results of different models

[0089] Model Accuracy Recall Precision AUC F1 Time InceptionV3 0.701 0.684 0.618 0.813 0.650 106.345 VGG16 0.686 0.795 0.653 0.805 0.701 108.012 ResNet50 0.698 0.542 0.701 0.847 0.660 107.453 ResNet34 0.688 0.540 0.700 0.839 0.681 108.753 VGGNet 0.702 0.550 0.655 0.832 0.651 105.424 DensNet121 0.684 0.574 0.640 0.814 0.606 111.652 ResNet108 0.675 0.702 0.690 0.821 0.686 107.379 NASNetMobile 0.795 0.648 0.712 0.871 0.732 152.613 MobileNet 0.769 0.655 0.722 0.874 0.751 125.410 MobileNetV2 0.759 0.647 0.724 0.863 0.756 130.031 MobileNetV3Small 0.692 0.643 0.694 0.812 0.701 127.721 MobileNetV3Large 0.665 0.613 0.684 0.846 0.717 129.285 VMamba 0.714 0.500 0.718 0.881 0.500 76.322 Mamba 0.703 0.512 0.706 0.850 0.501 77.034 VMamba-m 0.791 0.537 0.766 0.891 0.527 34.027

[0090] Table 2 shows the ablation experiment results.

[0091] Model Accuracy Recall Precision AUC F1 Time VMamba 0.714 0.500 0.718 0.871 0.500 76.322 VMamba+Focal 0.722 0.504 0.724 0.874 0.510 75.122 VMamba+SE 0.742 0.507 0.723 0.878 0.501 36.081 VMamba+Local 0.746 0.502 0.734 0.879 0.500 36.255 VMamba+Focall+SE 0.771 0.517 0.733 0.881 0.506 34.391 VMamba+Focall+Local 0.766 0.514 0.742 0.886 0.515 35.295 VMamba-m 0.791 0.537 0.766 0.891 0.527 34.027

[0092] It can be seen from the above data that in the diabetic retinopathy image classification task, the VMamba-m model proposed in the present invention improves the VSS module of the original VMamba model, adds an attention mechanism, enhances the overall expressiveness of the model, improves the details and architectural design of the implementation process, and improves the model performance and reasoning speed; the flexible use of the FocalLoss function enables the model to enhance its attention to rare categories and reduces the time complexity for different samples in the classification task. The VMamba-m model was evaluated on the Aptos2019 dataset released by the Kaggle platform. In the five-category task of lesion grading, the accuracy of the model proposed in the present invention was as high as 79.1%, the AUC value was 89.1%, and the iteration time per round was only 34.027s, which is one-third to one-quarter of the iteration time per round of most traditional CNN models. Comparative experiments and ablation experiments have proved that the model proposed in the present invention performs well in the five-category task of diabetic retinopathy.

[0093] The present invention also provides an electronic device comprising: one or more processors and a memory; wherein the memory is used to store one or more programs, and when the one or more programs are executed by the one or more processors, the one or more processors implement the above-mentioned deep learning-based diabetic retinopathy image classification method.

[0094] The present invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned deep learning-based diabetic retinopathy image classification method.

[0095] Those skilled in the art will appreciate that all or part of the functions of the various methods / modules in the above embodiments may be implemented via hardware or via computer programs. When all or part of the functions in the above embodiments are implemented via computer programs, the program may be stored in a computer-readable storage medium, which may include a read-only memory, random access memory, a magnetic disk, an optical disk, a hard disk, etc., and the program is executed by a computer to implement the above functions. For example, the program may be stored in a memory of a device, and when the program in the memory is executed by a processor, all or part of the above functions may be implemented.

[0096] In addition, when all or part of the functions in the above-mentioned embodiments are implemented by means of a computer program, the program can also be stored in a storage medium such as a server, another computer, a disk, an optical disk, a flash drive or a mobile hard disk, and saved to the memory of a local device by downloading or copying, or the system of the local device is updated. When the program in the memory is executed by the processor, all or part of the functions in the above-mentioned embodiments can be implemented.

[0097] The above description of the present invention using specific examples is intended only to facilitate understanding of the present invention and is not intended to limit the present invention. A person skilled in the art of the present invention may make several simple deductions, modifications, or substitutions based on the principles of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A deep learning-based diabetic retinopathy image classification system, characterized by: The system uses a trained VMamba-m model, which is obtained by improving the VMamba network model by replacing the VSS module of the VMamba network model with the VSS-SL module; Among them, the VSS-SL module is a multi-branch structure, including a VSS module, an SE attention module, and a Local attention module. The input data is input into the VSS module in one path, and the other path is input into the Local attention module and the SE attention module in sequence; wherein, the VSS module includes an LN normalization layer and a parallel first branch path and a second branch path. The first branch path uses a linear layer and an activation function for feature conversion, and the second branch path passes through a linear layer, a depth-separable convolution and an activation function in sequence, and then enters the SS2D module for in-depth feature extraction; after passing through the SS2D module, an LN normalization layer is used to standardize the output features, and the features standardized by the LN normalization layer are fused with the features output by the first branch path and the features output by the SE attention module; the fused features are input into the linear layer, and the features output by the linear layer are output after feature addition with the original input data; The Local attention module includes an initial 2D convolution layer, which performs a batch normalization operation; followed by a 2D convolution layer, which uses a 3×3 convolution kernel, a step size of 1, and a padding of 1, and then passes through a ReLU activation function; then a second convolution is performed, and the second convolution uses a 2D convolution layer, which uses a 3×3 convolution kernel, a step size of 1, and a padding of 1, and again passes through batch normalization and a ReLU activation function; then a 2D convolution layer is input, which uses a 1×1 convolution kernel and a step size of 1, followed by a ReLU function to continue to maintain nonlinearity, gradually extract high-order features, and finally output.

2. The deep learning-based diabetic retinopathy image classification system according to claim 1, characterized in that: The data processing flow of the SE attention module includes compression, excitation and scale weighting; the local attention module outputs the feature map X, which is first generated by the Transformer operation to generate the feature map U, and then the input feature map U is compressed into a vector through the global average pooling operation; the compressed vector is processed by the fully connected network to obtain the weights of different channels; finally, the scale weighting operation combines the weights obtained by the excitation operation with the original feature map Multiply channel by channel to get the weighted feature map .

3. The deep learning-based diabetic retinopathy image classification system according to claim 1, characterized in that: The expression of the Local attention module is: ； in, is the output feature, and are two locations in the input feature map, is a function that calculates the correlation between two positions, It's the position The function that transforms the characteristics of For all The mathematical symbol of ; is the normalization factor.

4. A deep learning-based diabetic retinopathy image classification method, characterized in that: The method comprises the following steps: Step 1. Construct the VMamba-m model described in any one of claims 1 to 3; Step 2. Train the VMamba-m model to obtain a trained VMamba-m model; Step 3. Input the retinal image data into the trained VMamba-m model for classification.

5. The deep learning-based diabetic retinopathy image classification method according to claim 4, characterized in that: The process of training the VMamba-m model is as follows: Step a. Build a dataset. Download the publicly available diabetic retinopathy classification dataset, Aptos2019, and preprocess the images to obtain a dataset for training the model. Then, divide the preprocessed dataset into a training set and a test set in proportion. Step b. Configure the training parameters and use the loss function during the training process; Step c. Inputting the constructed training set into the constructed VMamba-m model for testing and training to obtain the optimal diabetic retinopathy image classification model after training; Step d. After the training is completed, the test set is placed in the obtained optimal diabetic retinopathy image classification model for detection, the classification level of the diabetic retinopathy fundus image to be detected is obtained, and finally the recognition result is obtained.

6. The deep learning-based diabetic retinopathy image classification method according to claim 5, characterized in that: The method of preprocessing the image in step a includes: performing size normalization processing on the image and performing data enhancement processing on the image; wherein the data enhancement processing includes random rotation, random cropping, random flipping, brightness adjustment and contrast adjustment.

7. The deep learning-based diabetic retinopathy image classification method according to claim 5, characterized in that: The loss function used in the training process is the Focal loss function.

Images