Automatic classification model and classification method for diabetic retinopathy

Abstract

This invention discloses an automatic classification model and method for diabetic retinopathy. The automatic classification model includes convolutional input, feature extraction, and classification output. The convolutional input consists of 7×7 convolutions and max pooling, responsible for resizing the input image to extract shallow features. Feature extraction includes four alternately connected dense blocks and three transition layers, mainly responsible for extracting feature information of retinal lesions. The classification output consists of global average pooling and fully connected layers, responsible for outputting the classification result of diabetic retinopathy. This invention designs an attention mechanism (SC) based on retinal lesion features to increase the network's focus on lesion features and fully extract retinal lesion features. Using the lightweight network DenseNet121 as the backbone, the dense layers are improved by connecting the SC module after 3×3 convolutions, increasing the classifier's ability to express high-level semantic features and improving classification accuracy. This invention can achieve automatic classification of diabetic retinopathy levels.

Description

Technical Field

[0001] This invention belongs to the field of image classification and relates to an automatic classification model, method, device and storage medium for diabetic retinopathy. Background Technology

[0002] Diabetic retinopathy (DR) is one of the eye complications caused by diabetes. Early intervention and treatment of DR patients can significantly reduce the risk of blindness and alleviate the severity of the disease, which is of great significance to social development. The application of deep learning in the medical field is gradually becoming a mainstream trend. Computer-aided diagnosis is both efficient and accurate, not only alleviating the shortage of medical personnel but also improving the precision of examinations. Therefore, using deep learning for intelligent DR diagnosis provides a new solution for screening diabetic retinopathy.

[0003] With the widespread application of deep learning in the medical industry, computer-aided diagnosis using convolutional neural networks has become a trend. Ouyang Jihong et al. proposed that the two-branch mixed attention decision-making network (BiRAD-Net) has good diagnostic capabilities. See “Ouyang Jihong, Guo Zeqi, Liu Siguang. Two branch mixed attention decision-making network for the staging of diabetes retinopathy[J]. Journal of Jilin University (Engineering Edition), 2022, 52(03):648656.DOI:10.13229 / j.cnki.jdxbgxb20200813.” Zhao et al. proposed the BiRA-Net network model for bilinear fine-grained classification, which mines deep features. See "Zhao Z, Zhang K, Hao X, et al. Bira-net: Bilinear attention net for diabetic retinopathy grading[C] / / 2019IEEE International Conference on Image Processing(ICIP).IEEE,2019:1385-1389." Zhou et al. proposed a multi-task multi-grid framework, which deepens the network through a multi-grid structure to improve the model's classification accuracy. See "Zhou K, Gu Z, Liu W, et al. Multi-cell multi-task convolutional neural networks for diabeticretinopathy grading[C] / / 2018 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society(EMBC).IEEE,2018:2724-2727."Zheng Wen et al. fine-tuned the network using transfer learning and integrated an attention mechanism module to improve network performance. See “ZhengWen, Shen Qihao, Ren Jia. Recognition and grading of diabetes retinopathy based on improved DR Net algorithm[J]. Acta Optica Sinica, 2021, 41(22):72-83.” Liang Liming et al. introduced a cross-layer bilinear pooling model on the backbone network SEResNeXt to improve classification accuracy. See “LiangLiming, Peng Renjie, Feng Jun, Yin Jiang. Research on grading algorithm of diabetes retinopathy based on cross layer bilinear pooling[J]. Journal of Biomedical Engineering, 2022, 39(05):928-936.”

[0004] The aforementioned studies have some effect on DR classification, but the models are relatively complex, often sacrificing network depth and size to improve classification accuracy. They are not easy to deploy on mobile devices or other mobile platforms, and have low flexibility and practicality.

[0005] To address the issues of insufficient feature extraction for diabetic retinopathy (DR) lesions and the complexity of model budgeting, making them difficult to deploy on mobile and embedded devices, this invention designs a SC attention module and proposes an automatic lesion classification model for identifying DR types using a lightweight network backbone. This model aims to be flexibly deployed on mobile devices and accurately identify DR lesion types. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides an automatic classification model for diabetic retinopathy. By designing an attention module that focuses on fine-grained image features, it increases the effective attention to the lesion area. With a lightweight network as the backbone, it solves the problems existing in the prior art while ensuring the network's classification accuracy and being flexibly deployed to embedded mobile devices.

[0007] The second objective of this invention is to provide an automatic classification method for diabetic retinopathy.

[0008] A third objective of this invention is to provide an electronic device.

[0009] A fourth objective of this invention is to provide a computer storage medium.

[0010] The technical solution adopted in this invention is an automatic classification model for diabetic retinopathy, characterized by including convolutional input, feature extraction, and classification output;

[0011] The convolutional input consists of 7×7 convolution and downsampling (maxpool), and is responsible for resizing the input image to extract shallow features;

[0012] The feature extraction comprises four alternately connected dense blocks and three transition layers, which are mainly responsible for extracting feature information of retinal lesions;

[0013] The dense block contains several dense layers SC-Bottleneck, which are designed to increase feature attention to high-level semantic information and focus on the favorable areas of the lesion.

[0014] The classification output, consisting of a global average pooling layer and a fully connected layer, is responsible for outputting the classification results for diabetic retinopathy.

[0015] Furthermore, the dense blocks are intended to fully extract lesion features from the retinal image;

[0016] The four dense blocks contain 6, 12, 24, and 16 dense layers SC-Bottleneck, which are responsible for enhancing the network’s feature extraction of retinal lesions.

[0017] The dense layer SC-Bottleneck is processed by BN and ReLU and then fed into a 1×1 convolution to fix the output channel and reduce the output dimension. Then it is processed by BN and ReLU and fed into a 3×3 convolution for feature extraction. Then, the SC attention module is used to increase the feature attention of high-level semantic information and improve the network's feature extraction of lesion information.

[0018] The dense layer SC-Bottleneck, through the added SC attention mechanism, purposefully focuses on the extracted retinal feature information in both spatial and channel dimensions, concentrating on the favorable areas of the lesion, reducing the interference of complex background information on retinal lesions, and ensuring the network's accurate extraction of lesions.

[0019] Furthermore, the attention mechanism module SC is designed to increase the focus on the number of yellow rings, bright red patches, and lesions in the optic disc area within the effective region;

[0020] Each SC module consists of 3 attention layers and 1 convolutional module (Conv+BN+ERelu). Conv+ECA+ERelu consists of 1×1 traditional convolution, BN normalization, and ERElu activation function.

[0021] The SC attention module has three attention layers, which contain 64, 16, and 8 1×1 convolutional kernels respectively.

[0022] The attention layer consists of 1×1 convolution, ECA, and ERelu. 1×1 convolution and ECA are used for complementary attention in mixed dimensions, while ERelu performs non-linear processing to prepare for the next input.

[0023] The 1×1 convolution refines features in the spatial dimension, preserving the spatial information of the feature map without loss, and focusing on features in the spatial dimension.

[0024] The attention mechanism shares its learning parameters in the form of a fast one-dimensional convolution kernel, which has a light size while achieving mutual attention between channels, forming complementarity in spatial information, improving the network's attention to lesion features, and further improving network performance.

[0025] The convolution module adjusts the output of the original network through a 1×1 convolution to obtain the information weight distribution on a single spatial plane. Then, it performs a full multiplication operation with the original input feature map to obtain the weights of spatial and channel interaction. The network will purposefully focus on areas with large weights and learn useful features.

[0026] An automated classification method for diabetic retinopathy is performed according to the following steps:

[0027] S1. Input the preprocessed fundus retinal image into the convolutional layer to complete the extraction of shallow features and send it to the feature extraction module;

[0028] S2. After the dense block in the feature extraction module completes the extraction of high-level semantic information of the input image, it is sent to the prediction output.

[0029] S3, the classifier predicts and outputs the classification results for diabetic retinopathy.

[0030] Furthermore, in step S2, the feature extraction module comprises four alternately connected dense blocks and three transition layers. The calculation process of the dense layer SC-Bottleneck module within the dense block is as follows:

[0031] Y0 = Conv 1×1 (K i (1)

[0032] Y1 = Conv 3×3 (Y0) (2)

[0033] K i+1 =SC(Y1) (3)

[0034] Among them, K i(i takes values ​​of 1, 2, ..., n) represents the input to the dense layer SC-Bottleneck, and Y0 represents the input to input K. i via Conv 1×1 Convolution reduces the output dimension of the feature map, where Y1 represents the convolution of Y0. 3×3 Feature map extracted by convolution, K i+1 This represents the feature map that is finally output after passing through the SC attention module.

[0035] Further, in step S2, the SC attention module included in the SC-Bottleneck module is calculated according to the following formula:

[0036] F0 = ECA(Conv) 1×1 (X m-1 (4)

[0037] F1 = ECA(Conv) 1×1 (F0)) (5)

[0038] F2 = ECA(Conv) 1×1 (F1)) (6)

[0039] F3 = Conv 1×1 (F2) (7)

[0040]

[0041] Among them, X m-1 (m takes values ​​of 1, 2...n) represents the input of the SC attention module, and F0 represents the input X. m-1 via Conv 1×1 Convolution and ECA are used for the first layer of attention operation. The number of convolution kernels C1 is 64, and F1 indicates that F0 is processed by Convolution. 1×1 Convolution and ECA are used for the second layer of attention extraction, with C2 having 16 kernels. F2 represents the feature map for the third layer of attention operation on F1, with C3 having 8 kernels. F3 represents the feature map for the input F2 with a single kernel. 1×1 Convolution operations yield information weight distributions on a single spatial plane. X represents the product of the channel weights and the original feature map. m (m takes values ​​of 1, 2...n) represents the features finally extracted by the SC attention module.

[0042] An electronic device that uses the method described above to automatically classify OCT images of diabetic retinopathy.

[0043] A computer storage medium storing at least one program instruction, which is loaded and executed by a processor to implement the above-described automatic OCT image classification method for diabetic retinopathy.

[0044] The beneficial effects of this invention are:

[0045] This invention addresses the problems of insufficient retinal feature extraction, low classification accuracy, and difficulty in deploying the model algorithm to embedded mobile devices by designing an automatic classification model for retinal lesions, D-SCNet. First, a global attention module (SC) is designed, which is arranged alternately in a spatial and channel-based manner to enhance the network's focus on the number of yellow rings, bright red patches, and lesions in the optic disc region within the effective area. Then, using a lightweight DenseNet121 as the backbone network, a dense layer SC-Bottleneck is designed using the attention module SC. This layer is used to extract high-level semantic information, prompting the network to extract effective information and enhancing its ability to extract and represent lesion features. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a schematic diagram of the lesion classification model in an embodiment of the present invention.

[0048] Figure 2 This is a schematic diagram of the structure of the dense layer in the lesion classification model of this invention.

[0049] Figure 3 This is a schematic diagram of the SC attention module in the lesion classification model of this invention.

[0050] Figure 4 This is a retinal image heatmap of the lesion classification model in an embodiment of the present invention. Detailed Implementation

[0051] The technical solutions of the present invention will be clearly and completely described below with reference to the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0052] Example 1,

[0053] An automatic classification model for diabetic retinopathy, characterized by including convolutional input, feature extraction, and classification output;

[0054] The convolutional input, consisting of 7×7 convolution and downsampling (maxpool), is responsible for resizing the input image to extract shallow features.

[0055] Feature extraction consists of four densely connected alternating blocks and three transition layers, which are mainly responsible for extracting high-level semantic information from the input image;

[0056] The dense block contains several dense layers SC-Bottleneck, which aims to increase the feature attention to high-level semantic information and focus on the favorable areas of the lesion;

[0057] The classification output, consisting of a global average pooling layer and a fully connected layer, is responsible for outputting the classification results for diabetic retinopathy.

[0058] like Figure 2 As shown,

[0059] Dense blocks are used to fully extract lesion features from retinal images;

[0060] The four dense blocks contain 6, 12, 24, and 16 dense layers SC-Bottleneck, which are responsible for enhancing the network’s feature extraction of retinal lesions.

[0061] The dense layer SC-Bottleneck is processed by BN and ReLU and then fed into a 1×1 convolution to fix the output channel and reduce the output dimension. Then it is processed by BN and ReLU and fed into a 3×3 convolution for feature extraction. Then, the SC attention module is used to increase the feature attention to high-level semantic information and improve the network's feature extraction of lesion information.

[0062] The dense layer SC-Bottleneck, with the addition of the SC attention mechanism, purposefully focuses on the extracted retinal feature information in both spatial and channel dimensions, concentrating on the favorable areas of the lesion, reducing the interference of complex background information on retinal lesions, and ensuring the network's accurate extraction of lesions.

[0063] like Figure 3 As shown,

[0064] The attention mechanism module SC is designed to increase the focus on the number of yellow rings, bright red patches, and lesions in the optic disc area within the effective region;

[0065] Each SC module consists of 3 attention layers and 1 convolutional module (Conv+BN+ERelu). Conv+ECA+ERelu consists of 1×1 traditional convolution, BN normalization, and ERElu activation function.

[0066] The SC attention module has three attention layers, containing 64, 16, and 8 1×1 convolutional kernels respectively. The attention layer consists of 1×1 convolution, ECA, and ERelu. 1×1 convolution and ECA are used for complementary attention in mixed dimensions, while ERelu performs non-linear processing to prepare for the next input.

[0067] 1×1 convolution refines features in the spatial dimension without loss, preserving spatial information of the feature map and focusing on features in the spatial dimension; the attention mechanism shares its learning parameters in the form of a fast one-dimensional convolution kernel, achieving mutual attention between channels while having a light volume, forming complementarity in spatial information, improving the network's attention to lesion features, and further improving network performance.

[0068] The convolution module adjusts the output of the original network through a 1×1 convolution to obtain the information weight distribution on a single spatial plane. Then, it performs a full multiplication operation with the original input feature map to obtain the weights of spatial and channel interaction. The network will purposefully focus on areas with large weights and learn useful features.

[0069] Example 2,

[0070] An automated classification method for diabetic retinopathy is performed according to the following steps:

[0071] S1. Input the preprocessed fundus retinal image into the convolutional layer to complete the extraction of shallow features and send it to the feature extraction module;

[0072] S2. After the dense block in the feature extraction module completes the extraction of high-level semantic information of the input image, it is sent to the prediction output.

[0073] S3, the classifier predicts and outputs the classification results for diabetic retinopathy.

[0074] Furthermore, in step S2, the feature extraction module comprises four alternately connected dense blocks and three transition layers. The calculation process of the dense layer SC-Bottleneck module within the dense block is as follows:

[0075] Y0 = Conv 1×1 (K i (1)

[0076] Y1 = Conv 3×3 (Y0) (2)

[0077] K i+1=SC(Y1) (3)

[0078] Among them, K i (i takes values ​​of 1, 2, ..., n) represents the input to the dense layer SC-Bottleneck, and Y0 represents the input to input K. i via Conv 1×1 Convolution reduces the output dimension of the feature map, where Y1 represents the convolution of Y0. 3×3 Feature map extracted by convolution, K i+1 This represents the feature map that is finally output after passing through the SC attention module.

[0079] Further, in step S2, the SC attention module included in the SC-Bottleneck module is calculated according to the following formula:

[0080] F0 = ECA(Conv) 1×1 (X m-1 (4)

[0081] F1 = ECA(Conv) 1×1 (F0)) (5)

[0082] F2 = ECA(Conv) 1×1 (F1)) (6)

[0083] F3 = Conv 1×1 (F2) (7)

[0084]

[0085] Among them, X m-1 (m takes values ​​of 1, 2...n) represents the input of the SC attention module, and F0 represents the input X. m-1 via Conv 1×1 Convolution and ECA are used for the first layer of attention operation. The number of convolution kernels C1 is 64, and F1 indicates that F0 is processed by Convolution. 1×1 Convolution and ECA are used for the second layer of attention extraction, with C2 having 16 kernels. F2 represents the feature map for the third layer of attention operation on F1, with C3 having 8 kernels. F3 represents the feature map for the input F2 with a single kernel. 1×1 Convolution operations yield information weight distributions on a single spatial plane. X represents the product of the channel weights and the original feature map. m (m takes values ​​of 1, 2...n) represents the features finally extracted by the SC attention module.

[0086] To verify the effectiveness of the diabetic retinopathy classification method of this invention, a comparative experiment was conducted with existing models. The selected dataset was the EYEPACS competition dataset from the Kaggle platform, which categorizes lesion severity into five classes: Normal, Mild, Moderate, Severe, and Proliferative. To further verify the model performance, the Messidor-2 dataset was used as a second test set. After a series of preprocessing operations, the dataset was input into the classifier for the experiment.

[0087] The model networks compared are Li's algorithm (Li X, Xia H, Lu L. Eca-cbam: Classification of diabetic retinopathy: Classification of diabetic retinopathy by cross-combined attention mechanism [C] / / 2022the 6th International Conference on Innovation in Artificial Intelligence (ICIAI). 2022:78-82); Farag's method (Farag MM, Fouad M, Abdel-Hamid A T. Automatic severity classification of diabetic retinopathy based on denseNet and convolutional block attention module [J]. IEEE Access, 2022, 10:38299-38308); Liang's method (Liang Liming, Dong Xin, Li Renjie, He Anjun. Retinopathy classification algorithm based on attention mechanism multifeature fusion [J]. Optoelectronics Engineering, 2023, 50(01):100-112); and Kamothi's algorithm (Kamothi N, Thakur). The results of this invention (R. Detection of diabetic retinopathy using transferlearning approach[J]. BIOINFOLET, 2021, 18(1a): 51-54) are compared and verified from both subjective and objective perspectives.

[0088] Table 1. Specific Indicators of the D-SCNet Network Comparison Experiment

[0089]

[0090] Diabetic retinopathy exhibits small inter-class differences and complex lesion characteristics, leading to insufficient extraction of retinal lesion features and resulting in low classification accuracy. The proposed D-SCNet model achieves Acc, Spe, and Rec accuracy of 88.20%, 97.08%, and 88.00%, respectively, demonstrating superiority over other existing models. Table 1 shows that the method implemented in this invention achieves an accuracy of 88.20% in identifying retinopathy, representing improvements of 1.6%, 0.86%, 3.79%, and 10.6% compared to Li, Farag, Liang, and Kamothi models, respectively.

[0091] Visualizing the main areas of focus of the D-SCNet model during diagnosis makes deep learning models no longer "black boxes". Figure 4 The image clearly shows that the network can assign sufficient weights to minute pathological features, allowing the network to focus its attention on the optic disc and lesion area (which are outlined in the figure).

[0092] If the image reconstruction method described in this embodiment of the invention is implemented as a software functional module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the image reconstruction method described in this embodiment of the invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, external hard drives, ROM, RAM, magnetic disks, or optical disks.

[0093] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for constructing an automatic classification model for diabetic retinopathy, characterized in that, Includes convolutional input, feature extraction, and classification output; The convolutional input, consisting of 7×7 convolution and downsampling, is responsible for resizing the input image to extract shallow features. The feature extraction comprises four alternately connected dense blocks and three transition layers, responsible for extracting feature information of retinal lesions; The dense block contains several dense layers SC-Bottleneck, which are designed to increase feature attention to high-level semantic information and focus on favorable areas of lesions; the dense block is intended to fully extract lesion features from retinal images. The four dense blocks contain 6, 12, 24, and 16 dense layers SC-Bottleneck, which are responsible for enhancing the network’s feature extraction of retinal lesions. The dense layer SC-Bottleneck is processed by BN and ReLU and then fed into a 1×1 convolution to fix the output channel and reduce the output dimension. Then it is processed by BN and ReLU and fed into a 3×3 convolution for feature extraction. Then, the SC attention module is used to increase the feature attention to high-level semantic information and improve the network's feature extraction of lesion information. The dense layer SC-Bottleneck, with the addition of the SC attention mechanism, purposefully focuses on the extracted retinal feature information in the spatial and channel dimensions, focusing on the favorable area of ​​the lesion, reducing the interference of complex background information on retinal lesions, and ensuring the network's accurate extraction of lesions. The classification output, consisting of a global average pooling layer and a fully connected layer, is responsible for outputting the classification results for diabetic retinopathy.

2. The method for constructing an automatic classification model for diabetic retinopathy according to claim 1, characterized in that, The SC attention module is designed to increase the focus on the number of yellow rings, bright red patches, and lesions in the optic disc area within the effective region; Each SC attention module consists of 3 attention layers and 1 convolutional module Conv+BN+ERelu. Conv+ECA+ERelu consists of 1×1 traditional convolution, BN normalization and ERelu activation function. The SC attention module has three attention layers, which contain 64, 16, and 8 1×1 convolutional kernels, respectively. The attention layer consists of 1×1 convolution, ECA, and ERelu. 1×1 convolution and ECA are used for complementary attention in mixed dimensions, while ERelu performs non-linear processing to prepare for the next input. The 1×1 convolution refines features in the spatial dimension, preserving the spatial information of the feature map without loss, and focusing on features in the spatial dimension. The attention mechanism shares its learning parameters in the form of a fast one-dimensional convolution kernel, which has a light size while achieving mutual attention between channels, forming complementarity in spatial information, improving the network's attention to lesion features, and further improving network performance. The convolution module adjusts the output of the original network through a 1×1 convolution to obtain the information weight distribution on a single spatial plane. Then, it performs a full multiplication operation with the original input feature map to obtain the weights of spatial and channel interaction. The network will purposefully focus on areas with large weights and learn useful features.

3. An automatic classification method for diabetic retinopathy, characterized in that, Follow these steps: S1. Input the preprocessed fundus retinal image into the convolutional layer to complete the extraction of shallow features and send it to the feature extraction module; S2. After the dense block in the feature extraction module completes the extraction of high-level semantic information of the input image, it is sent to the prediction output. The feature extraction module consists of four alternately connected dense blocks and three transition layers. The computation process of the dense layer SC-Bottleneck module within the dense block is as follows: (1) (2) (3) in, , Let 1, 2...n represent the input to the dense layer SC-Bottleneck. Indicates input through Convolution reduces the dimensionality of the output feature map. Indicates to conduct Feature maps extracted by convolution, This represents the feature map that is finally output after passing through the SC attention module; The SC attention module included in the SC-Bottleneck module is calculated according to the following formula. (4) (5) (6) (7) (8) in, , Let 1, 2...n represent the input to the SC attention module. Indicates input through convolution, ECA Perform the first layer of attention operation, number of convolution kernels C 1 is 64, Indicates to through convolution, ECA Perform second-layer attention extraction, convolution kernel number C 2 is 16, Indicates to Feature maps for the third-layer attention operation, number of convolution kernels C 3 is 8. Indicates input Perform convolution with kernel 1 Convolution operations yield information weight distributions on a single spatial plane. This indicates that the channel weights are multiplied by the original feature map. , Let 1, 2...n represent the features finally extracted by the SC attention module; S3, the classifier predicts and outputs the classification results for diabetic retinopathy.

4. An electronic device, characterized in that, The method described in claim 3 is used to achieve automatic classification of OCT images of diabetic retinopathy.

5. A computer storage medium, characterized in that, The storage medium stores at least one program instruction, which is loaded and executed by a processor to implement the OCT image automatic classification method for diabetic retinopathy as described in claim 3.

Images