A method for grading nonproliferative diabetic retinopathy

Abstract

The present application relates to a kind of non-proliferative diabetic retinopathy grading model training methods, comprising: obtaining and data set is divided into training set and test set, data set includes fundus image;Fundus disc positioning and blood vessel positioning are carried out to fundus image, generate fundus disc area and blood vessel area;According to fundus disc area and blood vessel area, remove interference information in fundus image, generate the pretreatment lesion map of each fundus image;Resnet50 neural network is established, and the pretreatment lesion map of training set and test set is used to train neural network.The method of the present application can realize the automatic grading of non-proliferative diabetic retinopathy, and by removing fundus disc and blood vessel to reduce the interference to exudation, microaneurysm and blood vessel information extraction, improve grading accuracy.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of medical treatment, and particularly relates to a grading method for non-proliferative diabetic retinopathy. BACKGROUND

[0002] Diabetic retinopathy (DR) is one of the most common microvascular complications of diabetes, and is a series of ocular fundus lesions caused by retinal microvascular leakage and obstruction due to chronic progression of diabetes, such as microvascular, hard exudation, cotton wool spots, neovascularization, vitreous proliferation, macular edema and even retinal detachment. DR is divided into proliferative diabetic retinopathy and non-proliferative diabetic retinopathy according to whether there are abnormal neovascularization from the retina.

[0003] Most of the existing grading of non-proliferative diabetic retinopathy is achieved by manual inspection, but the efficiency is low and the human resources are consumed, so a grading method for non-proliferative diabetic retinopathy is needed. SUMMARY

[0004] Based on the above-mentioned shortcomings and deficiencies in the prior art, one of the purposes of the present application is to at least solve one or more of the above-mentioned problems in the prior art, in other words, one of the purposes of the present application is to provide a grading method for non-proliferative diabetic retinopathy which meets one or more of the above-mentioned needs.

[0005] In order to achieve the above-mentioned purposes of the application, the following technical solutions are adopted:

[0006] A training method for a non-proliferative diabetic retinopathy grading model, the method comprising the following steps:

[0007] S1, obtaining a plurality of data sets, and dividing the plurality of data sets into a training set and a test set, each data set comprising a plurality of fundus images of non-proliferative diabetic retinopathy;

[0008] S2, performing fundus optic disc positioning and blood vessel positioning on the fundus images to generate fundus optic disc regions and blood vessel regions;

[0009] S3, removing interference information in the fundus images according to the fundus optic disc regions and the blood vessel regions to generate a preprocessed lesion image of each fundus image; and S4, establishing a Resnet50 neural network, and training the neural network using the preprocessed lesion images of the training set and the test set.

[0010] As a preferred solution, the fundus optic disc positioning is specifically: using Hough space transformation and Canny edge detection to position the fundus images to obtain the fundus optic disc regions;

[0011] The blood vessel positioning specifically comprises: using morphological operation to locate the fundus image to obtain a blood vessel region.

[0012] As a preferred solution, the step S3 specifically comprises the following steps:

[0013] S31, selecting a point a with the highest replacement priority on the edge of the fundus optic disc region and the blood vessel region;

[0014] S32, searching for a best matching block with the highest matching degree in a to-be-replaced region centered on the point a, and covering the to-be-replaced region with the best matching block;

[0015] S33, repeating the above steps until the fundus optic disc region and the blood vessel region are completely replaced to obtain a preprocessed lesion map.

[0016] As a preferred solution, the search matching formula of the step S32 is specifically:

[0017]

[0018] Wherein, ψ b is the best matching block obtained by searching and matching, ψ a is the to-be-replaced region, ψ q is a sample block in the searching process, and d(.) represents the sum of squares of pixel color differences corresponding to and known in the target block ψ a and the sample block ψ q . The definition is as follows:

[0019] d(ψ a , ψ q ) = ∑ [(I R -I′ R ) 2 +(I G -I′ G ) 2 +(I B -I′ B ) 2 ];

[0020] I and I' correspond to pixel points in ψ a and ψ q , respectively.

[0021] As a preferred solution, the Resnet50 neural network specifically comprises a data input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a self-attention module, a global average pooling layer, and a full connection layer connected in sequence.

[0022] As a preferred solution, the ratio of the training set and the test set is 7:3.

[0023] As a preferred solution, the step S4 further comprises a step S40 before the step S4:

[0024] The pre-processed lesion maps are subjected to data augmentation, so that the number of pre-processed lesion maps for each lesion classification is the same.

[0025] In another aspect, the present application also provides a non-proliferative diabetic retinopathy grading algorithm, which uses the model trained according to any one of the above methods, and specifically comprises the following steps:

[0026] S1, performing fundus optic disc positioning and blood vessel positioning on the fundus image to generate a fundus optic disc region and a blood vessel region;

[0027] S2, removing interference information in the fundus image according to the fundus optic disc region and the blood vessel region to generate a pre-processed lesion map of the fundus image;

[0028] S3, using the model to predict the retinal lesion grading of the pre-processed lesion map.

[0029] In another aspect, the present application also provides a non-proliferative diabetic retinopathy grading system, which comprises:

[0030] A fundus optic disc positioning module for performing fundus optic disc positioning on the fundus image to generate a fundus optic disc region;

[0031] A blood vessel positioning module for performing blood vessel positioning on the fundus image to generate a blood vessel region;

[0032] A preprocessing module for removing interference information in the fundus image according to the fundus optic disc region and the blood vessel region to generate a pre-processed lesion map of the fundus image;

[0033] A model trained according to any one of the above methods for grading the pre-processed lesion map.

[0034] Compared with the prior art, the present application has the following beneficial effects:

[0035] The method of the present application can realize automatic grading of non-proliferative diabetic retinopathy;

[0036] The method of the present application removes the fundus optic disc and blood vessels to reduce interference with the extraction of exudation, microaneurysm and blood vessel information, and uses a neural network to extract lesion information, uses an attention mechanism to enable the network to filter out a small amount of important information from a large amount of information obtained and focus on these important information, which can effectively improve the grading effect of NPDR. BRIEF DESCRIPTION OF DRAWINGS

[0037] Figure 1 is a training flowchart of the non-proliferative diabetic retinopathy grading model of the embodiment of the present application;

[0038] Figure 2 is a fundus image of the embodiment of the present application;

[0039] Figure 3 is a pretreatment lesion map of an embodiment of the present application;

[0040] Figure 4 is a schematic diagram of step S3 of an embodiment of the present application;

[0041] Figure 5 is a connection structure diagram of a neural network of an embodiment of the present application. DETAILED DESCRIPTION

[0042] In order to more clearly illustrate the embodiments of the present application, the specific embodiments of the present application will be described below with reference to the accompanying drawings. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings and other embodiments can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0043] Embodiment: The embodiment provides a non-proliferative diabetic retinopathy grading method, a model capable of grading non-proliferative diabetic retinopathy is obtained through training, and then the model is used to grade the image of the retinopathy, so as to realize automatic grading of non-proliferative diabetic retinopathy.

[0044] The training method of the model is as shown in Figure 1 The method is trained by the following steps:

[0045] S1, obtain a plurality of data sets, the data sets can be obtained from public data sets, such as using the 2018 IDRID competition public data set as the data set for training, the data set includes a plurality of color fundus images respectively belonging to four categories of diabetic retinopathy of 0, 1, 2 and 3.

[0046] 30% of the obtained data set is selected as a test set, and the remaining 70% is selected as a training set. The division method is random sampling combination to reduce the random error of the network.

[0047] S2, the fundus image in the training set is positioned for the fundus optic disc and blood vessels, and the fundus optic disc area and blood vessel area of the training set are generated. The fundus optic disc and blood vessels have certain interference to the image of the retina, so it is not conducive to the extraction of lesion information, and positioning is performed here for subsequent removal.

[0048] Specifically, for the fundus image as shown in Figure 2 Hough space transformation and Canny edge detection method are used to pre-process the fundus image to obtain the fundus optic disc area, and the positioning of the fundus optic disc is completed. The blood vessel area is obtained by morphological operation on the fundus image, and the positioning of the blood vessel is completed.

[0049] S3, after the positioning of the interference information is completed, the interference information is removed according to the optic disc region and the blood vessel region, and each fundus image in the training set is converted into a preprocessed lesion image for detecting retinopathy, and the image of the preprocessed lesion image is as shown in Figure 3 .

[0050] Specifically, the present step is implemented by using the following steps:

[0051] S31, as shown in the schematic diagram of Figure 4 , Figure 4 The shadow area Φ in the figure represents a known area, that is, a normal fundus background area which is not blocked by the optic disc and blood vessels; Ω represents a region to be matched, that is, a blocked region which is blocked by the optic disc and blood vessels and is waiting to be replaced by the normal fundus background area; and 6Ω is the edge of the region to be matched, that is, the boundary edge of the optic disc, blood vessels and the normal fundus background.

[0052] The priority of each position is calculated in the above edge to obtain a point a with the highest priority;

[0053] P(a) = C(a)

[0054]

[0055] Wherein, C(a) is a confidence term, representing the confidence pixels of the target ψ a in the Φ area, |ψ a | represents the area of ψ a , and the greater C(a) is, the higher the priority is. The point a obtained by using the above formula is a point with the largest proportion of normal fundus background area in the template block ψ a .

[0056] S32, after the point a with the highest priority is found, the SSD matching principle finds the best matching block ψ b in the normal fundus background area which is most similar to ψ a , and the information in ψ b is copied and replaced by the information in the target block ψ a .

[0057] Specifically, the formula for searching the match is as follows:

[0058]

[0059] Wherein, d(.) represents the sum of squares of the color difference of the corresponding known pixels in the target block ψ a and the sample block ψ q . The definition is as follows:

[0060] d(ψ a , ψ q ) = ∑[(IR -I′ R ) 2 +(I G -I′ G ) 2 +(I B -I′ B ) 2 ];

[0061] I, I' respectively correspond to the pixel points in ψ a and ψ q .

[0062] After the matching is completed, the matched ψ b is used to replace the target block ψ a , then C(a) is updated, the next highest priority a point is found, and the above matching and replacing steps are repeated until the area to be matched Ω = 0, and the entire fundus optic disc and blood vessel area is replaced by the fundus background image to obtain the preprocessed lesion image.

[0063] The above process starts from the edge of the fundus optic disc and blood vessel area, and gradually replaces each small area with the most similar normal fundus background image, thereby eliminating the fundus optic disc and blood vessel area from the fundus background image.

[0064] S4, a Resnet50 neural network is established, the Resnet50 neural network comprises a self-attention mechanism, and a connection structure thereof is as shown in Figure 5 Fig. 3, comprising a data input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a self-attention module, a global average pooling layer, and a full connection layer connected in sequence.

[0065] The neural network is trained using a loss function, the learning rate is set to 0.001, the network uses a batch size of 16, the preprocessed lesion images of the training set and the test set are used to perform 100 iterations of training on the neural network, and the test set is used for testing. The predicted classification of the test set by the neural network is compared with the true labels of the test set, the accuracy of the grading is calculated, and the training of the grading model is completed.

[0066] After the training of the grading model is completed by the above method, the model can be used to grade the images of diabetic retinopathy, and the specific grading method is:

[0067] S1, after the fundus image is photographed, the same pre-processing method as step S2 in the above model training method is used to position the fundus optic disc and blood vessels of the fundus image, and the fundus optic disc area and blood vessel area are generated;

[0068] Then, step S2 is performed, which is the same as step S3 in the above model training method, to find the interference information in the fundus image using the optic disc region and the blood vessel region, remove the interference information, and generate a preprocessed lesion map of the fundus image that is convenient for accurate prediction;

[0069] Finally, step S3 is performed, that is, the trained model is used to predict the preprocessed lesion map, so as to obtain the retinal lesion grading of the original fundus image.

[0070] The embodiment also provides a non-proliferative diabetic retinopathy grading system, and the system specifically comprises:

[0071] An optic disc positioning module is configured to perform the positioning of the optic disc of the fundus image in the above method and generate an optic disc region.

[0072] A blood vessel positioning module is configured to perform the positioning of the blood vessel of the fundus image in the above method and generate a blood vessel region.

[0073] A preprocessing module is configured to perform the method of step S3 above, remove the interference information in the fundus image according to the optic disc region and the blood vessel region, and generate a preprocessed lesion map of the fundus image.

[0074] The system further comprises a non-proliferative diabetic retinopathy grading model trained according to the above method, and the preprocessed lesion map is input into the model, and the retinal lesion grading result of the fundus image corresponding to the image is output by the model.

[0075] It should be noted that the above embodiment only details the preferred embodiment and principle of the present application, and for those skilled in the art, the specific implementation manner can be changed according to the idea provided by the present application, and these changes should also be considered as the protection scope of the present application.

Claims

1. A training method for a non-proliferative diabetic retinopathy grading model, characterized in that, The method comprises the following steps: S1, acquiring a plurality of data sets, and dividing the plurality of data sets into a training set and a test set, each of the data sets comprising a plurality of non-proliferative diabetic retinopathy fundus images; S2, performing fundus optic disc positioning and blood vessel positioning on the fundus images to generate fundus optic disc regions and blood vessel regions; S3, removing interference information in the fundus images according to the fundus optic disc regions and the blood vessel regions to generate a preprocessed lesion map of each of the fundus images; The step S3 specifically comprises the following steps: S31, selecting a point a with the highest replacement priority on the edge of the fundus optic disc region and the blood vessel region; a shadow region Φ represents a known region, that is, a normal fundus background region not blocked by the fundus optic disc and the blood vessels; Ω represents a region to be matched, that is, a blocked region blocked by the fundus optic disc and the blood vessels and waiting to be replaced by the normal fundus background region; and δΩ is the edge of the region to be matched, that is, the junction edge of the fundus optic disc, the blood vessels and the normal fundus background; The priority of each position in the above edge is calculated to obtain the point a with the highest priority; ， ； Wherein, C(a) is a confidence term, indicating the target The confidence pixel in the Φ area, Indicates The area of the point a, the greater the C(a), the higher the priority; the point a obtained by using the above formula is the point at the center of the template block The point with the largest proportion of normal fundus background area in the Φ area S32, searching for a best matching block with the highest matching degree in a region to be replaced centered on the point a, and covering the region to be replaced with the best matching block; the search matching formula is specifically: ； wherein, is the best matching block found by the search, is the region to be replaced, is the sample block during the search, said d(.) denotes the difference between the pixel colors of the target block and the sample block and the corresponding and known pixel color difference; defined as follows: ； I, I' corresponding to and pixel points, respectively; S33, repeating the above steps until the fundus optic disc region and the blood vessel region are completely replaced to obtain the preprocessed lesion map; S4, establishing a Resnet50 neural network, and training the neural network using the preprocessed lesion maps of the training set and the test set.

2. The non-proliferative diabetic retinopathy grading model training method of claim 1, wherein, The fundus optic disc positioning specifically comprises: using Hough space transformation and Canny edge detection to position the fundus images to obtain the fundus optic disc positioning region; The blood vessel positioning specifically comprises: using morphological operation to position the fundus images to obtain the blood vessel region.

3. The non-proliferative diabetic retinopathy grading model training method of claim 1, wherein, The Resnet50 neural network specifically comprises a data input layer, a first convolutional layer, a maximum pooling layer, a second convolutional layer, a self-attention module, a global average pooling layer and a full connection layer connected in sequence.

4. The non-proliferative diabetic retinopathy grading model training method of claim 1, wherein, The ratio of the training set to the test set is 7:

3.

5. The non-proliferative diabetic retinopathy grading model training method of claim 1, wherein, The step S4 further comprises a step S40 before the step S4: Data augmentation is performed on the preprocessed lesion map, so that the number of preprocessed lesion maps of each lesion classification is the same.

6. A method of grading nonproliferative diabetic retinopathy, characterized by, The model trained by the method of any one of claims 1-5 specifically comprises the following steps: S1, performing fundus optic disc positioning and blood vessel positioning on the fundus images to generate fundus optic disc regions and blood vessel regions; S2, removing interference information in the fundus images according to the fundus optic disc regions and the blood vessel regions to generate a preprocessed lesion map of each of the fundus images; S3, using the model to predict the retinal lesion grade of the preprocessed lesion map.

7. A nonproliferative diabetic retinopathy grading system characterized by, It comprises: a fundus optic disc positioning module, configured to perform fundus optic disc positioning on the fundus images to generate fundus optic disc regions; a blood vessel positioning module, configured to perform blood vessel positioning on the fundus images to generate blood vessel regions; a preprocessing module, configured to remove interference information in the fundus image according to the optic disc region and the blood vessel region, to generate a preprocessed lesion map of the fundus image; a model trained according to any one of the methods of claims 1-5, configured to grade the preprocessed lesion map.

Images