Automatic segmentation method of diabetic retinopathy fundus images based on deep learning

Abstract

A deep learning-based automatic segmentation method for diabetic retinopathy fundus images includes the following steps: S1 resizing the diabetic retinopathy fundus image to 512*512, S2 starting image preprocessing, S3 passing the preprocessed image into a network model for node feature initialization, S4 feature transfer and fusion, S5 each node passing features to the upper layer node, segmenting the vascular structure in the image through deconvolution / interpolation / depooling methods, and when the last node of the first layer completes feature fusion, the resolution is restored to the same size as the original input image. S6 After the vascular structure segmentation is completed, a grayscale image of the vascular structure segmentation is output through a single layer of convolution. The method provides automatic segmentation of fundus vascular structures for medical assistance systems, solving the problems of slow image reading, unclear vascular ends, and incomplete segmentation caused by the easy loss of new small blood vessels. The present invention has the advantages of automatic extraction of fundus blood vessels, low computational resource consumption, and high segmentation accuracy.

Description

Technical Field

[0001] The present invention relates to a method for segmenting blood vessels in fundus images of diabetic retinopathy, and in particular to a method for automatic segmentation of fundus images of diabetic retinopathy based on deep learning, belonging to the field of medical image processing technology. Background Art

[0002] Diabetic retinopathy, also known as diabetic retinopathy, is the most common microvascular complication of diabetes. Among people with diabetes in my country, the prevalence of DR is 25% five years after diagnosis, rising to 75%-80% after 15 years. Over time, the disease worsens, with new, fragile blood vessels growing on the retinal surface. Rupture of these fragile vessels can cause intraocular bleeding, blurred vision, and, in severe cases, neovascular glaucoma, leading to permanent vision loss.

[0003] The purpose of medical image segmentation is to isolate the parts of a medical image that have special meaning. In the medical field, doctors mainly rely on medical film images and subjective judgment to diagnose diseases. Medical image segmentation is a key step in medical three-dimensional reconstruction and quantitative analysis. It is also an important technical prerequisite for quantifying lesion areas, selecting treatment methods, and radiotherapy. The accuracy of the segmentation results directly affects the subsequent treatment effect. At present, the main difficulties in medical image segmentation are as follows: the structure of medical tissue images is complex, and there are multiple imaging modalities, such as PET, MRI, CT, and ultrasound. Different segmentation methods should be selected according to different imaging methods and imaging sites. Another point is that medical images, unlike natural images, are more complex. Due to the single pixel of the image, the boundaries, shape, and other information of the lesion site are also relatively vague. Accurate segmentation of medical images remains a difficult problem.

[0004] Vascular structure in retinal fundus images is crucial for early detection of ophthalmic diseases. However, labeling and extracting vascular structures in diseased eyes is a time-consuming, labor-intensive, inefficient, and labor-intensive task. The recent rise of artificial intelligence, particularly the rapid development of neural networks, has provided new insights into the field of medical image processing.

[0005] Retinal vessel segmentation and diabetic retinopathy screening are mainly completed by analyzing fundus images. The early features of diabetic retinopathy are not easy to detect by observation, and the late stage of the disease will cause serious damage to the patient's vision. The retinal vessel segmentation work is extremely demanding and time-consuming.

[0006] Relying solely on manual labor is difficult. Therefore, the automatic extraction of vascular structure by the diabetic retinopathy vascular segmentation algorithm can greatly reduce the burden on the medical system, improve the work efficiency of medical personnel, and provide more extensive services to diabetic patients in various places, which has great practical significance. Summary of the Invention

[0007] A deep learning-based automatic segmentation method for diabetic retinopathy fundus images provides medical assistance systems with automatic segmentation of fundus vascular structures. It relies on extraction channels and filtering to improve the feature extraction effect of images, and extracts vascular structures through a neural network model. This can effectively simplify doctors' diagnosis and treatment of diabetic retinopathy and improve work efficiency.

[0008] The technical solution of the present invention is achieved as follows:

[0009] A deep learning-based automatic segmentation method for diabetic retinopathy fundus images includes the following steps:

[0010] S1. Resize the diabetic retinopathy fundus image to 512*512;

[0011] S2. Start image preprocessing: (1) extract the green channel portion; (2) remove the low-frequency component. Removing the low-frequency component means subjecting the original image to a 25*25 mean filter and subtracting the mean-filtered image from the green channel image; (3) perform linear filtering; (4) perform morphological erosion to eliminate noise;

[0012] S3. The preprocessed image is passed into the network model for node feature initialization: The network model is improved based on the Unet++ network model. It is a fan-shaped structure with five layers of upper and lower layers composed of residual blocks as units. Each residual block is regarded as a node. The first residual block on the left side of each layer is the encoder node. There are 5 encoder nodes in total, and the rest are decoder nodes. The first node on the left side of the first layer of the network model is the entrance of the entire model, and the last node on the right side of the first layer is the model exit. First, the preprocessed image is input into the first-layer encoder node of the network model for feature extraction. The extracted features will be sent to all decoder nodes in the same layer and the encoder nodes in the next layer. Then, the second-layer encoder node continues to extract features after receiving the features passed by the upper-layer encoder node, and passes the extracted features to all decoder nodes in the same layer and the encoder nodes in the next layer. This process is repeated to complete the feature initialization of all nodes in the network model.

[0013] S4, feature transfer and fusion: After all nodes complete feature initialization, each node performs feature fusion and feature transfer in real time. All nodes except the first layer perform the upsampling process of feature fusion. From the first node of the second layer to the last node of the fifth layer, each j-th node in the i-th layer will transfer the features extracted by this node to the j+1-th node of the upper layer, that is, the i-1-th layer. After receiving the features, the j+1-th node of the i-1-th layer performs a splicing operation with the original features and passes the spliced features to all nodes on the right side of the node in the same layer. After continuous real-time transfer and fusion, until it is transferred to the last node on the right side of the first layer, the features tend to be stable.

[0014] S5. Each node passes features to the upper layer node. The features of this layer are used to segment the blood vessel structure in the image through deconvolution / interpolation / unpooling methods. When the last node of the first layer completes feature fusion, the resolution is restored to the same size as the original input image, and the blood vessel structure segmentation is completed.

[0015] S6. After the vascular structure segmentation is completed, a grayscale image of the vascular structure segmentation is output through a single-layer convolution.

[0016] Its feature is also that the residual block in the S2 refers to a structure in which the residual module, channel attention and spatial attention are combined in series; the residual module extracts features; the spatial attention adjusts the position of the effective part of the image in the overall image, allowing the effective part to occupy the middle of the image, amplifying and filling the entire image; the channel attention focuses on the meaningful features in the feature map; for the overall structure of the model, there is no difference between the encoder node performing feature extraction first and adjusting the image first, and the order of the three parts of the residual module, channel attention and spatial attention can be adjusted arbitrarily.

[0017] It is also characterized in that the encoder node and the decoder node in S3 receive the residual block of the feature, and the convolution method uses a void convolution to increase the receptive field range without increasing the consumption of computing resources.

[0018] Compared with the prior art, the advantages of the present invention are obvious, mainly manifested in:

[0019] 1. A method for automatic segmentation of fundus blood vessels is proposed to enable the medical assistance system to automatically extract fundus blood vessels.

[0020] 2. Changes in model structure and convolution methods reduce resource consumption during training and calculation, and significantly improve speed.

[0021] 3. Different convolution methods of the upsampling structure greatly improve the segmentation effect of the blood vessel terminals and are very effective in recovering the new blood vessels at the terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The present invention has the following Figure 2 width.

[0023] Figure 1 This is a flow chart of the method for automatic segmentation of diabetic retinopathy fundus images based on deep learning of the present invention;

[0024] Figure 2 It is a schematic diagram of the network model structure. DETAILED DESCRIPTION

[0025] like Figure 1 、 2 The method for automatic segmentation of fundus images of diabetic retinopathy based on deep learning is shown, comprising the following steps:

[0026] S1. Resize the diabetic retinopathy fundus image to 512*512;

[0027] S2, start image preprocessing: (1) extract the green channel part; (2) remove the low frequency component, remove the low frequency component

[0028] The amount refers to making the original image pass 25*25 mean filtering, and subtracting the mean filtered image from the green channel image; (3) performing linear filtering; (4) morphological corrosion to eliminate noise;

[0029] S3. The pre-processed image is passed into the network model for node feature initialization: the network model is improved based on the Unet++ network model, and is a fan-shaped structure with five layers consisting of residual blocks as units.

[0030] Each residual block is regarded as a node. The first residual block on the left side of each layer is the encoder node. There are 5 encoder nodes in total, and the rest are decoder nodes. The first node on the left side of the first layer of the network model is the entrance of the entire model, and the last node on the right side of the first layer is the model exit. First, the preprocessed image is input into the first-layer encoder node of the network model for feature extraction. The extracted features will be sent to all decoder nodes in the same layer and the encoder nodes in the next layer. Then, after receiving the features passed by the upper-layer encoder nodes, the second-layer encoder nodes continue to extract features and pass the extracted features to all decoder nodes in the same layer and the encoder nodes in the next layer. This process is repeated to complete the feature initialization of all nodes in the network model.

[0031] S4, feature transfer and fusion: After all nodes complete feature initialization, each node performs feature fusion and feature transfer in real time. All nodes except the first layer perform the upsampling process of feature fusion. From the first node of the second layer to the last node of the fifth layer, each j-th node in the i-th layer will transfer the features extracted by this node to the j+1-th node of the upper layer, that is, the i-1-th layer. After receiving the features, the j+1-th node of the i-1-th layer performs a splicing operation with the original features and passes the spliced features to all nodes on the right side of the node in the same layer. After continuous real-time transfer and fusion, until it is transferred to the last node on the right side of the first layer, the features tend to be stable.

[0032] S5. Each node passes features to the upper layer node. The features of this layer are used to segment the blood vessel structure in the image through deconvolution / interpolation / unpooling methods. When the last node of the first layer completes feature fusion, the resolution is restored to the same size as the original input image, and the blood vessel structure segmentation is completed.

[0033] S6. After the vascular structure segmentation is completed, a grayscale image of the vascular structure segmentation is output through a single-layer convolution.

[0034] The residual block in S2 refers to a structure in which the residual module, channel attention and spatial attention are combined in series; the residual module extracts features; the spatial attention adjusts the position of the effective part of the image in the entire image, so that the effective part occupies the middle of the image, amplifies and fills the entire image; the channel attention focuses on the meaningful features in the feature map; for the overall structure of the model, there is no difference between the encoder node performing feature extraction first and adjusting the image first, and the order of the residual module, channel attention and spatial attention can be adjusted arbitrarily.

[0035] The encoder node and decoder node in S3 receive the residual block of the feature, and the convolution method uses a dilated convolution to increase the receptive field range without increasing the consumption of computing resources.

Claims

1. A deep learning-based automatic segmentation method for diabetic retinopathy fundus images, characterized in that: The steps include: S1. Resize the diabetic retinopathy fundus image to 512*512; S2. Start image preprocessing: (1) extract the green channel portion; (2) remove the low-frequency component. Removing the low-frequency component means subjecting the original image to a 25*25 mean filter and subtracting the mean-filtered image from the green channel image; (3) perform linear filtering; (4) perform morphological erosion to eliminate noise; S3. The preprocessed image is passed into the network model for node feature initialization: The network model is improved based on the Unet++ network model. It is a fan-shaped structure with five layers of upper and lower layers composed of residual blocks as units. Each residual block is regarded as a node. The first residual block on the left side of each layer is the encoder node. There are 5 encoder nodes in total, and the rest are decoder nodes. The first node on the left side of the first layer of the network model is the entrance of the entire model, and the last node on the right side of the first layer is the model exit. First, the preprocessed image is input into the first-layer encoder node of the network model for feature extraction. The extracted features will be sent to all decoder nodes in the same layer and the encoder nodes in the next layer. Then, the second-layer encoder node continues to extract features after receiving the features passed by the upper-layer encoder node, and passes the extracted features to all decoder nodes in the same layer and the encoder nodes in the next layer. This process is repeated to complete the feature initialization of all nodes in the network model. S4, feature transfer and fusion: After all nodes complete feature initialization, each node performs feature fusion and feature transfer in real time. All nodes except the first layer perform the upsampling process of feature fusion. From the first node of the second layer to the last node of the fifth layer, each j-th node in the i-th layer will transfer the features extracted by this node to the j+1-th node of the upper layer, that is, the i-1-th layer. After receiving the features, the j+1-th node of the i-1-th layer performs a splicing operation with the original features and passes the spliced features to all nodes on the right side of the node in the same layer. After continuous real-time transfer and fusion, until it is transferred to the last node on the right side of the first layer, the features tend to be stable. S5. Each node passes features to the upper layer node. The features of this layer are used to segment the blood vessel structure in the image through deconvolution / interpolation / unpooling methods. When the last node of the first layer completes feature fusion, the resolution is restored to the same size as the original input image, and the blood vessel structure segmentation is completed. S6. After the vascular structure segmentation is completed, a grayscale image of the vascular structure segmentation is output through a single-layer convolution.

2. The method for automatic segmentation of diabetic retinopathy fundus images based on deep learning according to claim 1 is further characterized by The residual block in S2 refers to a structure in which the residual module, channel attention, and spatial attention are combined in series; the residual module extracts features; Spatial attention adjusts the position of the effective part of the image in the entire image, allowing the effective part to occupy the middle of the image, enlarge and fill the entire image; channel attention focuses on the meaningful features in the feature map; for the overall structure of the model, there is no difference between the encoder node performing feature extraction first and adjusting the image first, and the order of the three parts of the residual module, channel attention and spatial attention can be adjusted arbitrarily.

3. The method for automatic segmentation of diabetic retinopathy fundus images based on deep learning according to claim 1, further comprising: The encoder node and decoder node in S3 receive the residual block of the feature, and the convolution method uses a dilated convolution to increase the receptive field range without increasing the consumption of computing resources.

Images