Optic disc extraction method from retinal images based on multi-attention U-Net fully convolutional network

Abstract

This invention relates to a method for optic disc extraction in retinal images based on a multi-attention U-Net fully convolutional network, comprising the following steps: Step S1: Acquire the original color retinal image and extract the region of interest (ROI) using the method of extracting the brightest area and blurring the brightest area; Step S2: Based on the extracted ROI image, perform RGB channel separation and histogram equalization and normalization on the separated red channel image; Step S3: Perform channel fusion on the red channel image, the histogram equalized image, and the normalized image to generate a new three-channel image; Step S4: Construct and train a retinal image optic disc segmentation model based on the multi-attention U-Net fully convolutional network; Step S5: Input the channel-fused image data into the retinal image optic disc segmentation model for retinal image optic disc extraction and segmentation. This invention achieves high-precision extraction and segmentation of the optic disc in retinal images.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method for extracting optic disc from retinal images based on a multi-attention U-Net fully convolutional network. Background Art

[0002] By observing the state of the retinal optic disc, doctors can obtain partial state information of the human body non-invasively. However, the information of the retinal optic disc generally needs to be extracted through retinal image segmentation technology, which is an image processing problem. Summary of the invention

[0003] In view of this, the purpose of the present invention is to provide a retinal image optic disc extraction method based on a multi-attention U-Net fully convolutional network, which realizes high-precision extraction and segmentation of the retinal image optic disc.

[0004] To achieve the above object, the present invention adopts the following technical solution:

[0005] A method for extracting optic disc from retinal images based on a multi-attention U-Net fully convolutional network comprises the following steps:

[0006] Step S1: obtaining an original color retinal image, and extracting a region of interest by using a method of extracting the brightest point and regionally blurring the brightest area;

[0007] Step S2: according to the extracted region of interest image, RGB channel separation is performed, and the separated red channel image is subjected to histogram equalization and standardization processing;

[0008] Step S3: performing channel fusion on the red channel image, the histogram equalization image and the standardized image to generate a new three-channel image;

[0009] Step S4: construct and train a retinal image optic disc segmentation model based on a multi-attention U-Net fully convolutional network;

[0010] Step S5 inputs the channel-fused image data into the retinal optic disc segmentation model to perform retinal image optic disc extraction and segmentation.

[0011] Furthermore, the step S2 is specifically as follows:

[0012] Step S21: separating the red channel according to the characteristic that the red channel has the strongest discrimination degree for the optic disc area;

[0013] Step S22: performing histogram equalization on the separated red channel image to further enhance the contrast;

[0014] Step S23: Standardize the separated red channel image to centralize the image pixel values.

[0015] Furthermore, the multi-attention U-Net fully convolutional network adopts the U-Net network model architecture, specifically: a spatial attention module is used to learn information on the spatial components of the image, a channel attention module is used to learn information on the channel components of the image and a modular DropBlock is added to alleviate the overfitting problem in the training process; the network includes a modular DropBlock, a batch normalization layer and a RELU activation function, a pooling layer, upsampling and short connections; after the image is processed by the network, a predicted segmentation image with a binary channel of 1 and the same size as the input data is generated, and the predicted segmentation image is used to calculate the output value of the loss function with the real segmentation image.

[0016] Furthermore, the retinal image optic disc segmentation model is obtained by training the image data generated by channel fusion after ROI extraction of the training set data in a multi-attention U-Net full convolutional network, specifically:

[0017] First, the pre-set parameter information is read at the beginning of model training;

[0018] Then, the input image list is read. Since the images are binarized when input into the network after preprocessing, the training samples are input into the network to predict the segmented image after the optimizer optimizes the learning rate, and the loss function value between the segmented image and the true image is calculated. The binary cross entropy loss function is used, and in order to alleviate the imbalance problem of samples, the Sigmoid function is used for pixel mapping.

[0019] Furthermore, the multi-attention U-Net full convolutional network uses a binary cross entropy loss function, and the pixel mapping adopts a Sigmoid function. The loss function is expressed as:

[0020]

[0021] Among them, P0 represents the probability of a pixel point being a optic disc structure under the true value image, P1 represents the probability of a pixel point being a non-optic disc structure under the true value image, N is the total number of pixels, is the probability that the pixel is output as a disc pixel under the Sigmoid function mapping, It is the probability that the pixel point is output as a non-disc pixel under the Sigmoid function mapping.

[0022] Compared with the prior art, the present invention has the following beneficial effects:

[0023] 1. The present invention uses ROI to extract the optic disc area and then extracts the red channel, obtains the histogram equalization image and the standardized image, and then fuses the three single-channel images. The above preprocessing of the image data can effectively enhance the contrast between the background and the optic disc structure, laying the foundation for the subsequent accurate segmentation of the optic disc blood vessels;

[0024] 2. The present invention adopts a dual attention mechanism module, namely a spatial attention module and a channel attention module, which makes fuller use of image spatial information and channel information, and introduces DropBlock in the convolution block, which can effectively prevent model misjudgment and overfitting problems caused by insufficient data;

[0025] 3. The multi-attention U-Net network adopted in the present invention fully utilizes the color information of the retinal image and combines it with the preprocessing strategy to achieve high-precision extraction and segmentation of the optic disc of the retinal image. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 is a flow chart of the method of the present invention;

[0027] Figure 2 It is a test flow chart in one embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0029] Please refer to Figure 1 The present invention provides a retinal disc segmentation method based on a multi-attention U-shaped fully convolutional neural network with ROI extraction and channel fusion, characterized in that it comprises the following steps:

[0030] Step S1: obtaining an original color retinal image, and extracting a region of interest by using a method of extracting the brightest point and regionally blurring the brightest area;

[0031] Step S2: according to the extracted region of interest image, RGB channel separation is performed, and the separated red channel image is subjected to histogram equalization and standardization processing;

[0032] Step S3: performing channel fusion on the red channel image, the histogram equalization image and the standardized image to generate a new three-channel image;

[0033] Step S4: construct and train a retinal image optic disc segmentation model based on a multi-attention U-Net fully convolutional network;

[0034] Step S5 inputs the channel-fused image data into the retinal optic disc segmentation model to perform retinal image optic disc extraction and segmentation.

[0035] Preferably, in this embodiment, the specific implementation method of using the preprocessing of extracting the red channel and combining histogram equalization and standardization to perform three-channel fusion to enhance the retinal image is: the original image is separated into three channels, the red channel image sensitive to the optic disc area is extracted, and then the red channel image is histogram equalized to achieve a larger color difference. Therefore, the red channel image is standardized to make the pixel data centralized and more in line with the data distribution law, enhance the generalization effect, and then the three single-channel preprocessed images are respectively used as independent channel image data for multi-channel fusion through channel fusion technology, which, on the one hand, enhances the contrast between the optic disc area and its surrounding tissue structure, and on the other hand, reduces the interference of non-optic disc structures on the learning of the optic disc area.

[0036] Preferably, the multi-attention U-Net fully convolutional network that is sensitive to multi-channel information uses the U-Net network model architecture to learn information on the spatial components of the image through a spatial attention module, uses a channel attention module to learn information on the channel components of the image, and adds a modular DropBlock to alleviate the overfitting problem in the training process; the network structure performs independent convolution kernel operations on the three channel components, and the network includes a modular DropBlock, a batch normalization layer and a RELU activation function, a pooling layer, upsampling and short connections; after the image is processed by the network, a binary predicted segmentation image with a channel of 1 and the same size as the input data is generated, and the predicted segmentation image is used to calculate the output value of the loss function with the true segmentation image.

[0037] Preferably, the multi-attention U-Net full convolutional network that is sensitive to multi-channel information uses a binary cross entropy loss function, and the mapping of pixels uses a Sigmoid function. The loss function is expressed as:

[0038]

[0039] Among them, P0 represents the probability of a pixel point being a optic disc structure under the true value image, P1 represents the probability of a pixel point being a non-optic disc structure under the true value image, N is the total number of pixels, is the probability that the pixel is output as a disc pixel under the Sigmoid function mapping, It is the probability that the pixel point is output as a non-disc pixel under the Sigmoid function mapping.

[0040] In this embodiment, the training process of the retinal image optic disc extraction method based on the multi-attention U-shaped full convolutional neural network with ROI extraction and channel fusion includes the following steps:

[0041] First, the training set images are extracted from ROI, the brightest point of the image is detected, and Gaussian blur is performed. The brightest area is determined by finding the brightest point, so as to obtain the ROI image of the optic disc; the ROI image is then channel-separated to extract the red channel image, and the red channel image is further enhanced by histogram equalization and standardization. Then, the red channel image, histogram equalization image and standardized image are respectively treated as a single-channel image for channel fusion using channel fusion technology to synthesize a high-contrast three-channel image. Finally, the model is trained through the multi-attention U-Net fully convolutional network to generate a retinal image optic disc segmentation model.

[0042] refer to Figure 2 ,In the testing step, after preprocessing, the retinal image optic disc segmentation model obtained by training ,is used to achieve the accurate extraction and segmentation of the retinal image optic disc structure.

[0043] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A retinal image optic disc extraction method based on a multi-attention U-Net fully convolutional network, characterized in that: The following steps are involved: Step S1: obtaining an original color retinal image, and extracting a region of interest by using the brightest point extraction and regional blurring of the brightest area method; Step S2: according to the extracted region of interest image, RGB channel separation is performed, and the separated red channel image is subjected to histogram equalization and standardization processing; Step S3: performing channel fusion on the red channel image, the histogram equalization image and the standardized image to generate a new three-channel image; Step S4: construct and train a retinal image optic disc segmentation model based on a multi-attention U-Net fully convolutional network; Step S5: inputting the channel-fused image data into the retinal optic disc segmentation model to perform retinal image optic disc extraction and segmentation; The step S2 is specifically as follows: Step S21: separating the red channel according to the characteristic that the red channel has the strongest discrimination degree for the optic disc area; Step S22: performing histogram equalization on the separated red channel image to further enhance the contrast; Step S23: standardize the separated red channel image to centralize the image pixel values; The multi-attention U-Net fully convolutional network adopts the U-Net network model architecture, specifically: a spatial attention module is used to learn information on the spatial components of the image, a channel attention module is used to learn information on the channel components of the image, and a modular DropBlock is added to alleviate the overfitting problem in the training process; the network includes a modular DropBlock, a batch normalization layer and a RELU activation function, a pooling layer, upsampling and a short connection; after the image is processed by the network, a binary predicted segmentation image with a channel of 1 and the same size as the input data is generated, and the predicted segmentation image is used to calculate the output value of the loss function with the real segmentation image; The retinal image optic disc segmentation model is obtained by training the image data generated by ROI extraction of the training set data and then channel fusion in a multi-attention U-Net full convolutional network, specifically: First, the pre-set parameter information is read at the beginning of model training; Then, the input image list is read. Since the images are binarized after preprocessing when they are input into the network, after the optimizer optimizes the learning rate, the training samples are input into the network to predict the segmented image, and the loss function value between it and the true value image is calculated. The binary cross entropy loss function is used, and in order to alleviate the imbalance problem of samples, the Sigmoid function is used for pixel mapping. The multi-attention U-Net full convolutional network uses a binary cross entropy loss function, and the pixel mapping uses a Sigmoid function. The loss function is expressed as: Among them, P 0n It represents the probability that pixel n is a pixel of the optic disc structure under the true value map, P 1n It indicates the probability that pixel n is a non-disc structure pixel under the true value image, N is the total number of pixels, is the probability that the pixel n is output as a disc pixel under the Sigmoid function mapping, is the probability that the pixel n is output as a non-disc pixel under the Sigmoid function mapping.

Images