A high-precision retinal blood vessel segmentation network structure and method

Abstract

The application discloses a high-precision retinal blood vessel segmentation network structure and method, belongs to the technical field of image analysis, and solves the problem of low segmentation precision caused by weak feature expression ability of a retinal image segmentation method designed artificially, and the application comprises a deformable convolution module, a multi-scale adaptive detail feature fusion module and a double-path upsampling module; the deformable convolution module comprises an encoder, n Res-Block-1 modules and a decoder, wherein n is an element in a set [3, 4, 6], and the decoder comprises a decoder submodule; the multi-scale adaptive detail feature fusion module comprises a feature map module, an upsampling module, a downsampling module, a replication module and a transformation module; and the double-path upsampling module comprises a capture module, an information module and a fusion module. The application is used for relieving useful information loss, restoring image resolution and improving precision in the retinal blood vessel segmentation network.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image analysis, and particularly relates to a high-precision retinal blood vessel segmentation network structure and method. BACKGROUND

[0002] The fundus image can reflect the retinal blood vessel structure, is a non-invasive way to observe human blood vessels, and can analyze the retinal blood vessel structure, which is helpful for the diagnosis of ophthalmic diseases such as diabetic retinopathy and age-related macular degeneration. Diabetic retinopathy is a microvascular complication of diabetes, and high blood sugar leads to swelling, leakage and occlusion of retinal blood vessels, which often leads to blindness; impaired microvascular reactivity and blood flow disorder are often considered to be the cause of diabetes complications. The feature of age-related macular degeneration is that the choroidal blood vessel structure grows from the inside to the macula, and is accompanied by increased vascular permeability and permeability, which leads to an increase in abnormal fluid in the retina or below, and when the abnormal fluid reaches the center of the macula, it will affect the human visual system. Automatic segmentation of the fundus image can observe the retinal blood vessel width, bending degree, blood vessel pattern and other shapes, providing an important basis for disease diagnosis. In summary, observation and analysis of retinal blood vessels are helpful for early diagnosis of ophthalmic diseases, so many researchers have studied retinal blood vessel detection algorithms to assist doctors in diagnosing diseases.

[0003] Due to the complex background of the fundus image and the large individual differences of the blood vessel structure, automatic segmentation of the retinal blood vessel is a very challenging task. The retinal blood vessels are intertwined and complex in structure, and the shape is extremely irregular, with very small ends. In addition, the fundus image has large noise, low foreground and background contrast, and uneven illumination, making it difficult for traditional algorithms to segment the retinal blood vessels from the image. Traditional segmentation algorithms not only need to design features manually, but also have weak feature expression ability, so that the fundus image segmentation accuracy cannot meet the diagnosis requirements. Although the current deep learning method can effectively capture the low-level detail features and high-level semantic features of the image, it has certain advantages in image segmentation under complex background, but the accuracy is not enough. SUMMARY

[0004] The purpose of the present application is:

[0005] To solve the problem of low segmentation accuracy caused by the weak feature expression ability of the retinal image segmentation method in the prior art, a high-precision retinal blood vessel segmentation network structure and method are provided.

[0006] The technical scheme adopted by the present application is as follows:

[0007] A high-precision retinal blood vessel segmentation network structure comprises a deformable convolution module, a multi-scale adaptive detail feature fusion module and a double-path upsampling module.

[0008] The deformable convolution module comprises an encoder, n Res-Block-1 modules, and a decoder, wherein n [3, 4, 6], and the decoder comprises a decoder submodule;

[0009] The multi-scale adaptive detail feature fusion module comprises a feature map module, an up-sampling module, a down-sampling module, a replication module, and a transformation module.

[0010] The dual-path up-sampling module comprises a capture module, an information module, and a fusion module.

[0011] Further, the encoder is a resnet34 model.

[0012] Further, the information module comprises spatial information and channel information.

[0013] A high-precision retinal blood vessel segmentation network method using the high-precision retinal blood vessel segmentation network structure described above, comprising:

[0014] On the basis of the U-Net network, deformable convolution, a multi-scale adaptive detail feature fusion module, and a dual-path up-sampling module are added to fuse multi-scale detail features, up-sample the feature map in space and channels, and relieve the loss of information in the up-sampling process.

[0015] The deformable convolution module is used to form a decoder submodule through deformable convolution, the decoder restores the feature map to the original input size, and finally a 1x1 convolution and a Sigmoid function are used to output a segmentation probability map.

[0016] The multi-scale adaptive detail feature fusion module is used to capture the network bottom layer detail features, and X R H×W , X1, X2, and X3 are feature maps of different scales, wherein H and W are the height and width of the feature map; through a unified scale operation, including up-sampling, down-sampling, and replication operation, the three feature maps of different scales are transformed to the same scale; deformable convolution is used to generate a random receptive field to guide the adaptive fusion of multi-scale detail features, and the deformable convolution is defined as formula (1) as follows:

[0017]

[0018] wherein, represents a grid, n is the number of grid pixels, n = {1, 2, …, N}, p n is the position of the pixel point in the grid, w is the weight, and Δp nis a position offset; three different scale feature maps are concatenated to obtain a feature map C, the feature map C is subjected to a 1*1 convolution operation to reduce the number of C channels, different scale receptive fields are captured through 3*3 and 5*5 deformable convolution, then 1*1 deformable convolution and a softmax function are used to obtain a 3-channel probability feature map S, finally, the channel feature map corresponding to the feature map S is multiplied by X1, X2 and X3 and added to obtain Y1, which is represented by formula (2) as follows:

[0019]

[0020]

[0021] wherein (i, j) represents a feature map position, S 1 , S 2 , S 3 is obtained from formula (3), Y is the output of the multi-scale adaptive detail feature fusion module;

[0022] A double-path upsampling module is used to capture high-resolution feature maps in space and channels simultaneously, and a deformable convolution is used to learn the fused high-resolution feature maps to reduce the loss of spatial information and channel information in the upsampling process; the input I is subjected to spatial upsampling to obtain a feature map U 1 , and channel upsampling to obtain U 2 , U 1 and U 2 are fused to obtain an output O, and the output of the DPUS is defined by formula (4) as follows:

[0023] D = Conv (Deconv (Conv (I))) + P (Conv (I)) (4)

[0024] wherein Conv represents convolution operation, Deconv represents deconvolution operation, and P represents PixelShuffle operation.

[0025] Further, the algorithm of the deformable convolution is as follows: the network input size is 48*48*3, the encoder is resnet34, and n* represents that n Res-Block-1 modules are used, wherein n [3, 4, 6]; the output of the MSADFF is added to the output of the DPUS, and then a deformable convolution is used to constitute a decoder submodule; the decoder restores the feature map to the original input size, and finally a 1*1 convolution and a Sigmoid function are used to output a segmentation probability map.

[0026] As described above, due to the adoption of the above technical solutions, the present application has the following advantages:

[0027] 1. The multi-scale adaptive detail feature fusion (MSADFF) module of the present application can adaptively fuse multi-scale features before skip connection, fully capture different scale detail features, and alleviate the loss of detail information.

[0028] 2. The present application designs a double-path upsampling (DPUS) module to replace the traditional upsampling module, compensates for information from space and channels, and restores image resolution. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The present application is a retinal blood vessel segmentation network structure schematic diagram;

[0030] Figure 2 The present application is a multi-scale adaptive detail feature fusion module schematic diagram;

[0031] Figure 3 The present application is a double-path upsampling module schematic diagram.

[0032] Figure 3 Among them:

[0033] Dconv-deconvolution; Conv-convolution; PixelShuffle-rearrange pixels; U-feature map; I-input; O-output. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical scheme and advantages of the present application more clear and obvious, the present application will be further described in detail below combined with examples. It should be understood that the specific examples described here are only used to explain the present application, and are not used to limit the present application.

[0035] As shown in Figure 1 A high-precision retinal blood vessel segmentation network structure of the present application, including deformable convolution module, multi-scale adaptive detail feature fusion module and double-path upsampling module, aims to fully fuse multi-scale detail features, reduce the loss of detail information in the segmentation process, and at the same time, upsample the feature map in space and channel, alleviate the loss of information in the upsampling process, and improve the segmentation accuracy of irregular blood vessels.

[0036] The deformable convolution module includes an encoder, n Res-Block-1 modules, and a decoder, where n [3,4,6], and the decoder includes a decoder submodule;

[0037] The multi-scale adaptive detail feature fusion module includes a feature map module, an upsampling module, a downsampling module, a replication module and a transformation module; the multi-scale adaptive detail feature fusion module can adaptively fuse multi-scale features before skip connection, fully capture different scale detail features, and alleviate the loss of detail information.

[0038] The dual-path up-sampling module includes a capture module, an information module and a fusion module. The dual-path up-sampling module replaces a conventional up-sampling module, compensates information in space and channels, and restores image resolution.

[0039] Preferably, the encoder is a resnet34 model.

[0040] Preferably, the information module includes spatial information and channel information.

[0041] The high-precision retinal blood vessel segmentation network method of the application uses the high-precision retinal blood vessel segmentation network structure described above, and includes:

[0042] On the basis of the U-Net network, deformable convolution, a multi-scale adaptive detail feature fusion module and a dual-path up-sampling module are added, multi-scale detail features are fused, feature maps are up-sampled in space and channels, and information loss in the up-sampling process is alleviated.

[0043] The deformable convolution module is used to form a decoder submodule through deformable convolution, the decoder restores the feature map to the original input size, and finally a 1x1 convolution and a Sigmoid function are used to output a segmentation probability map.

[0044] The multi-scale adaptive detail feature fusion module is used to capture network bottom layer detail features, and X is defined as X element of R H×W , X1, X2 and X3 are feature maps of different scales, wherein H and W are the height and width of the feature map; through a unified scale operation including up-sampling, down-sampling and copying operation, the three feature maps of different scales are transformed to the same scale; a deformable convolution is used to generate a random receptive field to guide the adaptive fusion of multi-scale detail features, and the deformable convolution is defined as formula (1) as follows:

[0045]

[0046] wherein, represents a grid, n is the number of grid pixels, n={1, 2,..., N}, p n is the position of the pixel in the grid, w is the weight, and Δp n is the position offset; three feature maps of different scales are obtained through channel concat to obtain a feature map C, a 1x1 convolution operation is performed on the feature map C to reduce the number of C channels, a 3x3 and 5x5 deformable convolution is used to capture different scale receptive fields, and then a 1x1 deformable convolution and a softmax function are used to obtain a 3-channel probability feature map S, finally the corresponding channel feature map of the feature map S is multiplied by X1, X2 and X3 and added, that is, Y1 is obtained, which is represented by formula (2) as follows:

[0047]

[0048]

[0049] where (i, j) denotes the feature map position, S 1 , S 2 , S 3 Y is obtained from formula (3), and Y is the output of the multi-scale adaptive detail feature fusion module;

[0050] A double-path upsampling module is adopted to simultaneously capture high-resolution feature maps in space and channels, and a deformable convolution is used to learn the fused high-resolution feature maps, so as to reduce the loss of spatial information and channel information in the upsampling process; the input I is upsampled in space to obtain a feature map U 1 , and the input I is upsampled in channels to obtain U 2 U 1 and U 2 are fused to obtain the output O, and the output of the DPUS is defined by formula (4):

[0051] D = Conv (Deconv (Conv (I))) + P (Conv (I)) (4)

[0052] where Conv represents convolution operation, Deconv represents deconvolution operation, and P represents PixelShuffle operation.

[0053] The algorithm of the deformable convolution is as follows: the network input size is 48*48*3, the encoder is resnet34, n* is used to represent that n Res-Block-1 modules are used, where n [3, 4, 6]; the output of the MSADFF is added to the output of the DPUS, and then the deformable convolution is used to constitute a decoder submodule; the decoder restores the feature map to the original input size, and finally, a 1*1 convolution and a Sigmoid function are used to output a segmentation probability map.

[0054] By using the above method, the experimental results of the application on the DRIVE / STARE data set show that the accuracy, sensitivity, specificity, AUC, F1-Score indexes of the proposed MILU-Net reach 95.67% / 96.50%, 82.20% / 82.39%, 97.63% / 98.25%, 98.05% / 98.65%, 82.87% / 83.88% respectively, compared with the original network structure, the AUC, F1-Score indexes are improved by 0.2%, 1.27% / 0.71%, 2.64%, and the sensitivity and F1-Score are superior to R2U-Net by 4.82%, 1.16% respectively.

[0055] The above merely describes preferred embodiments of the present application, and is not used to limit the present application, any modification, equivalent replacement and improvement within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A high-precision retinal blood vessel segmentation network method, characterized in that, A high-precision retinal blood vessel segmentation network structure is used, including a deformable convolution module, a multi-scale adaptive detail feature fusion module and a double-path upsampling module; The deformable convolution module includes an encoder, n Res-Block-1 modules and a decoder, wherein n [3, 4, 6], and the decoder includes a decoder submodule; the multi-scale adaptive detail feature fusion module includes a feature map module, an upsampling module, a downsampling module, a replication module and a transformation module; the double-path upsampling module includes a capture module, an information module and a fusion module; The encoder is a resnet34 model; The information module includes spatial information and channel information; The high-precision retinal blood vessel segmentation network method includes: On the basis of the U-Net network, deformable convolution, a multi-scale adaptive detail feature fusion module and a double-path upsampling module are added, multi-scale detail features are fused, and the feature map is upsampled in space and channel to relieve the loss of information in the upsampling process; The deformable convolution module is used, the decoder submodule is formed through deformable convolution, the decoder restores the feature map to the original input size, and finally, 1x1 convolution and a Sigmoid function are used to output a segmentation probability map; A multi-scale adaptive detail feature fusion module is adopted to capture the network bottom layer detail features, and X∈R H×W , , , are feature maps of different scales, wherein H , W are the height and width of the feature map; through a unified scale operation, including upsampling, downsampling, and copy operation, the three different scale feature maps are transformed to the same scale; a deformable convolution is used to generate a random receptive field to guide the adaptive fusion of multi-scale detail features, and the deformable convolution is defined as formula (1) as follows: （1）； wherein, represents a grid, is the number of grid pixels, , p n is the position of a pixel in the grid, is a weight, is a position offset; three different scale feature maps are obtained through channel concat to obtain a feature map C, the feature map C is subjected to a 1×1 convolution operation to reduce the number of C channels, different scales of receptive fields are captured through 3×3 and 5×5 deformable convolution, then 1×1 deformable convolution and a softmax function are used to obtain a 3-channel probability feature map S, and finally the feature map S corresponding to a channel feature map is multiplied by the feature map C corresponding to the channel feature map and added, that is, a feature map F is obtained. , , is represented by the following formula (2):​ （2）； （3）； where (i, j) represents a feature map position, , , is obtained from equation (3), is the output of the multi-scale adaptive detail feature fusion module; The dual-path up-sampling module is adopted to simultaneously capture high-resolution feature maps in space and channels, and a deformable convolution is used to learn the fused high-resolution feature maps, so as to reduce the loss of spatial information and channel information in the up-sampling process. The up-sampling is performed in space to obtain a feature map U 1 The up-sampling is performed in channels to obtain U 2 U 1 and U 2 are fused to obtain an output O, and the output of the DPUS is defined by formula (4). （4）； wherein Conv denotes a convolution operation, Deconv denotes a deconvolution operation, P denotes PixelShuffle operations.

2. The high-precision retinal blood vessel segmentation network method according to claim 1, characterized in that, The algorithm of the deformable convolution is: assuming that the network input size is 48*48*3, the encoder is resnet34, n* represents using n Res-Block-1 modules, wherein ; the MSADFF output is added to the DPU output, and then the deformable convolution is used to constitute a decoder submodule; the decoder restores the feature map to the original input size; finally, a 1*1 convolution and a Sigmoid function are used to output a segmentation probability map.

Images