Retinal vessel image segmentation method based on multi-scale attention gating network

Abstract

This invention relates to a retinal vessel image segmentation method based on a multi-scale attention-gated network, comprising: constructing a retinal vessel image segmentation network model; the retinal vessel image segmentation network model extracting features using multi-scale feature convolutional blocks with convolutional kernels of different sizes, effectively extracting and segmenting the entire vessel and terminal fine vessels using information at different feature scales, and using a non-local attention module to obtain richer global semantic information; then employing a multi-scale attention-gated network in the skip connection part, inputting feature maps of different dimensions, and selectively learning interrelated regions; training the retinal vessel image segmentation network model using a training dataset to obtain a trained retinal vessel image segmentation network model; and performing image segmentation of the retinal vessel image using the trained retinal vessel image segmentation network model. This method is beneficial for more accurate segmentation of retinal vessel images.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method for retinal vessel image segmentation based on a multi-scale attention gating network. Background Technology

[0002] Accurate segmentation of retinal vessel images can be used not only to assess and detect various ophthalmic diseases, but also to provide timely information on systemic diseases such as diabetes and blood disorders. Therefore, research on retinal vessel image segmentation is of great significance for the diagnosis of visually threatening diseases, and the accuracy of retinal vessel segmentation has a crucial impact on diagnostic efficiency.

[0003] Because the blood vessels in the eye have many tiny capillary structures, manual segmentation is extremely complex and susceptible to subjective influences. Therefore, automated and efficient segmentation of retinal blood vessels is crucial. Due to the small size of retinal blood vessels and the influence of factors such as lighting conditions, traditional algorithms do not achieve ideal segmentation accuracy, resulting in problems such as missing vessel terminals and unclear vessel edges.

[0004] Wu et al. proposed a multi-scale network structure for retinal vessel segmentation, obtaining more refined features. Yan et al., by analyzing the limitations of pixel loss, proposed a new loss function that emphasizes the proportion of fine vessels in the segmentation. Alom et al. combined a recursive residual structure with U-Net to construct R2U-Net, increasing the acquisition of contextual information. ZHUANG proposed LadderNet based on the U-Net network, constructing multiple paths from input to output through cascading. Guo et al. proposed randomly discarding convolutional blocks to prevent overfitting, constructing SD-UNet. Jiang et al. proposed a multi-scale information fusion module to capture vascular features of different thicknesses.

[0005] Due to the low contrast and uneven variation among tiny blood vessels in retinal images, achieving more accurate segmentation of retinal images has become a research hotspot. In recent years, supervised deep learning technology has developed rapidly. Because of its excellent performance in image processing, many researchers have begun to use deep learning for medical image segmentation. Using a U-shaped network structure for medical image segmentation has achieved good performance; its unique U-shaped structure effectively transmits extracted feature information, making it suitable for segmenting medical images on small datasets. However, deep neural networks, through continuous convolution and pooling operations, reduce image resolution, causing loss of details at the edges of retinal vessels and inaccurate segmentation of small blood vessels. Furthermore, as network depth increases, background noise in the encoder is passed to the decoder through skip connections, leading to incorrect pixel segmentation. Summary of the Invention

[0006] The purpose of this invention is to provide a retinal vessel image segmentation method based on a multi-scale attention gating network, which is beneficial for more accurate segmentation of retinal vessel images.

[0007] To achieve the above objectives, the technical solution adopted by this invention is: a retinal vessel image segmentation method based on a multi-scale attention gating network, comprising:

[0008] A retinal vessel image segmentation network model is constructed. The retinal vessel image segmentation network model uses multi-scale feature convolutional blocks and convolutional kernels of different sizes to extract features. It utilizes information from different feature scales to effectively extract and segment the whole vessel and the terminal fine vessels. It also uses a non-local attention module to obtain richer global semantic information. Then, in the skip connection part, a multi-scale attention gating network is used to selectively learn the interrelated regions by inputting feature maps of different dimensions.

[0009] The retinal vessel image segmentation network model is trained using the training dataset to obtain a trained retinal vessel image segmentation network model.

[0010] The retinal vessel image is segmented using a trained retinal vessel image segmentation network model.

[0011] Furthermore, in the retinal vessel image segmentation network model, each step of the encoder includes a multi-scale feature convolution block and a 2×2 max pooling operation, doubling the number of feature channels in each downsampling; each step of the decoder includes a 2×2 transposed convolution operation for upsampling, halving the number of feature channels, and concatenating with the corresponding feature map from the encoder; a non-local attention module is set between the encoder and the decoder.

[0012] Furthermore, the multi-scale feature convolutional block uses three different sizes of convolutional kernels (1×1, 3×3, and 5×5) to extract and fuse multi-scale features, thereby improving the ability to extract microvascular features. The convolutional results are fused into a combined feature map using concatenation, and then subjected to a 1×1 convolution to maintain the number of channels and input features. Figure 1 Then, DropBlock is used to effectively discard some semantic information in the multi-scale feature map to prevent the network from overfitting; the input feature map is added to the multi-scale feature map after DropBlock through 3×3 convolution to form a residual structure; ReLU activation function is added to the structure block to avoid overfitting.

[0013] Furthermore, the nonlocal attention module increases the correlation between global features by calculating the weighted sum of each position in the feature, establishing dependency relationships between feature pixels that are related at a greater distance, so as to better capture global semantic information. The nonlocal attention module performs linear mapping on the input feature map to generate three feature maps M1, M2, and M3 respectively. The multiplication matrix of M1 and M2 is obtained by reshaping and transposing, and the global spatial attention map A is obtained by the softmax function. The spatial attention map A is applied to M3 and the final nonlocal attention feature map is obtained by transposing and linear convolution. A residual connection is added at the output and added to the input feature map.

[0014] Furthermore, the calculation formula for the non-local attention module is as follows:

[0015]

[0016] Z i =A i y i +x i (2)

[0017] Where i and j are the output positions, the f function is used to calculate the similarity between positions i and j; the g function is used to calculate the representation of the feature map at point j; the response function C(x) is used to normalize the accumulated result; y i Z represents the generated nonlocal feature map. i This is the final spatial attention feature map obtained after passing through the residual network.

[0018] Furthermore, the multi-scale attention gating network is used to highlight the feature responses of the blood vessel portion and suppress the feature responses of the image background region. The multi-scale attention gating network includes multiple multi-scale attention gating modules. Each module first adjusts the number of channels and size of the encoded feature map x, the shallow encoded feature map y, and the feature map z to be upsampled to be consistent. Then, x and z are processed by max pooling, ReLU activation, a 1×1 convolution, and BN normalization, respectively. y is then processed by a 1×1 convolution. x and y are added together, processed by ReLU activation, a 1×1 convolution, and Sigmoid activation, and then multiplied by y. The result is added to z. The feature map is processed by ReLU activation, a 1×1 convolution, and Sigmoid activation, and then multiplied by y. If y is the first-level encoded feature map, only the feature maps of y and z are processed, and then they are fused together by concatenation, followed by ReLU activation, a 1×1 convolution, and BN normalization to obtain the output feature map. Since shallow and deep feature maps have different category and location information, shallow feature maps combined with deep feature maps can obtain rich semantic information and reduce the interference of useless information. A multi-scale attention gate module is added to the skip connection part of the neural network to spatially enhance the retinal features and reduce the impact of noise on retinal segmentation.

[0019] Furthermore, when training the retinal vessel image segmentation network model, the loss function used is the cross-entropy loss function, which is defined as follows:

[0020]

[0021] In the formula, i represents the category; y i p represents the probability of the true value. i This represents the probability value predicted by the model.

[0022] Compared with existing technologies, this invention has the following beneficial effects: This invention constructs a retinal vessel image segmentation network model. To improve the segmentation accuracy of blood vessels in fundus images, a multi-scale feature convolution method is used in the convolution part. Convolutional kernels of different scales are used for convolution and fusion respectively, which can more effectively learn information from feature maps of different scales and improve the ability to segment small blood vessels. A non-local attention module is added between the encoder and decoder to extract global semantic information of the retinal image. The skip connection part adopts a multi-scale attention gating network to fuse feature information at different depths, which can highlight the feature responses of blood vessels and suppress the feature responses of background regions, enhancing the network's representation ability and reducing the impact of noise on retinal segmentation. This invention can effectively improve the segmentation accuracy of retinal vessel images and improve the segmentation accuracy of blood vessel edge details and small blood vessels in the image. Attached Figure Description

[0023] Figure 1 This is an architecture diagram of the retinal blood vessel image segmentation network model in an embodiment of the present invention.

[0024] Figure 2 This is a structural diagram of the multi-scale feature convolution block in an embodiment of the present invention.

[0025] Figure 3 This is a structural diagram of the non-local attention module in an embodiment of the present invention.

[0026] Figure 4 This is a structural diagram of the multi-scale attention gate module in an embodiment of the present invention.

[0027] Figure 5 These are some visual examples of the segmentation results of this method and other segmentation models in the embodiments of the present invention.

[0028] Figure 6 These are some other visual examples of the segmentation results of this method and other segmentation models in the embodiments of the present invention. Detailed Implementation

[0029] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0030] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0031] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0032] like Figure 1 As shown, this embodiment provides a retinal vessel image segmentation method based on a multi-scale attention gating network, including:

[0033] 1) Construct a retinal vessel image segmentation network model; the retinal vessel image segmentation network model uses multi-scale feature convolutional blocks and convolutional kernels of different sizes to extract features, and uses information at different feature scales to effectively extract and segment the whole vessel and the terminal fine vessels, and uses a non-local attention module to obtain richer global semantic information; then, in the skip connection part, a multi-scale attention gating network is used to selectively learn the interrelated regions by inputting feature maps of different dimensions.

[0034] 2) The retinal vessel image segmentation network model is trained using the training dataset to obtain the trained retinal vessel image segmentation network model.

[0035] 3) Perform image segmentation on retinal blood vessel images using a trained retinal blood vessel image segmentation network model.

[0036] In the retinal vessel image segmentation network model, each step of the encoder includes a multi-scale feature convolution block and a 2×2 max pooling operation, doubling the number of feature channels in each downsampling; each step of the decoder includes a 2×2 transposed convolution operation for upsampling, halving the number of feature channels, and concatenating with the corresponding feature map from the encoder; a non-local attention module is set between the encoder and decoder.

[0037] 1. Multi-scale feature convolutional blocks

[0038] like Figure 2 As shown, the multi-scale feature convolutional block uses three different kernel sizes (1×1, 3×3, and 5×5) to extract and fuse multi-scale features, thereby improving the ability to extract microvascular features. The convolution results are fused into a combined feature map using concatenation, and then subjected to a 1×1 convolution to maintain the number of channels and input features. Figure 1 The algorithm then uses DropBlock to effectively discard some semantic information from the multi-scale feature maps, preventing overfitting. The input feature maps are added to the DropBlock-processed multi-scale feature maps through 3×3 convolutions to form residual structures. A ReLU activation function is added to the structure blocks to avoid overfitting. Compared to traditional algorithms, the improved module is more effective at learning information from feature maps at different scales, improving its ability to segment small blood vessels.

[0039] 2. Non-local attention module

[0040] The nonlocal attention module increases the correlation between global features by calculating a weighted sum at each location in the feature set, establishing dependencies between feature pixels that are related at relatively large distances, thereby better capturing global semantic information. In this embodiment, the nonlocal attention module is as follows: Figure 3As shown, H and W represent the height and width of the input feature map, respectively, and C is the number of channels. The non-local attention module performs a linear mapping on the input feature map to generate three feature maps M1, M2, and M3. The multiplication matrices of M1 and M2 are obtained through reshape and transpose, and the global spatial attention map A is obtained through the softmax function. The spatial attention map A is applied to M3 and transpose and linear convolution are used to obtain the final non-local attention feature map. A residual connection is added to the output and added to the input feature map.

[0041] The formula for calculating the non-local attention module is:

[0042]

[0043] Z i =A i y i +x i (2)

[0044] Where i and j are the output positions, the f function is used to calculate the similarity between positions i and j; the g function is used to calculate the representation of the feature map at point j; the response function C(x) is used to normalize the accumulated result; y i Z represents the generated nonlocal feature map. i This is the final spatial attention feature map obtained after passing through the residual network.

[0045] 3. Multi-scale attention gate module

[0046] Multi-scale attention gating networks are used to highlight the feature responses of blood vessels and suppress the feature responses of background regions in an image. A multi-scale attention gating network consists of multiple multi-scale attention gate modules. The structure of a multi-scale attention gate module is as follows: Figure 4As shown. The multi-scale attention gate module first adjusts the number of channels and size of the encoded feature map x, the shallow encoded feature map y, and the feature map z to be upsampled to be consistent. Then, x and z are respectively processed by max pooling, ReLU activation function, 1×1 convolution and BN normalization. y is processed by 1×1 convolution. Let x and y be added, processed by ReLU activation function, 1×1 convolution and Sigmoid activation operation, and then multiplied by y. The result is added to z, processed by ReLU activation function, 1×1 convolution and Sigmoid activation operation, and then multiplied by y. If y is the first-level encoded feature map, only the feature maps of y and z are processed. Then, they are fused together using concatenation, followed by ReLU activation, a 1×1 convolution, and BN normalization to obtain the output feature map. Since shallow and deep feature maps have different category and location information, shallow feature maps combined with deep feature maps can obtain rich semantic information and reduce the interference of useless information. Multi-scale attention gate modules are added to the skip connection part of the neural network to spatially enhance retinal features and reduce the impact of noise on retinal segmentation.

[0047] 4. Loss Function

[0048] Retinal image segmentation involves classifying each pixel in the image. When training a network model for retinal vessel image segmentation, the loss function used is the cross-entropy loss function, defined as follows:

[0049]

[0050] In the formula, i represents the category; y i p represents the probability of the true value. i This represents the probability value predicted by the model.

[0051] The performance of this invention is verified below using two publicly available datasets.

[0052] 1. Experimental Data and Evaluation

[0053] To evaluate the performance of the fundus image vessel segmentation algorithm, validation experiments were conducted on the publicly available retinal vessel segmentation datasets DRIVE and CHASE_DB1. Evaluations were performed using the following metrics: Accuracy (ACC), Sensitivity (SE), Specificity (SP), and Area Under Curve (AUC). AUC is an evaluation metric based on the segmentation model, representing the area under the receiver operating characteristic (ROC) curve; a higher value indicates better model classification performance. ACC represents the ability to correctly segment vessels and background, SE represents the ability to correctly classify vessel pixels, and SP represents the ability to correctly classify background pixels. The definitions of these performance metrics are as follows:

[0054]

[0055] In this model, true positives (TP) represent correctly segmented blood vessel pixels, and false positives (FP) represent incorrectly segmented blood vessel pixels. True negatives (TN) represent correctly segmented background pixels, and false negatives (FN) represent incorrectly segmented background pixels.

[0056] 2. Quantitative comparison

[0057] To evaluate the segmentation performance of the model of this invention, we compared its segmentation results with other fundus image vessel segmentation models. As shown in Table 1, the accuracy of the proposed model on the DRIVE dataset reached 96.90%, 79.72%, 98.55%, and 98.50%, respectively. On the CHASE_DB1 dataset, the accuracy of the proposed model reached 97.42%, 82.44%, 98.43%, and 98.83%, respectively. It can be seen that, except for a slight difference in Sp compared to other algorithms, the other indicators show good results, which represents a certain improvement in the accuracy of retinal image segmentation.

[0058] Table 1: Quantitative segmentation results on the DRIVE dataset

[0059]

[0060] 3 Qualitative comparison

[0061] Figure 5 , 6 Visual examples of the segmentation results of this invention compared to other advanced segmentation models are provided. Four samples were randomly selected from the test dataset for comparative analysis of the prediction results. The segmentation results show that the segmentation results of this invention are more complete in segmenting small blood vessels, and the connectivity of the blood vessels is high. It can effectively overcome the difficulties in retinal segmentation and achieve better segmentation results.

[0062] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for retinal vessel image segmentation based on a multi-scale attention-gated network, characterized in that, include: A retinal vessel image segmentation network model is constructed. The retinal vessel image segmentation network model uses multi-scale feature convolutional blocks and convolutional kernels of different sizes to extract features. It utilizes information from different feature scales to effectively extract and segment the whole vessel and the terminal fine vessels. It also uses a non-local attention module to obtain richer global semantic information. Then, in the skip connection part, a multi-scale attention gating network is used to selectively learn the interrelated regions by inputting feature maps of different dimensions. The retinal vessel image segmentation network model is trained using the training dataset to obtain a trained retinal vessel image segmentation network model. The retinal vessel image is segmented using a trained retinal vessel image segmentation network model. In the retinal vessel image segmentation network model, each step of the encoder includes a multi-scale feature convolution block and a 2×2 max pooling operation, doubling the number of feature channels in each downsampling; each step of the decoder includes a 2×2 transposed convolution operation for upsampling, halving the number of feature channels, and concatenating with the corresponding feature map from the encoder; a non-local attention module is set between the encoder and decoder. The nonlocal attention module increases the correlation between global features by calculating the weighted sum of each position in the feature, establishing dependency relationships between feature pixels that are related at relatively large distances, so as to better capture global semantic information. The nonlocal attention module performs linear mapping on the input feature map to generate three feature maps M1, M2 and M3 respectively. The multiplication matrix of M1 and M2 is obtained by reshaping and transposing, and the global spatial attention map A is obtained by the softmax function. The spatial attention map A is applied to M3 and transposed and linear convolution is used to obtain the final nonlocal attention feature map. A residual connection is added to the output and added to the input feature map. The multi-scale attention gating network is used to highlight the feature responses of the blood vessel region and suppress the feature responses of the image background region. The multi-scale attention gating network includes multiple multi-scale attention gating modules. Each module first adjusts the number of channels and size of the encoded feature map x, the shallow encoded feature map y, and the feature map z to be upsampled to be consistent. Then, x and z are processed by max pooling, ReLU activation, a 1×1 convolution, and BN normalization, respectively. y is then processed by a 1×1 convolution. x and y are added together, processed by ReLU activation, a 1×1 convolution, and Sigmoid activation, and then multiplied by y. The result is added to z, and then processed by ReLU activation. The ReLU activation function, a 1×1 convolution, and a Sigmoid activation operation are applied, and then multiplied by y. If y is the first-level encoded feature map, only the feature map operations of y and z are performed. Then, they are fused together by concatenation, followed by the ReLU activation function, a 1×1 convolution, and BN normalization to obtain the output feature map. Since shallow and deep feature maps have different category and location information, shallow feature maps combined with deep feature maps can obtain rich semantic information and reduce the interference of useless information. A multi-scale attention gate module is added to the skip connection part of the neural network to spatially enhance retinal features and reduce the impact of noise on retinal segmentation.

2. The retinal vessel image segmentation method based on a multi-scale attention gating network according to claim 1, characterized in that, The multi-scale feature convolutional block uses three different kernel sizes (1×1, 3×3, and 5×5) to extract and fuse multi-scale features, thereby improving the ability to extract microvascular features. The convolution results are fused into a combined feature map using concatenation, and then subjected to a 1×1 convolution to maintain the number of channels consistent with the input feature map. Then, DropBlock is used to effectively discard some semantic information in the multi-scale feature map to prevent network overfitting. The input feature map is added to the multi-scale feature map after DropBlock through a 1×1 convolution to form a residual structure. A ReLU activation function is added to the structure block to avoid overfitting.

3. The retinal vessel image segmentation method based on a multi-scale attention gating network according to claim 1, characterized in that, The calculation formula for the non-local attention module is as follows: Z i =A i y i +x i (2) Where i and j are the output positions, the f function is used to calculate the similarity between positions i and j; the g function is used to calculate the representation of the feature map at point j; the response function C(x) is used to normalize the accumulated result; y i Z represents the generated nonlocal feature map. i This is the final spatial attention feature map obtained after passing through the residual network.

4. The retinal vessel image segmentation method based on a multi-scale attention gating network according to claim 1, characterized in that, When training the retinal vessel image segmentation network model, the loss function used is the cross-entropy loss function, which is defined as follows: In the formula, i represents the category; y i p represents the probability of the true value. i This represents the probability value predicted by the model.

Images