Retinal vessel segmentation method based on double-space attention U-net model

Abstract

The invention discloses a retinal vessel segmentation method based on a double-space attention U-net model, and belongs to the field of computer vision and medical image processing. According to the method, on the basis of the U-Net network, a traditional convolution operation is replaced by traditional convolution + dynamic Snake Convolution (DE-Conv), so that the capturing capability of complex structures such as small curved blood vessels is remarkably enhanced. Meanwhile, a double space residual attention (DSRA) mechanism is provided, features are weighted in two dimensions of a channel and a space, and the overall performance of the segmentation model is improved. According to the technical scheme of the invention, by improving the feature extraction and attention distribution strategy of the network, the method fully adapts to the characteristics of micro structures such as retinal blood vessels, and significantly improves the segmentation precision and topological continuity. Experiments show that compared with an existing method, the method shows better accuracy, continuity and stability in a blood vessel segmentation task, and is suitable for various medical image analysis scenes such as clinical diagnosis and hemodynamic simulation.

Description

Technical Field

[0001] The present invention relates to the field of medical image processing, and specifically to a retinal blood vessel segmentation method based on a dual spatial attention U-net model, which aims to improve the perception ability and segmentation accuracy of complex structures in retinal blood vessel segmentation tasks, and is particularly suitable for scenarios such as medical diagnosis and ophthalmic disease monitoring. Background Art

[0002] With the development of medical imaging technology, automatic segmentation of retinal vessels has gradually become an important research direction in the field of medical image processing. The morphology and distribution of retinal vessels directly reflect the health of the eye and are of great significance for the early diagnosis of diseases such as diabetic retinopathy and hypertension. However, due to the complex morphology of retinal vessels, significant background interference, and low contrast of tiny vessels, traditional segmentation methods still have shortcomings in terms of accuracy and detail preservation.

[0003] At present, deep learning-based segmentation methods are widely used in retinal blood vessel segmentation, especially the encoder-decoder structure represented by U-Net. In the U-Net framework, jump connections can effectively integrate low-level detail information with high-level semantic information and improve the segmentation ability of the model. However, traditional U-Net still has the following problems:

[0004] 1. Insufficient ability to capture tiny blood vessels: Due to the limitations of traditional convolution operations, the model is prone to information loss or segmentation breaks when processing curved and complex tiny blood vessels;

[0005] 2. Single attention mechanism: Existing methods usually only use channel attention or spatial attention, which makes it difficult to focus on multi-dimensional features at the same time, resulting in insufficient precision in the recognition of vascular edges and topological structures;

[0006] 3. Redundant information problem of skip connection: The simple splicing method of skip connection may introduce background noise or invalid features, affecting the accuracy of segmentation results;

[0007] In response to the above problems, dynamic snake convolution, as a technology that can adaptively adjust the position of the convolution kernel, can significantly improve the model's perception of complex geometric structures (such as curved blood vessels). At the same time, the dual-space residual attention mechanism simulates the residual structure by combining the weighted strategies of the channel and space dimensions, effectively improving the feature fusion efficiency of the model. Based on this, the present invention proposes a retinal blood vessel segmentation method based on dynamic snake convolution and dual-space residual attention mechanism, aiming to improve the segmentation accuracy, enhance the ability to capture tiny structures, and improve the topological continuity of the segmentation results. Summary of the invention

[0008] In order to solve the problem that the existing traditional blood vessel segmentation methods are prone to segmentation breaks or detail loss when dealing with tiny curved structures, the present invention improves the traditional U-net network model to improve the segmentation accuracy and enhance the ability to capture tiny structures.

[0009] In order to verify the segmentation performance of the model, the technical solution of the present invention is:

[0010] Step 1: Randomly divide the original medical image dataset into 15 images each for the validation set and the test set;

[0011] Step 2: Preprocess the input medical image, including denoising, contrast enhancement and normalization;

[0012] Step 3: Construct a segmentation network based on dynamic snake convolution (DE-Conv), where: in the encoder and decoder of the U-Net network, the traditional convolution is replaced by traditional convolution + dynamic snake convolution, and the convolution kernel position is dynamically adapted to capture the complex vascular structure; the initial value of the dynamic offset is set to zero, and the offset range is limited to [-1, 1].

[0013] Step 4: Embed the dual spatial residual attention mechanism (DSRA) in the jump connection part, which includes: performing global average pooling and maximum pooling on the input features in the channel dimension to generate channel weights. Perform 1×1 convolution and activation operations on the spatial dimension features to generate a spatial attention map. Fuse the channel attention, spatial attention and feature information before convolution to obtain enhanced features.

[0014] Step 5: Input the enhanced training data set into the segmentation network established in steps 3 and 4 to train the model; during the model training process, set the batch size to 5, set the network learning rate to 0.0001, use the cross entropy loss function to calculate the loss, and set the epoch size to 200;

[0015] Step 6: After the blood vessel segmentation model is trained, the test set randomly divided in step 1 is input into the model to obtain the final segmentation map;

[0016] It can be seen from the above technical solution that compared with the prior art, it has the following technical advantages:

[0017] The method of the present invention introduces dynamic snake convolution and dual-space residual attention mechanism, which effectively improves the perception ability and segmentation accuracy of complex structures in retinal blood vessel segmentation tasks. Compared with existing methods, the present invention has significant advantages in extracting small blood vessels and curved feature structures, and has lower computational complexity, which is particularly suitable for medical diagnosis, ophthalmic disease monitoring and other scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0019] Figure 1 It is the overall flow chart of the method of the present invention;

[0020] Figure 2 It is a framework diagram of an improved U-net network model according to the method of the present invention;

[0021] Figure 3 is the improved attention mechanism module of the present invention; DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention, so that those skilled in the art can better understand the present invention and implement it, but the embodiments are not intended to limit the present invention.

[0023] The image enhancement process of the present invention is as follows Figure 1 The retinal blood vessel segmentation method based on the dual-space attention U-net model of the present invention specifically comprises the following steps:

[0024] Step 1: Obtain the public color retinal fundus vessel segmentation dataset DRIVE, randomly divide the dataset into 15 validation sets and 15 test sets respectively;

[0025] Step 2: Preprocess the input medical image, including denoising, contrast enhancement and normalization. First, use non-local means filtering (NL-Means) to remove random noise in the medical image, while retaining the edge details and texture information of the image. Then use the contrast-limited adaptive histogram equalization (CLAHE) method to enhance the contrast between blood vessels and background. Then use gamma to correct uneven illumination and enhance areas with low contrast grayscale. Standardize the corrected grayscale image, perform zero mean and unit normalization on the image, and normalize the image grayscale value to the standard normal distribution. This allows the network to better learn the feature distribution of the image and speed up the network training.

[0026] Step 3: Construct a segmentation network based on Dynamic Snake Convolution (DE-Conv). The specific implementation steps are as follows: In the encoder and decoder of the U-Net network, replace the traditional convolution with traditional convolution + dynamic snake convolution to enhance the model's ability to capture complex vascular structures; dynamic snake convolution dynamically adjusts the position of the convolution kernel by introducing a learnable offset to adapt to complex vascular morphology. The initial values ​​of all offsets are set to zero to ensure that DE-Conv is equivalent to traditional convolution in the initial state and stabilize the training process. Then limit the offset to the range of [-1, 1] pixels to prevent the convolution kernel from offsetting too much and causing feature extraction distortion. During the training process, the optimal offset is dynamically learned through back propagation, so that the convolution kernel can adaptively capture complex vascular structures. Then follow the steps below. Figure 2 As shown in the figure, a basic encoder-decoder network is constructed. The encoder consists of multiple convolutional layers and downsampling layers. Convolutional layer: Each convolutional layer uses a combination of traditional convolution and DE-Conv to extract multi-scale features; downsampling layer: Uses the maximum pooling operation to gradually reduce the size of the feature map and increase the receptive field. The decoder consists of multiple upsampling layers and convolutional layers. Upsampling layer: Through deconvolution or upsampling operations, the feature map size is gradually restored. Convolutional layer: Each convolutional layer also uses a combination of traditional convolution and DE-Conv to refine features and improve segmentation accuracy.

[0027] Step 4: Embed the dual spatial residual attention mechanism (DSRA) in the jump connection part, which includes: performing global average pooling and maximum pooling on the input features in the channel dimension to generate channel weights. Perform 1×1 convolution and activation operations on the spatial dimension features to generate a spatial attention map. Fuse the channel attention, spatial attention and feature information before convolution to obtain enhanced features.

[0028] In the U-Net network, feature loss will occur regardless of downsampling or upsampling. An attention mechanism is added to the original network structure to reduce the impact on image segmentation. However, there are many tiny blood vessels in the retina, and the lost information affects the segmentation of tiny blood vessels. Therefore, in this paper, the spatial attention mechanism is combined with the channel attention mechanism to unify the image size, solve the problem of feature weakening caused by sampling, increase spatial feature information, capture the spatial information of retinal blood vessels and reduce the impact of image noise. The effective features extracted by the model can be fully integrated with the encoder features, thereby avoiding the loss of some tiny blood vessels and edge information during the upsampling process. At the same time, a global pooling operation is performed on the output of the encoding layer, and the attention vector is calculated to calibrate the weight of the feature information, thereby improving the network's sensitivity to vascular features. The detailed principle of the attention mechanism is as follows. Figure 3 shown.

[0029] Among them, Pre-X(input) represents the unconvolved image of the same layer, and X(input) represents the image after convolution. The two inputs are subjected to average pooling and maximum pooling operations respectively. Among them, the average pooling operation is used to retain the global information of the image, and the maximum pooling operation is used to extract the texture information of the image. The feature maps obtained by average pooling and maximum pooling are then convolved by 1×1, and then the channels are superimposed and Concat is performed. After Sigmoid activation, the weight coefficient α is obtained, and then the feature map is calibrated with the weight coefficient α to obtain the output α×X(input). In the channel attention module, the maximum pooling and average pooling are fused again, and the channel coefficient β is obtained after 1×1 convolution and Sigmoid function; finally, the result of the feature map α×X(input) is calibrated with the channel coefficient β to obtain the final output feature map. This method can combine the original image and the convolution feature image, enhance the feature information and give spatial attention, bring more features and location information into the network, and obtain a more accurate segmentation effect, which plays a beneficial role in the segmentation of tiny blood vessels.

[0030] Step 5: Input the enhanced training data set into the segmentation network established in steps 3 and 4 to train the model; during the model training process, set the batch size to 5, set the network learning rate to 0.0001, select the Adam optimizer for network parameter optimization, use the cross entropy loss function to calculate the loss, and set the epoch size to 200; the Adam optimizer back-propagates to update the student model weights, and the process is as follows:

[0031]

[0032] m t is the first-order momentum estimate; v t is the first-order momentum estimate; β 1 , β 2 is the momentum coefficient (initial β 1 =0.9,β 2 =0.999), is the gradient of the loss function; η is the learning rate (the default is 0.001); ∈ is the flat parameter to prevent the denominator from being zero (∈=10 -8 ); γ is the weight decay coefficient (0.0001). The cross entropy loss function is used to measure the difference between the probability distribution predicted by the model and the true distribution.

[0033] Step 6: After the blood vessel segmentation model is trained, the test set randomly divided in step 1 is input into the model to obtain the final segmentation map;

[0034] Experimental verification and result analysis: Since different testers are too subjective in judging the same image, different testers have different criteria for judging images. Therefore, to ensure that the objective evaluation conclusion is fair and has certain data value, we should use an objective evaluation method combined with a subjective evaluation method, and use specified theorem formulas and mathematical expressions to evaluate, such as using objective test statistical parameters or prior knowledge to evaluate. The evaluation indicators used in this paper are: accuracy, specificity, sensitivity, AUC value, and F1 value.

[0035] The retinal image blood vessel segmentation method based on the improved U-Net network had an accuracy of 0.968, specificity of 0.988, sensitivity of 0.846, AUC value of 0.987, and F1 value of 0.829 on the test set. The model had a good segmentation effect and performed well in segmentation accuracy and segmentation details.

[0036] It should be understood that parts not elaborated in detail in this specification belong to the prior art.

[0037] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A retinal blood vessel segmentation method based on a dual spatial attention U-net model, characterized in that: The method comprises the following steps: Step 1: Randomly divide the original medical image dataset into 15 images each for the validation set and the test set; Step 2: Preprocess the input medical image, including denoising, contrast enhancement and normalization; Step 3: Construct a segmentation network based on dynamic snake convolution (DE-Conv), where: in the encoder and decoder of the U-Net network, the traditional convolution is replaced by traditional convolution + dynamic snake convolution, and the convolution kernel position is dynamically adapted to capture the complex vascular structure; the initial value of the dynamic offset is set to zero, and the offset range is limited to [-1, 1]; Step 4: Embed the dual spatial residual attention mechanism (DSRA) in the jump connection part, which includes: performing global average pooling and maximum pooling on the input features in the channel dimension to generate channel weights; performing 1×1 convolution and activation operations on the spatial dimension features to generate a spatial attention map; fusing the channel attention, spatial attention and feature information before convolution to obtain enhanced features; Step 5: Input the enhanced training data set into the segmentation network established in steps 3 and 4 to train the model; during the model training process, set the batch size to 5, set the network learning rate to 0.0001, use the cross entropy loss function to calculate the loss, and set the epoch size to 200; Step 6: After the vascular segmentation model is trained, the test set randomly divided in step 1 is input into the model to obtain the final segmentation map.

2. The method according to claim 1, characterized in that The dual spatial residual attention mechanism (DSRA) includes: global average pooling and maximum pooling of the pre-convolution features and the post-convolution features, convolution fusion of the two pooled feature maps, and then sigmoid activation function to obtain the weight coefficient α, and then calibrate the feature map with the weight coefficient α to obtain the output α×X(input), and then α×X(input) is added to the feature map. X(input) is fused after channel maximum and average operations respectively; the channel coefficient β is obtained after 1×1 convolution and Sigmoid function; finally, the result of the feature map α×X(input) is calibrated with the channel coefficient β to obtain the final output feature map.

Images