A retinal blood vessel image segmentation method
By integrating axial transformer and convolutional neural network into a retinal vessel image segmentation method, the problems of low accuracy, insufficient segmentation of small edge vessels and large noise interference in existing technologies are solved, achieving high-precision retinal vessel segmentation, which is suitable for computer-aided diagnosis.
Patent Information
- Application Number
- CN202310229670.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-10
- Publication Date
- 2026-01-27
- Estimated Expiration
- 2043-03-10
AI Technical Summary
Existing retinal vessel segmentation methods suffer from problems such as low segmentation accuracy, insufficient ability to segment small vessels at the edge of the eyeball, breakage at vessel branches, and excessive interference from image noise.
A retinal vessel image segmentation method integrating axial transformer and convolutional neural network is adopted. Through feature extraction, feature fusion and retinal vessel image segmentation, the image segmentation is performed by combining axial transformer and convolutional neural network. The cross-entropy loss function and Adam algorithm are used for optimization, and the softmax function is used for pixel classification.
It improves the accuracy of retinal vessel segmentation, enhances the segmentation ability of small blood vessels at the edge of the eyeball, reduces the breakage at blood vessel branches and image noise interference, and achieves high-precision retinal vessel segmentation with a sensitivity of 81.95% and a specificity of 97.27%, respectively, and an ROC curve of 97.93%, making it suitable for computer-aided diagnosis.
Smart Images
Figure CN116152273B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of medical image segmentation, specifically relating to a method for segmenting retinal blood vessel images. Background Technology
[0002] The morphological features of retinal vessels are closely related to pathological diseases such as glaucoma, diabetes, atherosclerosis, and hypertension. The low contrast and complex structure of vessels in retinal images, along with interference from lesion areas, make it difficult for existing retinal vessel segmentation algorithms to effectively segment delicate vessels. Therefore, designing automated and standardized high-precision retinal vessel segmentation methods is crucial for the early diagnosis of various diseases.
[0003] With the development of artificial intelligence and big data technologies, many deep learning-based retinal vessel segmentation methods have been proposed. Unlike traditional image segmentation methods, these methods do not require manual feature selection; they can automatically extract necessary features from images using big data and improve image segmentation accuracy through iterative optimization. Compared to traditional image segmentation methods, deep learning-based retinal vessel segmentation methods offer higher accuracy, faster segmentation speed, and lower cost. Therefore, using computers for medical image segmentation is a future trend in medical auxiliary diagnosis, and researching deep learning-based retinal vessel segmentation algorithms is of great significance.
[0004] However, due to the complex texture features of fundus images and the low distinguishability between retinal vessels and the fundus background, some current deep learning-based retinal vessel segmentation methods still suffer from problems such as low segmentation accuracy, insufficient ability to segment small vessels at the edge of the eyeball, vessel breakage at branch points, and excessive interference from image noise. Summary of the Invention
[0005] The purpose of this invention is to address the common problems in existing retinal vessel image segmentation, such as low segmentation accuracy, insufficient ability to segment small blood vessels at the edge of the eyeball, rupture at blood vessel branches, and excessive interference from image noise. This invention proposes a retinal vessel image segmentation method that integrates axial transformer and convolutional neural network.
[0006] To achieve the above objectives, the present invention employs the following technical solutions:
[0007] A method for retinal vessel image segmentation includes the following steps:
[0008] Step 1: Collect color retinal blood vessel RGB images as a dataset;
[0009] Step 2: Preprocess the retinal vessel images in the dataset;
[0010] Step 3: By performing feature extraction, feature fusion, and retinal vessel image segmentation on the preprocessed retinal vessel image, a retinal vessel image segmentation model is established to obtain the final retinal vessel image.
[0011] Furthermore, in step 2, the retinal vessel images within the dataset undergo preprocessing, specifically as follows:
[0012] Step 2.1: Convert the color retinal vessel RGB image to a grayscale image:
[0013] The three channels of a color retinal blood vessel RGB image are assigned different weights, and it is then converted into a grayscale image using the following formula:
[0014] P gray =0.15P red +0.75P green +0.1P blue ;
[0015] Among them, P gray For the converted grayscale image, P red P green P blue These are the red channel image, green channel image, and blue channel image of a color retinal blood vessel RGB image;
[0016] Step 2.2, compress the grayscale image using normalization:
[0017] The normalization formula is as follows:
[0018]
[0019] Among them, P norm The image is normalized, where μ and σ are the grayscale values of the image P. gray Standard deviation and variance of pixels;
[0020] Step 2.3: Use contrast-limited adaptive histogram equalization to improve the normalized image P. norm To enhance the details and contrast of the image;
[0021] Step 2.4: Use the Local Adaptive Gamma Transformation algorithm to adjust the image;
[0022] Step 2.5: Perform data augmentation on the images in the dataset to obtain sufficient sample images.
[0023] Furthermore, the data augmentation of the images in the dataset in step 2.5 specifically involves: applying translation, rotation, and noise to the images in the dataset.
[0024] Furthermore, in step 3, a retinal vessel image segmentation model is established, and the specific steps are as follows:
[0025] Step 3.1, Feature Extraction:
[0026] First, the preprocessed input image is transformed into a feature map containing high-dimensional information through a convolution with a kernel size of 3×3 and 16 channels. Its expression is as follows:
[0027] X0 = Conv 16_3×3 (X input );
[0028] Where X0 represents the feature image after convolution, X input Conv represents the preprocessed image input to the model. 16_3×3 This indicates a convolution calculation with a kernel size of 3×3 and 16 channels;
[0029] Secondly, the image is downsampled through a 2×2 max pooling layer. Each downsampling reduces the image size to 1 / 4 of the original size. After downsampling, global and local vascular features are extracted through an axial transformer and a 3×3 convolutional layer with doubled channels. The downsampling and feature extraction are performed four times. After each feature extraction, the ReLU nonlinear activation function is used to increase the model's nonlinear modeling ability and anti-interference ability.
[0030] The feature image X obtained by the convolutional neural network after the i-th downsampling. i The expression is as follows:
[0031] X i =Max(0,Conv) Up_3×3 Maxpool 2×2 (X i-1 )));
[0032] Among them, Conv Up_3×3 Maxpool represents a convolutional layer with a kernel size of 3×3 and double the number of channels. 2×2 This represents max pooling with a size of 2×2;
[0033] The feature image Y obtained by the axial transformer after the i-th downsampling is i The expression is as follows:
[0034] Y i =Max(0,Axial_T(Maxpool) 2×2 (Y i-1 )));
[0035] Where Axial_T represents the axial transformer, Maxpool 2×2 This represents max pooling of size 2×2, and when i=1, Y i-1 =X0;
[0036] Step 3.2, Feature Fusion:
[0037] First, the two smallest feature maps obtained through the axial transformer and convolutional neural network are concatenated and upsampling decoding begins. Upsampling is performed using deconvolution, halving the number of image channels and doubling the height and width with each upsampling step. After upsampling, the ReLU non-linear activation function is used to enhance the model's non-linear modeling ability and robustness. A total of four upsampling operations are performed. After the first upsampling, the expression for the feature image Z1 is:
[0038] Z1 = Max(0, Up_Conv) 3×3 (Connect(X4, Y4)));
[0039] Among them, Up_Conv 3×3 This indicates a deconvolution operation with a kernel size of 3×3 and half the number of channels; Connect indicates a concatenation operation between two images.
[0040] Then, the feature maps obtained during downsampling and the same size as those obtained during upsampling are stitched together using skip connections, fusing high-resolution features from different scales in the feature extraction part. A total of three image stitching operations are performed. After the first stitching, the expression for the obtained feature image W1 is as follows:
[0041] W1 = Max(Conv down_3×3 (Connect(Z1,X3)))+Max(Conv down_3×3 (Connect(Z1,Y3)));
[0042] Among them, Conv down_3×3 This indicates a convolution calculation with a kernel size of 3×3 and half the number of channels;
[0043] Step 3.3, retinal vessel image segmentation:
[0044] The training of retinal vessel image segmentation uses the cross-entropy loss function, and the loss function is optimized using the Adam algorithm. The initial learning rate is 0.01 and the momentum is 0.9.
[0045] The retinal vessel image is finely segmented using the Softmax activation function. The probability of each pixel in the image being a vessel or background is obtained by the softmax nonlinear function. A threshold of 0.5 is used. If the probability of a pixel being a vessel exceeds 0.5, the pixel is classified as a vessel; otherwise, it is classified as background, thus obtaining the final retinal vessel image.
[0046] Compared with the prior art, the present invention has the following advantages:
[0047] This invention effectively addresses issues such as low segmentation accuracy, insufficient ability to segment small blood vessels at the eyeball edge, rupture at blood vessel branches, excessive image noise interference, and interference from lesion regions on blood vessel segmentation. It achieves significant progress in global information extraction and detail information fusion. By fusing axial transformers and convolutional neural networks, this invention enables the retinal vessel segmentation model to possess excellent global and local blood vessel feature extraction capabilities, resulting in well-segmented retinal vessels. On the CHASE_DB1 dataset, it demonstrates superior overall segmentation performance, with sensitivity and specificity of 81.95% and 97.27%, respectively, an accuracy of 96.57%, and an ROC curve of 97.93%. This model can be used for computer-aided diagnosis in the medical field, achieving automated and standardized high-precision retinal vessel segmentation. Attached Figure Description
[0048] Figure 1 This is an image of three channels decomposed from a color RGB retinal image;
[0049] Figure 2 This invention converts a color retinal image into a grayscale image.
[0050] Figure 3 This is the image after contrast-limited adaptive histogram equalization processing according to the present invention;
[0051] Figure 4 The image obtained by this invention is after local adaptive gamma transformation;
[0052] Figure 5 This is a structural diagram of the axial tansformer in this invention;
[0053] Figure 6 This is the final segmentation result diagram of the present invention;
[0054] Figure 7 This is a general structural diagram of the present invention;
[0055] Figure 8 This is the ROC curve of the CHASE_DB1 dataset of this invention. Detailed Implementation
[0056] The invention will be further described below with reference to specific embodiments.
[0057] Example 1
[0058] like Figure 7 As shown, a method for retinal vessel image segmentation includes the following steps:
[0059] Step 1: Collect color retinal vessel RGB images as a dataset. In this embodiment, the data comes from the Image_12L images of the CHASE_DB1 dataset.
[0060] Step 2 involves preprocessing the RGB images of retinal vessels in the dataset: First, different weights are assigned to the RGB channels of the color retinal image to convert it into a grayscale image as the main research object; then, normalization and contrast-limited adaptive histogram equalization are used to improve grayscale details and enhance image contrast; next, the local adaptive gamma transformation algorithm is used to adjust the multi-scale bright and dark detail features of microvessels in the retinal image and further correct the image and smooth and denoise it; finally, the dataset is expanded by means of translation, rotation, and adding noise to improve the accuracy of vessel segmentation of the model and reduce overfitting.
[0061] Specifically, the following steps are included:
[0062] Step 2.1: Convert the color retinal vessel RGB image to a grayscale image:
[0063] The three channels of a color retinal RGB image have different levels of sharpness and brightness in their grayscale values. The green channel is the sharpest, the red channel is too bright, and the blue channel is too dark. To obtain a grayscale image without losing image information, different weights are assigned to the three channels of the color retinal blood vessel RGB image, and it is then converted to a grayscale image using the following conversion formula:
[0064] P gray =0.15P red +0.75P green +0.1P blue ;
[0065] Among them, P gray For the converted grayscale image, P red P green P blue These are the images decomposed from the red, green, and blue channels of a color retinal blood vessel RGB image, as shown below. Figure 1 As shown in the image, a larger weight is assigned to the green channel to obtain a clearer image of blood vessels, while appropriate weights are given to the red and blue channels to ensure no image information is lost and to adjust the image brightness appropriately. The resulting grayscale image is shown below. Figure 2 As shown;
[0066] Step 2.2, to reduce computational load, normalized compressed grayscale images are used:
[0067] The normalization formula is as follows:
[0068]
[0069] Among them, P norm The image is normalized, where μ and σ are the grayscale values of the image P. gray Standard deviation and variance of pixels;
[0070] Step 2.3: Use contrast-limited adaptive histogram equalization to improve the normalized image P. norm The details are improved and the image contrast is enhanced, resulting in an image like... Figure 3 As shown, contrast-limited adaptive histogram equalization can make blood vessels in an image clearer and suppress noise in the image;
[0071] Step 2.4: Use the Local Adaptive Gamma Transform algorithm to adjust the image, resulting in the image shown below. Figure 4 As shown, the local adaptive gamma change algorithm can make the details of the image, such as tiny blood vessels, clearer, and further correct the image and smooth the noise, reducing the contrast of the lesion area in the image.
[0072] Step 2.5 involves augmenting the images in the dataset using methods such as translation, rotation, and adding noise to obtain sufficient sample images, thereby improving the accuracy of blood vessel segmentation in the model and reducing overfitting.
[0073] Step 3: Establish a retinal vessel image segmentation model. First, increase the number of channels in the input retinal vessel image through convolution operations to obtain high-dimensional features. Second, downsample the image and extract global and local vessel features using an axial transformer and a convolutional neural network, respectively. Repeat downsampling four times, doubling the number of channels and halving the height and width each time to obtain feature maps at different levels. Then, concatenate the two smallest feature maps obtained by the axial transformer and convolutional neural network and begin upsampling decoding. Halve the number of channels and double the height and width each time upsampling. Use skip connections to concatenate the feature maps of the same size obtained during downsampling and upsampling, fusing high-resolution features from different scales in the feature extraction part to mitigate the spatial information loss caused by downsampling. Finally, enhance the model's anti-interference ability through a softmax nonlinear function and perform vessel and background pixel classification to obtain the segmented retinal vessel image.
[0074] The specific steps are as follows:
[0075] Step 3.1, Feature Extraction:
[0076] First, the preprocessed grayscale retinal image is transformed into a feature map containing high-dimensional information through a convolution with a kernel size of 3×3 and 16 channels. The expression is as follows:
[0077] X0 = Conv 16_3×3 (X input );
[0078] Where X0 represents the feature image after convolution, X input Conv represents the preprocessed image input to the model. 16_3×3 This indicates a convolution calculation with a kernel size of 3×3 and 16 channels;
[0079] Secondly, the image is downsampled using a 2×2 max-pooling layer, reducing the image size to 1 / 4 of its original size with each downsampling. After downsampling, global and local vascular feature extraction are performed using an axial transformer and a convolutional layer with a kernel size of 3×3 and doubled channel count, respectively. The downsampling and feature extraction are performed four times. After each feature extraction, a ReLU non-linear activation function is used to enhance the model's non-linear modeling ability and robustness. The structure of the axial transformer is as follows: Figure 5 As shown;
[0080] The size of the image after the i-th downsampling is 1 / 4 of the size of the image after the (i-1)-th downsampling, where X is the feature image X obtained by the convolutional neural network after the i-th downsampling. i The expression is as follows:
[0081] X i =Max(0,Conv) Up_3×3 Maxpool 2×2 (X i-1 )));
[0082] Among them, Conv Up_3×3 Maxpool represents a convolutional layer with a kernel size of 3×3 and double the number of channels. 2×2 This represents max pooling with a size of 2×2;
[0083] The feature image Y obtained by the axial transformer after the i-th downsampling is i The expression is as follows:
[0084] Y i =Max(0,Axial_T(Maxpool) 2×2 (Yi-1 )));
[0085] Where Axial_T represents the axial transformer, Maxpool 2×2 This represents max pooling of size 2×2, and when i=1, Y i-1 =X0;
[0086] Step 3.2, Feature Fusion:
[0087] First, the two smallest feature maps obtained through the axial transformer and convolutional neural network are concatenated and upsampling decoding begins. Upsampling is performed using deconvolution, halving the number of image channels and doubling the height and width with each upsampling step. After upsampling, the ReLU non-linear activation function is used to enhance the model's non-linear modeling ability and robustness. A total of four upsampling operations are performed. After the first upsampling, the expression for the feature image Z1 is:
[0088] Z1 = Max(0, Up_Conv) 3×3 (Connect(X4, Y4)));
[0089] Among them, Up_Conv 3×3 This indicates a deconvolution operation with a kernel size of 3×3 and half the number of channels; Connect indicates a concatenation operation between two images.
[0090] Then, by using skip connections, the feature maps obtained during downsampling and those obtained during upsampling are stitched together, fusing high-resolution features from different scales in the feature extraction part to mitigate the spatial information loss caused by downsampling. A total of three image stitching operations are performed. After the first stitching, the expression for the obtained feature image W1 is as follows:
[0091] W1 = Max(Conv down_3×3 (Connect(Z1,X3)))+Max(Conv down_3×3 (Connect(Z1,Y3)));
[0092] Among them, Conv down_3×3 This indicates a convolution calculation with a kernel size of 3×3 and half the number of channels;
[0093] Step 3.3, retinal vessel image segmentation:
[0094] The retinal vessel image segmentation model was trained using the cross-entropy loss function, with the Adam algorithm used to optimize the loss function. The initial learning rate was 0.01 and the momentum was 0.9.
[0095] The retinal vessel image is finely segmented using the Softmax activation function. The probability of each pixel being a vessel or background is obtained using the Softmax nonlinear function. A threshold of 0.5 is used; pixels with a probability exceeding 0.5 are classified as vessels, otherwise as background. This yields the final retinal vessel image, as shown in the image. Figure 6 As shown.
[0096] This invention combines an axial transformer with a convolutional neural network to extract features from retinal images. It effectively overcomes the problem of insufficient global information extraction capability caused by the limited receptive field of the convolutional kernel in existing algorithms. It can effectively segment retinal vessels and demonstrates excellent overall segmentation performance on the CHASE_DB1 dataset. Its sensitivity and specificity are 81.95% and 97.27%, respectively, and its accuracy reaches 96.57%. The overall performance of retinal vessel segmentation is excellent, and its accuracy is superior to existing algorithms.
[0097] Overall performance is represented by the receiver operating characteristic curve (ROC curve), which plots the false positive rate (FPR) on the horizontal axis and the true positive rate (TPR) on the vertical axis. Figure 8 As shown, the ROC curve value reaches 97.93%, which is at the level of current medical applications and can be used for computer-aided diagnosis.
Claims
1. A method for segmenting retinal blood vessel images, characterized in that, Includes the following steps: Step 1: Collect color retinal blood vessel RGB images as a dataset; Step 2: Preprocess the RGB images of retinal vessels in the dataset; Step 3: By performing feature extraction, feature fusion, and retinal vessel image segmentation on the preprocessed retinal vessel image, a retinal vessel image segmentation model is established to obtain the final retinal vessel image. The specific steps for establishing the retinal vessel image segmentation model in step 3 are as follows: Step 3.1, Feature Extraction: First, the preprocessed input image is transformed into a feature map containing high-dimensional information through a convolution with a kernel size of 3×3 and 16 channels. Its expression is as follows: X0=Conv 16_3×3 (X input ); Where X0 represents the feature image after convolution, X input Conv represents the preprocessed image input to the model. 16_3×3 This indicates a convolution calculation with a kernel size of 3×3 and 16 channels; Secondly, the image is downsampled through a 2×2 max pooling layer. Each downsampling reduces the image size to 1 / 4 of the original size. After downsampling, global and local vascular features are extracted through an axial transformer and a 3×3 convolutional layer with doubled channels. The downsampling and feature extraction are performed four times. After each feature extraction, the ReLU nonlinear activation function is used to increase the model's nonlinear modeling ability and anti-interference ability. The feature image X obtained by the convolutional neural network after the i-th downsampling. i The expression is as follows: ； Among them, Conv Up_3×3 Maxpool represents a convolutional layer with a kernel size of 3×3 and double the number of channels. 2×2 This represents max pooling with a size of 2×2; The feature image Y obtained by the axial transformer after the i-th downsampling is i The expression is as follows: ； Where Axial_T represents the axial transformer, Maxpool 2×2 This represents max pooling of size 2×2, and when i=1, Y i-1 =X0; Step 3.2, Feature Fusion: First, the two smallest feature maps obtained through the axial transformer and convolutional neural network are concatenated and upsampling decoding begins. Upsampling is performed using deconvolution, halving the number of image channels and doubling the height and width with each upsampling step. After upsampling, the ReLU non-linear activation function is used to enhance the model's non-linear modeling ability and robustness. A total of four upsampling operations are performed. After the first upsampling, the expression for the feature image Z1 is: ； Among them, Up_Conv 3×3 This indicates a deconvolution operation with a kernel size of 3×3 and half the number of channels; Connect indicates a concatenation operation between two images. Then, the feature maps obtained during downsampling and the same size as those obtained during upsampling are stitched together using skip connections, fusing high-resolution features from different scales in the feature extraction part. A total of three image stitching operations are performed. After the first stitching, the expression for the obtained feature image W1 is as follows: Among them, Conv down_3×3 This indicates a convolution calculation with a kernel size of 3×3 and half the number of channels.
2. The retinal vessel image segmentation method according to claim 1, characterized in that, Step 2 involves preprocessing the retinal vessel images within the dataset. The specific steps are as follows: Step 2.1: Convert the color retinal vessel RGB image to a grayscale image: The three channels of a color retinal blood vessel RGB image are assigned different weights, and it is then converted into a grayscale image using the following formula: ； Among them, P gray For the converted grayscale image, P red P green P blue These are the red channel image, green channel image, and blue channel image of a color retinal blood vessel RGB image; Step 2.2, compress the grayscale image using normalization: The normalization formula is as follows: ； Among them, P norm The image after normalization. The grayscale images P are respectively gray Standard deviation and variance of pixels; Step 2.3: Use contrast-limited adaptive histogram equalization to improve the normalized image P. norm To enhance the details and contrast of the image; Step 2.4: Use the Local Adaptive Gamma Transformation algorithm to adjust the image; Step 2.5: Perform data augmentation on the images in the dataset to obtain sufficient sample images.
3. The retinal vessel image segmentation method according to claim 2, characterized in that, The data augmentation of the images in the dataset in step 2.5 specifically involves translating, rotating, and adding noise to the images in the dataset.
4. The retinal vessel image segmentation method according to claim 1, characterized in that, It also includes retinal vessel image segmentation: The training of retinal vessel image segmentation uses the cross-entropy loss function, and the loss function is optimized using the Adam algorithm. The initial learning rate is 0.01 and the momentum is 0.
9. The retinal vessel image is finely segmented using the Softmax activation function. The probability of each pixel in the image being a vessel or background is obtained by the softmax nonlinear function. A threshold of 0.5 is used. If the probability of a pixel being a vessel exceeds 0.5, the pixel is classified as a vessel; otherwise, it is classified as background, thus obtaining the final retinal vessel image.