Boundary attention-assisted dynamic graph convolution method for retinal vessel image segmentation
Through the boundary attention-assisted dynamic graph convolution method, combined with preprocessing, dynamic graph convolution U-shaped network and boundary attention network, the problem of inaccurate vascular feature extraction in retinal vascular image segmentation is solved, achieving higher segmentation accuracy and effectiveness.
Patent Information
- Application Number
- CN202211187195.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-09-28
- Publication Date
- 2025-08-12
- Estimated Expiration
- 2042-09-28
AI Technical Summary
The existing retinal vascular image segmentation method cannot accurately extract vascular feature information, resulting in poor segmentation accuracy and prone to problems of disconnection and missegment of noise.
The boundary attention-assisted dynamic graph convolution method is adopted to obtain accurate segmentation by pre-processing the retinal blood vessel image, combining dynamic graph convolution U-shaped network and boundary attention network.
It improves the accuracy and effectiveness of retinal vascular image segmentation, reduces noise interference and tiny blood vessel disconnection, and improves the accuracy of image segmentation.
Smart Images
Figure BDA0003868217380000035 
Figure BDA0003868217380000071 
Figure BDA0003868217380000076
Abstract
Description
Technical Field
[0001] The present invention relates to an image processing method, and in particular to a boundary attention-assisted dynamic graph convolution retinal blood vessel image segmentation method. Background Art
[0002] Retinal vascular images are an important basis for assessing retinal health status. Therefore, accurate retinal vascular images need to be obtained. In the existing technology, the segmentation of vascular image information in retinal vascular images mainly adopts manual methods. However, due to the complexity of vascular images and the influence of subjective factors of workers, this method cannot obtain accurate vascular image information. With the development of technology, although some neural network-based image processing methods have been proposed, the existing algorithms are still unable to accurately extract the characteristic information of blood vessels, resulting in poor accuracy of the final vascular image segmentation, which shows that the blood vessels are disconnected in the image and noise is mistakenly segmented as vascular pixels.
[0003] Therefore, in order to solve the above technical problems, it is urgent to propose a new technical means. Summary of the Invention
[0004] Therefore, the purpose of the present invention is to provide a boundary attention-assisted dynamic graph convolution retinal vascular image segmentation method, which can accurately segment and extract the vascular feature map in the retinal vascular image, effectively reduce the interference of noise in the retinal image on the vascular pixels, reduce the disconnection of tiny blood vessels in the segmentation result image, and improve the accuracy and effectiveness of image segmentation.
[0005] The present invention provides a boundary attention-assisted dynamic graph convolution retinal vessel image segmentation method, comprising the following steps:
[0006] S1. Preprocess the retinal vascular image, flip and slice the preprocessed image to obtain an image slice set with a size of 48×48 pixels;
[0007] S2. Input the image slice set into the dynamic graph convolutional U-network to extract vascular features and output a vascular feature map;
[0008] S3. Input the vascular feature map extracted from the U-shaped network into the boundary attention network for processing to extract the shallow boundary feature map;
[0009] S4. The vascular feature map output by the dynamic graph convolutional U-type network is fused with the shallow boundary feature map output by the boundary attention network to obtain the retinal vascular image segmentation result.
[0010] Furthermore, in step S1, preprocessing the retinal blood vessel image includes:
[0011] S101. Converting the retinal vascular image from an RGB image to a grayscale image F1;
[0012] S102. Normalize the grayscale image F1 to obtain a normalized image F2;
[0013] S103. Performing adaptive histogram equalization on image F2 to obtain image F3;
[0014] S104. Perform gamma correction on the image F3 to obtain a corrected image F4;
[0015] S105. Flip and randomly crop the corrected image to obtain an image slice set with a size of 48×48 pixels, create an index for each slice image, and save the index value.
[0016] Furthermore, in step S2, the blood vessel feature extraction specifically includes:
[0017] S21. The dynamic graph convolutional U-network includes an encoder and a decoder. The encoder extracts image features layer by layer. The encoder consists of four identical layers connected sequentially from input to output. Each layer has two convolution sequences and a pooling downsampling layer. After each layer, the output feature map of the slice image is reduced in size by half, while the channel dimension is doubled.
[0018] Among them: the output feature map of the second convolution of each layer in the encoder is transmitted to the decoder through the skip connection;
[0019] S22. The decoder recovers image information layer by layer, with four layers of identical structure from input to output. Each layer has two convolution sequences, a dynamic graph convolution, and a deconvolution upsampling layer. The image information input to the decoder doubles in size and halves in channel dimension each time it passes through a layer.
[0020] Furthermore, step S2 also includes a dynamic graph convolution process, which specifically includes:
[0021] Dynamic graph convolution is specifically:
[0022] The Reshape function is used to convert the feature map X output by each layer of the decoder into a graph structure G, where G = (V, A). is the graph node matrix, N represents the number of graph nodes, and C represents the dimension of graph node features; is the adjacency matrix, which represents the connection relationship between nodes;
[0023] Use the Reshape function to convert X into a graph node matrix W, H, and C are the width, height, and number of channels of the feature map, respectively;
[0024] Use graph convolution formula to extract the structural features of vascular images:
[0025] Z = σ(AX′W); where σ is the ReLU activation function, is the parameter matrix of the original image space;
[0026] The structural features of the vascular image are restored to a vascular feature map and added to the output result of the decoder layer corresponding to the current dynamic graph convolution.
[0027] Furthermore, the adjacency matrix A is determined by the following formula:
[0028]
[0029] in: represents the dot product; θ(·) contains 1×1 convolution and Reshape function, W θ and W ρ represents the weight of the adjacency matrix; It is a diagonal matrix representing the channel attention coefficient of the two-dimensional feature map, It is the feature of X after global average pooling, ρ includes 1×1 convolution and activation function Sigmoid;
[0030] After determining the adjacency matrix A, it is normalized using the softmax function.
[0031] Further, step S3 specifically includes:
[0032] S31. The 2-channel feature map output by the dynamic graph convolutional U-network is normalized by the softmax function and used as the boundary attention feature map;
[0033] S32. Use binary cross entropy loss function to supervise the boundary area of the boundary attention feature map;
[0034] S33. Splicing the boundary attention feature map supervised by the boundary region with the original image to form a 3-channel image;
[0035] S34. Process the 3-channel image through a single-layer convolution sequence to extract shallow boundary features; wherein the single-layer convolution sequence includes a 1×1 convolution, a batch normalization layer, and a Relu activation function.
[0036] Furthermore, step S4 specifically includes:
[0037] S41. Concatenate the feature map output by the U-shaped network and the feature map output by the boundary attention network to obtain feature map M1;
[0038] S42. Processing the feature map M1 through a convolution sequence to obtain a feature map M2; wherein the convolution sequence includes a 3×3 convolution, a batch normalization function, and a Relu activation function;
[0039] S43. After performing global average pooling, 1×1 convolution, Relu activation function, 1×1 convolution, and Sigmoid activation function on the feature map M2, a feature map M3 is obtained.
[0040] S44. Process the feature map M2 and the feature map M3 to obtain the feature map M4:
[0041] Where: k = 0, 1, ..., C, C represents the number of image channels, Represents the pixel in the i-th row and j-th column of the k-th channel of the feature map M2;
[0042] S45. Project the feature map M4 into a 2-channel image through 1×1 convolution processing to obtain the final blood vessel image segmentation result.
[0043] The beneficial effects of the present invention are as follows: Through the present invention, the vascular feature map in the retinal vascular image can be accurately segmented and extracted, effectively reducing the interference of noise in the retinal image on the vascular pixels, reducing the disconnection of tiny blood vessels in the segmentation result image, and improving the accuracy and effectiveness of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments:
[0045] Figure 1 Flowchart of the present invention.
[0046] Figure 2 It is a schematic diagram of the U-shaped network structure of the present invention.
[0047] Figure 3 Schematic diagram of the dynamic graph convolution process of the present invention. DETAILED DESCRIPTION
[0048] The present invention is further described in detail below:
[0049] The present invention provides a boundary attention-assisted dynamic graph convolution retinal vessel image segmentation method, comprising the following steps:
[0050] S1. Preprocess the retinal vascular image, flip and slice the preprocessed image to obtain an image slice set with a size of 48×48 pixels;
[0051] S2. Input the image slice set into the dynamic graph convolutional U-network to extract vascular features and output a vascular feature map;
[0052] S3. Input the vascular feature map extracted from the U-shaped network into the boundary attention network for processing to extract the shallow boundary feature map;
[0053] S4. The vascular feature map output by the dynamic graph convolutional U-type network is fused with the shallow boundary feature map output by the boundary attention network to obtain the retinal vascular image segmentation result. Through the above method, the vascular feature map in the retinal vascular image can be accurately segmented and extracted, effectively reducing the interference of noise in the retinal image on the vascular pixels, reducing the disconnection of tiny blood vessels in the segmentation result image, and improving the accuracy and effectiveness of image segmentation.
[0054] In this embodiment, in step S1, preprocessing the retinal blood vessel image includes:
[0055] S101. Converting the retinal vascular image from an RGB image to a grayscale image F1;
[0056] S102. Normalize the grayscale image F1 to obtain a normalized image F2;
[0057] S103. Performing adaptive histogram equalization on image F2 to obtain image F3;
[0058] S104. Perform gamma correction on the image F3 to obtain a corrected image F4;
[0059] S105. The corrected image is flipped and randomly cropped to obtain a set of image slices of 48×48 pixels. An index is created for each slice image and the index value is saved. The specific processes in steps S101-S105 are all implemented using existing technologies and will not be described in detail here.
[0060] In this embodiment, in step S2, the blood vessel feature extraction specifically includes:
[0061] S21. The dynamic graph convolutional U-network includes an encoder and a decoder. The encoder extracts image features layer by layer. The encoder consists of four identical layers connected sequentially from input to output. Each layer has two convolution sequences and a pooling downsampling layer. After each layer, the output feature map of the slice image is reduced in size by half, while the channel dimension is doubled.
[0062] Among them: the output feature map of the second convolution of each layer in the encoder is transmitted to the decoder through the skip connection;
[0063] S22. The decoder recovers the image information layer by layer, and has four layers with the same structure from input to output, each layer having two convolution sequences, dynamic graph convolution, and deconvolution upsampling layers; wherein: the image information input to the decoder is doubled in size and halved in channel dimension after each layer. The encoder and decoder of the U-shaped network can both adopt existing technologies; such as Figure 2 As shown in the figure: after each layer of processing in the encoder, the number of channel bits is doubled, while the scale of the feature map is reduced. Generally, it is reflected in the figure that the blocks representing the image gradually become shorter and wider; after each layer of processing in the decoder, the number of channel bits is halved, while the scale of the feature map is doubled, which is reflected in the figure that the blocks representing the image gradually become longer and narrower; the output feature map of the second convolution of each layer in the encoder is transmitted to the decoder through a jump connection, that is, the feature map of the second convolution output of the first layer of the encoder is transmitted to the first layer of the decoder, and the feature map of the second convolution output of the second layer of the encoder is transmitted to the second layer of the decoder; by processing in this way, the accuracy of the final result can be ensured.
[0064] Step S2 also includes a dynamic graph convolution process, specifically including:
[0065] Dynamic graph convolution is specifically:
[0066] The Reshape function is used to convert the feature map X output by each layer of the decoder into a graph structure G. Since the feature map output by each layer of the decoder is 3-channel, the 3-channel feature map is converted into a 2-channel map through the Reshape function, where G = (V, A). is the graph node matrix, N represents the number of graph nodes, and C represents the dimension of graph node features; is the adjacency matrix, which represents the connection relationship between nodes;
[0067] Use the Reshape function to convert X into a graph node matrix W, H, and C are the width, height, and number of channels of the feature map, respectively;
[0068] Use graph convolution formula to extract the structural features of vascular images:
[0069] Z = σ(AX′W); where σ is the ReLU activation function, is the parameter matrix of the original image space;
[0070] The vascular image structural features are restored into a vascular feature map and added to the output of the decoder layer corresponding to the current dynamic graph convolution. Since the output of the dynamic graph convolution is a 2-channel structure, it is necessary to restore the 2-channel output of the dynamic graph convolution to a 3-channel map. The restoration process is implemented using existing technology.
[0071] Among them, the dynamic graph convolution has a total of four layers and corresponds one-to-one to the number of layers of the decoder. The output of each layer of the decoder is input into the corresponding dynamic graph convolution for processing. Then, the feature map output by the corresponding decoder and the feature map after graph restoration of the output of the dynamic graph convolution are added and input into the next layer of the encoder for processing.
[0072] In this embodiment, the adjacency matrix A is determined by the following formula:
[0073]
[0074] in: represents the dot product; θ(·) contains 1×1 convolution and Reshape function, W θ and W ρ represents the weight of the adjacency matrix; It is a diagonal matrix representing the channel attention coefficient of the two-dimensional feature map, It is the feature of X after global average pooling, ρ includes 1×1 convolution and activation function Sigmoid;
[0075] After determining the adjacency matrix A, it is normalized using the softmax function.
[0076] In this embodiment, step S3 specifically includes:
[0077] S31. The 2-channel feature map output by the dynamic graph convolutional U-network is normalized by the softmax function and used as the boundary attention feature map;
[0078] S32. Use a binary cross entropy loss function to supervise the boundary area of the boundary attention feature map; through this step, the boundary loss in the processing process of the U-shaped network can be reduced, and the accuracy of the final result can be improved;
[0079] Among them: The binary cross entropy loss function is:
[0080]
[0081] represents the predicted value of the boundary area, Annotations representing boundary areas;
[0082] S33. Splicing the boundary attention feature map supervised by the boundary region with the original image to form a 3-channel image;
[0083] S34. Process the 3-channel image through a single-layer convolution sequence to extract shallow boundary features; wherein the single-layer convolution sequence includes a 1×1 convolution, a batch normalization layer, and a Relu activation function.
[0084] In this embodiment, step S4 specifically includes:
[0085] S41. Concatenate the feature map output by the U-shaped network and the feature map output by the boundary attention network to obtain feature map M1;
[0086] S42. Processing the feature map M1 through a convolution sequence to obtain a feature map M2; wherein the convolution sequence includes a 3×3 convolution, a batch normalization function, and a Relu activation function;
[0087] S43. After performing global average pooling, 1×1 convolution, Relu activation function, 1×1 convolution, and Sigmoid activation function on the feature map M2, a feature map M3 is obtained.
[0088] S44. Process the feature map M2 and the feature map M3 to obtain the feature map M4:
[0089] Where: k = 0, 1, ..., C, C represents the number of image channels, Represents the pixel in the i-th row and j-th column of the k-th channel of the feature map M2;
[0090] S45. Project the feature map M4 into a 2-channel image through 1×1 convolution processing to obtain the final blood vessel image segmentation result.
[0091] The batch normalization function, ReLU activation function, etc. mentioned above are all existing technologies. The specific processing process of each convolution sequence is also existing technology, and their principles are not described in detail here.
[0092] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the purpose and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A boundary attention-assisted dynamic graph convolution retinal vessel image segmentation method, characterized by: The following steps are involved: S1. Preprocess the retinal vascular image, flip and slice the preprocessed image to obtain an image slice set with a size of 48×48 pixels; S2. Input the image slice set into the dynamic graph convolutional U-network to extract vascular features and output a vascular feature map; S3. Input the vascular feature map extracted from the U-shaped network into the boundary attention network for processing to extract the shallow boundary feature map; Step S3 specifically includes: S31. The 2-channel feature map output by the dynamic graph convolutional U-network is normalized by the softmax function and used as the boundary attention feature map; S32. Use binary cross entropy loss function to supervise the boundary area of the boundary attention feature map; S33. Splicing the boundary attention feature map supervised by the boundary region with the original image to form a 3-channel image; S34. Process the 3-channel image through a single-layer convolution sequence to extract shallow boundary features; wherein the single-layer convolution sequence includes a 1×1 convolution, a batch normalization layer, and a ReLU activation function; S4. Fuse the vascular feature map output by the dynamic graph convolutional U-network with the shallow boundary feature map output by the boundary attention network to obtain the retinal vascular image segmentation result; Step S4 specifically includes: S41. Concatenate the feature map output by the U-shaped network and the feature map output by the boundary attention network to obtain feature map M1; S42. Processing the feature map M1 through a convolution sequence to obtain a feature map M2; wherein the convolution sequence includes a 3×3 convolution, a batch normalization function, and a Relu activation function; S43. After performing global average pooling, 1×1 convolution, Relu activation function, 1×1 convolution, and Sigmoid activation function on the feature map M2, a feature map M3 is obtained. S44. Process the feature map M2 and the feature map M3 to obtain the feature map M4: Where: k = 0, 1, ..., C, C represents the number of image channels, Represents the pixel in the i-th row and j-th column of the k-th channel of the feature map M2; S45. Project the feature map M4 into a 2-channel image through 1×1 convolution processing to obtain the final blood vessel image segmentation result.
2. The boundary attention-assisted dynamic graph convolution retinal vessel image segmentation method according to claim 1, characterized in that: In step S1, preprocessing the retinal blood vessel image includes: S101. Converting the retinal vascular image from an RGB image to a grayscale image F1; S102. Normalize the grayscale image F1 to obtain a normalized image F2; S103. Performing adaptive histogram equalization on image F2 to obtain image F3; S104. Perform gamma correction on the image F3 to obtain a corrected image F4; S105. Flip and randomly crop the corrected image to obtain an image slice set with a size of 48×48 pixels, create an index for each slice image, and save the index value.
3. The boundary attention-assisted dynamic graph convolution retinal vessel image segmentation method according to claim 2, characterized in that: In step S2, the blood vessel feature extraction specifically includes: S21. The dynamic graph convolutional U-network includes an encoder and a decoder. The encoder extracts image features layer by layer. The encoder consists of four identical layers connected sequentially from input to output. Each layer has two convolution sequences and a pooling downsampling layer. After each layer, the output feature map of the slice image is reduced in size by half, while the channel dimension is doubled. Among them: the output feature map of the second convolution of each layer in the encoder is transmitted to the decoder through the skip connection; S22. The decoder recovers image information layer by layer, with four layers of identical structure from input to output. Each layer has two convolution sequences, a dynamic graph convolution, and a deconvolution upsampling layer. The image information input to the decoder doubles in size and halves in channel dimension each time it passes through a layer.
4. The boundary attention-assisted dynamic graph convolution retinal vessel image segmentation method according to claim 3, characterized in that: Step S2 also includes a dynamic graph convolution process, which specifically includes: Dynamic graph convolution is specifically: The Reshape function is used to convert the feature map X output by each layer of the decoder into a graph structure G, where G = (V, A). is the graph node matrix, N represents the number of graph nodes, and C represents the dimension of graph node features; is the adjacency matrix, which represents the connection relationship between nodes; Use the Reshape function to convert X into a graph node matrix W, H, and C are the width, height, and number of channels of the feature map, respectively; Use graph convolution formula to extract the structural features of vascular images: Z = σ(AX′W); where σ is the ReLU activation function, is the parameter matrix of the original image space; The structural features of the vascular image are restored to a vascular feature map and added to the output result of the decoder layer corresponding to the current dynamic graph convolution.
5. The boundary attention-assisted dynamic graph convolution retinal vessel image segmentation method according to claim 4, characterized in that: The adjacency matrix A is determined by the following formula: in: represents the dot product; θ(·) contains 1×1 convolution and Reshape function, W θ and W ρ represents the weight of the adjacency matrix; It is a diagonal matrix representing the channel attention coefficient of the two-dimensional feature map, It is the feature of X after global average pooling, ρ includes 1×1 convolution and activation function Sigmoid; After determining the adjacency matrix A, it is normalized using the softmax function.