Enhanced Feature Consistency Preprocessing Method for Retinal Vessel Image Segmentation

Abstract

An enhanced feature consistency preprocessing method for retinal vessel image segmentation provided by the present invention includes the following steps: S1. Obtain a retinal vessel image with labels, and preprocess the retinal vessel image with labels; S2. Construct an image segmentation network based on enhanced feature consistency preprocessing. The image segmentation network includes an enhanced feature consistency preprocessing network and a segmentation network. The output end of the enhanced feature consistency preprocessing network is connected to the input end of the segmentation network, and the segmentation network outputs a segmentation result; S3. Input the preprocessed retinal vessel image with labels into the image segmentation network based on enhanced feature consistency preprocessing to train the image segmentation network based on enhanced feature consistency preprocessing; S4. Obtain a real-time collected retinal vessel image, and input the real-time retinal vessel image into the trained image segmentation network based on enhanced feature consistency preprocessing to obtain a segmentation result after processing.

Description

Technical Field

[0001] The present invention relates to an image segmentation method, and particularly to an enhanced feature consistency preprocessing retinal vessel image segmentation method. Background Art

[0002] Retinal vessel image segmentation is of great significance in the diagnosis of retinal vascular diseases. The morphological structure of retinal vessels can significantly reflect early signs of various eye diseases. By segmenting retinal vessel images, vascular structure information such as vessel diameter, branch angle, and branch length can be obtained, which helps subsequent objective and standard quantitative analysis of related diseases.

[0003] Due to the differences between local vascular pixels in retinal vessel images, this difference includes two situations: The first situation: Due to unsuitable lighting conditions, the contrast between vessels and the background is low, making some vascular pixels blurred. The second situation: Due to factors such as retinal lesions and exudates, the characteristics of local vascular pixels may change greatly. Therefore, the existing retinal vessel image segmentation methods cannot accurately segment retinal vessel images, that is, their segmentation results have low accuracy. The reasons are as follows: (1) The structural redundancy information of retinal vessels is not fully utilized. (2) The problem that the characteristics of vascular pixels in the same region may have significant differences due to factors such as inappropriate lighting, exudate occlusion, and retinal lesions is ignored. Especially, vessels affected by noise are prone to cause the segmentation network to obtain semantic features with noise, thus easily resulting in the missegmentation of such vessels.

[0004] Therefore, in order to solve the above technical problems, it is urgent to propose a new technical means. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide an enhanced feature consistency preprocessing retinal vessel image segmentation method to solve the above technical problems.

[0006] An enhanced feature consistency preprocessing retinal vessel image segmentation method provided by the present invention includes the following steps:

[0007] S1. Obtain a retinal vessel image with labels and preprocess the retinal vessel image with labels;

[0008] S2. Construct an image segmentation network based on enhanced feature consistency preprocessing. The image segmentation network includes an enhanced feature consistency preprocessing network and a segmentation network. The output end of the enhanced feature consistency preprocessing network is connected to the input end of the segmentation network, and the segmentation network outputs a segmentation result;

[0009] S3. Input the preprocessed retinal vessel image with labels into the image segmentation network based on enhanced feature consistency preprocessing to train the image segmentation network based on enhanced feature consistency preprocessing;

[0010] S4. Obtain the real-time collected retinal vessel image, and input the real-time retinal vessel image into the trained image segmentation network based on enhanced feature consistency preprocessing to obtain the segmentation result.

[0011] Furthermore, the enhanced feature consistency preprocessing network includes a dynamic edge convolution module, a dimension transformation module, and a local class activation mapping module;

[0012] The output end of the dynamic edge convolution module is connected to the input end of the dimension transformation module, the output end of the dimension transformation module is connected to the input end of the local class activation mapping module, and the output end of the local class activation mapping module outputs the preprocessing result to the input end of the segmentation network;

[0013] The dynamic edge convolution module maps the preprocessed retinal vessel image into a graph structure;

[0014] The dimension transformation module converts the graph structure output by the dynamic edge convolution module into a feature map and inputs the feature map into the local class activation mapping module;

[0015] The local class activation mapping module maps the feature map into an activation map and inputs the activation map into the segmentation network.

[0016] Furthermore, the dynamic edge convolution module includes three layers of edge convolution, and the feature dimensions of the three layers of edge convolution from the input to the output direction are 32, 64, and 128 respectively;

[0017] The dynamic edge convolution module converts the input retinal vessel image into a graph node matrix;

[0018] Use the K-nearest neighbor algorithm to find the central node x i 's K-neighborhood N K (x), calculate the edge weight between the central node x i and its neighbor node x j ∈N K (x):

[0019] e′ ijm =LeakyReLU(θ m ·x i +φ m ·x j )

[0020] Where: represents the inner product, θ m and φ m represent the same as the central node xi Trainable parameters with the same dimension, M represents the number of filters in the dynamic edge convolution module, and m represents the m-th filter; LeakyReLU(·) represents the activation function;

[0021] Calculate node x i The feature x' of the m-th dimension im :

[0022]

[0023] Form a graph node matrix with the features of all dimensions of all nodes, and then process this graph node matrix through a dimension transformation module to form a feature map X in , and then the feature map X in is input into the local class activation mapping module.

[0024] Furthermore, the segmentation network is a U-net network.

[0025] Furthermore, the loss function of the image segmentation network based on enhanced feature consistency preprocessing is L:

[0026] L = Loss seg + Loss pa ;

[0027] The loss function Loss of the enhanced feature consistency preprocessing network pa is:

[0028]

[0029] where: is the predicted value of the enhanced feature consistency preprocessing network, is the generated label γ > 0 is an adjustable factor;

[0030]

[0031] The loss function Loss of the segmentation network seg is:

[0032]

[0033] where: represents the prediction result of the segmentation network, represents the ground truth annotation.

[0034] Furthermore, the processing process of the local class activation mapping module is as follows:

[0035] A c = Conv 1×1 (X' in )

[0036] A c represents the activation map, X' in represents the matrix feature map X in the feature map processed by the dimensionality transformation module;

[0037] Calculate the classification probability S c :

[0038] S c = Sigmoid(AvgPool S×S (A c ));

[0039] S represents the number of chunks, A c is divided into S×S chunks, B represents the number of classes;

[0040] Convert the segmentation label of class B into a vector L c , and then the vector L c is processed by max pooling operation as the generated label for each chunk

[0041] Furthermore, preprocess the retinal vessel image with labels by randomly slicing the retinal vessel image.

[0042] Advantages of the present invention: Through the present invention, the graph structure conversion is carried out through the dynamic edge convolution module, and the structural redundancy information of blood vessels is simulated and transmitted by means of the graph structure, so as to effectively prevent the loss of structural redundancy information in the image, and the relationship between pixels and classes is established through local class activation mapping to enhance the consistency of intra-class features and reduce the influence degree of noise on the segmentation result, thereby reducing the mis-segmentation of occluded blood vessels and blurred blood vessels;

[0043] After preprocessing by the enhanced feature consistency preprocessing network, it can effectively reduce the specificity of blood vessels in the input image in advance and highlight the features conducive to segmentation, so as to help the segmentation network better understand the input image, thereby improving the performance of the segmentation network and ensuring the final segmentation accuracy. Brief Description of the Drawings

[0044] The present invention will be further described below in conjunction with the drawings and embodiments:

[0045] Figure 1 is the flow schematic diagram of the present invention.

[0046] Figure 2 is the structural schematic diagram of the image segmentation network based on enhanced feature consistency preprocessing of the present invention.

[0047] Figure 3 is the schematic diagram of the principle of the enhanced feature consistency preprocessing network of the present invention. Detailed Implementation Modes

[0048] The following further elaborates on the present invention in detail:

[0049] A method for enhancing feature consistency preprocessing of retinal vessel image segmentation provided by the present invention includes the following steps:

[0050] S1. Obtain retinal vessel images with labels and preprocess the retinal vessel images with labels;

[0051] S2. Construct an image segmentation network based on enhanced feature consistency preprocessing. The image segmentation network includes an enhanced feature consistency preprocessing network and a segmentation network. The output end of the enhanced feature consistency preprocessing network is connected to the input end of the segmentation network, and the segmentation network outputs a segmentation result. Among them, the segmentation network is a U-net network, that is, the English abbreviation of the U-shaped Network based on deep learning;

[0052] S3. Input the preprocessed retinal vessel images with labels into the image segmentation network based on enhanced feature consistency preprocessing to train the image segmentation network based on enhanced feature consistency preprocessing;

[0053] S4. Obtain real-time collected retinal vessel images and input the real-time retinal vessel images into the trained image segmentation network based on enhanced feature consistency preprocessing to obtain a segmentation result; Through the above method, the graph structure is converted through the dynamic edge convolution module, and the graph structure is used to simulate and transmit the structural redundant information of blood vessels, thereby effectively preventing the loss of structural redundant information in the image. And the relationship between pixels and classes is established through local class activation mapping to enhance the consistency of intra-class features and reduce the influence of noise on the segmentation result, thereby reducing the mis-segmentation of occluded blood vessels and blurred blood vessels;

[0054] After preprocessing by the enhanced feature consistency preprocessing network, it can effectively reduce the specificity of blood vessels in the input image in advance and highlight the features beneficial to segmentation, so as to help the segmentation network better understand the input image, thereby improving the performance of the segmentation network and ensuring the final segmentation accuracy.

[0055] In this embodiment, the enhanced feature consistency preprocessing network includes a dynamic edge convolution module, a dimension transformation module, and a partial class activation mapping module. Among them, the dynamic edge convolution module is a graph-based dynamic edge convolution network, with the full English name being Dynamic EdgeConv, abbreviated as Dec. The full name of the partial class activation mapping module should be Partial Class Activation Mapping, abbreviated as PCAM, which is a method designed for fully supervised pixel-level semantic segmentation tasks. It can generate activation maps to represent the weighted relationship between pixels and classes, and indirectly improve the within-class consistency through the guidance of the loss function. The dimension transformation module is a Reshape module;

[0056] The output end of the dynamic edge convolution module is connected to the input end of the dimension transformation module, the output end of the dimension transformation module is connected to the input end of the partial class activation mapping module, and the output end of the partial class activation mapping module outputs the preprocessing result to the input end of the segmentation network;

[0057] The dynamic edge convolution module maps the preprocessed retinal blood vessel image into a graph structure;

[0058] The dimension transformation module converts the graph structure output by the dynamic edge convolution module into a feature map and inputs the feature map into the partial class activation mapping module;

[0059] The partial class activation mapping module maps the feature map into an activation map and inputs the activation map into the segmentation network.

[0060] In this embodiment, the dynamic edge convolution module includes three layers of edge convolution, and the feature dimensions of the three layers of edge convolution from the input to the output direction are 32, 64, and 128 respectively;

[0061] The dynamic edge convolution module converts the input retinal blood vessel image into a graph node matrix;

[0062] Use the K-nearest neighbor algorithm to find the K-nearest neighborhood N i of the central node x K (x), and calculate the edge weight between the central node x i and its neighbor node x j ∈N K (x):

[0063] e′ ijm =LeakyReLU(θ m ·x i +φ m ·x j )

[0064] Where: represents the inner product, and θ m and φ m represent the vectors related to the central node xi Trainable parameters with the same dimension, M represents the number of filters of the dynamic edge convolution module, and m represents the m-th filter; LeakyReLU(·) represents the activation function;

[0065] Calculate node x i The feature x' of the m-th dimension im :

[0066]

[0067] Form a graph node matrix with the features of all dimensions of all nodes, and then form a feature map X after processing the graph node matrix through a dimension transformation module in , and then the feature map X in is input into the local class activation mapping module.

[0068] In this embodiment, the loss function of the image segmentation network based on enhanced feature consistency preprocessing is L:

[0069] L = Loss seg + Loss pa ;

[0070] The loss function Loss of the enhanced feature consistency preprocessing network pa is:

[0071]

[0072] Where: is the predicted value of the enhanced feature consistency preprocessing network, is the generated label γ > 0 is an adjustable factor;

[0073]

[0074] The loss function Loss of the segmentation network seg is:

[0075]

[0076] Where: represents the prediction result of the segmentation network, represents the true annotation.

[0077] In this embodiment, the processing process of the local class activation mapping module is as follows:

[0078] A c = Conv 1×1 (X in );

[0079] A c represents the activation map, Xin Represents the feature map output by the dimensionality transformation module;

[0080] Calculate the classification probability S c :

[0081] S c = Sigmoid(AvgPool S×S (A c ));

[0082] S represents the number of chunks, A c is divided into S×S chunks, and B represents the number of categories;

[0083] Convert the segmentation label of class B to vector L c , and then vector L c After performing max pooling operation, it is used as the generated label for each chunk As Figure 3 shown:

[0084] The principle of PCAM and DEc to enhance feature consistency is as Figure 3 shown. After constructing the graph, the feature of node x1 is updated by its largest learnable edge weight. To reduce the influence of the noisy node x4 on x1, the graph structure output by dynamic edge convolution is converted into a feature map and input into PCAM. After convolution calculation of the weighted relationship between pixels and corresponding categories, the pixel points converted from node x1 are given the opportunity to adjust their own features. Then, after adaptive average pooling and activation function, the prediction result is output. The prediction result is compared with the true label, and backpropagation adjusts the weighted relationship between pixels and corresponding categories to improve the global representation of the same category, and then adjusts the edge weights between nodes, and further adjusts the relationship between pixels in this process, such as weakening the association relationship between x1 and x4. When the next image is input, dynamic edge convolution has the ability to better adjust the edge weights between noisy nodes and vascular nodes in a similar situation.

[0085] The whole process of the network composed of PCAM and DEC automatically learning and enhancing intra-class feature consistency is as Figure 3 shown. Through backpropagation, the activation map output by the 1×1 convolutional layer in PCAM and the edge weights between nodes can be automatically adjusted. In this process, the relationship between pixels is adjusted, and the global representation of the same category is improved, thereby reducing the influence degree of node x4 on x1.

[0086] In this embodiment, the preprocessing of the retinal vessel image with labels is to perform random slicing on the retinal vessel image.

[0087] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An enhanced feature consistency preprocessing method for retinal vessel image segmentation, characterized in that: Including the following steps: S1. Obtain a retinal vessel image with labels, and preprocess the retinal vessel image with labels; S2. Construct an image segmentation network based on enhanced feature consistency preprocessing. The image segmentation network includes an enhanced feature consistency preprocessing network and a segmentation network. The output end of the enhanced feature consistency preprocessing network is connected to the input end of the segmentation network, and the segmentation network outputs a segmentation result; S3. Input the preprocessed retinal vessel image with labels into the image segmentation network based on enhanced feature consistency preprocessing to train the image segmentation network based on enhanced feature consistency preprocessing; S4. Obtain a real-time acquired retinal vessel image, and input the real-time retinal vessel image into the trained image segmentation network based on enhanced feature consistency preprocessing to obtain a segmentation result through processing; The enhanced feature consistency preprocessing network includes a dynamic edge convolution module, a dimension transformation module, and a local class activation mapping module; The output end of the dynamic edge convolution module is connected to the input end of the dimension transformation module, the output end of the dimension transformation module is connected to the input end of the local class activation mapping module, and the output end of the local class activation mapping module outputs a preprocessing result to the input end of the segmentation network; The dynamic edge convolution module maps the preprocessed retinal vessel image into a graph structure; The dimension transformation module converts the graph structure output by the dynamic edge convolution module into a feature map and inputs the feature map into the local class activation mapping module; The local class activation mapping module maps the feature map into an activation map and inputs the activation map into the segmentation network; The dynamic edge convolution module includes three layers of edge convolution. The feature dimensions of the three layers of edge convolution from the input to the output direction are 32, 64, and 128 respectively; The dynamic edge convolution module converts the input retinal vessel image into a graph node matrix; Use the K-nearest neighbor algorithm to find the central node x i 's K-nearest neighborhood N K (x), calculate the central node x i and its neighbor node x j ∈ N K (x)'s edge weights: e′ ijm = LeakyReLU(θ m ·x i + φ m ·x j ) where: · denotes the inner product, θ m and φ m represent trainable parameters with the same dimension as the central node x i M represents the number of filters of the dynamic edge convolution module, m represents the m-th filter; LeakyReLU(·) represents the activation function; Computing node x i Feature x' of the m-th dimension im : The features of all dimensions of all nodes are combined into a graph node matrix, and then the graph node matrix is processed by a dimension transformation module to form a feature map X in , and then the feature map X in is input into the local class activation mapping module.

2. The method for preprocessing retinal vessel image segmentation to enhance feature consistency according to claim 1, wherein: The segmentation network is a U-net network.

3. The method for preprocessing retinal vessel image segmentation to enhance feature consistency according to claim 2, wherein: The loss function of the image segmentation network based on enhanced feature consistency preprocessing is L: L = Loss seg + Loss pa ; Loss function of the preprocessing network for enhancing feature consistency pa is as follows: Wherein: is the predicted value of the enhanced feature consistency preprocessing network, is the generated label γ > 0 is an adjustable factor; Loss function of the segmentation network seg is as follows: Wherein: represents the prediction result of the segmentation network, represents the ground truth annotation.

4. The method for preprocessing retinal vessel image segmentation to enhance feature consistency according to claim 3, wherein: The processing process of the local class activation mapping module is as follows: A c = Conv 1×1 (X in ); A c represents the activation map, X in represents the feature map output by the dimensionality transformation module; Calculate the classification probability S c : S c = Sigmoid(AvgPool S×S (A c )); S represents the number of cut blocks, A c is divided into S×S blocks, and B represents the number of categories; Convert the segmentation labels of class B into a vector L c , and then for the vector L c perform a max-pooling operation and use the result as the generated label for each block 5. The method for preprocessing retinal vessel image segmentation to enhance feature consistency according to claim 1, wherein: Preprocessing the retinal vessel image with labels is to perform random slicing on the retinal vessel image.

Images