Fundus vascular segmentation method based on fast label extraction and adaptive topology enhancement
Through deep learning algorithms with fast label extraction and adaptive topology enhancement, the problem of poor distinction of artificial annotation and thick and thin blood vessels in the prior art is solved, and efficient fundus vascular segmentation and increased sensitivity to small blood vessels are achieved.
Patent Information
- Application Number
- CN202210265279.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-03-17
- Publication Date
- 2025-05-23
- Estimated Expiration
- 2042-03-17
AI Technical Summary
The existing deep learning fundus vascular segmentation method relies on a large amount of manual annotation, which consumes a lot of manpower and material costs, and cannot effectively distinguish between coarse blood vessels and fine blood vessels.
A deep learning fundus vascular segmentation algorithm based on fast label extraction and adaptive topology enhancement is proposed, and training tags are automatically generated using optimal directional gradient flux filtering and vascular skeleton tracking, and the vascular continuity and sensitivity to small blood vessels are improved through adaptive topology enhancement loss function.
It greatly reduces the human and material resources consumption of manual labeling, and improves the sensitivity of deep learning models to small blood vessels and the vascular coherence of segmentation results.
Smart Images
Figure CN114612448B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of medical image processing, specifically to the automatic extraction of fundus vascular labels to train a fundus vascular deep learning segmentation model to achieve automatic segmentation of fundus vascular. Background Art
[0002] Numerous studies have shown that fundus images obtained through non-invasive examinations can provide important evidence for the diagnosis of diseases such as diabetic retinopathy, cardiovascular disease, and hypertension. The segmentation of retinal vessels plays a vital role in clinical diagnosis for the subsequent analysis of vascular characteristics (such as tortuosity and width). However, manual segmentation of vessels from fundus images heavily relies on the doctor's experience and is a very time-consuming task. Therefore, a high-precision and high-speed automatic vessel segmentation method is urgently needed.
[0003] Fundus image blood vessel segmentation algorithms have always attracted the attention of scholars at home and abroad, and new methods have been proposed. Generally speaking, the existing fundus blood vessel segmentation algorithms can be divided into three categories: (1) Segmentation methods based on traditional image processing usually perform segmentation based on the morphological characteristics of blood vessels, such as matching filtering, blood vessel tracking, morphological methods, and multi-scale methods. (2) Segmentation methods based on machine learning do not require pre-defined rules, but distinguish between blood vessels and non-blood vessels through learning. (3) Segmentation methods based on deep learning are a branch of machine learning methods, which complete blood vessel segmentation by training specific deep network models with data.
[0004] Segmentation methods are divided into supervised methods and unsupervised methods. Among the existing deep learning fundus segmentation methods, almost all methods are supervised methods. These supervised methods require a large number of fundus images manually annotated by medical experts to support the training of deep learning models. And labeling the categories of images pixel by pixel requires a lot of manpower and material costs. In order to overcome this problem, some unsupervised deep learning methods have been proposed. However, the segmentation effect of unsupervised methods is generally not as good as that of supervised learning methods.
[0005] Considering that blood vessels are linear topological structures, some studies have proposed deep learning methods for topological enhancement. These topological enhancement methods have a certain effect on the topological consistency of blood vessel segmentation results, but they do not distinguish between thick and thin blood vessels. Summary of the invention
[0006] In order to solve the strong dependence of supervised deep learning segmentation methods on manual labels, this paper proposes a deep learning fundus vascular segmentation algorithm based on fast label extraction and adaptive topological enhancement, which automatically generates training labels using optimal directional gradient flux filtering, vascular skeleton tracking and other steps, greatly reducing the human and material resource consumption of manual labeling. In addition, we consider the difference between thick and thin blood vessels and propose an adaptive topological enhancement loss function to improve the vascular continuity and sensitivity to small blood vessels of the deep learning model vascular segmentation results.
[0007] In order to achieve the above object, the method of the present invention comprises the following steps:
[0008] Step 1: Perform the best directional gradient flux filtering on the training set images to obtain the best directional gradient flux response, which includes the best blood vessel measurement value M and the corresponding blood vessel direction of each pixel;
[0009] Step 2: Use the adaptive threshold method on the vascular metric value in the best directional gradient flux response to segment the image into the main vascular structure map;
[0010] Step 3: Use the local maximum algorithm to search for blood vessels on the blood vessel metric values to obtain the blood vessel skeleton graph s;
[0011] Step 4: Count the difference in average grayscale values of the neighborhoods on both sides of each vascular skeleton s, and delete the vascular skeletons with large differences to eliminate false positives at the edges of the optic disc and pathological tissue;
[0012] Step 5: Using the vascular skeleton as a constraint, remove the false positives in the main vascular structure map to obtain the coarse vascular label g thick ;
[0013] Step 6: Use the vascular skeleton to supplement the small blood vessels missing in the main blood vessel map, perform AND operations with the main blood vessel structure map, and perform morphological algorithms (opening and closing operations) to improve the image and obtain a fine binary blood vessel label g; at the same time, obtain the small blood vessel label g thin and thick blood vessel labels g thick ;
[0014] Step 7: Use the fundus image and the corresponding fine binary vascular map g as training set labels and pass them into the deep learning model. Combined with the adaptive topology enhancement loss function, a deep learning model specifically for fundus image vascular segmentation is trained.
[0015] Step 8: Save the learned model. When you need to process a new fundus image blood vessel segmentation task, input the image into the model and output the segmentation result.
[0016] Furthermore, in step 1, the optimal directional gradient flux response is a symmetric matrix Q(x,r), and through similar diagonalization, its two eigenvalues λ about position x can be obtained: 1 (x,r),λ 2 (x,r) and the corresponding eigenvector ω 1 (x,r),ω 2 (x,r); then the optimal directional gradient flux response can be decomposed as follows:
[0017] Q(x,r)=λ 1 (x,r)ω 1 (x,r)ω 1 T (x,r)+λ 2 (x,r)ω 2 (x,r)ω 2 T (x,r)
[0018] Let λ 1 ≤λ 2 , then for a position x in the training set image, the feature vector ω 1 represents the vascular direction, ω 2 Represents the direction of the blood vessel, and the blood vessel measurement value M(x) at position x is calculated according to the following formula:
[0019]
[0020] Where R scale is a series of different scales used for multi-scale detection, and r is the corresponding scale.
[0021] Furthermore, in step 2, the adaptive threshold method is used to avoid the influence of uneven light intensity and contrast on the segmentation. The adaptive threshold method calculates the average value of the pixel window around each pixel, and calculates a suitable threshold for each pixel based on the given blood vessel foreground ratio. Then, the threshold method is used to segment the blood vessel into a binary image, in which the foreground is the main blood vessel image.
[0022] Furthermore, in step 3, each pixel in the image is checked. For a pixel x, in the direction of its blood vessel ω 2 Orthogonal direction On the graph, take two points A and B with a distance ζ from point x:
[0023]
[0024]
[0025] Search for the local maximum on the line segment connecting A and B; set the local maximum point to 1, otherwise set it to 0. After traversing every pixel in the image, the local maximum map can be obtained; remove the intersections in the local maximum map within the 8-neighborhood to obtain a local maximum map without intersections; finally, only retain the skeletons with skeleton lengths greater than a given threshold of 20 in the local maximum map without intersections to obtain a vascular skeleton map without background noise.
[0026] Furthermore, in step 4, first, the neighborhoods on both sides of each skeleton are intercepted. Specifically for a single vascular skeleton, the neighborhood of the skeleton is obtained by taking a neighborhood with a radius of l centered on each skeleton pixel and taking the union of all pixel neighborhoods. Since the neighborhood of the skeleton can be divided into two parts by the skeleton curve and the extension line of its two endpoints, the neighborhoods on both sides of the skeleton can be obtained. Then, the average grayscale value on both sides of each vascular skeleton is calculated by counting the grayscale values of all pixels in the neighborhood and summing them up, and then dividing them by the number of pixels in the neighborhood. Finally, the skeletons whose average grayscale values on both sides differ by more than a given threshold of 0.3 are deleted to complete the false positive removal of the vascular skeleton.
[0027] Furthermore, in step 5, the main vessel image pixels far from the skeleton are excluded based on the skeleton, which is equivalent to retaining only the vessel pixels in the neighborhood of the skeleton in the main vessel image, and obtaining the coarse vessel label g thick .
[0028] Furthermore, in step 6, the vessel skeleton is expanded by two pixels and then aligned with the thick vessel label g thick The combined image is opened (the image is corroded first, then expanded) to eliminate the fine noise in the image, and then closed (the image is expanded first, then corroded) to eliminate the holes in the image, while smoothing the blood vessel edges; the thin blood vessel label g thin =gg thick .
[0029] Furthermore, in step 7, the fundus image is predicted by the model to obtain the prediction result y, and the difference between y and the label g is calculated by the proposed loss function L. The loss function not only considers the correctness of each pixel, but also takes into account the difference between thick and thin blood vessels, and strengthens the weights of thick and thin blood vessels respectively through parameters α and β:
[0030]
[0031] Where y is the predicted value and g is the label value. is the Hadamard product. bce is the discrete cross entropy loss function:
[0032]
[0033] Where N represents the total number of pixels, g i and i Represent the label and predicted value of the i-th pixel respectively.
[0034] The beneficial effects of the present invention are: using a method based on optimal directional gradient flux to automatically extract blood vessels in fundus images as labels to train deep learning models, while eliminating the influence of pathological areas, low-quality imaging areas and optic disc edges in fundus images on blood vessel labels. Using automatically extracted labels to train deep learning models greatly improves the dependence of deep learning models on manual labels and saves manpower and material resources. In addition, the adaptive topological enhancement loss function proposed by this method can effectively improve the blood vessel coherence and sensitivity to small blood vessels in the segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] Figure 1 is a flow chart of the method of the present invention.
[0036] Figure 2a It is an image of the fundus.
[0037] Figure 2b It is a combination Figure 2a Automatically generated thick and thin blood vessel label map.
[0038] Figure 3a Another fundus image
[0039] Figure 3b It is combined with deep learning model Figure 3a Schematic diagram of the blood vessel segmentation results. DETAILED DESCRIPTION
[0040] The present invention will be further described below in conjunction with the accompanying drawings.
[0041] like Figure 1 As shown in FIG. 1 , the deep learning fundus vessel segmentation algorithm based on fast label extraction and adaptive topology enhancement includes the following steps:
[0042] Step 1: Perform the best directional gradient flux filtering on the training set images to obtain the best directional gradient flux response, which includes the best blood vessel measurement value M and the corresponding blood vessel direction of each pixel;
[0043] Step 2: Use the adaptive threshold method on the vascular metric value in the best directional gradient flux response to segment the image into the main vascular structure map;
[0044] Step 3: Use the local maximum algorithm to search for blood vessels on the blood vessel metric values to obtain the blood vessel skeleton graph s;
[0045] Step 4: Count the difference in average grayscale values of the neighborhoods on both sides of each vascular skeleton s, and delete the vascular skeletons with large differences to eliminate false positives at the edges of the optic disc and pathological tissue;
[0046] Step 5: Using the vascular skeleton as a constraint, remove the false positives in the main vascular structure map to obtain the coarse vascular label g thick ;
[0047] Step 6: Use the vascular skeleton to supplement the small blood vessels missing in the main blood vessel map, perform AND operations with the main blood vessel structure map, and perform morphological algorithms (opening and closing operations) to improve the image and obtain a fine binary blood vessel label g; at the same time, obtain the small blood vessel label g thin and thick blood vessel labels g thick As shown in the figure, Figure 2b yes Figure 2a The thicker lines represent thicker blood vessel labels, and the thinner lines represent thinner blood vessel labels.
[0048] Step 7: Use the fundus image and the corresponding fine binary vascular map g as training set labels and pass them into the deep learning model. Combined with the adaptive topology enhancement loss function, a deep learning model specifically for fundus image vascular segmentation is trained.
[0049] Step 8: Save the learned model. When you need to process a new fundus image blood vessel segmentation task, input the image into the model and output the segmentation result. Figure 3a The prediction result after the fundus image of is input into the model is Figure 3b .
[0050] Furthermore, in step 1, the optimal directional gradient flux response is a symmetric matrix Q(x,r), and through similar diagonalization, its two eigenvalues λ about position x can be obtained: 1 (x,r),λ 2 (x,r) and the corresponding eigenvector ω 1 (x,r),ω 2 (x,r); then the optimal directional gradient flux response can be decomposed as follows:
[0051] Q(x,r)=λ 1 (x,r)ω 1 (x,r)ω 1 T (x,r)+λ 2 (x,r)ω 2 (x,r)ω 2 T (x,r)
[0052] Let λ 1 ≤λ 2, then for a position x in the training set image, the feature vector ω 1 represents the vascular direction, ω 2 Represents the direction of the blood vessel, and the blood vessel measurement value M(x) at position x is calculated according to the following formula:
[0053]
[0054] Where R scale is a series of different scales used for multi-scale detection, and r is the corresponding scale.
[0055] Embodiment 1:
[0056] In step 1, take the range R of multi-scale detection scale =[1,8].
[0057] Furthermore, in step 2, the adaptive threshold method is used to avoid the influence of uneven light intensity and contrast on the segmentation. The adaptive threshold method calculates the average value of the pixel window around each pixel, and calculates a suitable threshold for each pixel based on the given blood vessel foreground ratio. Then, the threshold method is used to segment the blood vessel into a binary image, in which the foreground is the main blood vessel image.
[0058] Furthermore, in step 3, each pixel in the image is checked. For a pixel x, in the direction of its blood vessel ω 2 Orthogonal direction On the graph, take two points A and B with a distance ζ from point x:
[0059]
[0060]
[0061] Search for the local maximum on the line segment connecting A and B; set the local maximum point to 1, otherwise set it to 0. After traversing every pixel in the image, the local maximum map can be obtained; remove the intersections in the local maximum map within the 8-neighborhood to obtain a local maximum map without intersections; finally, only retain the skeletons with skeleton lengths greater than a given threshold of 20 in the local maximum map without intersections to obtain a vascular skeleton map without background noise.
[0062] Embodiment 2:
[0063] In step 3, the search range ζ=10.
[0064] Furthermore, in step 4, first, the neighborhoods on both sides of each skeleton are intercepted. Specifically for a single vascular skeleton, the neighborhood of the skeleton is obtained by taking a neighborhood with a radius of l centered on each skeleton pixel and taking the union of all pixel neighborhoods. Since the neighborhood of the skeleton can be divided into two parts by the skeleton curve and the extension line of its two endpoints, the neighborhoods on both sides of the skeleton can be obtained. Then, the average grayscale value on both sides of each vascular skeleton is calculated by counting the grayscale values of all pixels in the neighborhood and summing them up, and then dividing them by the number of pixels in the neighborhood. Finally, the skeletons whose average grayscale values on both sides differ by more than a given threshold of 0.3 are deleted to complete the false positive removal of the vascular skeleton.
[0065] Furthermore, in step 5, the main vessel image pixels far from the skeleton are excluded based on the skeleton, which is equivalent to retaining only the vessel pixels in the neighborhood of the skeleton in the main vessel image, and obtaining the coarse vessel label g thick .
[0066] Furthermore, in step 6, the vessel skeleton is expanded by two pixels and then aligned with the thick vessel label g thick The combined image is opened (the image is corroded first, then expanded) to eliminate the fine noise in the image, and then closed (the image is expanded first, then corroded) to eliminate the holes in the image, while smoothing the blood vessel edges; the thin blood vessel label g thin =gg thick .
[0067] Furthermore, in step 7, the fundus image is predicted by the model to obtain the prediction result y, and the difference between y and the label g is calculated by the proposed loss function L. The loss function not only considers the correctness of each pixel, but also takes into account the difference between thick and thin blood vessels, and strengthens the weights of thick and thin blood vessels respectively through parameters α and β:
[0068]
[0069] Where y is the predicted value and g is the label value. is the Hadamard product. bce is the discrete cross entropy loss function:
[0070]
[0071] Where N represents the total number of pixels, g i and i Represent the label and predicted value of the i-th pixel respectively.
[0072] Embodiment 4:
[0073] In step 7, the training set, validation set, and test set are divided into two parts in a ratio of 10:1:10.
[0074] Finally, it should be noted that the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, those skilled in the art may modify the technical solutions described in the aforementioned embodiments, or replace some of the technical features therein with equivalents to obtain other embodiments, which shall all fall within the scope of protection of the present invention.
Claims
1. Fundus vascular segmentation method based on fast label extraction and adaptive topology enhancement, Features The steps include: Step 1: Perform optimal directional gradient flux filtering on the training set images to obtain the optimal directional gradient flux response, which includes the optimal blood vessel measurement value M and the corresponding blood vessel direction of each pixel; including the following steps: The optimal directional gradient flux response is a symmetric matrix Q(x,r), which can be obtained by similar diagonalization with respect to the position x by two eigenvalues λ 1 (x,r),λ 2 (x,r) and the corresponding eigenvector ω 1 (x,r),ω 2 (x,r); then the optimal directional gradient flux response is decomposed as follows: Q(x,r)=λ 1 (x,r)ω 1 (x,r)ω 1 T (x,r)+λ 2 (x,r)ω 2 (x,r)ω 2 T (x,r) Let λ 1 ≤λ 2 , then for a position x in the training set image, the feature vector ω 1 represents the vascular direction, ω 2 Represents the direction of the blood vessel, and the blood vessel measurement value M(x) at position x is calculated according to the following formula: Where R scale is a series of different scales used for multi-scale detection, and r is the corresponding scale; Step 2: Use the adaptive threshold method on the vascular metric value in the best directional gradient flux response to segment the image into the main vascular structure map; Step 3: Use the local maximum algorithm to search for blood vessels on the blood vessel metric value to obtain the blood vessel skeleton s; Step 4: Count the difference in average grayscale values of the neighborhoods on both sides of each vascular skeleton s, and delete the vascular skeletons with large differences to eliminate false positives at the edges of the optic disc and pathological tissue; Step 5: Using the vascular skeleton as a constraint, remove the false positives in the main vascular structure map to obtain the coarse vascular label g thick ; Step 6: Use the vascular skeleton to supplement the small blood vessels missing in the main blood vessel map, perform operations with the main blood vessel structure map, and perform morphological algorithms to improve the image to obtain a fine binary blood vessel label g; at the same time, obtain the small blood vessel label g thin and thick blood vessel labels g thick ; comprising the following steps: The vascular skeleton is expanded by two pixels and then combined with the thick vessel label g thick The combined image is opened to eliminate the fine noise in the image. The opening operation is to corrode the image first and then expand it. The holes in the image are then eliminated by closing the image. The closing operation is to expand the image first and then corrode it. At the same time, the blood vessel edge is smoothed. The thin blood vessel label g thin =gg thick ; Step 7: Use the fundus image and the corresponding fine binary vascular map g as training set labels and pass them into the deep learning model. Combined with the adaptive topological enhancement loss function, a deep learning model specifically for fundus image vascular segmentation is trained; including the following steps: The fundus image is predicted by the model to obtain the prediction result y. The difference between y and the fine binary vessel label g is calculated by the proposed loss function L. The loss function not only considers the correctness of each pixel, but also takes into account the difference between thick and thin blood vessels. The weights of thick and thin blood vessels are strengthened respectively through the parameters α and β: Where y is the predicted value and g is the label value. is the Hadamard product, L bce is the discrete cross entropy loss function: Where N represents the total number of pixels, g i and i Represent the label and predicted value of the i-th pixel respectively; Step 8: Save the deep learning model obtained in step 7. When a new fundus image blood vessel segmentation task needs to be processed, input the image into the model and output the segmentation result.
2. The method for segmenting fundus blood vessels according to claim 1, Its characteristics are In the step 2, using an adaptive threshold method on the blood vessel measurement value in the best directional gradient flux response to segment the image into a main blood vessel structure map comprises the following steps: Adaptive thresholding method is used to avoid the influence of uneven light intensity and contrast on segmentation; The adaptive threshold method calculates the average value of the pixel window around each pixel point, and calculates the appropriate threshold for each pixel based on the given blood vessel foreground ratio; then the blood vessel measurement is segmented into a binary image through the threshold method, in which the foreground is the main blood vessel image.
3. The method for segmenting fundus blood vessels according to claim 1, Its characteristics are In the step 3: searching for blood vessels on the blood vessel metric values using a local maximum algorithm to obtain a blood vessel skeleton graph s; the steps include: Check each pixel in the training set image. For a pixel x, in the direction of its blood vessel ω 2 Orthogonal direction On the left, take two points with a distance of Point A, B: Search for the local maximum on the line segment connecting A and B; set the local maximum point to 1, otherwise set it to 0; after traversing each pixel in the training set image, the local maximum map can be obtained; remove the intersections in the local maximum map within the 8-neighborhood to obtain a local maximum map without intersections; finally, only retain the skeletons with skeleton lengths greater than a given threshold of 20 in the local maximum map without intersections to obtain a vascular skeleton map without background noise.
4. The method for segmenting fundus blood vessels according to claim 1, Its characteristics are In the step 4: counting the difference in average grayscale values of the neighborhoods on both sides of each vascular skeleton s, deleting vascular skeletons with large differences to eliminate false positives at the edges of the optic disc and pathological tissue; including: First, the neighborhoods on both sides of each skeleton are intercepted. Specifically for a single vascular skeleton, the neighborhood of the skeleton is obtained by taking a neighborhood with a radius of l centered on each skeleton pixel and taking the union of all pixel neighborhoods. Since the neighborhood of the skeleton is divided into two parts by the skeleton curve and the extension line of its two endpoints, the neighborhoods on both sides of the skeleton can be obtained. Then, the average grayscale value on both sides of each vascular skeleton is calculated by counting the grayscale values of all pixels in the neighborhood and summing them up, and then dividing them by the number of neighborhood pixels. Finally, the skeletons whose average grayscale value difference on both sides exceeds the given threshold of 0.3 are deleted, thus completing the false positive removal of the vascular skeleton.