A thick and thin blood vessel classification method for high-precision blood vessel segmentation model training application

Abstract

A method for classifying thick and thin blood vessels for high-precision blood vessel segmentation model training application includes the following steps: Step 1: extracting blood vessel skeletons from fundus blood vessel labels; Step 2: calculating the ratio of blood vessel skeletons to blood vessel area in the window where each blood vessel skeleton pixel is located; Step 3: using a threshold to divide the blood vessel skeleton into thick blood vessel skeletons and thin blood vessel skeletons; Step 4: according to the classification result of the blood vessel skeleton, using the 8-neighborhood voting method to confirm the classification of other blood vessel pixels; Step 5: using a morphological method to reclassify the blood vessels located in the intersection area of ​​thick and thin blood vessel breaks to ensure the overall structure of the blood vessels. The method of the present invention is not only suitable for the training of neural network fundus blood vessel segmentation models based on deep learning, but also suitable for improving the segmentation performance of other supervised fundus blood vessel segmentation algorithms.

Description

Technical Field

[0001] The present invention relates to the technical field of retinal blood vessel segmentation, and in particular to a thick and thin blood vessel classification method for high-precision blood vessel segmentation model training application. Background Art

[0002] Retinal vascular structure contains important information that helps ophthalmologists detect and diagnose various retinal diseases, such as diabetic retinopathy, age-related macular degeneration, hypertensive retinopathy, etc. See reference [1] Mohamed Q, Gillies MC, Wong TY. Management of diabetic retinopathy: a systematic review. Jama, 2007, 298(8): 902–916. Changes in vascular morphology, such as shape, tortuosity and width, provide accurate early detection for many diseases, see reference [2] Srinidhi CL, Aparna P, Rajan J. Recent advances in retinal vessel segmentation. Journal of medical systems, 2017, 41(4): 70. Therefore, the segmentation of retinal vessels is particularly important for the diagnosis, treatment and surgical planning of retinal diseases, as described in the literature [3] Cheung CY, Zheng Y, Hsu W, et al. Retinal vascular tortuosity, blood pressure, and cardiovascular risk factors. Ophthalmology, 2011, 118(5): 812–818.

[0003] At present, the main method for fundus vessel segmentation is based on supervised learning, that is, using manually annotated labels to assist the model in training and learning parameters. Reference [4] Zhun Fan, Zhun Fan, Jiahong Wei, et al. Evolutionary Neural Architecture Search for Retinal Vessel Segmentation. arXiv preprint arXiv: 2001.06678, 2020. regards the U-Net model as a search space based on the encoder-decoder framework, and applies neural architecture search to retinal vessel segmentation. However, it treats fundus vessels equally with thick vessels when segmenting, which will cause the segmentation results to be biased towards thick vessels, reducing the accuracy of the segmentation results. Because the samples of thick and thin vessels in the fundus vessels are unbalanced, and the contrast between thin vessels and the background is lower than that of thick vessels, the segmentation of thin vessels is much more difficult than that of thick vessels.

[0004] An effective way to solve this problem is to treat thick blood vessels differently from thin blood vessels. Reference [5] Khan Bahadar Khan, Amir A Khaliq, Muhammad Shahid. A Morphological Hessian Based Approach for Retinal Blood Vessels Segmentation and Denoising Using Region Based Otsu Thresholding. PLoS ONE, 2016, 11(7): e0158996. The Hessian matrix is ​​used to extract and enhance thick and thin blood vessel images at two different scales, and then the region-based Otsu threshold is used to segment the blood vessels. However, the thin blood vessel image actually contains thick blood vessels, and only the thick blood vessels are actually enhanced.

[0005] Reference [6] Zengqiang Yan, Xin Yang, Kwang-Ting Cheng. A Three-Stage Deep Learning Model for Accurate Retinal Vessel Segmentation. IEEE Journal of Biomedical and Health Informatics, 2019, 23(4): 1427-1436. Realizing that the difficulty of segmenting thick and thin blood vessels is very different, the vascular skeleton is first extracted from the vascular label map, and the minimum inscribed circle centered on the vascular skeleton pixel and bounded by the vascular wall is calculated, and its diameter is used as the vascular thickness. For skeleton pixels whose vascular thickness is lower than a fixed threshold, all pixels covered by the minimum inscribed circle centered on it are represented as thin vessel pixels, and the remaining vessel pixels are classified as thick vessel pixels. The image of thick and thin blood vessels classified by this method has many breakpoints, which destroys the overall structure of the blood vessels and is not conducive to the subsequent segmentation of fundus blood vessels.

[0006] The fundus blood vessel dataset does not have label information of thick and thin blood vessel types, which makes the deep learning-based neural network treat the errors caused by thick and thin blood vessel pixels equally during training due to the lack of blood vessel type information, resulting in the inability to effectively improve the accuracy of thin blood vessel segmentation; and the deep learning-based neural network treats the errors caused by thick and thin blood vessel pixels equally due to the lack of blood vessel type information during training, resulting in the inability to effectively improve the accuracy of thin blood vessel segmentation. A method for quickly classifying thick and thin blood vessels is urgently needed, which can not only ensure the correct classification of thick and thin blood vessels, but also ensure the integrity of the overall structure of thick and thin blood vessels, and no cross-fracture areas of thick and thin blood vessels. Summary of the invention

[0007] The present invention proposes a method for classifying thick and thin blood vessels for high-precision blood vessel segmentation model training application. First, the blood vessel skeleton is extracted from the blood vessel label, and the ratio of the blood vessel area of ​​the label to the blood vessel skeleton area in the 8×8 circular window is calculated. The blood vessel pixels are classified into thick blood vessel pixels and thin blood vessel pixels according to the threshold; finally, the thick blood vessel spots and thin blood vessel spots located in the thick and thin blood vessel cross-fracture area are removed using the connected domain method, so as to obtain the best thick and thin blood vessel classification results, which is convenient for the subsequent blood vessel segmentation model to learn the thick and thin blood vessel features and achieve high-precision segmentation of fundus blood vessels. The invention is not only applicable to the training of neural network fundus blood vessel segmentation models based on deep learning, but also applicable to improving the segmentation performance of other supervised fundus blood vessel segmentation algorithms.

[0008] The technical solution adopted by the present invention is:

[0009] A thick and thin blood vessel classification method for high-precision blood vessel segmentation model training application includes the following steps:

[0010] Step 1: Extract the vascular skeleton from the fundus vascular labels;

[0011] Step 2: In the window where each vascular skeleton pixel is located, calculate the ratio of the vascular skeleton to the vascular area;

[0012] Step 3: Use the threshold to divide the vascular skeleton into a thick vascular skeleton and a thin vascular skeleton;

[0013] Step 4: Based on the classification results of the vascular skeleton, the classification of other vascular pixels is confirmed using the 8-neighborhood voting method;

[0014] Step 5: Use morphological methods to reclassify the blood vessels located in the intersection area of ​​thick and thin blood vessels to ensure the overall vascular structure.

[0015] In step 1, a morphological method is used to corrode the blood vessel boundary pixels, but the blood vessels are not allowed to break, and the retained pixels are combined into a blood vessel skeleton.

[0016] In step 2, the ratio of the vascular skeleton to the vascular area is calculated in the 8×8 circular window where each vascular skeleton pixel is located, as shown in formula (1):

[0017]

[0018] In formula (1): represents the blood vessel shrinkage ratio, represents the blood vessel skeleton pixels, represents the original vessel pixels, represents the number of vessel skeleton pixels in the 8×8 circular window, and represents the number of vessel pixels in the 8×8 circular window.

[0019] In step 4, the 8-neighborhood voting method includes the following steps:

[0020] S4.1: Input the classified vascular skeleton image, the number of iterations T and the total number of vascular pixels S;

[0021] S4.2: Compare the current iteration number t with T. If it is smaller, proceed to S4.3. Otherwise, jump to S4.8.

[0022] S4.3: Compare the number of voted pixels s with S. If it is smaller, proceed to S4.4. Otherwise, jump to S4.7.

[0023] S4.4: Calculate the number X of thick blood vessel pixels and the number Y of thin blood vessel pixels in the 8-neighborhood of the blood vessel pixel I;

[0024] S4.5: Compare the magnitudes of X and Y. If X > Y, then I is a thick blood vessel pixel; if X < Y, then I is a thin blood vessel pixel; if X = Y = 4, then I is a thick blood vessel pixel;

[0025] S4.6: Increment s by 1 and jump to step S4.3;

[0026] S4.7: Increment t by 1 and jump to step S4.2;

[0027] S4.8: Obtain the blood vessel appearance image that has been classified properly.

[0028] In step 4, classify other blood vessel pixels into thick blood vessel pixels and thin blood vessel pixels to obtain a preliminary classification result.

[0029] Step 5 specifically includes the following steps:

[0030] S5.1: Input the preliminary thick and thin blood vessel classification image and the number of iterations T;

[0031] S5.2: Compare the current iteration number t with T. If t is smaller, proceed to S5.3; otherwise, jump to S5.7;

[0032] S5.3: Extract the thick blood vessel image I thick and the thin blood vessel image I thin ;

[0033] S5.4: In I thick find small regions with a connected region smaller than 35 and classify them as thin blood vessels;

[0034] S5.5: In I thin find spots with a connected region smaller than 35 and classify them as thick blood vessels.

[0035] S5.6: Update the thick and thin blood vessel classification in the thick and thin blood vessel classification image;

[0036] S5.7: Obtain the final blood vessel appearance classification result.

[0037] In step 5, use the connected domain method to remove spots. In the thick blood vessel image, find spots with a connected region smaller than 35 and classify them as thin blood vessels. In the thin blood vessel image, find spots with a connected region smaller than 35 and classify them as thick blood vessels. Iterate 5 times to obtain the final classification result.

[0038] For the method for classifying thick and thin blood vessels for high-precision blood vessel segmentation model training application of the present invention, the technical effects are as follows:

[0039] 1) Compared with other thick and thin blood vessel classification methods, the thick and thin blood vessel classification method of the present invention can correctly classify blood vessels into thick and thin blood vessels, and ensure the relatively complete structure of thick and thin blood vessels. It plays an important role in effectively improving the training efficiency and reasoning accuracy of the neural network fundus blood vessel segmentation model based on deep learning. At the same time, it is also applicable to the training of other non-neural network fundus blood vessel segmentation algorithms.

[0040] 2) This invention is not only applicable to the training of neural network fundus vascular segmentation models based on deep learning, but also applicable to improving the segmentation performance of other supervised fundus vascular segmentation algorithms. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1(a) is the original image of the DRIVE dataset;

[0042] Figure 1(b) is the label map of the DRIVE dataset;

[0043] Figure 2 This is a schematic diagram of the thick and thin blood vessel classification process;

[0044] Figure 2 In the figure, red pixels represent thick blood vessels and blue pixels represent thin blood vessels.

[0045] Figure 3(a) is the result of vascular skeleton extraction (vascular label);

[0046] FIG3( b ) is a diagram showing the vascular skeleton extraction result (vascular skeleton).

[0047] FIG4( a ) is a schematic diagram showing the calculation of the ratio of the original blood vessel to the blood vessel skeleton area;

[0048] FIG4( b ) is an enlarged view of point C in FIG4( a ).

[0049] Figure 5 It is the histogram of the ratio of vascular skeleton to original blood vessel.

[0050] Figure 6 This is a skeleton diagram of thick and thin blood vessels.

[0051] Figure 7 This is a flow chart for the classification of thick and thin blood vessels.

[0052] Figure 8(a) is a preliminary classification result diagram;

[0053] FIG8( b ) is an enlarged view of point A in FIG8( a ).

[0054] Figure 8(c) is the result of morphological processing.

[0055] FIG8( d ) is an enlarged view of point B in FIG8( c ).

[0056] Fig. 9 This is the morphological processing flow chart.

[0057] Fig.10 This is a flow chart for the classification of thick and thin blood vessels. DETAILED DESCRIPTION

[0058] In view of the fact that the fundus vascular data set does not have label information of thick and thin blood vessel types, the neural network based on deep learning is not guided by the blood vessel type information during training, resulting in the error caused by the thick and thin blood vessel pixels being treated equally, resulting in the problem that the accuracy of thin blood vessel segmentation cannot be effectively improved. The present invention proposes a thick and thin blood vessel classification method for high-precision blood vessel segmentation model training application. The method first extracts the blood vessel skeleton from the blood vessel label, and calculates the ratio of the blood vessel area of ​​the label to the blood vessel area of ​​the skeleton in the 8×8 circular area, and then divides the blood vessel skeleton into a thick blood vessel skeleton and a thin blood vessel skeleton according to the threshold, and then uses the 8-neighborhood voting method to iterate 40 times, classifies other blood vessel pixels into thick blood vessel pixels and thin blood vessel pixels, and obtains the preliminary classification result. In view of the situation that there are thin blood vessel spots mixed in long thick blood vessels and thick blood vessel spots mixed in long thin blood vessels in the preliminary classification results, the connected domain method is used to remove the spots, find the spots with a connected area less than 35 in the thick blood vessel image and classify them as thin blood vessels, find the spots with a connected area less than 35 in the thin blood vessel image and classify them as thick blood vessels, iterate 5 times, and obtain the final classification result.

[0059] The specific steps include:

[0060] Step 1: Extract the vascular skeleton from the fundus vascular label:

[0061] Using morphological methods, the blood vessel boundary pixels are eroded, but the blood vessels are not allowed to break, and the retained pixels are combined into the blood vessel skeleton;

[0062] Step 2: In the 8×8 circular window where each vascular skeleton pixel is located, calculate the ratio of the labeled blood vessel to the vascular skeleton area, as shown in formula (1):

[0063]

[0064] Among them: represents the blood vessel shrinkage ratio, represents the blood vessel skeleton pixels, Represents the original blood vessel pixels, represents the number of blood vessel skeleton pixels in the 8×8 circular window, represents the number of blood vessel pixels in the 8×8 circular window. As shown in Figure 4(a) and Figure 4(b), in Figure 4(a) and Figure 4(b), red represents the blood vessel skeleton, white represents other blood vessel pixels, and the blue circle represents the 8×8 circular area.

[0065] Step 3: Based on the histogram of the ratio of the vascular skeleton to the original vascular skeleton, Figure 5As shown in Figure 2, the classification threshold can be roughly determined to be between 0.3 and 0.7. After multiple experiments on the DRIVE dataset, the classification effect is best when the threshold is t = 0.363. Using the threshold t = 0.363, the vascular skeleton is divided into a thick vascular skeleton and a thin vascular skeleton. Figure 6 As shown, red is the thick blood vessel skeleton, blue is the thin blood vessel skeleton, and white is other blood vessel pixels.

[0066] Step 4: Based on the coarse and fine classification results of the vascular skeleton, the 8-neighborhood voting method is used and iterated 40 times to confirm the classification of other vascular pixels. For a pixel point where the coarse and fine vascular pixels account for half of each in the 8-neighborhood, this method tends to consider the pixel point as a coarse vascular pixel. The specific process is as follows: Figure 7 Figure 8(a) and Figure 8(b) are preliminary classification results, where red pixels represent thick blood vessels and blue pixels represent thin blood vessels.

[0067] Step 5: Although most of the thick and thin blood vessels are classified correctly in the preliminary classification results, there are some impurities in the classification results. For example, a blue spot representing a thin blood vessel appears in a small area in the middle of a long thick blood vessel, as shown in the image within the light blue box in Figure 8(a). This situation destroys the overall structure of the blood vessel and is not conducive to the subsequent segmentation of the blood vessel.

[0068] To address this problem, the present invention uses a morphological method to remove these impurity points while ensuring that the classification of thick and thin blood vessels is basically correct. The specific process is as follows: Fig. 9 As shown in Figure 8(c) and Figure 8(d), the spots of thick vessels and thin vessels are basically disappeared. In this way, the blood vessels are classified into thick vessels and thin vessels, while the relatively complete structure of thick and thin vessels is guaranteed.

Claims

1. A method for classifying thick and thin blood vessels for high-precision blood vessel segmentation model training application, characterized in that It includes the following steps: Step 1: Extract the vascular skeleton from the fundus vascular labels; Step 2: Calculate the ratio of the vascular skeleton to the vascular area within the window where each vascular skeleton pixel is located; Step 3: Use a threshold to divide the vascular skeleton into thick vascular skeletons and thin vascular skeletons; Step 4: According to the classification result of the vascular skeleton, use the 8-neighborhood voting method to confirm the classification of other vascular pixels; Step 5: Use a morphological method to reclassify the blood vessels located in the broken and crossed areas of thick and thin blood vessels; In the said Step 4, the 8-neighborhood voting method includes the following steps: S4.1: Input the classified vascular skeleton image, the number of iterations T, and the total number of vascular pixels S; S4.2: Compare the current iteration number t with T. If t is smaller, proceed to S4.3; otherwise, jump to S4.8; S4.3: Compare the number of voted pixels s with S. If s is smaller, proceed to S4.4; otherwise, jump to S4.7; S4.4: Calculate the number of thick vascular pixels X and the number of thin vascular pixels Y within the 8-neighborhood of the vascular pixel I; S4.5: Compare X with Y. If X > Y, then I is a thick vascular pixel; if X < Y, then I is a thin vascular pixel; if X = Y = 4, then I is a thick vascular pixel; S4.6: Increment s by 1 and jump to step S4.3; S4.7: Increment t by 1 and jump to step S4.2; S4.8: Obtain the classified vascular image in good condition; In the said Step 4, other vascular pixels are classified into thick vascular pixels and thin vascular pixels to obtain a preliminary classification result; In the said Step 5, it specifically includes the following steps: S5.1: Input the preliminary thick and thin vascular classification image and the number of iterations T; S5.2: Compare the current iteration number t with T. If t is smaller, proceed to S5.3; otherwise, jump to S5.7; S5.3: Extracting a thick vessel image I from the preliminary thick and thin vessel classification image thick With fine blood vessel image I thin； S5.4: In I thick Find small areas with connected areas less than 35 and classify them as capillaries; S5.5: In I thin Find the spots with connected areas less than 35 and classify them as thick blood vessels; S5.6: Update the thick and thin vascular classification in the thick and thin vascular classification image; S5.7: Obtain the final vascular classification result.

2. A thick and thin blood vessel classification method for high-precision blood vessel segmentation model training application according to claim 1, characterized in that: In the said Step 1, use a morphological method to erode the vascular boundary pixels, and the remaining pixels are combined into a vascular skeleton.

3. The thick and thin blood vessel classification method for high-precision blood vessel segmentation model training application according to claim 1, characterized in that: According to the thick and thin vascular classification method for high-precision vascular segmentation model training application described in Claim 1, it is characterized in that: in the said Step 2, within the circular window where each vascular skeleton pixel is located, calculate the ratio of the vascular skeleton to the vascular area, as shown in formula (1): （1）； In formula (1): Represents the percentage of blood vessel shrinkage. represents the blood vessel skeleton pixels, represents the original blood vessel pixel, Represents the number of blood vessel skeleton pixels within the 8×8 circular window, Represents the number of blood vessel pixels within an 8×8 circular window.

4. The thick and thin blood vessel classification method for high-precision blood vessel segmentation model training application according to claim 1, characterized in that: In the said Step 5, use the connected component method to remove the spots. Find the spots with a connected area less than 35 in the thick vascular image and classify them as thin blood vessels, and find the spots with a connected area less than 35 in the thin vascular image and classify them as thick blood vessels. Iterate 5 times to obtain the final classification result.

Images