Automatic separation method of blood vessel hierarchy based on retinal blood vessel image

Abstract

The application discloses a hierarchical structure automatic separation method based on a retinal blood vessel image. The method comprises the following steps: acquiring original retinal blood vessel data and performing pretreatment, extracting key points on a blood vessel surface, and using a unit normal vector of a plane where the key points are located to represent a growth direction of a vertical blood vessel cluster at the key points; a three-dimensional image of the retinal blood vessel is divided into retinal blood vessel segments by a cube, the main axis direction of the blood vessel segment in each cube is calculated, and the hierarchical structure category to which the blood vessel segment in the current cube belongs is judged according to the included angle between the main axis direction and the unit normal vector of the nearest key point. The application accurately extracts the spatial structure of the blood vessel while reducing the calculation amount through binarization, cavity filling and skeletonization of the blood vessel, the morphological processing reduces the influence of noise points on the extraction of the key points, the robustness and the application range of the method are improved by using the rolling ball method, and the method is still effective when irregular and non-smooth retinal blood vessel networks are processed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of medical image processing, and in particular to a blood vessel hierarchical structure automatic separation method based on a retinal blood vessel image. BACKGROUND

[0002] Many major retinal diseases are often accompanied by changes in retinal blood vessel structure, including retinopathy of prematurity (ROP), diabetic retinopathy (DR), age-related macular degeneration (AMD), etc. These diseases can cause visual impairment and even blindness in patients.

[0003] Using optical imaging technology to analyze and quantify the three-dimensional multi-layer structure of retinal blood vessels is an important way to study the mechanism of disease occurrence and develop effective treatment methods. Therefore, the automatic separation method of the hierarchical structure of retinal blood vessels has important research significance and application value.

[0004] Currently, there are two commonly used methods to study the fine hierarchical structure of the entire retinal blood vessel. One is to cut the retina into a "four-leaf clover" shape and flatten it, and then use confocal or multiphoton fluorescence microscopy for imaging. The other method is to maintain the spatial structure of the retina unchanged, use tissue transparentization technology and light sheet fluorescence microscopy for imaging, and obtain complete three-dimensional retinal blood vessel images. The former method needs to cut and flatten the blood vessels, which reduces the difficulty of separating the hierarchical structure of the blood vessels, but the disadvantage is that it destroys the original three-dimensional spatial structure of the retinal blood vessels in the form of a bowl, and the cutting causes damage to part of the blood vessels, affecting the quantification results of the blood vessels. The second method preserves the three-dimensional spatial structure of the retinal blood vessels, but it is difficult to separate the blood vessel structure, so a special hierarchical structure automatic separation method needs to be developed for this type of image data. SUMMARY

[0005] To solve the problems in the background art, the present application provides a blood vessel hierarchical structure automatic separation method based on a retinal blood vessel image. The present application accurately extracts the spatial structure of the blood vessels while reducing the computational load by binarizing, cavity filling and skeletonizing the blood vessels. The morphological processing reduces the influence of noise points on surface extraction. The rolling ball method is used to extract the surface key points, and the unit normal vector of the surface where the key points are located is calculated, which improves the robustness and applicability of the algorithm. The method has good hierarchical structure separation effect for irregular and non-smooth retinal blood vessel networks and good robustness.

[0006] The technical scheme adopted by the present application is as follows:

[0007] 1) Use light sheet fluorescence microscopy imaging method to obtain three-dimensional image data of retinal blood vessels;

[0008] 2) pre-processing the three-dimensional image data of the retinal blood vessels to obtain pre-processed three-dimensional image data of the retinal blood vessels;

[0009] 3) extracting key points from the pre-processed three-dimensional image data of the retinal blood vessels by using a rolling ball method to obtain a plurality of key points on the surface of the retinal blood vessels, selecting an approximate plane where each key point is located, and calculating a unit normal vector of the approximate plane where each key point is located as a unit normal vector of each key point;

[0010] 4) dividing the pre-processed three-dimensional image data of the retinal blood vessels obtained in step 2) into a plurality of retinal blood vessel segments, and calculating a principal axis direction of each retinal blood vessel segment by using a principal component analysis method;

[0011] 5) finding a key point closest to the center of each retinal blood vessel segment, taking the unit normal vector of the closest key point, calculating the included angle between the principal axis direction of each retinal blood vessel segment and the unit normal vector of the closest key point of the center of the retinal blood vessel segment, and classifying each retinal blood vessel segment according to the included angle by using a decision rule to obtain a vertically growing blood vessel cluster and a parallel growing blood vessel cluster in the entire retinal blood vessels, thereby realizing separation of the hierarchical structure of the retinal blood vessels.

[0012] The step 3) specifically comprises: for the pre-processed three-dimensional image data of the retinal blood vessels, performing triangulation on the pre-processed three-dimensional image data of the retinal blood vessels by using a rolling ball method to obtain a plurality of triangulation points, and taking each triangulation point as a key point of the retinal blood vessels.

[0013] For each key point of the retinal blood vessels, connecting each key point with a plurality of key points around the key point to form a plane and thereby obtaining an approximate plane corresponding to each key point, and calculating a unit normal vector of the approximate plane corresponding to each key point as a unit normal vector of each key point.

[0014] The key point around each key point is a key point located in a circular region with the key point of the retinal blood vessels as the center and R as the radius.

[0015] The step 4) specifically comprises: constructing a three-dimensional space for the pre-processed three-dimensional image data of the retinal blood vessels, the three-dimensional space containing the three-dimensional image of the entire retinal blood vessels, dividing the three-dimensional space into a plurality of cubic cells of equal size, and dividing the three-dimensional image of the entire retinal blood vessels into a plurality of retinal blood vessel segments by the plurality of cubic cells.

[0016] The gray value summation method is used to determine whether each cube contains a retinal blood vessel fragment. When a cube contains a retinal blood vessel fragment, the principal component analysis method is used to calculate the principal axis direction of the retinal blood vessel fragment in each cube. When a cube does not contain a retinal blood vessel fragment, no processing is performed.

[0017] In step 5), the determination rule is specifically:

[0018] When the angle between the principal axis direction of the retinal blood vessel fragment and the unit normal vector of the nearest key point of the retinal blood vessel fragment is greater than or equal to 0 degrees and less than 45 degrees, the retinal blood vessel fragment is a vertical blood vessel cluster.

[0019] When the angle between the principal axis direction of the retinal blood vessel fragment and the unit normal vector of the nearest key point of the retinal blood vessel fragment is greater than or equal to 45 degrees and less than or equal to 90 degrees, the retinal blood vessel fragment is a parallel blood vessel cluster.

[0020] When the angle between the principal axis direction of the retinal blood vessel fragment and the unit normal vector of the nearest key point of the retinal blood vessel fragment exceeds 90 degrees, the supplementary angle of the angle is taken and the determination rule is determined again.

[0021] In step 2), the preprocessing is specifically to sequentially perform binarization, cavity filling, skeletonization and morphological processing on the original three-dimensional image data of the retinal blood vessels.

[0022] In step 3), the unit normal vector of the plane where each key point is located is calculated as the unit normal vector of the corresponding key point of each plane by the least square method.

[0023] The binarization specifically adopts an adaptive threshold method to segment the blood vessels and background in the three-dimensional image data of the retinal blood vessels.

[0024] The skeletonization processing specifically adopts a K3M 3D skeletonization method for skeletonization processing.

[0025] The blood vessel cavity filling specifically adopts a blood vessel cavity filling network based on deep learning for processing.

[0026] The morphological processing specifically adopts a voxel connected domain method for morphological processing, which reduces the influence of noise points on key point extraction.

[0027] The center of each retinal blood vessel fragment is specifically the center of the cube where the retinal blood vessel fragment is located.

[0028] The beneficial effects of the present application are: the spatial structure of the blood vessels is accurately extracted while the calculation amount is reduced, the morphological processing effectively reduces the influence of noise points on the surface extraction, the key points of the blood vessel surface are extracted by using the rolling ball method, the unit normal vector of the plane where the key points are located is calculated by using the least square method, the robustness and application range of the algorithm are improved, and the irregular and non-smooth retinal blood vessel network has good hierarchical structure separation effect. BRIEF DESCRIPTION OF DRAWINGS

[0029] Figure 1 The flow chart of the hierarchical structure automatic separation based on the retinal blood vessel image of the present application.

[0030] Figure 2 The orthographic view of the retinal blood vessel.

[0031] Figure 3 The retinal blood vessel image after binarization processing.

[0032] Figure 4 The retinal blood vessel image after skeletonization and morphological processing.

[0033] Figure 5 The key point image of the retinal blood vessel surface.

[0034] Figure 6 The unit normal vector image of the key point of the entire retinal blood vessel surface.

[0035] Figure 7 The hierarchical structure automatic separation result image of the local area retinal blood vessel.

[0036] Figure 8 The 2D projection image of the hierarchical structure automatic separation of the retinal blood vessel.

[0037] Figure 9 The 2D projection image of the hierarchical structure automatic separation of the retinal blood vessel of different eyeballs. DETAILED DESCRIPTION

[0038] The present application will be further described in detail below in combination with the drawings and specific embodiments.

[0039] As shown in Figure 1 , the method comprises the following steps:

[0040] 1) S1, adopt the light sheet fluorescence microscopic imaging method to acquire the three-dimensional image data of the retinal blood vessel, the three-dimensional image data is a 3D gray scale image composed of a plurality of voxels, the size is 2048x2048x1350, the actual size corresponding to a single voxel is 1.625 μm x 1.625 μm x 2.00 μm, and the original image orthographic view of the retinal blood vessel is as shown in Figure 2 ;

[0041] 2) S2, the three-dimensional image data of the retinal blood vessels is binarized by using a method of adaptive threshold value, and the blood vessels and the background in the three-dimensional image data of the retinal blood vessels are segmented; S3, since the large blood vessels only have the blood vessel walls, have a large signal strength and a gray value, and the cavities exist in the blood vessels, therefore, after binarization, the cavities of the blood vessels need to be filled by means of a deep learning network, so as to ensure that the complete binarized blood vessels can be extracted, and the binarized image of the retinal blood vessels is as shown in Figure 3 ;

[0042] S4, after the data processed by the cavity filling of the blood vessels, the skeleton line of the blood vessels is extracted by using the 3D skeletonization method of K3M; S5, the size of the retinal blood vessel skeleton image is 2048x2048x1350, and then the method of voxel connected domain is adopted to perform morphological processing, so as to remove the connected targets with small volumes, so as to eliminate the noise points and obtain the three-dimensional image data of the retinal blood vessels after the preprocessing, and the three-dimensional image data of the retinal blood vessels after the preprocessing is as shown in Figure 4 ;

[0043] 3) S6, for the three-dimensional image data of the retinal blood vessels after the preprocessing, the three-dimensional image data of the retinal blood vessels after the preprocessing is triangulated by using the rolling ball method, a plurality of triangular division points are obtained, each triangular division point is taken as a key point of the retinal blood vessels, and the surface of the retinal blood vessels formed by the key points is as shown in Figure 5 ;

[0044] S7, for each key point of the retinal blood vessels, the key point and a plurality of key points around the key point are connected to form a surface, and then the approximate plane corresponding to each key point is obtained; wherein, a circular region is established with the key point of the retinal blood vessels as the center and with R as the radius, and the key points located in the circular region belong to the key points around the key point; then the normal vector of the approximate plane corresponding to each key point is calculated by using the least square method, and the normal vectors of all the key points are normalized, as shown in Figure 6 , the unit normal vector of each key point is obtained;

[0045] 4) S8, a three-dimensional space is constructed for the three-dimensional image data of the retinal blood vessels after the preprocessing, the three-dimensional space contains the three-dimensional image of the entire retinal blood vessels, the three-dimensional space is divided into a plurality of cubic regions with equal sizes, the size of the cubic region is 15x15x15, and the three-dimensional image of the entire retinal blood vessels is divided into a plurality of retinal blood vessel segments by the plurality of cubic regions;

[0046] The gray value summation method is adopted to judge whether each cubic region contains a retinal blood vessel segment or not, when the cubic region contains a retinal blood vessel segment, the principal axis direction of the retinal blood vessel segment in each cubic region is calculated by using the principal component analysis method; when the cubic region does not contain a retinal blood vessel segment, no processing is performed.

[0047] 5) S9, find the unit normal vector of the nearest key point to the center of each retinal vessel segment, calculate the angle between the main axis direction of each retinal vessel segment and the unit normal vector of the nearest key point to its own center, wherein the center of each retinal vessel segment is the center of the cube where the retinal vessel segment is located; after calculating the angle, classify each retinal vessel segment according to the angle using a decision rule to obtain the vertically growing vascular tufts and the parallel growing vascular tufts in the entire retinal vessel, and realize the hierarchical structure separation of the retinal vessel.

[0048] In step 5), the decision rule is specifically:

[0049] When the angle between the main axis direction of the retinal vessel segment and the unit normal vector of the nearest key point to its own center is greater than or equal to 0 degrees and less than 45 degrees, the retinal vessel segment is a vertical vascular tuft.

[0050] When the angle between the main axis direction of the retinal vessel segment and the unit normal vector of the nearest key point to its own center is greater than or equal to 45 degrees and less than or equal to 90 degrees, the retinal vessel segment is a parallel vascular tuft.

[0051] When the angle between the main axis direction of the retinal vessel segment and the unit normal vector of the nearest key point to its own center exceeds 90 degrees, the supplementary angle of the angle is taken and the decision rule is used again for determination.

[0052] The hierarchical structure separation result of the retinal vessel in the local area is shown in FIG. 6, Figure 7 After obtaining the category of the hierarchical structure of each vessel segment in the cube, the hierarchical structure of the entire retinal vessel is restored, and the vertical vascular tufts and the parallel vascular tufts are saved as two independent image data, as shown in FIG. 7, Figure 8 The left side is the 2D projection of the vertical vascular tufts, and the right side is the 2D projection of the parallel vascular tufts, Figure 9 the 2D projections of the hierarchical structure separation of the retinal vessels of different eyeballs.

Claims

1. An automatic hierarchical structure separation method based on retinal blood vessel images, characterized in that: The method includes the following steps: 1) Three-dimensional image data of retinal vessels were obtained using light-sheet fluorescence microscopy. 2) Preprocess the three-dimensional image data of retinal vessels to obtain preprocessed three-dimensional image data of retinal vessels; 3) The rolling ball method is used to extract key points from the preprocessed 3D image data of retinal vessels, and multiple key points of retinal vessels are obtained. The plane where each key point is located is selected, and the unit normal vector of the plane where each key point is located is calculated as the unit normal vector of each key point. 4) Divide the preprocessed retinal vessel three-dimensional image data obtained in step 2) into several retinal vessel segments, and calculate the principal axis direction of each retinal vessel segment using the principal component analysis method. 5) After finding the key point closest to the center of each retinal vascular segment, take the unit normal vector of the nearest key point, calculate the angle between the principal axis direction of each retinal vascular segment and the unit normal vector of the nearest key point at its center, and use the judgment rule to classify each retinal vascular segment to obtain the vertically growing vascular plexus and the parallel growing vascular plexus in the entire retinal blood vessels, so as to realize the separation of the hierarchical structure of retinal blood vessels. In step 5), the determination rule is specifically as follows: When the angle between the principal axis of a retinal vascular segment and the unit normal vector of its nearest key point is greater than or equal to 0 degrees and less than 45 degrees, the retinal vascular segment is a vertical vascular plexus. When the angle between the principal axis of a retinal vascular segment and the unit normal vector of its nearest key point is greater than or equal to 45 degrees and less than or equal to 90 degrees, the retinal vascular segment is a parallel vascular plexus. When the angle between the principal axis of a retinal vascular segment and the unit normal vector of its nearest key point exceeds 90 degrees, the supplementary angle is taken and the judgment is made again according to the judgment rule.

2. The automatic hierarchical structure separation method based on retinal vascular images according to claim 1, characterized in that: Step 3) specifically involves: using the rolling ball method to triangulate the preprocessed retinal vessel 3D image data to obtain multiple triangulation points, with each triangulation point serving as a key point of the retinal vessel. For each key point of a retinal vessel, connect each key point with several surrounding key points to obtain the plane corresponding to each key point. Calculate the unit normal vector of the plane corresponding to each key point, which is then used as the unit normal vector of each key point.

3. The automatic hierarchical structure separation method based on retinal vascular images according to claim 1, characterized in that: Step 4) specifically involves: constructing a three-dimensional space for the preprocessed three-dimensional image data of retinal vessels, the three-dimensional space containing the three-dimensional image of the entire retinal vessels, dividing the three-dimensional space into several cubes of equal size, and dividing the three-dimensional image of the entire retinal vessels into several retinal vessel segments by several cubes. The grayscale summation method is used to determine whether each cube contains a retinal vascular segment. When a cube contains a retinal vascular segment, the principal component analysis method is used to calculate the principal axis direction of the retinal vascular segment in each cube; when a cube does not contain a retinal vascular segment, no processing is performed.

4. The automatic hierarchical structure separation method based on retinal vessel images according to claim 1, characterized in that: In step 2), the preprocessing specifically involves performing binarization, cavity filling, skeletonization, and morphological processing on the original three-dimensional image data of retinal vessels in sequence.

5. The automatic hierarchical structure separation method based on retinal vessel images according to claim 1, characterized in that: In step 3), the unit normal vector of the plane containing each key point is calculated using the least squares method as the unit normal vector of the corresponding key point of each plane.

6. The automatic hierarchical structure separation method based on retinal vascular images according to claim 4, characterized in that: The binarization specifically involves using an adaptive thresholding method to segment the blood vessels and background in the three-dimensional image data of retinal blood vessels.

7. The automatic hierarchical structure separation method based on retinal vascular images according to claim 4, characterized in that: The skeletonization process is specifically performed using the K3M 3D skeletonization method.

8. The automatic hierarchical structure separation method based on retinal vascular images according to claim 4, characterized in that: The morphological processing described herein specifically employs the voxel connected domain method for morphological processing.

9. The automatic hierarchical structure separation method based on retinal vascular images according to claim 3, characterized in that: The center of each retinal vascular segment is specifically the center of the cube containing the retinal vascular segment.

Images