[0038] The invention is a complete liver segmentation method for MSCTP sequence images. The liver segmentation of the present invention is divided according to the regions to which different branches of the portal vein blood vessels belong, and the liver segmentation mainly includes five parts: liver segmentation, blood vessel segmentation and thinning, blood vessel grading, liver segmentation and three-dimensional reconstruction.
[0039] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and examples.
[0040] Such as figure 1 As shown, the method of the present invention includes the following steps:
[0041] (1) Load the portal vein phase and arterial phase sequence images of abdominal MSCTP to automatically segment the liver. The flowchart is as follows image 3 As shown, the specific steps are as follows:
[0042] (1.1) Pretreatment. Data cropping is performed on the serial images of the portal vein and arterial phases, bilinear interpolation is used to interpolate the cropped data, and then anisotropic filtering is used to denoise.
[0043] Because there are many redundant data in the original data, which affects the speed of subsequent processing, the minimum box method is used to cut out the target (patient) data: find the coordinates x corresponding to the minimum and maximum target points in the x, y, and z directions respectively min , X max , Y min , Y max ,z min And z max , Using size (x max -x min +3)×(y max -y min +3)×(z max -z min +3) from the starting point (x min -1, y min -1, z min-1) Frame the target along the direction parallel to the axial direction, and the data in this box is the cropped data. Generally, the cropped data only accounts for 40%-80% of the original uncropped data. Since the scanning interval is 1.25mm and the pixel size in the layer is 0.703125mm×0.703125mm, the bilinear method is used to interpolate the data before registration, so that the length of the voxel points in three directions is 0.703125mm.
[0044] Researchers have proposed a powerful smoothing method—anisotropic filtering. Its greatest advantage is that it can remove image noise while avoiding blurring of the boundary area. The idea of anisotropic filtering originates from the solution of the thermal diffusion equation, which controls the diffusion behavior by introducing image features and designing appropriate diffusion coefficients. (For details, see Krissian K, Malandain G, Ayache N (1996) Directional Anisotropic Diffusion Applied to Segmentation of Vessels in 3DImages. INRIA France, RR-3064.)
[0045] (1.2) Coarse segmentation. Taking the pre-processed arterial phase sequence image as the standard, the preprocessed portal vein phase sequence image is registered using B-spline-based elastic registration; then the adaptive threshold segmentation is used for coarse segmentation to obtain the initial liver segmentation result.
[0046] Due to the deformation between the portal vein phase and the arterial phase images, in order to better count the two-dimensional grayscale distribution of the liver, the images need to be registered. The present invention adopts the B-spline-based elastic registration proposed by existing researchers. (For details, see Rueckert D, Sonoda L, Hayes C, et al. Nonrigid registration using free-form deformations: application to breast MR images[J]. IEEE Transactions on Medical Imaging, 1999, 18(8): 712~721.)
[0047] The two-dimensional histogram of the liver is estimated by counting the CT value distribution of the region of interest in the sequence images of the arterial phase and the portal vein, and the normal liver parenchyma is preliminarily segmented on this basis. (For details, see Jun Masumoto, Masatoshi Hori, Yoshinobu Sato, et al. Automated liver Segmentation Using Multi-slice CT Images[J]. Systems and Computer in Japan, 2003; 34(9): 2150-2161.)
[0048] (1.3) Fine segmentation. The hole filling algorithm is used to remove the small holes and wrong connections in the results of the coarse liver segmentation; the area growth algorithm is combined to remove the excess tissue and further fill the internal holes; finally the contour correction is performed to obtain the liver contour image.
[0049] Because some blood vessels or lesions exist on the surface of the liver, they cannot be obtained by filling the internal holes, and the segmentation results need to be corrected. The present invention combines gray-scale features and adopts a morphological closing operation based on contour points to extract under-segmented regions, blood vessels and lesions on the liver surface. The idea is to first judge whether the current point is a boundary point, if it is, then perform morphological operations, otherwise skip; then combine the gray-scale features to distinguish and retain the newly added points after the closing operation. The use of morphological operations based on contour points can effectively reduce the amount of calculation, especially when the morphological operation radius and data volume are relatively large.
[0050] (1.4) Value mask. Using the liver contour obtained by fine segmentation as a template, the preprocessed portal vein phase sequence image is masked to obtain the liver image.
[0051] The results of each step are as Figure 4 As shown, the original images A and B of the arterial phase and the venous phase are preprocessed to obtain C and D. E is the result of registering D with C as the standard. Use C and E to perform adaptive threshold segmentation to obtain coarse segmentation F, and remove Excess tissue and fill the internal holes to get G, H is the final result.
[0052] (2) Enhanced segmentation and refinement of blood vessels. The liver image obtained by segmentation in step (1) is enhanced and segmented into portal vein vessels, and then the portal vein centerline is obtained by refinement. In the patent "A method and system for modeling hepatic portal vein vessel tree", the following enhanced segmentation and refinement of blood vessels are discussed in detail.
[0053] (2.1) The intrahepatic portal vein is enhanced. The specific steps are:
[0054] (a) Adjust the window width and window level of the liver image, and normalize the gray value to obtain a normalized liver image;
[0055] (b) Count and analyze the normalized liver image histogram to determine the grayscale range of interest, that is, the grayscale range of hepatic portal vein vessels;
[0056] (c) Perform multi-scale filtering based on Hessian matrix on the liver image in the gray range of interest to obtain an enhanced hepatic portal vein image.
[0057] (2.2) Segmentation of hepatic portal vein. Segmentation methods such as region growth/level set/fuzzy connection are used to segment the hepatic portal vein from the enhanced image.
[0058] Researchers have proposed various segmentation methods for blood vessels, such as region growth/level set/fuzzy connection methods. (For details, see Kirbas C, Quek F. A review of vessel extraction techniques and algorithms [J]. ACM Computing Surveys, 2004, 36(6): 81-121.)
[0059] Attached Figure 5 Shown is a coronal maximum density projection map of the segmentation result of hepatic portal vein obtained from a set of MSCTP data.
[0060] (2.3) Extract the centerline of the hepatic portal vein vascular tree from the segmented hepatic portal vein. Extracting the centerline of a blood vessel is a common method for analyzing blood vessels. The centerline can effectively represent the structure of the blood vessel and provide the radius of each point of the blood vessel. The steps are:
[0061] (a) Hole filling. Due to the inconsistency of the internal gray scale of the hepatic portal vein or other reasons, there will be holes in the hepatic portal vein segmented in step (2.2). These holes will affect the work of extracting the center line, and these holes must be filled.
[0062] In the segmented binary hepatic portal vein volume data (assuming that the background is marked as 0 and the target is the hepatic portal vein is marked as 1), select a voxel point in the background, use this point as the seed point, and use the markers as 0 as the growth condition. Use the area growth method to get a maximum connected background. The voxel points belonging to the connected background are all marked as 0, otherwise they are marked as 1, that is, the holes are filled.
[0063] (b) Use the three-dimensional topology refinement method to obtain the initial centerline of the hepatic portal vein.
[0064] At present, researchers have done a lot of research on extracting centerline. Because the topology refinement method can better retain the topological structure information of the original target, the three-dimensional topology refinement method proposed by Palágyi K is used to obtain the centerline of the blood vessel. (For details, please refer to Palágyi K, Kuba A. A3D6-subiteration thinning algorithm for extracting medial lines, Pattern Recognition Letter. 1998; 19: 613-627.)
[0065] (c) Use DFS (Depth First Search, depth first search algorithm) to detect the ring that exists in the center line. Due to the image quality or the segmentation method, the segmented hepatic portal vein may have a closed ring structure, which will result in a ring in the center line extracted by topological refinement. In order to obtain effective portal vein centerline data, the ring needs to be removed. The method of detecting the ring is divided into the following two steps:
[0066] ①Calculate the DFS on the center line. Take any branch terminal point s on the center line as the initial point, and use the Dijkstra algorithm based on minimum heap to calculate the shortest distance from each point on the center line to point s. The distance adopts the exact Euclidean distance. (For details, see Shih Frank Y, Wu Yi Ta. Three dimensional Euclidean distance transformation and its application to shortest pathplanning[J]. Pattern Recognition, 2004, 37(1): 79-92)
[0067] ②Detect the ring existing in the center line. If there is a ring structure in the centerline, then there are at least two paths from one branch tip to another. The performance on the center line DFS is that there is a point q, and the distance value of the points on the center line on the neighborhood is not greater than the distance value of the q point. Through this feature, the ring on the center line can be detected. After finding the local maximum distance value, trace back to the same nearest source point continuously along the decreasing direction of the distance value, and record this ring with graph structure G=(V, E). Among them, E is the branch point on the ring, and V is the edge formed by the points on the central path directly connecting the two branch points.
[0068] (2.4) Analyze each loop in the center path and untie the loop, then pruning the center line data to remove the false branches, and get the portal vein center line. (For details, see Palágyi K, Kuba A.A 3D6-subiteration thinning algorithm for extracting medial lines, Pattern Recognition Letter. 1998; 19: 613-627.)
[0069] Through the steps in (2), the coordinate information and radius information of the hepatic portal vein centerline can be obtained, and the maximum density projection of the hepatic portal vein centerline obtained in this embodiment is performed, that is, the result map of the topology refinement, such as Figure 6 Shown, where Figure 6 (a) is the initial centerline obtained by using 3D topology refinement, Figure 6 (b) is the final centerline obtained by unlooping and pruning.
[0070] (3) Interactive vascular classification. The present invention adopts a generalized cylinder (Generalized Cylinders) model to reconstruct the surface model of the portal vein vascular tree, and performs interactive vascular classification on the basis of the model. The generalized column represents a columnar object, which includes a space curve and a cross-sectional equation with the curve as a parameter. The present invention takes the centerline of hepatic portal vein as the generalized column space curve, constructs the cross-sectional equation of the generalized column through the centerline coordinate information and radius information, and finally uses OSG (OpenSceneGraph) to reconstruct the blood vessel surface model and perform interactive grading and marking of blood vessels.
[0071] (3.1) Construct a simplified tree data structure of the centerline of the portal vein.
[0072]In order to conveniently and quickly access the coordinates of the portal vein centerline and the radius corresponding to each coordinate, a tree data structure of the portal vein centerline needs to be constructed. Generally, the tree data structure takes each data unit (in the present invention, the point of the center line) as a node in the tree, and establishes the relationship between the nodes, so as to achieve the purpose of traversing and operating the tree. However, in order to reflect the branch information and travel information of the portal vein, a simplified tree data structure must be constructed for the centerline of the portal vein. The present invention classifies the points of the portal vein centerline according to the following rules: if there is one point in the space 26 neighborhood of the point, then that point is a leaf node; if there are two points in the space 26 neighborhood of the point, the point is a connection Point; if there are more than three points in the neighborhood of the point's space 26, the point is a branch node. Among them, the leaf node with the largest radius information is taken as the root node of the tree data structure, and then all the branch nodes and leaf nodes in the above classification are added to the tree data structure according to the parent-child and sibling relationship, and finally all the connection points are added In the tree structure, the construction of simplified tree data structure is completed. After the above steps, it can be determined that each node can access the following information: the coordinates and radius information of the node, the parent node of the node, the first child node and the first sibling node of the node, and the relationship between the node and the parent node Coordinates and radius information of all connection points (if the node does not have any one or more of the above information, then the missing information will be blank). Figure 7 It is a schematic diagram of the simplified tree data structure of the portal vein centerline.
[0073] (3.2) Fitting of center line data.
[0074] Usually the coordinate vector directions of each pair of adjacent connecting points in the centerline data may have a large deviation, which will cause the blood vessel centerline to have a jagged structure; in addition, the radius data of the centerline data will also be due to the calculation error of the thinning step. And the phenomenon that the radius changes between two adjacent points is too large. In order to remove the jagged structure and generate a blood vessel model with continuous and smooth changes in direction and radius, it is necessary to fit the centerline data. The fitting operation is only for the connection points without changing the position of the nodes.
[0075] A polynomial fitting method is used to fit the coordinates x, y, z and radius R of all connection points between the two branch points. For the centerline branch with the number of connection points greater than 15, the 5th degree polynomial fitting is adopted; for the centerline branch with the number of connection points less than or equal to 15, the 2nd degree polynomial fitting is adopted.
[0076] (3.3) Surface reconstruction of blood vessels.
[0077] Based on the assumption that the cross-section of a non-pathological blood vessel is circular, a regular dodecagon is used to approximate the cross-sectional circle of the blood vessel. The radius and curvature of the tubular structure are determined according to the coordinates and radius of the centerline point after fitting. The method of vascular surface reconstruction is as follows.
[0078] (a) Set a regular dodecagon with a radius of 1 on the xy plane as a unit cross-section polygon;
[0079] (b) Traverse the portal vein centerline tree data structure, place a unit section polygon at each centerline point, adjust the position, deflection direction and radius of the unit section polygon according to the coordinates and radius of each centerline point, and draw different centers Section polygon at the line point;
[0080] (c) Filling the vertices of the cross-sectional polygon obtained by the above calculation along the centerline direction by continuously filling the quadrilateral string, thereby constructing the blood vessel surface;
[0081] (d) At the leaf nodes, use a hemisphere to cover the top.
[0082] There are two factors that affect the effect of angiogenesis:
[0083] (a) The accuracy of the polygon that approximates the cross section of the tubular structure, that is, the number of sides of the polygon;
[0084] (b) Adjust the polygon position, the deflection angle and the radius sampling rate, that is, how many centerline points are spaced to adjust the position, deflection direction and radius of the section polygon.
[0085] In order to obtain a good generation effect, it is necessary to increase the number of sides and the sampling rate of the polygon that approximates a circle. At the same time, in order to obtain a fast generation speed, but can not increase the above two factors unlimitedly, the present invention selects a regular 12-sided polygon as the cross-sectional polygon. At the same time, the position, deflection direction and radius are performed once at each centerline point. The adjustment of, can quickly build a blood vessel model with a better visualization effect.
[0086] (3.4) Vascular interaction grade marking.
[0087] In order to achieve the segmentation of the liver, it is necessary to perform interactive grading of the blood vessels, that is, the user determines the grading of the blood vessels by clicking on the three-dimensional model, and then segment the liver based on the blood vessel grading data. In clinical research, the liver segmentation method proposed by Couinaud is widely used, which divides the liver into eight segments according to the different branches of the portal vein in the liver. This example is based on Couinaud's segmentation, combined with clinical prior knowledge to classify and label the portal vein, and obtain the result of hepatic portal vein centerline classification. The result is like Figure 8 Shown.
[0088] (4) Liver segmentation. Perform distance transformation and Voronoi algorithm calculation on the grading result of the hepatic portal vein centerline obtained in step (3) to obtain the Voronoi diagram, and use the liver contour to mask the Voronoi diagram to achieve liver segmentation. Attached Picture 9 Shows the flow chart of liver segmentation, the steps are:
[0089] (4.1) Distance transformation.
[0090] To calculate the area of the different branches of the portal vein, it is necessary to calculate the closest distance between the liver tissue and the different branches. The different grades of the portal vein centerline obtained in step (3) correspond to different gray values, and the method based on the generalized distance change is used to calculate the distance from each point on the centerline to other points on the centerline. This distance adopts Euclidean distance, and the obtained distance transform image is as Picture 10 Shown. (For details, see Pedro FFelzenszwalb and Daniel P Huttenlocher. Distance transforms of sampledfunctions. Technical report, Cornell University, September 2004)
[0091] (4.2) Calculate the Voronoi diagram.
[0092] Voronoi diagram is an effective method to solve the problem of space proximity. It realizes the division of spatial regions by constructing a three-dimensional Voronoi diagram. The present invention uses graded marked portal vein centerline data as a label image, and a distance transformation image as a sample image. Calculate the boundary division of different areas of the distance image and divide the areas. Assign the corresponding gray value to the corresponding area through the different gray value of the label image. The Voronoi diagram obtained is as Picture 11 Shown.
[0093] (4.3) The liver contour value mask.
[0094] Use the liver contour obtained in step (1) to mask Voronoi, so that the area inside the liver contour is preserved, and the outer gray value of the liver area is assigned to 0. The liver segmentation results obtained are as follows Picture 12 Shown.
[0095] (5) Three-dimensional reconstruction. Use OSG to reconstruct the three-dimensional liver segmentation results.
[0096] The liver segmentation results are obtained through the above steps, and the liver segments with different gray levels are colored and rendered by OSG, and the 3D reconstruction results obtained are as follows Figure 13 Shown.
[0097] The structure diagram of the liver segmentation system provided by the present invention is as follows figure 2 Shown. The system includes a liver segmentation module 100, a blood vessel enhancement segmentation and thinning module 200, a blood vessel classification module 300, a liver segmentation module 400 and a three-dimensional reconstruction module 500.
[0098] The liver segmentation module 100 loads the MSCTP portal vein phase and arterial phase sequence images, preprocesses them, and automatically segments them to obtain liver contours. The module is divided into 4 sub-modules, namely the loading image and preprocessing module 110, the liver coarse segmentation module 120, the liver fine segmentation module 130, and the value mask module 140. The image loading and preprocessing module 110 is used to load the abdominal MSCTP sequence image and perform cropping, interpolation and filtering processing on it. The rough segmentation module 120 is used to segment the initial contour of the liver. The fine segmentation module 130 performs hole filling and contour correction on the liver contour obtained by the coarse segmentation module 120. The value mask module 140 uses the liver contour image obtained by the liver fine segmentation module 130 to mask the original image value to obtain a liver image. The liver segmentation module 100 specifically completes the processing process of the above step (1).
[0099] The blood vessel enhancement segmentation and thinning module 200 uses the liver image obtained by the liver segmentation module 100 to perform three-dimensional blood vessel enhancement on the hepatic portal vein in the liver, and perform blood vessel segmentation and thinning. The module is divided into three sub-modules, namely, a blood vessel enhancement module 210, a blood vessel segmentation module 220, and a blood vessel thinning module 230.
[0100] The blood vessel enhancement module 210 is used to enhance the intrahepatic portal vein and transmit it to the blood vessel segmentation module 220; that is, the function of the above step (2.1) is completed.
[0101] The blood vessel segmentation module 220 segments the hepatic portal vein from the enhanced hepatic portal vein image, and transmits it to the blood vessel thinning module 230; that is, the function of the above step (2.2) is completed.
[0102] The blood vessel thinning module 230 extracts the center line of the hepatic portal vein vascular tree from the segmented hepatic portal vein: then analyzes and breaks each ring in the center path, and then prunes the center line data to remove the pseudo branches to obtain the portal vein center Line; that is, the function to complete the above steps (2.3) and (2.4).
[0103] The blood vessel classification module 300 uses the portal vein centerline obtained by the blood vessel enhancement segmentation and thinning module 200 to reconstruct the surface of the blood vessel and interactively classify the portal vein. The module is divided into a tree data structure building module 310 and a centerline data fitting module 320 , The blood vessel reconstruction module 330 and the blood vessel interactive classification module 340.
[0104] The tree data structure construction module 310 is used to construct a simplified tree data structure of the center line of the portal vein and transmit it to the center line data fitting module 320; that is, the function of the above step (3.1) is completed.
[0105] The centerline data fitting module 320 performs centerline data fitting on the simplified tree data structure, and transmits the fitted data to the blood vessel reconstruction module 330; that is, the function of the above step (3.2) is completed.
[0106] The blood vessel reconstruction module 330 uses the fitted data to reconstruct the blood vessel surface to obtain a three-dimensional blood vessel model, and transmits it to the blood vessel interactive classification module 340; that is, the function of the above step (3.3) is completed.
[0107] The blood vessel interactive grading module 340 determines the grading situation of the blood vessel by clicking on the three-dimensional model, and performs the blood vessel interactive grading mark; that is, the function of the above step (3.4) is completed.
[0108]The liver segmentation module 400 implements liver segmentation according to the portal vein centerline grading data obtained by the blood vessel grading module 300. The module is divided into three sub-modules, namely a distance transformation module 410, a Voronoi diagram calculation module 420, and a liver contour value mask module 430. The distance transformation module 410 mainly performs distance transformation on the portal vein centerline grading data obtained by the blood vessel grading module. The Voronoi diagram calculation module 420 calculates the result obtained by the distance transformation module to obtain the initial segmentation of the liver. The liver contour value masking module 430 uses the liver contour obtained by the liver fine segmentation module 130 to mask the result obtained by the Voronoi diagram calculation module 420 to realize liver segmentation. The liver segmentation module 400 specifically completes the processing process of step (4).
[0109] The three-dimensional reconstruction module 500 uses OSG to reconstruct the three-dimensional segmentation result of the liver according to the liver segmentation result obtained by the liver segmentation module 400.
[0110] The present invention is not limited to the above-mentioned specific embodiments. According to the disclosure of the present invention, those skilled in the art can adopt other various specific embodiments to implement the present invention. Therefore, whenever the design structure and ideas of the present invention are adopted, simple Changes or altered designs fall into the protection scope of the present invention.
