Vessel segmentation method, system and storage medium for liver image

Abstract

The present invention relates to the field of medical image segmentation technology, and provides a method, system, and storage medium for vascular segmentation of liver images. The method comprises: S1, obtaining an image containing all vascular information through image acquisition; S2, highlighting the vascular lumen and liver parenchymal cells and changing the liver lobe exterior to a black background through image preprocessing; S3, first cutting the image data set into three-dimensional data blocks, reading them into memory in batches, and then performing semantic segmentation of the vascular vessels using an automatic segmentation algorithm based on region growing to obtain continuous and complete vascular vessels; S4, using an automatic vascular tracking algorithm to track different types of vascular vessels and complete vascular classification. The present invention solves the problems of insufficient memory and slow computing speed in TB data processing by using parallel computing and dynamic data reading methods. It can automatically segment different types of vascular vessels in the complete liver lobe and at submicron resolution without losing hepatic sinusoidal signals.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image segmentation, and in particular to a liver vessel segmentation method based on parallel computing and dynamic data reading. Background Art

[0002] The liver contains different types of blood vessels, including the hepatic vein, portal vein, bile duct, lymphatic vessels, and hepatic sinusoids. By labeling the hepatic sinusoidal endothelial cells and using microscopic optical sectioning tomography, image data containing all vascular information can be obtained at a submicron resolution across the entire liver lobe. The image data includes data for the entire liver lobe measuring 6.0 cm × 7.2 cm × 9.0 cm, with a voxel resolution of 0.3 × 0.3 × 1 μm. 3 The original acquired images are in unsigned sixteen-bit format (uint16), and storing these image data requires approximately 9TB of space. Since these massive image data cannot be read into the memory at one time, the existing image processing algorithm needs to downsample the image data so that it can be read into the memory for subsequent image reconstruction. The image reconstruction method described above can fully obtain the hepatic vein, portal vein, bile duct, lymphatic vessel and hepatic artery in the liver lobe. Due to the downsampling of the image data, the above method has two problems:

[0003] (1) Although images of the hepatic vein, portal vein, bile duct, lymphatic vessels, and hepatic artery within the entire liver lobe can be obtained, the above operation is performed on the downsampled data because the original data far exceeds the memory capacity. Therefore, these vascular signals cannot be obtained at submicron resolution, which reduces the resolution of the vascular vessels.

[0004] (2) Since the diameter of the hepatic sinusoids is 3-5 microns, if the downsampled image exceeds the diameter of the hepatic sinusoids, it will lead to the loss of hepatic sinusoid signals in the complete liver lobe, and therefore the hepatic sinusoid structure in the complete liver lobe cannot be reconstructed. Summary of the Invention

[0005] The purpose of the present invention is to propose a vascular segmentation method, system and storage medium for liver images to solve the problem that existing image processing algorithms cannot obtain vascular signals at submicron resolution and lose liver sinusoidal signals when downsampling image data.

[0006] To achieve the above objectives, the present invention adopts the following specific technical solutions:

[0007] In one aspect, the present invention provides a method for segmenting blood vessels in a liver image based on parallel computing and dynamic data reading, comprising the following steps:

[0008] S1. Image acquisition: Hepatic sinusoidal endothelial cells are labeled and imaged using microscopic optical sectioning tomography to obtain images containing all vascular information, forming an image dataset. The vascular walls of all vessels and the cell membranes of hepatic parenchymal cells in the image serve as foreground signals, while the vascular lumen and the interior of hepatic parenchymal cells serve as background signals.

[0009] S2. Image preprocessing: First, perform intensity inversion on the images in the image dataset, so that the vascular lumen and the interior of the hepatic parenchymal cells become foreground signals, and the vascular wall and the cell membrane of the hepatic parenchymal cells become background signals. Then, use a threshold segmentation algorithm to segment the liver lobe to obtain a liver lobe region image. Perform a logical AND operation on the liver lobe region image and the intensity-inverted image, keeping the vascular lumen and hepatic parenchymal cells highlighted and the exterior of the liver lobe as a black background.

[0010] S3, vascular semantic segmentation: First, the image dataset is cut into 3D data blocks and read into memory in batches. Then, an automatic segmentation algorithm based on region growing is used to segment the vessels to obtain continuous and complete vessels. The vascular types include hepatic vein, portal vein, lymphatic vessel, bile duct, hepatic artery, and hepatic sinusoid.

[0011] S4. Vessel tracking: An automatic vessel tracking algorithm is used to track different types of vessels to complete the classification of vessels at the scale of the entire liver lobe and sub-micron resolution.

[0012] Preferably, the specific process of segmenting the vessels using the automatic segmentation algorithm based on region growing is as follows:

[0013] S31, automatically setting seed points at equal intervals in the three-dimensional data block;

[0014] S32. Segmenting the three-dimensional data block using a region growing algorithm for each seed point to obtain complete and continuous blood vessels and liver parenchymal cells in the three-dimensional data block;

[0015] S33. Calculating the outer contours of the vessels and hepatic parenchymal cells using the Euclidean method, shrinking the outer contours of the vessels and hepatic parenchymal cells toward their respective centers, thereby separating the hepatic parenchymal cells and vessels, each becoming an independent connected domain, while all connected vessels remain as an independent connected domain;

[0016] S34, using the volume threshold method to remove independent hepatocytes;

[0017] S35. Calculate the outer contour of the hepatic blood vessels again using the Euclidean method, and then superimpose the outer contour of the hepatic blood vessels onto the existing blood vessels to expand the vessels outward to their original size. Combine all three-dimensional data blocks to achieve vascular segmentation of the complete liver lobe at a sub-resolution scale.

[0018] Preferably, the specific process of using the vessel automatic tracking algorithm to track different types of vessels is as follows:

[0019] S41, using a circular structure with a radius R1 to erode the image after the vascular semantic segmentation to obtain a result V1, where the erosion radius R1 is larger than the vascular diameter of the hepatic sinusoids and smaller than the diameters of the lymphatic vessels, bile ducts, and hepatic arteries, ensuring that only the hepatic sinusoids are removed and the bile ducts, lymphatic vessels, and hepatic arteries are intact;

[0020] S42, using a radius R2 to erode the circular structure of the image after the vessel semantic segmentation to obtain a result V2, where the erosion radius R2 is larger than the vessel diameters of the lymphatic vessels, hepatic arteries, and portal veins, ensuring that only the terminal branches of the portal vein and hepatic vein are removed, while the main trunks of the portal vein and hepatic vein are retained;

[0021] S43, performing an XOR operation on the result V1 and the result V2 to obtain a result V3, where the result V3 includes the lymphatic vessels, bile duct, hepatic artery, portal vein, and terminal branches of the hepatic vein;

[0022] S44, downsampling the result V3 to obtain a result V3_ds, and reading the result V3_ds into the memory;

[0023] S45. Find independent connected domains in the result V3_ds, determine the seed points of the bile duct, lymphatic vessel, and hepatic artery, and upsample the seed points;

[0024] S46, based on the seed points of bile duct, lymphatic vessel and hepatic artery, a parallel dynamic algorithm is used to track bile duct, lymphatic vessel and hepatic artery within the entire liver lobe at submicron resolution.

[0025] S47. Downsample the result V2 to obtain a result V2_ds.

[0026] S48. Find independent connected domains in the result V2_ds, determine the seed points of the main hepatic vein and the main portal vein, and upsample the seed points;

[0027] S49, based on the seed points of the main hepatic vein and the main portal vein, a parallel dynamic algorithm is used to track the main hepatic vein and the main portal vein within the entire liver lobe at submicron resolution;

[0028] S410, performing a logical OR operation on the main hepatic vein and the terminal branches of the hepatic vein, and tracking the hepatic vein within the entire liver lobe at a sub-micron resolution using a parallel dynamic algorithm;

[0029] S411. Perform a logical OR operation on the portal vein trunk and the terminal branches of the portal vein, and track the portal vein within the entire liver lobe at a submicron resolution scale using a parallel dynamic algorithm.

[0030] Preferably, the processing process of the parallel dynamic algorithm is as follows:

[0031] S461. Based on the three-dimensional coordinates of the seed points of the bile duct, lymphatic vessel, and hepatic artery, respectively, search the hard disk for three-dimensional data blocks corresponding to the seed points of the bile duct, lymphatic vessel, and hepatic artery to obtain a set of three-dimensional data blocks.

[0032] S462: Read the 3D data block set into memory, use a 3D region growing algorithm to track the vascular semantic segmentation results of each 3D data block in the 3D data block set, and determine whether the current 3D data block is a terminal block. If so, end the tracking; if not, record the number of the 3D data block to be tracked and the corresponding seed point;

[0033] S463: Determine whether a tracking conflict occurs in the 3D data block to be tracked. If a tracking conflict occurs, all seed points are tracked sequentially. If no tracking conflict occurs, all seeds are tracked in parallel.

[0034] S464. Finally, the hepatic artery, lymphatic vessels, bile duct, portal vein, and venous terminal branches of the hepatic vein were tracked within the liver lobe at submicron resolution.

[0035] Preferably, after step S4, the method further includes the step of liver lobule segmentation, which is specifically as follows:

[0036] S51, downsampling of the hepatic vein;

[0037] S52, calculating the skeleton of the hepatic vein using a skeleton extraction algorithm;

[0038] S53. Calculate the diameter of the hepatic vein using the Kalman-Palagyi algorithm based on the skeleton of the hepatic vein;

[0039] S54, screening the terminal branches of the hepatic vein according to the diameter of the hepatic vein to obtain downsampled terminal branches of the hepatic vein;

[0040] S55, calculating the skeleton of the terminal branch of the hepatic vein using a skeleton extraction algorithm, and performing upsampling to restore it to its original size;

[0041] S56. Finding the position of the corresponding complete liver lobe based on the skeleton coordinates of the terminal branch of the hepatic vein, and cutting the data around the terminal branch of the hepatic vein to obtain a data block containing the terminal branch of the hepatic vein and the hepatic sinusoids at the original resolution;

[0042] S57, using a 3D corrosion method to operate the data block obtained in step S56 to obtain the terminal branches of the hepatic vein and the hepatic sinusoids at the original resolution;

[0043] S58. Calculating the skeleton of the hepatic sinusoids using a skeleton extraction algorithm, and converting the skeleton of the hepatic sinusoids into a graph structure; wherein the nodes in the graph structure are divided into terminal nodes and connecting nodes, the terminal nodes are end nodes of the graph structure, and the connecting nodes are intermediate nodes in the graph structure;

[0044] S59, performing a logical AND operation on the terminal branch of the hepatic vein and the terminal node in the graph structure to find the hepatic sinusoid directly connected to the terminal branch of the hepatic vein;

[0045] S510, record the intermediate nodes connected to the terminal node as vector<a,b> , calculate the vector by the cosine theorem<a,b> Vectors corresponding to the terminal branches of the hepatic vein<c,d> The angle between

[0046] S511. When the calculated angle falls within the angle threshold range, the hepatic sinusoid is considered to belong to the terminal branch of the hepatic vein, and step S511 is repeated for tracking. When the calculated angle does not fall within the angle threshold range, tracking is stopped.

[0047] Preferably, after executing step S511, the method further includes performing a pruning operation on the hepatic sinusoids, specifically as follows:

[0048] Calculate the distance between the end of the traced hepatic sinusoid and the terminal branch of the hepatic vein. If the distance exceeds the distance threshold, delete the terminal node and set the intermediate node connected to the terminal node as the terminal node. Repeat the judgment in sequence until the distance between the end of the hepatic sinusoid and the terminal branch of the hepatic vein does not exceed the distance threshold.

[0049] Preferably, in step S3, the image data set is cut into 38,000 three-dimensional data blocks, and the voxel resolution of each three-dimensional data block is 500×500×500 voxel.

[0050] On the other hand, the present invention provides a computer storage medium storing a computer program, which implements the above-mentioned vessel segmentation method when executed by a processor.

[0051] On the other hand, the present invention provides a three-dimensional cell image segmentation system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the above-mentioned vascular segmentation method when executing the computer program.

[0052] Compared with the existing technology, the present invention can automatically segment different types of blood vessels (including hepatic arteries, hepatic veins, lymphatic vessels, bile ducts, portal veins and hepatic sinusoids) at the scale of a complete liver lobe and sub-micron resolution without losing the hepatic sinusoid signal. In addition, the present invention can also automatically segment the liver lobule. Thanks to the original fine imaging results and the automatic segmentation algorithm based on regional growth, the segmented vascular structure is continuous and complete at the scale of a complete liver lobe and sub-micron resolution. At the scale of a complete liver lobe and sub-micron resolution, when using the automatic segmentation algorithm based on regional growth to process a set of data with an original size of 9TB (20000×26000×9000 voxels) on a server (CPU with 48 cores and 96 threads, memory with 500G), the preprocessing time of the original structural information data is 7.6 hours, with an average speed of 1.7×10 8 voxel / s; semantic segmentation takes 7.4 hours, with an average speed of 1.7×10 8 voxel / s; vascular tracing takes 20 hours, with an average speed of 6.5×10 7 voxel / s. Image preprocessing, vessel semantic segmentation, and vessel tracing utilize parallel computing. More processor cores and more memory reduce computation time. Furthermore, due to the algorithm's dynamic data readout, the amount of data processed and computation time are linearly related. The average speed above can be used to estimate the required processing time. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 3 is a flow chart of a method for segmenting blood vessels in liver images based on parallel computing and dynamic data reading according to an embodiment of the present invention.

[0054] Figure 2 The original image and the pre-processed image according to the embodiment of the present invention;

[0055] Figure 2 In the figure, (A) shows a slice in the original image; (B) shows the local image after preprocessing; (C) shows the acquired liver lobe edge.

[0056] Figure 3 This is a schematic diagram of the vascular semantic segmentation process according to an embodiment of the present invention;

[0057] Figure 3In the figure, (A) shows the segmentation of the complete liver lobe; (B) shows the seed points set in batches for each three-dimensional data block; (C) shows the segmentation result obtained using the region growing algorithm; (D) shows the connected liver parenchymal cells and blood vessels; (E) shows the disconnected liver parenchymal cells and blood vessels; (F) shows the deleted liver parenchymal cells; (G) shows the restored liver sinusoids; (H) shows the blood vessels obtained after semantic segmentation; (I) shows the semantic segmentation result for the complete liver lobe.

[0058] Figure 4 is a schematic diagram of vessel tracking and tracking results provided according to an embodiment of the present invention;

[0059] Figure 4 In the figure, (A) shows the process of vascular tracking within the entire liver lobe; (B) shows the process of the parallel dynamic algorithm; (C) shows the tracking results of the hepatic artery, bile duct and lymphatic vessel; (D) shows the tracking results of the hepatic vein and portal vein.

[0060] Figure 5 is a schematic diagram of a liver lobule segmentation process according to an embodiment of the present invention;

[0061] Figure 5 In the figure, (A) shows the skeleton of the extracted hepatic vein; (B) the classification result of the hepatic vein according to the diameter of the hepatic vein; (C) shows the terminal branches of the hepatic vein; (D) shows the corresponding area found in the complete liver lobe according to the skeleton of the portal vein; (E) shows the graph structure of the hepatic sinusoids in the corresponding area; (F) shows the angle between the two vectors; (G) shows the screened hepatic sinusoids; (H) shows the pruning operation of the hepatic sinusoids; (I) shows the actual segmented liver lobule result. DETAILED DESCRIPTION

[0062] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In the following description, identical modules are denoted by identical reference numerals. In the case of identical reference numerals, their names and functions are also identical. Therefore, their detailed description will not be repeated.

[0063] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not constitute a limitation of the present invention.

[0064] A first aspect of the present invention provides a method for segmenting blood vessels in a liver image based on parallel computing and dynamic data reading.

[0065] Figure 1The flowchart of the vascular segmentation method of the liver image based on parallel computing and dynamic data reading according to an embodiment of the present invention is shown.

[0066] like Figure 1 As shown, the present invention provides a method for segmenting blood vessels in a liver image based on parallel computing and dynamic data reading according to an embodiment, comprising the following steps:

[0067] S1. Image acquisition: Hepatic sinusoidal endothelial cells are labeled and imaged using microscopic optical sectioning tomography to obtain images containing all vascular information, forming an image dataset. The vascular walls of all vessels and the cell membranes of hepatic parenchymal cells in the image serve as foreground signals, while the vascular lumen and the interior of the hepatic parenchymal cells serve as background signals.

[0068] The present invention uses CD31 antibody to label liver sinusoidal endothelial cells. Therefore, the vascular walls of all vessels and the cell membranes of liver parenchymal cells in the collected image data are foreground signals, and the vascular lumen and the interior of liver parenchymal cells are background signals. Figure 2 As shown in (A) in .

[0069] S2. Image preprocessing: First, perform intensity inversion on the images in the image dataset, so that the vascular lumen and the interior of the liver parenchymal cells become foreground signals, and the vascular wall and the cell membrane of the liver parenchymal cells become background signals; then use the threshold segmentation algorithm to segment the liver lobe to obtain the liver lobe region image; perform a logical AND operation on the liver lobe region image and the intensity inverted image, keeping the vascular lumen and liver parenchymal cells highlighted and the outside of the liver lobe as a black background.

[0070] The preprocessing actually includes two steps: the first step is image flipping, which turns the vascular lumen and liver parenchymal cells in the image into foreground signals, and the vascular wall and cell membrane of liver parenchymal cells in the image into background signals, such as Figure 2 As shown in (B) in the figure. In the second step, since image flipping will also highlight the background at the edge, the threshold segmentation method is used to segment the liver lobe to obtain the liver lobe area image, as shown in Figure 2 As shown in (C) in the image above, after acquiring the liver lobe region image, a logical AND operation is performed on the liver lobe region image and the intensity-inverted image, maintaining a black background outside the liver lobe. After preprocessing, the vascular lumen and liver parenchymal cells become highlighted. This image processing step takes 4 hours.

[0071] S3. Vascular semantic segmentation: First, the image dataset is cut into three-dimensional data blocks and read into memory in batches. Then, an automatic segmentation algorithm based on region growing is used to segment the vessels to obtain continuous and complete vessels. Among them, the vascular types include hepatic vein, portal vein, lymphatic vessel, bile duct, hepatic artery and hepatic sinusoid.

[0072] In order to obtain a complete and continuous liver vasculature at the submicron resolution scale of the entire liver lobe, the present invention first cuts the entire data set into 38,000 blocks, each of which is a 3D data block of 500×500×500 voxel, so that it can be read into the memory in batches, such as Figure 3 As shown in (A) in the figure, the region growing method is then used for vascular segmentation to obtain continuous and complete vascular segments. The region growing algorithm requires manual specification of a starting seed point to begin segmentation. For the entire data segmentation, over 30,000 seed points need to be manually set, which is a tremendous amount of work. To achieve an automatic segmentation algorithm for liver vascular segmentation, the present invention has developed an automatic segmentation algorithm based on region growing, based on the structure of the liver vascular system.

[0073] The automatic segmentation algorithm consists of five steps:

[0074] The first step is to automatically set seed points at equal intervals in the 3D data block, such as Figure 3 As shown in (B);

[0075] The second step is to use the region growing algorithm to segment the data blocks for each seed point, such as Figure 3 As shown in (C).

[0076] Because the middle of the hepatocytes is also highlighted, after the first and second steps, complete and continuous blood vessels and hepatocytes can be obtained in the three-dimensional data block. There are two types of hepatocytes: the first type is independent hepatocytes, and the second type is hepatocytes connected to the hepatic sinusoids. By tracking batches of seed points, the hepatic sinusoids and hepatocytes can be obtained. When the hepatocytes are connected to the hepatic sinusoids through the cell membrane, this connection relationship can be abstracted as a straight line and a circle tangent to each other, and the contact area between the hepatic sinusoids and the hepatocytes is smaller than the contact area inside the hepatic sinusoids, such as Figure 3 As shown in (D) in the figure.

[0077] The third step is to calculate the outer contours of blood vessels and hepatic parenchymal cells by the Euclidean method, and shrink the outer contours of blood vessels and hepatic parenchymal cells toward their respective centers. Figure 3 As shown in (E) in the figure, because the connection between hepatocytes and vessels is tangential, the hepatocytes and vessels are separated after both connected regions shrink toward the center, while the connection between the vessels is not severed. After the separation, the hepatocytes and vessels are separated into independent connected domains, while all connected vessels remain as a single connected domain.

[0078] The fourth step is to use the volume threshold method to remove independent hepatocytes, such as Figure 3 As shown in (F) in .

[0079] Calculate the volume of all independent connected domains. Because hepatocytes are relatively small, while connected vessels are relatively large, a volume threshold is set. When the volume of a connected domain is less than this threshold, it is considered a hepatocyte and deleted, thereby removing independent hepatocytes.

[0080] Step 5: Calculate the outer contour of the hepatic vessels again using the Euclidean method, and then superimpose the outer contour of the hepatic vessels onto the existing vessels to expand the vessels outward and restore them to their original size. Figure 3 As shown in (G) and (H) in Figure 3, since there are dense seed points between each 3D data block, the block operation does not affect the final result. After each 3D data block is automatically segmented, all 3D data blocks are combined to achieve vascular segmentation of the complete liver lobe at sub-micron resolution, as shown in Figure 3. Figure 3 As shown in (I) in .

[0081] Because the automatic segmentation algorithm is fully automated and each block is independent of each other, parallel computing is used to speed up the calculation. On a 48-core 96-thread server, the complete semantic segmentation result takes 7 hours to run.

[0082] S4. Vessel tracking: An automatic vessel tracking algorithm is used to track different types of vessels to complete the classification of vessels at the scale of the entire liver lobe and sub-micron resolution.

[0083] After achieving semantic segmentation of the blood vessels, the segmentation results of the hepatic vein, portal vein, lymphatic vessels, bile ducts, hepatic arteries, and hepatic sinusoids are obtained. The venous ends of the hepatic vein and portal vein are smaller and closer to the bile duct.

[0084] The prerequisite for vascular classification within an entire liver lobe and at submicron resolution is that the vessels must be internally connected, with different vessels being independent and disconnected. However, lymphatic vessels, bile ducts, and the hepatic artery often accompany the portal vein, so these vessels are likely to touch each other. Furthermore, branches of the portal vein and hepatic vein may touch each other at their ends. These contacts can lead to errors in vascular segmentation. Given the massive amount of data at the terabyte level, manually identifying where the vessels connect, finding the contact points, and then manually disconnecting them is a tremendous amount of work.

[0085] In order to distinguish different types of vessels in the whole liver lobe and at submicron resolution, the present invention develops an automatic vessel tracking algorithm, such as Figure 4 As shown in (A) in the figure. Since the hepatic vein and portal vein in the liver are larger in diameter, while the lymphatic vessels, bile ducts, and hepatic arteries are smaller in diameter, the hepatic sinusoids have the smallest diameter. Therefore, the present invention adopts a divide-and-conquer approach to tracing vessels in the liver, as follows:

[0086] Step 1: Use a circular structure with a radius of R1 to erode the image after vascular semantic segmentation to obtain the result V1. The erosion radius of R1 is larger than the vascular diameter of the hepatic sinusoids and smaller than the diameters of the lymphatic vessels, bile ducts, and hepatic arteries, thereby ensuring that only the hepatic sinusoids are eroded and the bile ducts, lymphatic vessels, and hepatic arteries are preserved intact.

[0087] Step 2: Use radius R2 to erode the image after vascular semantic segmentation to obtain result V2. This radius is larger than the vascular diameters of the lymphatic vessels, hepatic arteries, and portal veins, thereby obtaining the main trunks of the portal vein and hepatic vein, while the terminal branches of the portal vein and hepatic vein will be eroded.

[0088] Step 3. Perform an XOR operation on the results of V1 and V2 to achieve the subtraction of the results of V1 and V2, and obtain the result V3. The result V3 includes the terminal branches of the lymphatic vessels, bile ducts, hepatic arteries, portal veins and hepatic veins.

[0089] The portal vein and hepatic vein are the main blood vessels of the liver. The terminal branches of the hepatic vein are called central veins, while the terminal branches of the portal vein do not have specific names.

[0090] Step 4: Downsample the result V3 to obtain the result V3_ds, and read the result V3_ds into the memory.

[0091] Step 5. Find the independent connected domains in the result V3_ds, determine the seed points of the bile duct, lymphatic vessel, and hepatic artery, and upsample the seed points of the bile duct, lymphatic vessel, and hepatic artery.

[0092] Step 6: Using the upsampled seed points of the bile duct, lymphatic vessel, and hepatic artery, a parallel dynamic algorithm is used to achieve vascular tracking within the entire liver lobe at submicron resolution, such as Figure 4 As shown in (B) in the figure.

[0093] The principle of the parallel dynamic algorithm is as follows:

[0094] Step 6.1. According to the three-dimensional coordinates of the seed points of the bile duct, lymphatic vessel and hepatic artery, search for the three-dimensional data blocks corresponding to the seed points of the bile duct, lymphatic vessel and hepatic artery in the hard disk to obtain a three-dimensional data block set.

[0095] For example, the three-dimensional data block corresponding to the hepatic artery is found based on the hepatic artery seed point, and the three-dimensional data block corresponding to the lymphatic vessel is found based on the lymphatic vessel seed point.

[0096] Step 6.2. Read the three-dimensional data block set into the memory, use the three-dimensional region growing algorithm to track the vascular semantic segmentation results of each three-dimensional data block in the three-dimensional data block set, and at the same time determine whether the current three-dimensional data block is a terminal block. If so, end the tracking. If not, record the number of the three-dimensional data block to be tracked and the corresponding seed point.

[0097] Step 6.3: After completing the tracking of the 3D data block, save the results to the hard disk. Simultaneously, determine whether the 3D data block is a terminal block. If so, tracking ends. If not, record the number of the 3D data block to be tracked and the corresponding seed point. Because given a seed point and block number, the region growing method still maintains block independence. This invention utilizes a parallel algorithm for parallel tracking, thereby accelerating computational speed.

[0098] Step 6.4: Determine whether a tracking conflict occurs in the three-dimensional data block to be tracked. If a tracking conflict occurs, all seed points are tracked sequentially. If no tracking conflict occurs, all seed points are tracked in parallel.

[0099] Since the tracking process is parallel, when multiple seed points track the same 3D data block at the same time, the tracking results of different seed points cannot be synchronized, so conflicts will occur. Therefore, first determine whether the 3D data block to be tracked conflicts. If a conflict occurs, select one of the seed points and return to step S461 for tracking, and the other seed points will wait for the next round of tracking; if there is no conflict, all seed points will directly return to step S461 for parallel tracking;

[0100] After a parallel dynamic algorithm, the present invention tracks the hepatic artery, lymphatic vessels, and terminal branches of the hepatic artery and vein in the complete liver lobe at submicron resolution.

[0101] Step 7. Downsample the result V2 to obtain the result V2_ds.

[0102] Step 8. Find the independent connected domains in the result V2_ds, determine the seed points of the hepatic vein and the portal vein trunk, and upsample the seed points of the hepatic vein and the portal vein trunk.

[0103] Step 9: Use a parallel dynamic algorithm to track the hepatic vein main trunk HV_main and the portal vein main trunk PV_main in the entire liver lobe and at submicron resolution.

[0104] Step 10: Perform a logical OR operation on the main hepatic vein HV_main and the central hepatic vein HV_CV, and then track the hepatic vein HV within the entire liver lobe at a submicron resolution using a parallel dynamic algorithm.

[0105] Step 11: Perform a logical OR operation on the portal vein main trunk PV_main and the portal vein terminal branch PV_CV, and then track the portal vein PV within the entire liver lobe at a submicron resolution scale using a parallel dynamic algorithm.

[0106] After the above steps, the present invention realizes the tracking of the hepatic vein, portal vein, lymphatic vessel, bile duct and hepatic artery in the complete liver lobe and at sub-micron resolution.

[0107] The present invention solves the problem of not being able to find the connection point when the hepatic artery, lymphatic vessels and bile duct are connected to the portal vein in a complete liver lobe through Step 1, Step 2 and Step 3. The present invention downsamples massive data through Step 4 and Step 5, and searches for seed points in the blood vessels in the downsampled data. After upsampling, the seed points are converted into coordinates under the complete liver lobe, thereby achieving seed point setting and solving the problem of finding specific blood vessel seed points in a complete liver lobe and at a sub-micron resolution scale. Through the parallel dynamic algorithm of Step 6, the blood vessels can be tracked when the data is not fully read into the memory, solving the problems of insufficient memory and slow computing speed. The problem of not being able to distinguish between the hepatic vein and the portal vein when they are connected at the terminal branches is solved through Step 10 and Step 11. After the above steps, the tracking of different blood vessels in a complete liver lobe and at a sub-micron resolution scale is finally achieved, such as Figure 4 As shown in (C) and (D).

[0108] The hepatic lobule is the smallest structural unit of liver tissue. The typical structure of a mouse hepatic lobule is a central vein surrounded by numerous venous branches. Based on the anatomical structure of the hepatic lobule, the hepatic lobule tissue in mice is organized as hepatic sinusoids radiating from the central vein. The present invention can segment the hepatic lobule after obtaining detailed hepatic sinusoidal structures and portal veins.

[0109] The steps for segmenting the liver lobule are as follows:

[0110] The first step is to downsample the hepatic vein (2×2×2voxel).

[0111] The second step is to calculate the skeleton of the hepatic vein using the skeleton extraction algorithm, such as Figure 5 As shown in (A) in .

[0112] The skeleton extraction algorithm is an existing technology, and the reference is "Building skeleton models via 3-Dmedial surface / axis thinning algorithms".

[0113] The third step is to calculate the diameter of the hepatic vein using the Kalman-Palagyi algorithm based on the skeleton of the hepatic vein, such as Figure 5 As shown in (B) in the figure.

[0114] The Kalman-Palagyi algorithm considers the blood vessel branch as a cylinder, and the corresponding diameter calculation formula is as follows:

[0115]

[0116] The diameter of the blood vessels can be calculated.

[0117] Step 4: Screen the central vein of the hepatic vein according to the diameter of the hepatic vein to obtain the central vein of the hepatic vein with downsampling, such as Figure 5 As shown in (C).

[0118] Step 5: Calculate the skeleton of the central vein of the hepatic vein using a skeleton extraction algorithm and perform upsampling to restore it to its original size.

[0119] Step 6: Find the position of the corresponding complete liver lobe according to the skeleton coordinates of the central vein of the hepatic vein, and cut around the central vein of the hepatic vein to obtain the data block containing the terminal branches of the hepatic vein and the hepatic sinusoids at the original resolution, such as Figure 5 As shown in (D) in the figure.

[0120] Step 7: Use a 3D corrosion method to operate the data block obtained in step S56 to obtain the terminal branches of the hepatic vein and the hepatic sinusoids at the original resolution.

[0121] The cut data block contains the central vein and hepatic sinusoids at the original resolution. Set a structure with a radius R, which is larger than the diameter of the hepatic sinusoids but smaller than the diameter of the central vein. Then use 3D erosion to obtain the central vein and hepatic sinusoids of the hepatic vein at the original resolution.

[0122] Step 8: Calculate the skeleton of the hepatic sinusoids through the skeleton extraction algorithm and transform the skeleton of the hepatic sinusoids into a graph structure. The nodes in the graph structure are divided into terminal nodes and connection nodes, such as Figure 5 As shown in (E), the terminal node is the end node of the graph structure, and the connecting node is the intermediate node in the graph structure. At this point, the present invention has obtained the central vein shape, the central vein main axis corresponding to the central vein, and the corresponding hepatic sinusoidal graph structure.

[0123] Step 9. Perform a logical AND operation on the central vein of the hepatic vein and the terminal nodes in the graph structure to find the hepatic sinusoid directly connected to the central vein of the hepatic vein.

[0124] Step 10: Record the intermediate nodes connected to the terminal nodes of the hepatic sinusoid as vectors<a,b> , and then calculate the vector using the cosine theorem<a,b> The vector corresponding to the central vein<c,d> The angle between Figure 5 As shown in (F) in .

[0125] In step 11, when the calculated angle falls within the angle threshold range, the hepatic sinusoid is considered to belong to the central vein, and step S511 is repeated for tracking. When the calculated angle does not fall within the angle threshold range, tracking is stopped. Figure 5 As shown in (G) in .

[0126] In the above steps, the quality of the liver lobule segmentation result is related to the set angle. If the threshold is set high, the segmentation area of ​​the liver lobule will become smaller, while if the threshold is set low, the segmentation area of ​​the liver lobule will become larger. In order to solve the strict dependence of the threshold angle setting, the present invention adds a pruning operation. By detecting the distance between the end of the hepatic sinusoid and the central vein, the terminal nodes that exceed the distance threshold are pruned. Figure 5 If the distance between the end of the hepatic sinusoid and the central vein exceeds the distance threshold, the terminal node is deleted, and the intermediate node connected to the terminal node is set as the terminal node. The judgment is repeated in sequence until the distance between the end of all hepatic sinusoids and the central vein is kept within a reasonable range, as shown in Figure 1. Figure 5 As shown in (I) in .

[0127] In the above algorithm, since the segmentation of each liver lobule is still performed independently, the present invention adopts a parallel operation method to speed up the operation of the algorithm.

[0128] The present invention solves the problems of insufficient memory and slow computing speed in the TB data processing process through parallel computing and dynamic data reading methods. It can automatically segment different types of blood vessels, including hepatic arteries, hepatic veins, lymphatic vessels, bile ducts, portal veins and hepatic sinusoids, at the scale of complete liver lobes and sub-micron resolution. In addition, the present invention can also automatically segment liver lobules. Thanks to the original fine imaging results and the region-growing algorithm adopted, the segmented vascular structure is continuous and complete on the complete liver lobe and at sub-resolution. At the complete liver lobe and sub-micron resolution, when the above algorithm is used to process a set of data with an original size of 9TB (20000×26000×9000 voxels) on a server (CPU with 48 cores and 96 threads, memory with 500G), the preprocessing time of the original structural information data is 7.6 hours, with an average speed of 1.7×10^ 8 voxel / s; semantic segmentation takes 7.4 hours, with an average speed of 1.7×10^ 8voxel / s; vascular tracing takes 20 hours, with an average speed of 6.5×10^ 7 voxel / s. Preprocessing, semantic segmentation, and vessel tracing are parallelized. Theoretically, more processor cores and more memory reduce computation time. Furthermore, due to the algorithm's dynamic data readout, the amount of data processed and computation time are linearly related. The average speed above can be used to estimate the required processing time.

[0129] A second aspect of the present invention provides a computer storage medium, in which a computer program is stored. When the computer program is executed by a processor, the method steps of the first aspect of the present application are implemented.

[0130] The storage medium is a memory, and the memory can be a non-volatile storage medium, which can exemplarily include but is not limited to read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM) or flash memory, for example, it can be any of the following: embedded multi media card (EMMC), Nor Flash, Nand Flash, etc.

[0131] Exemplarily, the memory may further include a cache device for caching data, such as a signal queue. The cache device may be a volatile storage medium, and may exemplarily include but is not limited to random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), DDR2, DDR3, enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DRAM).

[0132] The third aspect of the present invention provides a vascular segmentation system for liver images based on parallel computing and dynamic data reading. The vascular segmentation system includes a memory, a processor, and a computer program stored in the memory and running on the processor. When the processor executes the computer program, the steps of the method of the first aspect of the present application are implemented.

[0133] For example, the memory may include a memory card of a smartphone, a storage component of a tablet computer, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0134] Exemplarily, the processor may be a central processing unit (CPU), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and may control other components in the system to perform desired functions. For example, the processor may include one or more embedded processors, processor cores, microprocessors, logic circuits, hardware finite state machines (FSMs), digital signal processors (DSPs), or combinations thereof.

[0135] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0136] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

[0137] The above specific embodiments of the present invention do not constitute a limitation on the scope of protection of the present invention. Any other corresponding changes and modifications made based on the technical concept of the present invention should be included in the scope of protection of the claims of the present invention.

Claims

1. A method for vascular segmentation of liver images based on parallel computing and dynamic data reading, characterized in that: The steps include: S1. Image acquisition: Hepatic sinusoidal endothelial cells are labeled and imaged using microscopic optical sectioning tomography to obtain images containing all vascular information, forming an image dataset. The vascular walls of all vessels and the cell membranes of hepatic parenchymal cells in the image serve as foreground signals, while the vascular lumen and the interior of hepatic parenchymal cells serve as background signals. S2. Image preprocessing: First, perform intensity inversion on the images in the image dataset, so that the vascular lumen and the interior of the hepatic parenchymal cells become foreground signals, and the vascular wall and the cell membrane of the hepatic parenchymal cells become background signals. Then, use a threshold segmentation algorithm to segment the liver lobe to obtain a liver lobe region image. Perform a logical AND operation on the liver lobe region image and the intensity-inverted image, keeping the vascular lumen and hepatic parenchymal cells highlighted and the exterior of the liver lobe as a black background. S3, vascular semantic segmentation: First, the image dataset is cut into 3D data blocks and read into memory in batches. Then, an automatic segmentation algorithm based on region growing is used to segment the vessels to obtain continuous and complete vessels. The vascular types include hepatic vein, portal vein, lymphatic vessel, bile duct, hepatic artery, and hepatic sinusoid. S4. Vessel Tracking: An automatic vessel tracking algorithm is used to track different types of vessels, thereby classifying the vessels at a sub-micron resolution within the entire liver lobe. The specific process of using the automatic vessel tracking algorithm to track different types of vessels is as follows: S41, using a circular structure with a radius R1 to erode the image after the vascular semantic segmentation to obtain a result V1, where the erosion radius R1 is larger than the vascular diameter of the hepatic sinusoids and smaller than the diameters of the lymphatic vessels, bile ducts, and hepatic arteries, ensuring that only the hepatic sinusoids are removed and the bile ducts, lymphatic vessels, and hepatic arteries are intact; S42, using a radius R2 to erode the circular structure of the image after the vessel semantic segmentation to obtain a result V2, where the erosion radius R2 is larger than the vessel diameters of the lymphatic vessels, hepatic arteries, and portal veins, ensuring that only the terminal branches of the portal vein and hepatic vein are removed, while the main trunks of the portal vein and hepatic vein are retained; S43, performing an XOR operation on the result V1 and the result V2 to obtain a result V3, where the result V3 includes the lymphatic vessels, bile duct, hepatic artery, portal vein, and terminal branches of the hepatic vein; S44, downsampling the result V3 to obtain a result V3_ds, and reading the result V3_ds into the memory; S45. Find independent connected domains in the result V3_ds, determine the seed points of the bile duct, lymphatic vessel, and hepatic artery, and upsample various seed points; S46, based on the seed points of bile duct, lymphatic vessel and hepatic artery, a parallel dynamic algorithm is used to track bile duct, lymphatic vessel and hepatic artery within the entire liver lobe at submicron resolution. S47, downsampling the result V2 to obtain a result V2_ds; S48. Find independent connected domains in the result V2_ds, determine the seed points of the main hepatic vein and the main portal vein, and upsample the seed points; S49, based on the seed points of the main hepatic vein and the main portal vein, a parallel dynamic algorithm is used to track the main hepatic vein and the main portal vein within the entire liver lobe at submicron resolution; S410, performing a logical OR operation on the main hepatic vein and the terminal branches of the hepatic vein, and tracking the hepatic vein within the entire liver lobe at a sub-micron resolution using a parallel dynamic algorithm; S411. Perform a logical OR operation on the portal vein trunk and the terminal branches of the portal vein, and track the portal vein within the entire liver lobe at a submicron resolution scale using a parallel dynamic algorithm.

2. The vessel segmentation method according to claim 1, wherein: The specific process of segmenting blood vessels using the automatic segmentation algorithm based on region growing is as follows: S31, automatically setting seed points at equal intervals in the three-dimensional data block; S32. Segmenting the three-dimensional data block using a region growing algorithm for each seed point to obtain complete and continuous blood vessels and liver parenchymal cells in the three-dimensional data block; S33. Calculating the outer contours of the vessels and hepatic parenchymal cells using the Euclidean method, shrinking the outer contours of the vessels and hepatic parenchymal cells toward their respective centers, thereby separating the hepatic parenchymal cells and vessels, each becoming an independent connected domain, while all connected vessels remain as an independent connected domain; S34, using the volume threshold method to remove independent hepatocytes; S35. Calculate the outer contour of the hepatic blood vessels again using the Euclidean method, and then superimpose the outer contour of the hepatic blood vessels onto the existing blood vessels to expand the vessels outward to their original size. Combine all three-dimensional data blocks to achieve vascular segmentation of the complete liver lobe at a sub-resolution scale.

3. The vessel segmentation method according to claim 1, wherein: The processing of the parallel dynamic algorithm is as follows: S461. Based on the three-dimensional coordinates of the seed points of the bile duct, lymphatic vessel, and hepatic artery, respectively, search the hard disk for three-dimensional data blocks corresponding to the seed points of the bile duct, lymphatic vessel, and hepatic artery to obtain a set of three-dimensional data blocks. S462: Read the 3D data block set into memory, use a 3D region growing algorithm to track the vascular semantic segmentation results of each 3D data block in the 3D data block set, and determine whether the current 3D data block is a terminal block. If so, end the tracking; if not, record the number of the 3D data block to be tracked and the corresponding seed point; S463: determining whether a tracking conflict occurs in the three-dimensional data block to be tracked. If a tracking conflict occurs, all seed points are tracked sequentially. If no tracking conflict occurs, all seed points are tracked in parallel. S464. Finally, the hepatic artery, lymphatic vessels, bile duct, portal vein, and venous terminal branches of the hepatic vein were tracked within the liver lobe at submicron resolution.

4. The vascular segmentation method according to any one of claims 1 to 3, characterized in that: After step S4, the process further includes the following steps: S51, downsampling of the hepatic vein; S52, calculating the skeleton of the hepatic vein using a skeleton extraction algorithm; S53. Calculate the diameter of the hepatic vein using the Kalman-Palagyi algorithm based on the skeleton of the hepatic vein; S54, screening the terminal branches of the hepatic vein according to the diameter of the hepatic vein to obtain downsampled terminal branches of the hepatic vein; S55, calculating the skeleton of the terminal branch of the hepatic vein using a skeleton extraction algorithm, and performing upsampling to restore it to its original size; S56. Find the position of the corresponding complete liver lobe according to the skeleton coordinates of the terminal branch of the hepatic vein, and cut around the terminal branch of the hepatic vein to obtain a data block containing the terminal branch of the hepatic vein and the hepatic sinusoids at the original resolution; S57, using a 3D corrosion method to operate the data block obtained in step S56 to obtain the terminal branches of the hepatic vein and the hepatic sinusoids at the original resolution; S58. Calculating the skeleton of the hepatic sinusoids using a skeleton extraction algorithm, and converting the skeleton of the hepatic sinusoids into a graph structure; wherein the nodes in the graph structure are divided into terminal nodes and connecting nodes, the terminal nodes are end nodes of the graph structure, and the connecting nodes are intermediate nodes in the graph structure; S59, performing a logical AND operation on the terminal branch of the hepatic vein and the terminal node in the graph structure to find the hepatic sinusoid directly connected to the terminal branch of the hepatic vein; S510, record the intermediate nodes connected to the terminal node as vector<a,b> , calculate the vector by the cosine theorem<a,b> Vectors corresponding to the terminal branches of the hepatic vein<c,d> The angle between S511. When the calculated angle falls within the angle threshold range, the hepatic sinusoid is considered to belong to the terminal branch of the hepatic vein, and step S511 is repeated for tracking. When the calculated angle does not fall within the angle threshold range, tracking is stopped.

5. The vessel segmentation method according to claim 4, wherein: After executing step S511, the process further includes performing a pruning operation on the hepatic sinusoids, specifically as follows: Calculate the distance between the end of the traced hepatic sinusoid and the terminal branch of the hepatic vein. If the distance exceeds the distance threshold, delete the terminal node and set the intermediate node connected to the terminal node as the terminal node. Repeat the judgment in sequence until the distance between the end of the hepatic sinusoid and the terminal branch of the hepatic vein does not exceed the distance threshold.

6. The vessel segmentation method according to claim 1, wherein: In step S3, the image data set is cut into 38,000 three-dimensional data blocks, and the voxel resolution of each three-dimensional data block is 500×500×500 voxel.

7. A computer storage medium, characterized in that The computer storage medium stores a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 6.

8. A three-dimensional cell image segmentation system, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor implements the method according to any one of claims 1 to 6 when executing the computer program.

Images