A method and device for hepatic vein and portal vein segmentation

Abstract

The present invention provides a method and device for segmenting hepatic veins and portal veins. The method includes: inputting the original medical image into a first segmentation model to obtain a liver mask; inputting the liver mask into a second segmentation model to obtain a segmented image region including 6 categories, namely background, hepatic vein vessels, outermost mask of hepatic vein vessels, outermost mask of portal vein vessels, portal vein vessels, and intersection region of hepatic veins and portal vein vessels; correcting the segmented image region based on the connectivity of the vessels to obtain an image including 2 categories, namely hepatic vein vessels and portal vein vessels. By obtaining segmented image regions of 6 different categories, the present invention can keep the main veins of hepatic veins and portal vein vessels separated as much as possible; moreover, by segmenting the outermost mask of hepatic vein vessels and the outermost mask of portal vein vessels, the noise interference caused by unclear boundaries due to partial volume effect can be reduced, so the segmentation accuracy of hepatic veins and portal vein vessels is significantly improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical imaging, and particularly relates to a method and device for segmenting hepatic veins and portal veins. Background Art

[0002] In recent years, the number of patients with liver diseases has been showing a continuous upward trend, seriously endangering human health. Since many liver diseases need to be treated by liver resection surgery or ablation, during the pre-operative planning stage, doctors need to segment the liver based on the morphological trends of hepatic veins and portal veins in medical images to accurately resect or eliminate the diseased area during the operation as much as possible, and retain more liver function for the patient. However, in the current surgical planning work, the vast majority of medical image reading tasks are completed manually by doctors, which poses a huge challenge to the doctor's energy, experience, and patience. Therefore, accurate automatic segmentation of hepatic veins and portal veins helps doctors obtain the overall information of liver blood vessels, which is of great significance for liver disease diagnosis and liver surgery.

[0003] Compared with the segmentation of organs in medical images, blood vessel tissues have the following inherent characteristics: irregular branch shapes, individual differences in overall distribution, uneven sizes, and there are also many adhesions and intersections between hepatic veins and portal veins. Therefore, the segmentation of hepatic veins and portal veins is still a huge challenge in the field of medical image processing. At the same time, since most medical data are three-dimensional data, a huge amount of computation is required in the deep learning network model. The rise of deep learning has brought new opportunities for the segmentation of hepatic veins and portal veins. Existing methods for segmenting hepatic veins and portal veins mostly consider blood vessel segmentation and the classification of hepatic veins and portal veins as two different tasks, and design systems and deep learning network models accordingly to ensure the segmentation results of hepatic veins and portal veins. For example, by setting two parallel branches at the output end of the deep learning network model, the upper branch is used to extract the features of hepatic veins and portal veins different from the background, and the lower branch is used to extract the features for distinguishing hepatic veins and portal veins; or using a two-stage method, first segment the liver blood vessels, and then judge the blood vessel adhesion points to separate the two types of blood vessels, hepatic veins and portal veins; or by extracting rich inter-layer information of adjacent CT sequence layers through a graph neural network, and combining a sequence attention association fusion module designed based on the attention mechanism to achieve the segmentation of three-dimensional medical data layer by layer, but it is still based on two-dimensional image segmentation and fails to obtain sufficient three-dimensional structural information. In addition, due to the huge parameters of the three-dimensional deep learning network model, during the segmentation task, data chunking operations are performed, and then the segmentation results of each chunk are stitched together. This not only requires a huge amount of computation, but also these methods do not fully utilize (or even destroy) the structural characteristics and prior information of hepatic veins and portal veins themselves. Summary of the Invention

[0004] To solve the above problems existing in the prior art, the present invention provides a method and device for hepatic vein and portal vein segmentation.

[0005] To achieve the above object, the present invention adopts the following technical solutions.

[0006] In a first aspect, the present invention provides a method for hepatic vein and portal vein segmentation, including the following steps:

[0007] Input the original medical image into the first segmentation model to obtain a liver mask;

[0008] Input the liver mask into the second segmentation model to obtain a segmentation image region including 6 categories: background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and intersection region of hepatic vein and portal vein blood vessels;

[0009] Based on the connectivity of blood vessels, correct the segmentation image region to obtain an image including 2 categories: hepatic vein blood vessels and portal vein blood vessels.

[0010] Further, the method further includes a step of preprocessing the liver mask:

[0011] Crop the minimum bounding box region of the liver mask;

[0012] Based on the cropped region, adjust the window width and window level, and normalize the pixel value of each pixel point to [0, 1] according to the following formula:

[0013]

[0014] In the formula, I(x, y) and I w (x, y) are the pixel values of the pixel point with coordinates (x, y) before and after normalization respectively, W is the window width, and C is the window level;

[0015] Adjust the liver region image to a fixed size, and perform denoising filtering on the image.

[0016] Further, the second segmentation model adopts a 3D ResUnet network structure.

[0017] Furthermore, a Focus module is provided at the input end of the 3D ResUnet.

[0018] Furthermore, a combination of dilated convolutions for expanding the receptive field is provided before the first 3 downsampling layers of the 3D ResUnet.

[0019] Further, in the total loss function for training the second segmentation model, the weights of the loss functions for the background, hepatic vein vessels, outermost mask of hepatic vein vessels, outermost mask of portal vein vessels, portal vein vessels, and the intersection region of hepatic vein and portal vein vessels are 1, 1, 0.5, 0.5, 1, and 1 respectively.

[0020] Further, the method for correcting the segmented image region includes:

[0021] S1. Set the pixel values of the pixel points in the six regions of the background, hepatic vein vessels, outermost mask of hepatic vein vessels, outermost mask of portal vein vessels, portal vein vessels, and the intersection region of hepatic vein and portal vein vessels to 0, 1, 2, 3, 4, and 5 respectively;

[0022] S2. Eliminate the image regions with connected components smaller than the set threshold, and eliminate the image regions with pixel point pixel values of 5;

[0023] S3. Respectively extract the largest connected components of the image regions with pixel point pixel values of 1 and 4 to obtain the main hepatic vein and the main portal vein;

[0024] S4. Modify the pixel values of the pixel points in the remaining vascular connected components with pixel values of 1 and 4 outside the two main veins to the pixel values 1 or 4 of the main vein connected to them;

[0025] S5. Repeat step S4 until the pixel values of all the remaining vascular connected components outside the two main veins are modified;

[0026] S6. Calculate the mean of the pixel values of the non-0 pixel points in the 5×5×5 region centered on each non-0 pixel point, and replace the pixel value of the corresponding pixel point with the mean;

[0027] S7. Output the pixel points with pixel values greater than 0 and less than 2.5 as the hepatic vein vessel category, and output the pixel points with pixel values greater than or equal to 2.5 as the portal vein vessel category.

[0028] Further, the method further includes a step of smoothing the corrected image: performing morphological correction on the corrected image using three-dimensional opening operation and three-dimensional closing operation.

[0029] In a second aspect, the present invention provides a hepatic vein and portal vein segmentation device, including:

[0030] A first segmentation module, configured to input the original medical image into the first segmentation model to obtain a liver mask;

[0031] A second segmentation module, configured to input the liver mask into a second segmentation model to obtain a segmented image region including six categories: background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels;

[0032] A segmentation correction module, configured to correct the segmented image region based on the connectivity of blood vessels to obtain an image including two categories: hepatic vein blood vessels and portal vein blood vessels.

[0033] Further, the apparatus further includes a liver mask preprocessing module, configured to:

[0034] Crop the minimum bounding box region of the liver mask;

[0035] Adjust the window width and window level based on the cropped region, and normalize the pixel value of each pixel point to [0, 1] according to the following formula:

[0036]

[0037] In the formula, I(x, y) and I w (x, y) are the pixel values of the pixel point with coordinates (x, y) before and after normalization respectively, W is the window width, and C is the window level;

[0038] Adjust the liver region image to a fixed size, and perform denoising filtering on the image.

[0039] Further, the second segmentation model adopts a 3D ResUnet network structure.

[0040] Furthermore, a Focus module is provided at the input end of the 3D ResUnet.

[0041] Furthermore, a combination of dilated convolutions for expanding the receptive field is provided before the first three downsampling layers of the 3D ResUnet.

[0042] Further, in the total loss function of the training of the second segmentation model, the weights of the loss functions of the background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels are 1, 1, 0.5, 0.5, 1, and 1 respectively.

[0043] Further, the segmentation correction module is specifically configured to:

[0044] S1. Set the pixel values of the pixel points in six regions of the background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels to 0, 1, 2, 3, 4, and 5 respectively;

[0045] S2. Remove the image regions where the connected components are smaller than the set threshold, and remove the image regions where the pixel value of the pixel points is 5.

[0046] S3. Respectively extract the largest connected components of the image regions where the pixel values of the pixel points are 1 and 4 to obtain the main hepatic veins and the main portal veins.

[0047] S4. Modify the pixel values of the pixel points of the remaining vascular connected components where the pixel values are 1 and 4 outside the two main veins to the pixel values 1 or 4 of the main veins connected to them.

[0048] S5. Repeat step S4 until the pixel values of all the remaining vascular connected components outside the two main veins are modified.

[0049] S6. Calculate the mean value of the pixel values of the non-zero pixel points within the 5×5×5 region centered on each non-zero pixel point, and replace the pixel value of the corresponding pixel point with the mean value.

[0050] S7. Output the pixel points with pixel values greater than 0 and less than 2.5 as the hepatic vein vessel category, and output the pixel points with pixel values greater than or equal to 2.5 as the portal vein vessel category.

[0051] Furthermore, the device further includes a smoothing processing module for performing morphological correction on the corrected image by using three-dimensional opening operation and three-dimensional closing operation.

[0052] Compared with the prior art, the present invention has the following beneficial effects.

[0053] The present invention inputs the liver mask output by the first segmentation model into the second segmentation model to obtain a segmented image region including 6 different categories: background, hepatic vein vessels, outermost mask of hepatic vein vessels, outermost mask of portal vein vessels, portal vein vessels, and intersection region of hepatic vein and portal vein vessels, and corrects the segmented image region based on the connectivity of the vessels. Finally, an image with 2 categories of hepatic vein vessels and portal vein vessels is obtained, realizing the automatic segmentation of hepatic vein and portal vein vessels. The present invention uses the second segmentation model to obtain a segmented image region including not only hepatic vein and portal vein vessels, but also 6 different categories: background, outermost mask of hepatic vein vessels, outermost mask of portal vein vessels, and intersection region of hepatic vein and portal vein vessels, which can keep the main veins of hepatic vein and portal vein separated as much as possible; moreover, by segmenting the outermost mask of hepatic vein vessels and the outermost mask of portal vein vessels, the noise interference caused by unclear boundaries due to partial volume effect can be reduced, so the segmentation accuracy of hepatic vein and portal vein vessels is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of a method for segmenting hepatic vein and portal vein according to an embodiment of the present invention.

[0055] Figure 2 It is a schematic diagram of the network structure of the second segmentation model.

[0056] Figure 3 It is a schematic diagram of the structure of the Focus module.

[0057] Figure 4 It is a schematic diagram of the dilated convolution combination structure.

[0058] Figure 5 It is a block diagram of a hepatic vein and portal vein segmentation device according to an embodiment of the present invention. Detailed implementation manners

[0059] To make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the present invention will be further described below with reference to the accompanying drawings and specific implementation manners. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0060] Figure 1 It is a flowchart of a hepatic vein and portal vein segmentation method according to an embodiment of the present invention, including the following steps:

[0061] Step 101: Input the original medical image into the first segmentation model to obtain a liver mask;

[0062] Step 102: Input the liver mask into the second segmentation model to obtain a segmented image area including 6 categories: background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and intersection area of hepatic vein and portal vein blood vessels;

[0063] Step 103: Correct the segmented image area based on the connectivity of the blood vessels to obtain an image including 2 categories: hepatic vein blood vessels and portal vein blood vessels.

[0064] In this embodiment, step 101 is mainly used to obtain a liver mask including hepatic vein and portal vein blood vessels from the input original medical image. The input original medical image includes but is not limited to CT images and MRI images. In this embodiment, the liver mask is obtained by inputting the original medical image into a trained liver segmentation network, that is, the first segmentation model. Since the hepatic vein and portal vein blood vessels to be segmented are both included in the area covered by the liver mask, first segmenting the liver mask and then processing based on the cropped liver mask can reduce the data processing burden.

[0065] In this embodiment, step 102 is mainly used to obtain segmented image regions of six categories by using the second segmentation model. These six categories of segmented image regions not only include hepatic veins and portal veins, but also include four other categories outside the hepatic veins and portal veins. Since the ultimate goal is only to segment the hepatic veins and portal veins, the prior art generally only outputs images (masks) of these two categories. In this embodiment, when training the second segmentation model, six labels are used in the dataset to label the background, hepatic vein vessels, outermost mask of hepatic vein vessels, outermost mask of portal vein vessels, portal vein vessels, and the intersection region of hepatic veins and portal veins (obtained by taking the intersection of the hepatic veins and portal veins after three-dimensional dilation operation). The trained second segmentation model can output images of the above six categories. Such processing is mainly considered from the following two aspects: First, by independently labeling the outermost mask of hepatic veins, the outermost mask of portal veins, and the intersection region of hepatic veins and portal veins, the main veins of hepatic veins and portal veins can be kept separate in labeling, which is convenient for result correction through the connectivity of the two types of blood vessels in post-processing. Second, by setting up two categories, namely the outermost mask of hepatic veins and the outermost mask of portal veins, the influence of noise on training can be reduced. The reason why the noise influence can be reduced is that due to the partial volume effect in CT images, the blood vessel edges are not clear at the pixel level, so the labeling of blood vessel edges by the labeling personnel is affected by subjective factors and is not as accurate as the labeling of the main veins of blood vessels. By separately setting the two categories of the outermost mask of hepatic vein vessels and the outermost mask of portal vein vessels, these labeling points with lower certainty can be classified separately, and the influence of noise on training can be reduced by setting a lower category weight in the loss function. Therefore, by labeling images of six categories, compared with the prior art that only labels two categories, the segmentation accuracy can be significantly improved.

[0066] In this embodiment, step 103 belongs to a post-processing step, which is mainly used to correct the segmented image regions obtained in step 102 to obtain images of hepatic veins and portal veins. Step 102 outputs images of six categories, but we ultimately only need to output images of hepatic veins and portal veins. Therefore, it is necessary to correct the images of six categories, reclassify them, and only output images of two categories, namely hepatic veins and portal veins. This embodiment mainly corrects based on the connectivity of blood vessels. Specifically, it is corrected according to the characteristics that the hepatic veins and portal veins are completely connected and separated from each other (excluding regions with labels 2 and 3). For example, regions with too small connected components are removed, and the intersection region of hepatic veins and portal veins is removed, etc. A specific correction method will be given in the following embodiments.

[0067] As an optional embodiment, the method further includes a step of preprocessing the liver mask:

[0068] Crop the minimum bounding rectangular cuboid region of the liver mask;

[0069] Based on the cropped region, adjust the window width and window level, and normalize the pixel value of each pixel point to [0, 1] according to the following formula:

[0070]

[0071] In the formula, I(x, y) and I w (x, y) are the pixel values of the pixel point with coordinates (x, y) before and after normalization, W is the window width, and C is the window level;

[0072] Adjust the liver region image to a fixed size and perform denoising filtering on the image.

[0073] This embodiment provides a technical solution for preprocessing the liver mask. First, obtain the minimum bounding rectangular cuboid region of the mask according to the three-dimensional boundary extreme values of the liver mask, and crop this region from the original image. Then, adjust the window width and window level of the liver region. The following is a specific adjustment method: obtain the average CT value of the liver according to the liver mask, denoted as m; use m + 50 as the window level C and 200 as the window width W to adjust the pixel values of the liver region, and perform normalization processing according to the above formula to convert all pixel values to [0, 1]. Subsequently, adjust the liver region image to a fixed size according to the morphological characteristics of the liver. For example, the size of the liver region can be adjusted to 128×224×256. Finally, perform denoising filtering on the image. There are many filtering methods. For example, three-dimensional anisotropic diffusion filtering can be used. Anisotropic diffusion filtering is a filtering algorithm based on partial differential equations, which can eliminate noise while retaining image details and edges, thereby improving image quality. In addition to three-dimensional anisotropic diffusion filtering, common filtering methods such as Gaussian filtering, mean filtering, and bilateral filtering can also be selected.

[0074] As an optional embodiment, the second segmentation model adopts a 3D ResUnet network structure.

[0075] This embodiment presents a network structure for implementing the second segmentation model. The second segmentation model in this embodiment adopts a 3D ResUnet network structure. 3D ResUnet is a 3D Unet network model combined with a residual structure. Among them, the residual structure is a structural body embedded in a convolutional neural network proposed to solve the problem of vanishing gradients. Vanishing gradients is a very common problem in deep learning. Briefly, when training a CNN, the gradient starts from the last layer and needs to pass through each intermediate layer before reaching the initial layer; but often the gradient approaches 0, making the machine learning speed slow or even completely stop, thus making it difficult to complete the training of the model. The residual structure creates an alternative path for gradient transmission to skip intermediate layers and directly reach the initial layer. This enables people to train extremely deep models with better performance. Currently, the residual structure has been more and more widely used.

[0076] As an optional embodiment, a Focus module is provided at the input end of the 3D ResUnet.

[0077] This embodiment presents an improved structure of the second segmentation model. In this embodiment, a Focus module is added at the input end of the 3D ResUnet, as Figure 2 shown in the area selected by the dashed box 1 in the figure. The Focus structure increases the channel dimension while reducing the Z-axis dimension to ensure that data accuracy is not lost, thereby reducing the computational complexity. At the last layer of the network, the original size of the Z-axis of the image is restored by doubling the upsampling of the Z-axis. The following takes a two-dimensional image as an example to illustrate. The principle of the Focus structure is specifically as Figure 3 shown. First, the original image with a size of 1×8×8 is sampled at intervals in the X and Y directions to generate 4 images with a size of 1×4×4; then these 4 images are stacked in the channel dimension, that is, a new image of 4×4×4 is generated, achieving the purpose of reducing the computational complexity while ensuring data accuracy. In this embodiment, only the Z-axis is sampled. Therefore, after passing through the Focus structure, the image size changes from 1×128×224×256 to 2×64×224×256.

[0078] As an optional embodiment, a dilated convolution combination for expanding the receptive field is provided before the first 3 downsampling layers of the 3D ResUnet.

[0079] This embodiment presents another improved structure of the second segmentation model. In this embodiment, a dilated convolution combination for expanding the receptive field is provided before the first 3 downsampling layers of the 3D ResUnet, as Figure 2 shown in the dashed box 2 in the figure. The specific structure of the dilated convolution combination is as Figure 4As shown in the figure, it consists of 4 dilated convolutions with different intervals and a global average pooling layer. The feature layer enters the convolution kernel with different intervals for calculation. The interval is the difference in sampling position coordinates of adjacent sampling points of the convolution kernel. It can be seen that the larger the interval, the sparser the sampling, and the larger the receptive field. The calculation formula of the receptive field size is:

[0080] size=(d-1)×(k-1)+k

[0081] Where d represents the number of intervals and k represents the size of the convolution kernel. For example, the number of intervals of the dilated convolution combination can be 1, 6, 12, and 18 respectively. Then the four sets of feature layers generated are superimposed with the results of the global pooling layer in the channel dimension, and finally restored to the feature dimension at the time of input through a 1×1×1 convolution. It is worth noting that since the input and output sizes of the dilated convolution combination are consistent, any number of dilated convolution combinations can be inserted at any position in the network, and the specific number of interval combinations can also be adjusted according to actual conditions.

[0082] As an optional embodiment, in the total loss function of the second segmentation model training, the weights of the loss functions of the background, hepatic vein, the outermost mask of the hepatic vein, the outermost mask of the portal vein, the portal vein, and the intersection area of ​​the hepatic vein and the portal vein are 1, 1, 0.5, 0.5, 1, and 1, respectively.

[0083] This embodiment gives the weights of the loss functions of the six different types of segmented regions in the total loss function of the second segmentation model training. As mentioned above, separately setting the outermost mask of the hepatic vein and the outermost mask of the portal vein is conducive to reducing the impact of noise. This effect is achieved by reducing the weights of these two categories in the loss function during model training. This embodiment sets the weights of these two categories to half of the other four categories. Specifically, the weights of these two categories are 0.5, and the weights of the other four categories are 1.

[0084] As an optional embodiment, the method for correcting the segmented image region includes:

[0085] S1, setting the pixel values ​​of the six areas of background, hepatic vein, outermost mask of hepatic vein, outermost mask of portal vein, portal vein, and intersection of hepatic vein and portal vein to 0, 1, 2, 3, 4, and 5 respectively;

[0086] S2, eliminate image areas where the connected domain is smaller than a set threshold, and eliminate image areas where the pixel value is 5;

[0087] S3, extracting the maximum connected domain of the image area where the pixel values ​​are 1 and 4 respectively, to obtain the main hepatic vein and the main portal vein;

[0088] S4. Modify the pixel values of the pixel points in the remaining vessel connected regions with pixel values of 1 and 4 outside the two main veins to the pixel values 1 or 4 of the pixel points of the main veins connected thereto.

[0089] S5. Repeat step S4 until the pixel values of all pixel points in the remaining vessel connected regions outside the two main veins are modified.

[0090] S6. Calculate the mean value of the pixel values of the non-0 pixel points in the 5×5×5 region centered on each non-0 pixel point, and replace the pixel value of the corresponding pixel point with the mean value.

[0091] S7. Output the pixel points with pixel values greater than 0 and less than 2.5 as the hepatic vein vessel category, and output the pixel points with pixel values greater than or equal to 2.5 as the portal vein vessel category.

[0092] This embodiment provides a technical solution for correcting the output result of the second segmentation model. In step S1, the pixel values of 6 mask image regions are first set to six integer values from 0 to 5 respectively, which is equivalent to dividing the output mask image into six gray levels, and each region mask corresponds to one gray level. In step S2, the segmentation result is first screened for connected components, and the regions with too small connected components are removed as incorrect results, and the intersection region of the hepatic vein and portal vein vessels with a pixel value of 5 is removed. Starting from step S3, according to the respective connectivity characteristics of the hepatic vein and portal vein vessels, the data after removal is corrected. During this correction process, only the hepatic vein and portal vein vessels with pixel values of 1 and 4 are targeted, excluding the outermost masks of the hepatic vein vessels and portal vein vessels with pixel values of 2 and 3. Even if measures such as expanding the receptive field are taken (such as the combination of dilated convolutions has a significant effect of expanding the receptive field), there may still be insufficient acquisition of the main vein features in the terminal vessel branches far from the main vein, resulting in incorrect vessel classification. This correction process is based on the fact that the hepatic vein and portal vein vessels are completely connected to each other. After ignoring the regions with pixel values of 2 and 3, it can be considered that the hepatic vein and portal vein vessels are in a state of being separated from each other but completely connected to themselves at this time. Therefore, in this embodiment, according to the vessel connectivity characteristics, the misclassified vessels are corrected. In step S3, the largest connected region of each of the two types of vessels is first extracted as their respective main veins. In step S4, the pixel values of the remaining vessel connected components outside the two main veins are corrected. To facilitate understanding of the correction method, an example is given below. Suppose there are two remaining vessel connected components A and B after extracting the two main veins, the pixel value of A is 1, and the pixel value of B is 4. If the pixel value of the main vein connected to A is 4 instead of 1, then the pixel values of all pixel points of A are changed to 4. Similarly, if the pixel value of the main vein connected to B is 1 instead of 4, then the pixel values of all pixel points of B are changed to 1. That is, the pixel values of A and B are modified to the pixel values of the main veins connected to them. Step S4 is repeatedly executed until the pixel values of all the remaining vessel connected components outside the two main veins are modified (step S5). In step S6, the mean value of the non-zero pixel values in a 5×5×5 region (other different sizes and shapes of regions can also be selected) centered on each pixel point with a non-zero pixel value is calculated, and the corresponding pixel point is replaced with the mean value. In step S7, all pixel points are divided into two categories with a threshold of 2.5. The category with a smaller pixel value is the hepatic vein vessel, and the category with a larger pixel value is the portal vein vessel.

[0093] As an optional embodiment, the method further includes a step of smoothing the corrected image: performing morphological correction on the corrected image by using three-dimensional opening operation and three-dimensional closing operation.

[0094] This embodiment provides a technical solution for smoothing the corrected image. In this embodiment, morphological correction of the image is performed by performing three-dimensional opening operation and three-dimensional closing operation. The three-dimensional opening operation first performs a three-dimensional erosion operation and then a three-dimensional dilation operation to disconnect some weak spatial connections. The three-dimensional closing operation first performs a three-dimensional dilation operation and then a three-dimensional erosion operation to fill some small gaps and holes. The erosion operation and the dilation operation are both basic operation units of morphological processing. Performing three-dimensional opening operation and three-dimensional closing operation on the hepatic veins and portal veins in the corrected segmentation result respectively can make the contour of the segmentation result smoother.

[0095] Figure 5 FIG. is a schematic diagram of the composition of a hepatic vein and portal vein segmentation device according to an embodiment of the present invention. The device includes:

[0096] The first segmentation module 11 is configured to input the original medical image into the first segmentation model to obtain a liver mask;

[0097] The second segmentation module 12 is configured to input the liver mask into the second segmentation model to obtain a segmentation image region including 6 categories: background, hepatic vein vessels, outermost mask of hepatic vein vessels, outermost mask of portal vein vessels, portal vein vessels, and intersection region of hepatic vein and portal vein vessels;

[0098] The segmentation correction module 13 is configured to correct the segmentation image region based on the connectivity of the vessels to obtain an image including 2 categories: hepatic vein vessels and portal vein vessels.

[0099] The device of this embodiment can be used to execute Figure 1 the technical solution of the method embodiment shown. The implementation principle and technical effect are similar and will not be elaborated here. The same is true for the subsequent embodiments and will not be further described.

[0100] As an optional embodiment, the device further includes a liver mask preprocessing module, configured to:

[0101] Crop the minimum bounding box region of the liver mask;

[0102] Adjust the window width and window level based on the cropped region, and normalize the pixel value of each pixel point to [0, 1] according to the following formula:

[0103]

[0104] In the formula, I(x, y) and I w (x, y) are the pixel values of the pixel point with coordinates (x, y) before and after normalization respectively, W is the window width, and C is the window level;

[0105] Adjust the liver region image to a fixed size and perform denoising filtering on the image.

[0106] As an alternative embodiment, the second segmentation model 12 adopts a 3D ResUnet network structure.

[0107] As an alternative embodiment, a Focus module is provided at the input end of the 3D ResUnet.

[0108] As an alternative embodiment, a combination of dilated convolutions for expanding the receptive field is provided before the 3 downsampling layers of the 3D ResUnet.

[0109] As an alternative embodiment, in the total loss function for training the second segmentation model, the weights of the loss functions for the background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels are 1, 1, 0.5, 0.5, 1, and 1 respectively.

[0110] As an alternative embodiment, the segmentation correction module 13 is specifically configured to:

[0111] S1. Set the pixel values of the pixel points in the 6 regions of the background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels to 0, 1, 2, 3, 4, and 5 respectively;

[0112] S2. Remove the image regions where the connected component is smaller than the set threshold, and remove the image regions where the pixel value of the pixel point is 5;

[0113] S3. Respectively extract the largest connected components of the image regions where the pixel value of the pixel point is 1 and 4 to obtain the main hepatic vein and the main portal vein;

[0114] S4. Modify the pixel values of the pixel points in the remaining blood vessel connected components with pixel values of 1 and 4 other than the two main veins to the pixel values 1 or 4 of the main vein to which they are connected;

[0115] S5. Repeat step S4 until the pixel values of all the remaining blood vessel connected components other than the two main veins are modified;

[0116] S6. Calculate the mean of the pixel values of the non-0 pixel points in the 5×5×5 region centered on each non-0 pixel point, and replace the pixel value of the corresponding pixel point with the mean;

[0117] S7. Output the pixel points with pixel values greater than 0 and less than 2.5 as the hepatic vein blood vessel category, and output the pixel points with pixel values greater than or equal to 2.5 as the portal vein blood vessel category.

[0118] As an optional embodiment, the device further includes a smoothing processing module, which is configured to perform morphological correction on the corrected image by using three-dimensional opening operation and three-dimensional closing operation.

[0119] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for segmenting hepatic veins and portal veins, characterized in that, It includes the following steps: Input the original medical image into the first segmentation model to obtain a liver mask; Input the liver mask into the second segmentation model to obtain a segmented image region including 6 categories: background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels; Based on the connectivity of blood vessels, correct the segmented image region to obtain an image including 2 categories: hepatic vein blood vessels and portal vein blood vessels; The method for correcting the segmented image region includes: S1. Set the pixel values of the pixel points in the 6 regions of background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels to 0, 1, 2, 3, 4, and 5 respectively; S2. Eliminate the image regions with a connectivity domain smaller than the set threshold, and eliminate the image regions with a pixel value of 5 for pixel points; S3. Respectively extract the largest connected domain of the image regions with pixel values of 1 and 4 for pixel points to obtain the main hepatic vein and the main portal vein; S4. Modify the pixel values of the pixel points in the remaining blood vessel connected domains with pixel values of 1 and 4 outside the two main veins to the pixel values of the main veins they are connected to, and correct the incorrect blood vessel categories; S5. Repeat step S4 until the pixel values of all the remaining blood vessel connected domains connected to the two main veins outside the two main veins are modified; S6. Calculate the mean value of the pixel values of the non-0 pixel points in the 5×5×5 region centered on each non-0 pixel point, and replace the pixel value of the corresponding pixel point with the mean value; S7. Output the pixel points with pixel values greater than 0 and less than 2.5 as the hepatic vein blood vessel category, and output the pixel points with pixel values greater than or equal to 2.5 as the portal vein blood vessel category.

2. The method for segmenting hepatic veins and portal veins according to claim 1, characterized in that, The method further includes a step of preprocessing the liver mask: Crop the minimum bounding box region of the liver mask; Based on the cropped region, adjust the window width and window level, and normalize the pixel value of each pixel point to [0,1] according to the following formula: Wherein, I(x,y) and I w (x,y) are the pixel values of the pixel point with coordinates (x,y) before and after normalization respectively, W is the window width, and C is the window level; Adjust the liver region image to a fixed size, and perform denoising filtering on the image.

3. The method for segmenting hepatic veins and portal veins according to claim 1, characterized in that, The second segmentation model adopts a 3D ResUnet network structure.

4. The method for segmenting hepatic veins and portal veins according to claim 3, characterized in that, The input end of the 3D ResUnet is provided with a Focus module.

5. The method for segmenting hepatic veins and portal veins according to claim 3, characterized in that, Before the first 3 downsampling layers of the 3D ResUnet, there is a combination of dilated convolutions for expanding the receptive field.

6. The method for segmenting hepatic veins and portal veins according to claim 1, characterized in that, In the total loss function of the training of the second segmentation model, the weights of the loss functions of the background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels are 1, 1, 0.5, 0.5, 1, and 1 respectively.

7. The method for segmenting hepatic veins and portal veins according to claim 1, characterized in that, The method further includes a step of smoothing the corrected image: performing morphological correction on the corrected image using three-dimensional opening operation and three-dimensional closing operation.

8. A device for segmenting hepatic veins and portal veins, characterized in that, The device includes: A first segmentation module for inputting the original medical image into the first segmentation model to obtain a liver mask; The second segmentation module is used to input the liver mask into the second segmentation model to obtain a segmented image region including six categories: background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels; The segmentation correction module is used to correct the segmented image region based on the connectivity of blood vessels to obtain an image including two categories: hepatic vein blood vessels and portal vein blood vessels; The method for correcting the segmented image region includes: S1. Set the pixel values of the pixel points in the six regions of background, hepatic vein blood vessels, outermost mask of hepatic vein blood vessels, outermost mask of portal vein blood vessels, portal vein blood vessels, and the intersection region of hepatic vein and portal vein blood vessels to 0, 1, 2, 3, 4, and 5 respectively; S2. Eliminate the image regions with a connected domain smaller than the set threshold, and eliminate the image regions with a pixel value of 5 for pixel points; S3. Extract the largest connected domain of the image regions with pixel values of 1 and 4 for pixel points respectively to obtain the main hepatic vein and the main portal vein; S4. Modify the pixel values of the pixel points in the remaining blood vessel connected domains with pixel values of 1 and 4 outside the two main veins to the pixel values of the main veins connected thereto to correct the incorrect blood vessel categories; S5. Repeat step S4 until the pixel values of the pixel points in all the remaining blood vessel connected domains connecting the main veins outside the two main veins are modified; S6. Calculate the mean value of the pixel values of the non-zero pixel points in the 5×5×5 region centered on each non-zero pixel point, and replace the pixel value of the corresponding pixel point with the mean value; S7. Output the pixel points with pixel values greater than 0 and less than 2.5 as the hepatic vein blood vessel category, and output the pixel points with pixel values greater than or equal to 2.5 as the portal vein blood vessel category.

Images