A selective hepatobiliary blood flow occlusion calibration method and system based on three-dimensional reconstruction

Abstract

The present invention relates to a selective hepatobiliary blood flow occlusion calibration method and system based on three-dimensional reconstruction, which includes extracting a feature map of hepatobiliary tomographic images through a backbone network, inputting the feature map into a classification branch to obtain a probability distribution matrix of each pixel point, calculating the gradient of each point in the hepatobiliary tomographic image to obtain an edge intensity map, obtaining an edge uncertainty probability map by using the probability distribution matrix and the edge intensity map, performing image segmentation on the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; determining the number of images to be inserted between two adjacent tomographic images, and determining the region of interest in the inserted images according to the regions of interest of the same type in two adjacent tomographic images and the edge uncertainty probability map; performing three-dimensional reconstruction by using the segmented regions of interest and the regions of interest in the inserted images, and determining the blood flow occlusion marking position by using the relationship between the blood vessels and the patient region.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction, and specifically to a method and system for calibrating selective hepatobiliary blood flow occlusion based on three-dimensional reconstruction. Background Art

[0002] Hepatobiliary blood flow occlusion is an important measure to reduce bleeding during hepatobiliary surgery, mainly including total hepatobiliary blood flow occlusion and selective hepatobiliary occlusion. Compared with total hepatobiliary blood flow occlusion, selective hepatobiliary blood flow occlusion can maximize the protection of normal hepatobiliary tissues and reduce the possibility of postoperative hepatobiliary failure. Although selective hepatobiliary blood flow occlusion can maximize the health of patients, traditional selective hepatobiliary blood flow occlusion methods often rely on doctors' experience and touch, and it is difficult to achieve precise positioning and control, especially in complex hepatobiliary surgeries. Three-dimensional reconstruction technology can construct an accurate three-dimensional model of the liver blood vessels based on the CT or MRI medical images of patients. With the help of the three-dimensional reconstruction model, doctors can clearly observe the anatomical structure of the liver blood vessels, including the distribution of the hepatic artery, portal vein and their branches. Moreover, doctors can simulate different blood flow occlusion schemes on the three-dimensional model, evaluate their impact on liver blood supply, and thus select the best occlusion position and degree. However, limited by the complexity of CT, MRI and the human body structure, in three-dimensional reconstruction, if the liver and gallbladder, blood vessels and diseased parts cannot be accurately segmented, the established three-dimensional model will not be accurate enough. Moreover, when performing three-dimensional reconstruction, images need to be inserted between CT, MRI slices, etc., which will accumulate errors into the finally established three-dimensional model, resulting in loss of details, etc., which all affect the accuracy of blood flow occlusion marking. Summary of the Invention

[0003] Aiming at the problems of large errors and missing details in the three-dimensional model when determining the position of hepatobiliary blood flow occlusion marking based on the three-dimensional model, the present invention proposes a method for calibrating selective hepatobiliary blood flow occlusion based on three-dimensional reconstruction, and the method includes the following steps:

[0004] Extract the feature map of the hepatobiliary tomographic image through the backbone network, input the feature map into the classification branch to obtain the probability distribution matrix of each pixel point, calculate the gradient of each point in the hepatobiliary tomographic image to obtain the edge intensity map, use the probability distribution matrix and the edge intensity map to obtain the edge uncertainty probability map, and segment the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest;

[0005] Determine the number of images to be inserted between two adjacent tomographic images, and determine the region of interest in the inserted image according to the regions of interest of the same type in two adjacent tomographic images and the edge uncertainty probability map;

[0006] Perform three-dimensional reconstruction using the region of interest obtained by segmentation and the region of interest of the inserted image, and determine the blood flow blockage marker position based on the relationship between the blood vessels and the patient region.

[0007] Preferably, obtaining the edge uncertainty probability map using the probability distribution matrix and the edge intensity map is specifically as follows:

[0008] Obtain the classification probability of each pixel point from the probability distribution matrix, take the second-largest probability as the background and the second-largest probability of the pixel point where the largest probability and the second-largest probability meet the preset conditions as the edge feature value, and the edge feature values of other pixel points are 0, and construct an edge feature map;

[0009] Multiply the edge feature map and the edge intensity map bit by bit to obtain the edge uncertainty probability map.

[0010] Preferably, segmenting the medical tomographic image to obtain the region of interest according to the edge uncertainty probability map and the probability distribution matrix is specifically as follows:

[0011] For the pixel points in the edge feature map with non-zero edge feature values, calculate the weighted sum of the values in the edge uncertainty probability map and the second-largest probability in the probability distribution matrix. If the result of the weighted sum is greater than the largest probability or the preset value, the category of the pixel point is the category corresponding to the second-largest probability, otherwise the category of the pixel point is the category corresponding to the largest probability;

[0012] For the pixel points in the edge feature map with zero edge feature values, determine the category of the pixel points according to the largest probability in the probability distribution matrix;

[0013] Determine the region of interest based on the classification of the pixel points in the hepatobiliary tomographic image.

[0014] Preferably, determining the region of interest in the inserted image according to the regions of interest of the same type in two adjacent tomographic images and the edge uncertainty probability map is specifically as follows:

[0015] Obtain the areas and centroids of the regions of interest of the same type in two adjacent tomographic images, and connect the two centroids;

[0016] Downsample the region of interest in the tomographic image with a larger area of the region of interest, upsample the region of interest in the tomographic image with a smaller area of the region of interest, merge the results of upsampling and downsampling, and determine the pixel values of the region of interest in the inserted image according to the merged result;

[0017] Downsample the edge uncertainty probability map of the tomographic image with a larger area of the region of interest, upsample the edge uncertainty probability map of the tomographic image with a smaller area of the region of interest, merge the results of upsampling and downsampling, and determine the edge probability of the region of interest in the inserted image according to the merged result;

[0018] Obtain the pixel points where the edge probability of the region of interest in the inserted image meets the condition. If the pixel point is located in the region of interest in the inserted image, then remove the pixel point from the region of interest;

[0019] Determine the centroid of the region of interest in the inserted image, and move the region of interest of the inserted image based on the centroid and the connection line of the centroid.

[0020] Preferably, the method for determining the blood flow blockage marking position by using the relationship between blood vessels and the patient area is specifically as follows:

[0021] Obtain the blood vessel in the three-dimensional model that is closest to the patient area, and add a blood flow blockage position marking at a preset distance from the blood vessel to the patient area.

[0022] In addition, the present invention also provides a selective hepatobiliary blood flow blockage calibration system based on three-dimensional reconstruction. The system includes the following modules:

[0023] A segmentation module, which is used to extract the feature map of the hepatobiliary tomographic image through the backbone network, input the feature map into the classification branch to obtain the probability distribution matrix of each pixel point, calculate the gradient of each point in the hepatobiliary tomographic image to obtain the edge intensity map, use the probability distribution matrix and the edge intensity map to obtain the edge uncertainty probability map, and perform image segmentation on the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest;

[0024] A tomographic interpolation module, which is used to determine the number of images to be inserted between two adjacent tomographic images, and determine the region of interest in the inserted image according to the regions of interest of the same type in two adjacent tomographic images and the edge uncertainty probability map;

[0025] A three-dimensional reconstruction and marking module, which is used to perform three-dimensional reconstruction by using the segmented region of interest and the region of interest of the inserted image, and determine the blood flow blockage marking position by using the relationship between blood vessels and the patient area.

[0026] Preferably, the method for obtaining the edge uncertainty probability map by using the probability distribution matrix and the edge intensity map is specifically as follows:

[0027] Obtain the classification probability of each pixel point from the probability distribution matrix, take the second largest probability as the edge feature value for the pixel points where the second largest probability is the background and the largest probability and the second largest probability meet the preset conditions, and the edge feature value of other pixel points is 0, and construct an edge feature map;

[0028] Multiply the edge feature map and the edge intensity map bit by bit to obtain the edge uncertainty probability map.

[0029] Preferably, the method for performing image segmentation on the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest is specifically as follows:

[0030] For the pixel points in the edge feature map with non-zero edge feature values, calculate the weighted sum of the values in the edge uncertainty probability map and the second-largest probability in the probability distribution matrix. If the result of the weighted sum is greater than the maximum probability or a preset value, the category of the pixel point is the category corresponding to the second-largest probability; otherwise, the category of the pixel point is the category corresponding to the maximum probability.

[0031] For the pixel points in the edge feature map with edge feature values of 0, determine the category of the pixel points according to the maximum probability in the probability distribution matrix.

[0032] Determine the region of interest based on the classification of the pixel points in the hepatobiliary tomographic images.

[0033] Preferably, the method for determining the region of interest in the inserted image according to the regions of interest of the same category in two adjacent tomographic images and the edge uncertainty probability map is as follows:

[0034] Obtain the areas and centroids of the regions of interest of the same category in two adjacent tomographic images, and connect the two centroids.

[0035] Downsample the region of interest in the tomographic image with a larger area of the region of interest, upsample the region of interest in the tomographic image with a smaller area of the region of interest, merge the results of the upsampling and downsampling, and determine the pixel values of the region of interest in the inserted image according to the merged result.

[0036] Downsample the edge uncertainty probability map of the tomographic image with a larger area of the region of interest, upsample the edge uncertainty probability map of the tomographic image with a smaller area of the region of interest, merge the results of the upsampling and downsampling, and determine the edge probability of the region of interest in the inserted image according to the merged result.

[0037] Obtain the pixel points in the inserted image whose edge probabilities of the region of interest meet the conditions. If the pixel point is located in the region of interest in the inserted image, then remove the pixel point from the region of interest.

[0038] Determine the centroid of the region of interest in the inserted image, and move the region of interest in the inserted image based on the centroid and the connection line of the centroids.

[0039] Preferably, the method for determining the blood flow blockage marking position by using the relationship between blood vessels and the patient region is as follows:

[0040] Obtain the blood vessel in the three-dimensional model that is closest to the patient region, and add a blood flow blockage position mark at a preset distance from the patient region on the blood vessel.

[0041] When performing tomographic image segmentation, the present invention more accurately segments the region of interest by combining the probability distribution matrix and the edge uncertainty probability map. In addition, by using the region of interest information and the edge uncertainty probability map of adjacent images, the region of interest in the inserted image is more accurately estimated, thereby optimizing the continuity of the region of interest in three-dimensional space. Additionally, the present invention can automatically determine the position of the blood flow blocking marker according to the relationship between the blood vessels and the patient area. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 is a flowchart of Example 1;

[0043] Figure 2 is a schematic diagram of a hepatobiliary tomographic image;

[0044] Figure 3 is a schematic diagram of the gradient intensity of the edge region;

[0045] Figure 4 is the edge intensity map of the hepatobiliary tomographic image;

[0046] Figure 5 is a flowchart of inserting an image into adjacent tomographic images;

[0047] Figure 6 is the effect diagram of 3D reconstruction;

[0048] Figure 7 is the effect diagram of 3D reconstruction after reducing the liver transparency;

[0049] Figure 8 is a schematic diagram of the position of the blocking marker. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0050] In this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising a..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0051] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0052] Figure 1 The first embodiment of the present invention is shown as Figure 1 shown, and the method includes the following steps:

[0053] S1. Extract the feature map of the hepatobiliary tomographic image through the backbone network, input the feature map into the classification branch to obtain the probability distribution matrix of each pixel point, calculate the gradient of each point in the hepatobiliary tomographic image to obtain the edge intensity map, use the probability distribution matrix and the edge intensity map to obtain the edge uncertainty probability map, and perform image segmentation on the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest;

[0054] Medical tomographic images can clearly show the shape, size, position, and structure of internal organs and tissues in the human body and have become an important technology in medicine. Hepatobiliary tomographic images are cross-sectional images including the liver, gallbladder, etc. obtained by means such as CT or MRI. The hepatobiliary tomographic images include, but are not limited to, CT and MRI images, and their contents are not limited to the liver, gallbladder, etc., and may also include, for example, blood vessels, bones, etc. In one embodiment, the hepatobiliary tomographic image is a liver tomographic image or a gallbladder tomographic image, as Figure 2 shown.

[0055] Extract the feature map of the hepatobiliary tomographic image by using the backbone network. The backbone network includes, but is not limited to, ResNet, VGG, DenseNet, etc. After obtaining the feature map, input the feature map into the classification branch to obtain the probability distribution matrix of each pixel point. Among them, the classification branch is used to classify each pixel point, such as the liver, tumor, blood vessel, etc. The probability distribution matrix is H×W×C, where H is the height of the hepatobiliary tomographic image, W is the width of the hepatobiliary tomographic image, and C is the number of categories. The number of categories is the number to be classified. In one example, for the hepatobiliary tomographic image, the liver region, blood vessel region, background region, and tumor region need to be segmented, so the number of categories is 4. The background region is neither the liver, nor the blood vessel and the tumor. Of course, the number of categories is not limited to the above and may also include, for example, bones, skin, etc. A pixel point in a hepatobiliary tomographic image corresponds to a vector in the probability distribution matrix, and the vector length is C. For example, [0.6, 0.2, 0.1, 0.1] are the probabilities of the liver region, blood vessel region, background region, and tumor region respectively. In an alternative embodiment, models such as U-Net or Swin-Unet are used to obtain the probability distribution matrix of the hepatobiliary tomographic image.

[0056] Edges are important features in image segmentation. Simply relying on the probability distribution of pixels for segmentation may lead to blurred edges or inaccurate segmentation. The boundaries of organs and lesion areas such as the liver, gallbladder, and tumors usually show sudden changes in pixel values in the image, that is, areas with relatively large gradients. By calculating the gradient, it helps to improve the accuracy of segmentation. After obtaining the gradient using operators such as Sobel, Prewitt, and Canny, the gradient value or gradient intensity is converted into an edge intensity map. Preferably, the edge intensity map is further normalized after maximum filtering. In the more marginal parts of the hepatobiliary tomographic images, the corresponding values in the edge intensity map are larger.

[0057] Edge regions are often difficult to segment because the pixel classes in these regions may be unclear. The probability distribution matrix provides the probability that each pixel belongs to different classes, reflecting the prediction confidence of the model for pixel classes, while the edge intensity map provides the intensity of the pixel located at the edge, reflecting the edge characteristics of the pixel position. Figure 3 shows the gradient intensity map obtained after calculating the gradient of the original matrix. From Figure 3 it can be seen that the pixel points located at the boundary of the black region have relatively high gradient intensities or gradient values. In the probability distribution matrix, the edge regions are very likely to be misclassified, but the farther or closer the boundary is to the region of interest, the smaller the possibility of being misclassified. For example, Figure 3 in, the maximum probabilities of pixel point 1 and pixel point 2 may both correspond to the liver, but the probability that pixel point 1 is the background is less than the probability that pixel point 2 is the background. For example, the probability distributions of pixel points 1 and 2 are [0.6, 0.3, 0.1] and [0.5, 0.4, 0.1] respectively. In the conventional way, both pixel points 1 and 2 are segmented as the liver. The present invention uses gradient information to classify pixel points 1 and 2 into the appropriate categories. Figure 4 shows Figure 2 of the edge intensity map. Combining the probability distribution matrix and the edge intensity map realizes a more comprehensive evaluation or calculation of the edge. In one embodiment, the value in the edge uncertainty probability map is determined by the method of edge probability × (1 - maximum classification probability). In another embodiment, after obtaining the edge uncertainty probability map, the values less than the preset value are set to 0. For example, the preset value is 0.1, and all the values of the points less than 0.1 in the edge uncertainty probability map are set to 0.

[0058] In another embodiment, the method of obtaining the edge uncertainty probability map by using the probability distribution matrix and the edge intensity map is specifically as follows:

[0059] Obtain the classification probability of each pixel from the probability distribution matrix. Take the second-largest probability as the edge feature value for the pixel whose second-largest probability corresponds to the background and the largest probability and the second-largest probability satisfy the preset conditions, and set the edge feature value of other pixels to 0 to construct an edge feature map;

[0060] Multiply the edge feature map and the edge intensity map bit by bit to obtain an edge uncertainty probability map.

[0061] Extract the classification probability of each pixel from the probability distribution matrix, and find the largest probability value and the second-largest probability value of each pixel. For each pixel, if the category corresponding to the second-largest probability is the background and the difference between the largest probability and the second-largest probability satisfies the preset conditions, preferably the preset condition is that the difference is less than a certain threshold such as 0.1 or 0.2, etc., then take the second-largest probability value of the pixel as the edge feature value; otherwise, set the edge feature value of the pixel to 0. Combine the edge feature values of all pixels to form an edge feature map; further, multiply the edge feature map and the edge intensity map bit by bit, and the result obtained is the edge uncertainty probability map. For example, the probability that pixel P belongs to the liver is 0.6, the probability that it belongs to the background is 0.3, and the probability that it belongs to blood vessels is 0.1. Since the second-largest probability is the background and assuming the threshold is 0.2, the difference between the largest probability and the second-largest probability is 0.3 which is greater than 0.2 and satisfies the preset conditions, then the edge feature value of pixel P is 0.3; the edge feature values of other pixels that do not meet the above conditions are set to 0. For example, if the second-largest probability corresponding to pixel Q is not the background, then the edge feature value of pixel Q is 0. The edge feature values of all pixels constitute the edge feature map. Further assume that the value in the edge intensity map of pixel P is 0.6, then in the edge uncertainty probability map, the value of pixel P is 0.18 and the value of pixel Q is 0. Among them, the pixel points P and Q are pixel points in the hepatobiliary tomographic image. Since the hepatobiliary tomographic image, the edge feature map, and the edge intensity map have the same size and corresponding positions, the corresponding edge feature values and edge probability values can be determined according to the positions of pixel points P and Q in the hepatobiliary tomographic image.

[0062] The edge uncertainty probability map represents the uncertainty of the pixel being located in the edge region. Use the edge uncertainty probability map to correct or optimize the probability distribution matrix, so as to obtain a more accurate segmentation result. In another embodiment, the method for segmenting a medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest is specifically as follows:

[0063] For the pixel points in the edge feature map whose edge feature values are not 0, calculate the weighted sum of the values in the edge uncertainty probability map and the second-largest probability in the probability distribution matrix. If the result of the weighted sum is greater than the largest probability or the preset value, then the category of the pixel point is the category corresponding to the second-largest probability; otherwise, the category of the pixel point is the category corresponding to the largest probability;

[0064] For the pixel points with edge feature values of 0 in the edge feature map, determine the category of the pixel points according to the maximum probability in the probability distribution matrix;

[0065] Determine the region of interest based on the classification of the pixel points in the hepatobiliary tomographic image.

[0066] The pixel points with non-zero edge feature values in the edge uncertainty probability map are the pixel points that may be misclassified. Calculate the weighted sum of the values in the edge uncertainty probability map and the second-largest probability in the probability distribution matrix. For example Figure 3 In the example, the probability distributions of pixel points 1 and 2 are [0.6, 0.3, 0.1] and [0.5, 0.4, 0.1] respectively. Assuming that the gradient intensities are both 0.8, the values in the edge uncertainty probability maps of the two are 0.24 and 0.32 respectively, and the weighted sums are 0.54 and 0.72 respectively. The weighted sum of pixel point 1, 0.54, is less than the maximum probability 0.6, so the category of pixel point 1 is the category corresponding to the maximum probability 0.6. The weighted sum of pixel point 2, 0.72, is greater than the maximum probability 0.5, so the category of pixel point 2 is the category corresponding to the second-largest probability 0.4, that is, the background. In the above example, the weights are all 1, and other weights can also be used. The determination of these weights is related to, for example, the specific gradient calculation method. It is not limited to comparing the weighted sum with the maximum probability, and it can also be determined according to the relationship between the weighted sum and a preset value. For example, in the above example, the preset value is 0.6, etc.

[0067] The pixel points with edge feature values of 0 in the edge uncertainty probability map are not the pixel points that may be misclassified. Classify the pixel points directly according to the classification corresponding to the maximum probability in the probability distribution matrix. After all pixel points are classified, the region of interest can be obtained. The region of interest includes but is not limited to the hepatobiliary region, blood vessel region, lesion region, etc.

[0068] S2. Determine the number of images to be inserted between two adjacent tomographic images, and determine the region of interest in the inserted images according to the regions of interest of the same type in two adjacent tomographic images and the edge uncertainty probability map;

[0069] The distance between medical tomographic images is relatively far. Generally speaking, it is greater than the distance between two pixel points on the tomographic image. If images are not inserted between two adjacent tomographic images, the constructed three-dimensional model will be distorted relatively large. Determine the number of images to be inserted according to the distance between adjacent tomographic images and the required resolution. If the distance between adjacent images is large, or a higher resolution is required, more images can be inserted.

[0070] In the inserted image, only the regions of interest in adjacent tomographic images are retained. For two adjacent regions of interest, linear interpolation, morphological interpolation, deep learning, or other methods are preferably used. In linear interpolation, the contribution weights of adjacent images to the inserted image are adjusted using the edge uncertainty probability map. For example, if image A and image B are adjacent tomographic images, in the regions with high edge uncertainty in image A, the contribution weight of image A to the inserted image is reduced, and the contribution weight of image B is increased. When using the deep learning method for interpolation, the edge uncertainty probability map is used as an additional input channel to guide the network to learn the interpolation rules for the uncertainty regions.

[0071] In another embodiment, as Figure 5 shown, determining the region of interest in the inserted image based on the regions of interest of the same type in two adjacent tomographic images and the edge uncertainty probability map specifically includes:

[0072] S201, obtaining the areas and centroids of the regions of interest of the same type in two adjacent tomographic images, and connecting the two centroids;

[0073] S202, downsampling the region of interest in the tomographic image with a larger area of the region of interest, upsampling the region of interest in the tomographic image with a smaller area of the region of interest, combining the results of upsampling and downsampling, and determining the pixel values of the region of interest in the inserted image according to the combined result;

[0074] S203, downsampling the edge uncertainty probability map of the tomographic image with a larger area of the region of interest, upsampling the edge uncertainty probability map of the tomographic image with a smaller area of the region of interest, combining the results of upsampling and downsampling, and determining the edge probability of the region of interest in the inserted image according to the combined result;

[0075] S204, obtaining the pixel points whose edge probabilities in the region of interest in the inserted image meet the conditions, and if the pixel points are located in the region of interest in the inserted image, then removing the pixel points from the region of interest;

[0076] S205, determining the centroid of the region of interest in the inserted image, and moving the region of interest in the inserted image based on the centroid and the connection line of the centroids.

[0077] Obtaining the areas and centroids of the regions of interest of the same type in two adjacent tomographic images, and connecting the two centroids for subsequent centroid movement operations. In one embodiment, during interpolation, only one type of region of interest is reconstructed each time, and the reconstruction results of all types are combined into the inserted image.

[0078] Downsample the edge uncertainty probability map of the tomographic image with a relatively large area of the region of interest, upsample the edge uncertainty probability map of the tomographic image with a relatively small area of the region of interest, and combine the results of the upsampling and downsampling, such as by averaging or weighted averaging. According to the combined result, determine the edge of the region of interest in the inserted image.

[0079] Similarly, downsample the region of interest of the tomographic image with a relatively large area of the region of interest, upsample the region of interest of the tomographic image with a relatively small area of the region of interest, and combine the results of the upsampling and downsampling, such as by averaging or weighted averaging. According to the combined result, determine the pixel value of the region of interest in the inserted image. The pixel points that meet the conditions are preferably greater than 0 and less than a certain value, such as 0.2 or 0.3. In another embodiment, the certain value is preferably the average value of the values greater than 0 in the combined result of the edge uncertainty probability map. If the conditions are met, the pixel points are removed from the region of interest.

[0080] The amplitudes of the upsampling and downsampling in the above process are determined according to the distances between the inserted image and the two adjacent tomographic images above and below. For example, if the area of the region of interest of the upper tomographic image is 10 and the area of the region of interest of the lower tomographic image is 20, and one needs to be inserted in the middle position, then the area is 15. Upsample the region of interest of the upper tomographic image so that the area of the region of interest after upsampling is closest to 15. Similarly, downsample the region of interest of the lower tomographic image so that the area of the region of interest after downsampling is closest to 15. The upsampling methods include bilinear interpolation, bicubic interpolation, deconvolution, etc., and the downsampling methods include max pooling, average pooling, etc.

[0081] According to the line connecting the centroids of the regions of interest of two adjacent tomographic images, move the region of interest of the inserted image to make it in a suitable position. Preferably, align the centroids of the regions of interest of the two adjacent tomographic images and the centroid of the region of interest of the inserted image.

[0082] S3. Use the region of interest obtained by segmentation and the region of interest of the inserted image for three-dimensional reconstruction, and determine the position of the blood flow blockage marker based on the relationship between the blood vessels and the patient region.

[0083] Use the region of interest obtained by segmentation and the region of interest of the inserted image to perform three-dimensional reconstruction in ways such as voxels and surfaces. Figure 6 It is a schematic diagram of three-dimensional reconstruction. Figure 7Schematic diagram of three-dimensional reconstruction after reducing the liver transparency. In the three-dimensional reconstruction model, analyze the relationship between blood vessels and lesion areas, such as measuring the distance, angle, overlap degree, etc. between the lesion area and blood vessels. According to the relationship between blood vessels and lesion areas, determine the blood flow blocking marker position. Preferably, obtain the blood vessel closest to the patient area in the three-dimensional model, and add a blood flow blocking position marker at a preset distance from the blood vessel to the patient area, as Figure 8 shown.

[0084] Embodiment 2 provides a selective hepatobiliary blood flow blocking calibration system based on three-dimensional reconstruction. The system includes the following modules:

[0085] The segmentation module is used to extract the feature map of the hepatobiliary tomographic image through the backbone network, input the feature map into the classification branch to obtain the probability distribution matrix of each pixel point, calculate the gradient of each point in the hepatobiliary tomographic image to obtain the edge intensity map, use the probability distribution matrix and the edge intensity map to obtain the edge uncertainty probability map, and perform image segmentation on the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest;

[0086] The tomographic interpolation module is used to determine the number of images to be inserted between two adjacent tomographic images, and determine the region of interest in the inserted images according to the regions of interest of the same type in two adjacent tomographic images and the edge uncertainty probability map;

[0087] The three-dimensional reconstruction and marking module is used to perform three-dimensional reconstruction using the segmented region of interest and the region of interest of the inserted images, and determine the blood flow blocking marker position using the relationship between blood vessels and the patient area.

[0088] Preferably, the obtaining of the edge uncertainty probability map using the probability distribution matrix and the edge intensity map is specifically as follows:

[0089] Obtain the classification probability of each pixel point from the probability distribution matrix, use the second largest probability as the background and the second largest probability of the pixel point where the largest probability and the second largest probability meet the preset conditions as the edge feature value, and the edge feature value of other pixel points is 0, and construct an edge feature map;

[0090] Multiply the edge feature map and the edge intensity map bit by bit to obtain the edge uncertainty probability map.

[0091] Preferably, the performing of image segmentation on the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest is specifically as follows:

[0092] For the pixel points in the edge feature map with non-zero edge feature values, calculate the weighted sum of the values in the edge uncertainty probability map and the second-largest probability in the probability distribution matrix. If the result of the weighted sum is greater than the maximum probability or a preset value, the class of the pixel point is the class corresponding to the second-largest probability; otherwise, the class of the pixel point is the class corresponding to the maximum probability.

[0093] For the pixel points in the edge feature map with edge feature values of 0, determine the class of the pixel points according to the maximum probability in the probability distribution matrix.

[0094] Determine the region of interest based on the classification of the pixel points in the hepatobiliary tomographic images.

[0095] Preferably, the method for determining the region of interest in the inserted image according to the regions of interest of the same class in two adjacent tomographic images and the edge uncertainty probability map is as follows:

[0096] Obtain the areas and centroids of the regions of interest of the same class in two adjacent tomographic images, and connect the two centroids.

[0097] Downsample the region of interest in the tomographic image with a larger area of the region of interest, upsample the region of interest in the tomographic image with a smaller area of the region of interest, merge the results of the upsampling and downsampling, and determine the pixel values of the region of interest in the inserted image according to the merged result.

[0098] Downsample the edge uncertainty probability map of the tomographic image with a larger area of the region of interest, upsample the edge uncertainty probability map of the tomographic image with a smaller area of the region of interest, merge the results of the upsampling and downsampling, and determine the edge probability of the region of interest in the inserted image according to the merged result.

[0099] Obtain the pixel points in the inserted image whose edge probabilities of the region of interest meet the conditions. If the pixel points are located in the region of interest in the inserted image, then remove the pixel points from the region of interest.

[0100] Determine the centroid of the region of interest in the inserted image, and move the region of interest in the inserted image based on the centroid and the connection line of the centroids.

[0101] Preferably, the method for determining the blood flow blockage marking position by using the relationship between blood vessels and the patient area is as follows:

[0102] Obtain the blood vessel in the three-dimensional model that is closest to the patient area, and add a blood flow blockage position mark at a preset distance from the patient area on the blood vessel.

[0103] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of adding a necessary general hardware platform, and of course, it can also be implemented by a combination of hardware and software. Based on such an understanding, the above technical solutions, in essence, or the parts that contribute to the prior art can be embodied in the form of a computer product. The present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk memory, CD-ROM, optical memory, etc.) that contain computer-usable program code.

[0104] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Other embodiments can also be adopted. Although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments or equivalently replace some of the technical features. These modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A selective hepatobiliary blood flow occlusion calibration method based on three-dimensional reconstruction, characterized in that: The method comprises the following steps: Extracting a feature map of the hepatobiliary tomography image through the backbone network, inputting the feature map into the classification branch to obtain a probability distribution matrix for each pixel, calculating the gradient of each point in the hepatobiliary tomography image to obtain an edge intensity map, using the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and performing image segmentation on the medical tomography image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; Determine the number of images that need to be inserted between two adjacent tomographic images, and determine the region of interest in the inserted image according to the same type of regions of interest of the two adjacent tomographic images and the edge uncertainty probability map; The region of interest obtained by segmentation and the region of interest of the inserted image are used for three-dimensional reconstruction, and the position of the blood flow blocking mark is determined by using the relationship between the blood vessel and the patient area; The edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain an edge uncertainty probability map; The image segmentation of the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest is specifically as follows: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

2. The method according to claim 1, characterized in that The determining of the region of interest in the inserted image according to the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically as follows: Obtain the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connect the two centroids; Down-sampling the region of interest of the tomographic image with a large area of ​​the region of interest, up-sampling the region of interest of the tomographic image with a small area of ​​the region of interest, merging the results of the up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; Down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the up-sampling and down-sampling results, and determining the edge probability of the region of interest inserted into the image according to the merged result; Obtaining a pixel point whose edge probability satisfies a condition in the region of interest in the inserted image, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; The centroid of the region of interest in the inserted image is determined, and the region of interest in the inserted image is moved based on the centroid and a line connecting the centroids.

3. The method according to claim 1, characterized in that The method of determining the blood flow blocking mark position by using the relationship between the blood vessel and the patient area is specifically as follows: The blood vessel closest to the patient area in the three-dimensional model is obtained, and a blood flow blocking position mark is added at a preset distance from the blood vessel to the patient area.

4. A selective hepatobiliary blood flow occlusion calibration system based on three-dimensional reconstruction, characterized in that: The system includes the following modules: A segmentation module is used to extract a feature map of the hepatobiliary tomography image through a backbone network, input the feature map into a classification branch to obtain a probability distribution matrix for each pixel, calculate the gradient of each point in the hepatobiliary tomography image to obtain an edge intensity map, use the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and perform image segmentation on the medical tomography image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; A tomographic interpolation module, used to determine the number of images that need to be inserted between two adjacent tomographic images, and to determine the region of interest in the inserted image according to the same region of interest of the two adjacent tomographic images and the edge uncertainty probability map; A three-dimensional reconstruction and marking module is used to perform three-dimensional reconstruction using the region of interest obtained by segmentation and the region of interest inserted into the image, and determine the blood flow blocking mark position using the relationship between the blood vessel and the patient area; The edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain an edge uncertainty probability map; The image segmentation of the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest is specifically as follows: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

5. The system according to claim 4, characterized in that The determining of the region of interest in the inserted image according to the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically as follows: Obtain the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connect the two centroids; Down-sampling the region of interest of the tomographic image with a large area of ​​the region of interest, up-sampling the region of interest of the tomographic image with a small area of ​​the region of interest, merging the results of the up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; Down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the up-sampling and down-sampling results, and determining the edge probability of the region of interest inserted into the image according to the merged result; Obtaining a pixel point whose edge probability satisfies a condition in the region of interest in the inserted image, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; The centroid of the region of interest in the inserted image is determined, and the region of interest in the inserted image is moved based on the centroid and a line connecting the centroids.

6. The system according to claim 4, characterized in that The method of determining the blood flow blocking mark position by using the relationship between the blood vessel and the patient area is specifically as follows: The blood vessel closest to the patient area in the three-dimensional model is obtained, and a blood flow blocking position mark is added at a preset distance from the blood vessel to the patient area.

Images