Selective hepatobiliary blood flow occlusion calibration method and system based on three-dimensional reconstruction

Abstract

The invention relates to a selective hepatobiliary blood flow occlusion calibration method and system based on three-dimensional reconstruction, and the method comprises the steps: extracting a feature map of a hepatobiliary cross-sectional image through a backbone network, inputting the feature map to obtain a classification branch, and obtaining a probability distribution matrix of each pixel point; calculating the gradient of each point in the liver and gall cross-sectional image to obtain an edge intensity graph, obtaining an edge uncertainty probability graph by using the probability distribution matrix and the edge intensity graph, and performing image segmentation on the medical cross-sectional image according to the edge uncertainty probability graph and the probability distribution matrix to obtain a region of interest; determining the number of images needing to be inserted between two adjacent cross-sectional images, and determining a region of interest in the inserted image according to the similar regions of interest of the two adjacent cross-sectional images and the edge uncertainty probability graph; and performing three-dimensional reconstruction by using the region of interest obtained by segmentation and the region of interest of the inserted image, and determining the position of the blood flow occlusion mark by using the relationship between the blood vessel and the patient region.

Description

Technical Field

[0001] The present invention relates to the field of three-dimensional reconstruction, and in particular to a selective hepatobiliary blood flow blocking calibration method and system based on three-dimensional reconstruction. Background Art

[0002] Hepatobiliary blood flow occlusion is an important measure to reduce bleeding in hepatobiliary surgery, which mainly includes total hepatobiliary blood flow occlusion and selective hepatobiliary blood flow occlusion. Compared with total hepatobiliary blood flow occlusion, selective hepatobiliary blood flow occlusion can protect normal hepatobiliary tissue to the greatest extent and reduce the possibility of postoperative hepatobiliary failure. Although selective hepatobiliary blood flow occlusion can protect the health of patients to the greatest extent, the traditional selective hepatobiliary blood flow occlusion method often relies on the doctor's experience and feel, and it is difficult to achieve accurate positioning and control, especially in complex hepatobiliary surgery. Three-dimensional reconstruction technology can construct an accurate three-dimensional model of liver blood vessels based on the patient's CT or MRI medical images. With the help of the three-dimensional reconstructed model, doctors can clearly observe the anatomical structure of liver blood vessels, including the distribution of the hepatic artery, portal vein and its branches, and doctors can simulate different blood flow occlusion schemes on the three-dimensional model to evaluate their effects on liver blood supply, so as to select the best occlusion position and degree. However, due to the limitations of CT and MRI as well as the complexity of the human body structure, in three-dimensional reconstruction, if the liver, gallbladder, blood vessels and diseased parts cannot be accurately segmented, the established three-dimensional model will not be accurate enough. In addition, images need to be inserted between CT, MRI and other slices during three-dimensional reconstruction. These will accumulate errors in the final three-dimensional model, resulting in loss of details, etc., which will affect the accuracy of blood flow blockage marking. Summary of the invention

[0003] In view of the problems of large error and missing details in the three-dimensional model when determining the position of the hepatobiliary blood flow blocking mark based on the three-dimensional model, the present invention proposes a selective hepatobiliary blood flow blocking calibration method based on three-dimensional reconstruction, which includes the following steps: Extracting a feature map of the hepatobiliary tomography image through the backbone network, inputting the feature map into a classification branch to obtain a probability distribution matrix of each pixel point, calculating the gradient of each point in the hepatobiliary tomography image to obtain an edge intensity map, using the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and performing image segmentation on the medical tomography image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; Determine the number of images that need to be inserted between two adjacent tomographic images, and determine the region of interest in the inserted image according to the same type of regions of interest of the two adjacent tomographic images and the edge uncertainty probability map; The region of interest obtained by segmentation and the region of interest of the inserted image are used for three-dimensional reconstruction, and the position of the blood flow blocking mark is determined by using the relationship between the blood vessel and the patient area.

[0004] Preferably, the edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain the edge uncertainty probability map.

[0005] Preferably, the image segmentation of the medical tomographic image to obtain the region of interest according to the edge uncertainty probability map and the probability distribution matrix is ​​specifically: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

[0006] Preferably, the determining of the region of interest in the inserted image according to the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically: Obtain the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connect the two centroids; Down-sampling the region of interest of the tomographic image with a large area of ​​the region of interest, up-sampling the region of interest of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; Down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the edge probability of the region of interest in the inserted image according to the merged result; Obtaining a pixel point whose edge probability satisfies a condition in the region of interest in the inserted image, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; The centroid of the region of interest in the inserted image is determined, and the region of interest in the inserted image is moved based on the centroid and a line connecting the centroids.

[0007] Preferably, the method of determining the blood flow blocking mark position by utilizing the relationship between the blood vessel and the diseased area is specifically as follows: The blood vessel closest to the patient area in the three-dimensional model is obtained, and a blood flow blocking position mark is added at a preset distance from the blood vessel to the patient area.

[0008] In addition, the present invention also provides a selective hepatobiliary blood flow blocking calibration system based on three-dimensional reconstruction, the system comprising the following modules: A segmentation module is used to extract a feature map of the hepatobiliary tomography image through a backbone network, input the feature map into a classification branch to obtain a probability distribution matrix for each pixel, calculate the gradient of each point in the hepatobiliary tomography image to obtain an edge intensity map, use the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and perform image segmentation on the medical tomography image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; A tomographic interpolation module, used to determine the number of images that need to be inserted between two adjacent tomographic images, and to determine the region of interest in the inserted image according to the same region of interest of the two adjacent tomographic images and the edge uncertainty probability map; The three-dimensional reconstruction and marking module is used to perform three-dimensional reconstruction using the region of interest obtained by segmentation and the region of interest inserted into the image, and to determine the blood flow blocking mark position using the relationship between the blood vessel and the diseased area.

[0009] Preferably, the edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain the edge uncertainty probability map.

[0010] Preferably, the image segmentation of the medical tomographic image to obtain the region of interest according to the edge uncertainty probability map and the probability distribution matrix is ​​specifically: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

[0011] Preferably, the determining of the region of interest in the inserted image according to the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically: Obtain the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connect the two centroids; Down-sampling the region of interest of the tomographic image with a large area of ​​the region of interest, up-sampling the region of interest of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; Down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the edge probability of the region of interest in the inserted image according to the merged result; Obtaining a pixel point whose edge probability satisfies a condition in the region of interest in the inserted image, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; The centroid of the region of interest in the inserted image is determined, and the region of interest in the inserted image is moved based on the centroid and a line connecting the centroids.

[0012] Preferably, the method of determining the blood flow blocking mark position by utilizing the relationship between the blood vessel and the diseased area is specifically as follows: The blood vessel closest to the patient area in the three-dimensional model is obtained, and a blood flow blocking position mark is added at a preset distance from the blood vessel to the patient area.

[0013] The present invention can segment the region of interest more accurately by combining the probability distribution matrix and the edge uncertainty probability map when performing tomographic image segmentation; in addition, the region of interest information of adjacent images and the edge uncertainty probability map are used to more accurately estimate the region of interest inserted into the image, thereby optimizing the continuity of the region of interest in three-dimensional space. In addition, the present invention can automatically determine the position of the blood flow blocking mark according to the relationship between the blood vessel and the patient area. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a flow chart of Embodiment 1; Figure 2 It is a schematic diagram of hepatobiliary cross-sectional images; Figure 3 Schematic diagram of gradient intensity in edge area; Figure 4 It is the edge intensity map of hepatobiliary tomography image; Figure 5 A flow chart for inserting an image between adjacent tomographic images; Figure 6 This is the effect of 3D reconstruction; Figure 7 This is the 3D reconstruction effect after lowering the liver perspective; Figure 8 Schematic diagram of the blocking mark position. DETAILED DESCRIPTION

[0015] In this article, relational terms such as first and second, etc. are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] Figure 1 A first embodiment of the present invention is shown, Figure 1 As shown, the method comprises the following steps: S1, extracting a feature map of a hepatobiliary tomographic image through a backbone network, inputting the feature map into a classification branch to obtain a probability distribution matrix of each pixel point, calculating the gradient of each point in the hepatobiliary tomographic image to obtain an edge intensity map, using the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and performing image segmentation on the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; Medical tomographic images can clearly show the shape, size, position and structure of internal organs and tissues of the human body, and have become an important technology in medical treatment. Hepatobiliary tomographic images are cross-sectional images of the liver and gallbladder obtained by CT or MRI. The hepatobiliary tomographic images include but are not limited to CT and MRI images, and their contents are not limited to the liver, gallbladder, etc., and may also include, for example, blood vessels, bones, etc. In one embodiment, the hepatobiliary tomographic image is a liver tomographic image or a gallbladder tomographic image, such as Figure 2 shown.

[0018] The feature map of the hepatobiliary tomography image is extracted by using a backbone network, and the backbone network includes but is not limited to ResNet, VGG, DenseNet, etc. After obtaining the feature map, the feature map is input into the classification branch to obtain a probability distribution matrix for each pixel point, wherein the classification branch is used to classify each pixel point, for example, liver or tumor, blood vessel, etc., and the probability distribution matrix is ​​H×W×C, H is the height of the hepatobiliary tomography image, W is the width of the hepatobiliary tomography image, and C is the number of categories, which is the number of categories to be classified. In a real-time hepatobiliary tomography image, the liver area, blood vessel area, background area, and tumor area are to be segmented, and the number of categories is 4, and the background area is neither liver, blood vessel, nor tumor. Of course, the number of categories is not limited to the above, and can also include, for example, bones, skin, etc. A pixel point in a hepatobiliary tomography image corresponds to a vector in the probability distribution matrix, and the vector length is C, for example [0.6, 0.2, 0.1, 0.1], which are the probabilities of the liver area, blood vessel area, background area, and tumor area, respectively. In an alternative embodiment, a model such as U-Net or Swin-Unet is used to obtain a probability distribution matrix of the hepatobiliary tomographic image.

[0019] Edges are important features of image segmentation. Segmentation based solely on the probability distribution of pixels may result in blurred edges or inaccurate segmentation. The boundaries of organs and lesion areas such as the liver, gallbladder, and tumors are usually manifested as sudden changes in pixel values ​​in the image, that is, areas with large gradients. Calculating the gradient helps improve the accuracy of segmentation. After obtaining the gradient using operators such as Sobel, Prewitt, and Canny, the gradient value or gradient intensity is converted into an edge intensity map. Preferably, the edge intensity map is further normalized after maximum filtering. The closer the edge part is to the hepatobiliary tomography image, the larger the corresponding value in the edge intensity map.

[0020] Edge areas are often difficult to segment because the pixel categories in these areas may be unclear. The probability distribution matrix provides the probability that each pixel belongs to a different category, reflecting the model's prediction confidence of the pixel category, while the edge intensity map provides the intensity of the pixel at the edge, reflecting the edge characteristics of the pixel position. Figure 3 shows the gradient strength graph obtained after calculating the gradient of the original matrix, from Figure 3 It can be seen that the pixels at the border of the black area have higher gradient strength or gradient value. In the probability distribution matrix, the edge area is likely to be misclassified, but the farther or closer the border is to the region of interest, the less likely it is to be misclassified. Figure 3In the above example, the pixel 1 and the pixel 2 may both correspond to the liver with the highest probability, but the probability of the pixel 1 being the background is lower than the probability of the pixel 2 being the background. For example, the probability distributions of the pixels 1 and 2 are [0.6, 0.3, 0.1] and [0.5, 0.4, 0.1]. According to the conventional method, the pixels 1 and 2 are both segmented as the liver. The present invention uses the gradient information to classify the pixels 1 and 2 into the appropriate categories. Figure 4 Shows Figure 2 The edge strength map is combined with the probability distribution matrix and the edge strength map to achieve a more comprehensive evaluation or calculation of the edge. In one embodiment, the value in the edge uncertainty probability map is determined by edge probability × (1-maximum classification probability). In another embodiment, after obtaining the edge uncertainty probability map, the value less than the preset value is set to 0. For example, the preset value is 0.1, and the values ​​of all points less than 0.1 in the edge uncertainty probability map are set to 0.

[0021] In yet another embodiment, the edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain the edge uncertainty probability map.

[0022] Extract the classification probability of each pixel from the probability distribution matrix, and find the maximum probability value and the second largest probability value of each pixel. For each pixel, if the category corresponding to the second largest probability is the background, and the difference between the maximum probability and the second largest probability meets the preset conditions, preferably the preset conditions are that the difference is less than a certain threshold value such as 0.1 or 0.2, then the second largest probability value of the pixel is used as the edge feature value; otherwise, the edge feature value of the pixel is set to 0. The edge feature values ​​of all pixels are combined to form an edge feature map; further, the edge feature map and the edge intensity map are bitwise multiplied, and the result is the edge uncertainty probability map. For example, the probability that pixel P belongs to the liver is 0.6, the probability that it belongs to the background is 0.3, and the probability that it belongs to the blood vessel is 0.1. Since the second largest probability is the background, assuming that the threshold is 0.2, the difference between the maximum probability and the second largest probability is 0.3 greater than 0.2, and the preset conditions are met, then the edge feature value of pixel P is 0.3; the edge feature values ​​of other pixels that do not meet the above conditions are set to 0, for example, the second largest probability of pixel Q does not correspond to the background, then the edge feature value of pixel Q is 0. The edge feature values ​​of all pixels constitute the edge feature map. Assuming further that the value of pixel P in the edge intensity map is 0.6, then in the edge uncertainty probability map, the value of pixel P is 0.18 and the value of pixel Q is 0. Wherein, the pixels P and Q are pixels in the hepatobiliary tomography image. Since the hepatobiliary tomography image and the edge feature map and edge intensity map have the same size and corresponding positions, the corresponding edge feature values ​​and edge probability values ​​can be determined according to the positions of the pixels P and Q in the hepatobiliary tomography image.

[0023] The edge uncertainty probability map indicates the uncertainty of the pixel being located in the edge area, and the edge uncertainty probability map is used to correct or optimize the probability distribution matrix, thereby obtaining a more accurate segmentation result. In another embodiment, the image segmentation of the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest is specifically: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

[0024] The pixels whose edge feature values ​​are not 0 in the edge uncertainty probability map are pixels that may be misclassified. The weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated, for example Figure 3In the above example, the probability distribution of pixels 1 and 2 is [0.6, 0.3, 0.1], [0.5, 0.4, 0.1]. Assuming that the gradient strength is 0.8, the values ​​in the edge uncertainty probability maps of the two are 0.24 and 0.32 respectively, and the weighted sums are 0.54 and 0.72 respectively. The weighted sum of pixel 1 0.54 is less than the maximum probability 0.6, and the category of pixel 1 is the category corresponding to the maximum probability 0.6. The weighted sum of pixel 2 0.72 is greater than the maximum probability 0.5, and the category of pixel 2 is the category corresponding to the second largest probability 0.4, that is, the background. In the above example, the weights are all 1, and other weights can also be used. The determination of these weights is related to, for example, the specific gradient calculation method. It is not limited to comparing the weighted sum with the maximum probability, and can also be determined based on the relationship between the weighted sum and the preset value. For example, in the above example, the preset value is 0.6.

[0025] Pixels with edge feature values ​​of 0 in the edge uncertainty probability map are not pixels that may be misclassified. The pixels can be directly classified according to the classification corresponding to the maximum probability in the probability distribution matrix. Finally, after all pixels are classified, the region of interest can be obtained, which includes but is not limited to the liver and gallbladder region, blood vessel region, lesion region, etc.

[0026] S2, determining the number of images that need to be inserted between two adjacent tomographic images, and determining the region of interest in the inserted image according to the same type of regions of interest of the two adjacent tomographic images and the edge uncertainty probability map; The distance between medical tomograms is relatively far, generally greater than the distance between two pixels on the tomogram. If images are not inserted between two adjacent tomograms, the constructed 3D model will be greatly distorted. The number of images to be inserted is determined based on the distance between adjacent tomograms and the required resolution. If the distance between adjacent images is large or a higher resolution is required, more images can be inserted.

[0027] In the inserted image, only the regions of interest in the adjacent tomographic images are retained. For two adjacent regions of interest, linear interpolation, morphological interpolation or deep learning are preferably used. In linear interpolation, the edge uncertainty probability map is used to adjust the contribution weight of the adjacent image to the inserted image. For example, image A and image B are adjacent tomographic images. In the area where the edge uncertainty of image A is high, the contribution weight of image A to the inserted image is reduced, and the contribution weight of image B is increased. When using the deep learning method for interpolation, the edge uncertainty probability map is used as an additional input channel to guide the network to learn the interpolation law of the uncertainty area.

[0028] In another embodiment, if Figure 5 As shown, the determination of the region of interest in the inserted image based on the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically as follows: S201, obtaining the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connecting the two centroids; S202, down-sampling the region of interest of the tomographic image with a large area, up-sampling the region of interest of the tomographic image with a small area, merging the results of up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; S203, down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the edge probability of the region of interest to be inserted into the image according to the merged result; S204, obtaining a pixel point whose edge probability of the region of interest in the inserted image meets a condition, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; S205, determining the centroid of the region of interest in the inserted image, and moving the region of interest in the inserted image based on the centroid and the centroid line.

[0029] The area and centroid of the same type of regions of interest of two adjacent tomographic images are obtained, and the two centroids are connected for subsequent centroid movement operations. In one embodiment, when interpolating, only one type of region of interest is reconstructed each time, and all types of reconstruction results are merged into an interpolated image.

[0030] Down-sample the edge uncertainty probability map of the tomographic image with a larger area of ​​interest, up-sample the edge uncertainty probability map of the tomographic image with a smaller area of ​​interest, merge the up-sampling and down-sampling results, such as averaging or weighted averaging, and determine the edge of the area of ​​interest in the inserted image based on the merged result.

[0031] Similarly, the region of interest of the tomographic image with a larger region of interest is downsampled, and the region of interest of the tomographic image with a smaller region of interest is upsampled, and the results of the upsampling and downsampling are merged, for example, averaged or weighted averaged, and the pixel value of the region of interest in the inserted image is determined according to the merged result. The pixel points that meet the conditions are preferably greater than 0 and less than a certain value, such as 0.2 or 0.3. In another embodiment, the certain value is preferably the average value of the values ​​greater than 0 in the merged result of the edge uncertainty probability map. If the conditions are met, the pixel points are removed from the region of interest.

[0032] In the above process, the amplitude of upsampling and downsampling is determined according to the distance between the inserted image and the upper and lower adjacent tomographic images. For example, the area size of the region of interest of the upper tomographic image is 10, and the area size of the region of interest of the lower tomographic image is 20. If one needs to be inserted in the middle, the area is 15. The area of ​​the region of interest of the upper tomographic image is upsampled so that the area of ​​the region of interest after upsampling is closest to 15. Similarly, the area of ​​the region of interest of the lower tomographic image is downsampled so that the area of ​​the region of interest after downsampling is closest to 15. Upsampling methods include bilinear interpolation, bicubic interpolation, deconvolution, etc., and downsampling methods include maximum pooling, average pooling, etc.

[0033] According to the centroid line of the regions of interest of two adjacent tomographic images, the region of interest of the inserted image is moved to a suitable position. Preferably, the centroids of the regions of interest of the two adjacent tomographic images and the centroid of the region of interest of the inserted image are aligned.

[0034] S3, performing three-dimensional reconstruction using the region of interest obtained by segmentation and the region of interest of the inserted image, and determining the position of the blood flow blocking mark using the relationship between the blood vessel and the diseased area.

[0035] Using the region of interest obtained by segmentation and the region of interest inserted into the image, three-dimensional reconstruction is performed using voxels, surfaces, etc. Figure 6 Schematic diagram of 3D reconstruction. Figure 7 The figure is a schematic diagram of the three-dimensional reconstruction after the liver transparency is reduced. In the three-dimensional reconstruction model, the relationship between the blood vessels and the lesion area is analyzed, for example, the distance, angle, overlap degree, etc. between the lesion area and the blood vessels are measured, and the blood flow blocking mark position is determined according to the relationship between the blood vessels and the lesion area. Preferably, the blood vessel closest to the lesion area in the three-dimensional model is obtained, and the blood flow blocking position mark is added at the preset distance from the blood vessel to the lesion area, such as Figure 8 shown.

[0036] Embodiment 2 provides a selective hepatobiliary blood flow occlusion calibration system based on three-dimensional reconstruction, the system comprising the following modules: A segmentation module is used to extract a feature map of the hepatobiliary tomography image through a backbone network, input the feature map into a classification branch to obtain a probability distribution matrix for each pixel, calculate the gradient of each point in the hepatobiliary tomography image to obtain an edge intensity map, use the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and perform image segmentation on the medical tomography image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; A tomographic interpolation module, used to determine the number of images that need to be inserted between two adjacent tomographic images, and to determine the region of interest in the inserted image according to the same region of interest of the two adjacent tomographic images and the edge uncertainty probability map; The three-dimensional reconstruction and marking module is used to perform three-dimensional reconstruction using the region of interest obtained by segmentation and the region of interest inserted into the image, and to determine the blood flow blocking mark position using the relationship between the blood vessel and the diseased area.

[0037] Preferably, the edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain the edge uncertainty probability map.

[0038] Preferably, the image segmentation of the medical tomographic image to obtain the region of interest according to the edge uncertainty probability map and the probability distribution matrix is ​​specifically: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

[0039] Preferably, the determining of the region of interest in the inserted image according to the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically: Obtain the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connect the two centroids; Down-sampling the region of interest of the tomographic image with a large area of ​​the region of interest, up-sampling the region of interest of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; Down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the edge probability of the region of interest in the inserted image according to the merged result; Obtaining a pixel point whose edge probability satisfies a condition in the region of interest in the inserted image, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; The centroid of the region of interest in the inserted image is determined, and the region of interest in the inserted image is moved based on the centroid and a line connecting the centroids.

[0040] Preferably, the method of determining the blood flow blocking mark position by utilizing the relationship between the blood vessel and the diseased area is specifically as follows: The blood vessel closest to the patient area in the three-dimensional model is obtained, and a blood flow blocking position mark is added at a preset distance from the blood vessel to the patient area.

[0041] Through the description of the above implementation methods, technicians in this field can clearly understand that each implementation method can be implemented by adding a necessary general hardware platform, and of course can also be implemented by combining hardware and software. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a computer product, and the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0042] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it, and other embodiments may also be used. Although the present invention has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical solutions recorded in the aforementioned embodiments, or replace some of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A selective hepatobiliary blood flow occlusion calibration method based on three-dimensional reconstruction, characterized in that: The method comprises the following steps: Extracting a feature map of the hepatobiliary tomography image through the backbone network, inputting the feature map into a classification branch to obtain a probability distribution matrix of each pixel point, calculating the gradient of each point in the hepatobiliary tomography image to obtain an edge intensity map, using the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and performing image segmentation on the medical tomography image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; Determine the number of images that need to be inserted between two adjacent tomographic images, and determine the region of interest in the inserted image according to the same type of regions of interest of the two adjacent tomographic images and the edge uncertainty probability map; The region of interest obtained by segmentation and the region of interest of the inserted image are used for three-dimensional reconstruction, and the position of the blood flow blocking mark is determined by using the relationship between the blood vessel and the patient area.

2. The method according to claim 1, characterized in that The edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain the edge uncertainty probability map.

3. The method according to claim 2, characterized in that The image segmentation of the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest is specifically as follows: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

4. The method according to claim 1, characterized in that The determining of the region of interest in the inserted image according to the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically as follows: Obtain the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connect the two centroids; Down-sampling the region of interest of the tomographic image with a large area of ​​the region of interest, up-sampling the region of interest of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; Down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the edge probability of the region of interest in the inserted image according to the merged result; Obtaining a pixel point whose edge probability satisfies a condition in the region of interest in the inserted image, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; The centroid of the region of interest in the inserted image is determined, and the region of interest in the inserted image is moved based on the centroid and a line connecting the centroids.

5. The method according to claim 1, characterized in that The method of determining the blood flow blocking mark position by using the relationship between the blood vessel and the patient area is specifically as follows: The blood vessel closest to the patient area in the three-dimensional model is obtained, and a blood flow blocking position mark is added at a preset distance from the blood vessel to the patient area.

6. A selective hepatobiliary blood flow occlusion calibration system based on three-dimensional reconstruction, characterized in that: The system includes the following modules: A segmentation module is used to extract a feature map of the hepatobiliary tomography image through a backbone network, input the feature map into a classification branch to obtain a probability distribution matrix for each pixel, calculate the gradient of each point in the hepatobiliary tomography image to obtain an edge intensity map, use the probability distribution matrix and the edge intensity map to obtain an edge uncertainty probability map, and perform image segmentation on the medical tomography image according to the edge uncertainty probability map and the probability distribution matrix to obtain a region of interest; A tomographic interpolation module, used to determine the number of images that need to be inserted between two adjacent tomographic images, and to determine the region of interest in the inserted image according to the same region of interest of the two adjacent tomographic images and the edge uncertainty probability map; The three-dimensional reconstruction and marking module is used to perform three-dimensional reconstruction using the region of interest obtained by segmentation and the region of interest inserted into the image, and to determine the blood flow blocking mark position using the relationship between the blood vessel and the diseased area.

7. The system according to claim 6, characterized in that The edge uncertainty probability map is obtained by using the probability distribution matrix and the edge strength map, specifically: Obtain the classification probability of each pixel from the probability distribution matrix, take the second highest probability of the pixel with the second highest probability as the background and the maximum probability and the second highest probability satisfying the preset conditions as the edge feature value, and the edge feature value of other pixels is 0, and construct an edge feature map; The edge feature map and the edge strength map are bitwise multiplied to obtain the edge uncertainty probability map.

8. The system according to claim 7, characterized in that The image segmentation of the medical tomographic image according to the edge uncertainty probability map and the probability distribution matrix to obtain the region of interest is specifically as follows: For pixels whose edge feature values ​​in the edge feature map are not 0, the weighted sum of the value in the edge uncertainty probability map and the second largest probability in the probability distribution matrix is ​​calculated. If the result of the weighted sum is greater than the maximum probability or the preset value, the category of the pixel is the category corresponding to the second largest probability, otherwise the category of the pixel is the category corresponding to the maximum probability; For the pixel point whose edge feature value is 0 in the edge feature map, the category of the pixel point is determined according to the maximum probability in the probability distribution matrix; The region of interest is determined based on the classification of pixels in the hepatobiliary tomography image.

9. The system according to claim 6, characterized in that The determining of the region of interest in the inserted image according to the same type of regions of interest of two adjacent tomographic images and the edge uncertainty probability map is specifically as follows: Obtain the area and centroid of the same type of regions of interest of two adjacent tomographic images, and connect the two centroids; Down-sampling the region of interest of the tomographic image with a large area of ​​the region of interest, up-sampling the region of interest of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the pixel value of the region of interest in the inserted image according to the merging result; Down-sampling the edge uncertainty probability map of the tomographic image with a large area of ​​the region of interest, up-sampling the edge uncertainty probability map of the tomographic image with a small area of ​​the region of interest, merging the results of up-sampling and down-sampling, and determining the edge probability of the region of interest in the inserted image according to the merged result; Obtaining a pixel point whose edge probability satisfies a condition in the region of interest in the inserted image, and if the pixel point is located in the region of interest in the inserted image, removing the pixel point from the region of interest; The centroid of the region of interest in the inserted image is determined, and the region of interest in the inserted image is moved based on the centroid and a line connecting the centroids.

10. The system according to claim 6, characterized in that The method of determining the blood flow blocking mark position by using the relationship between the blood vessel and the patient area is specifically as follows: The blood vessel closest to the patient area in the three-dimensional model is obtained, and a blood flow blocking position mark is added at a preset distance from the blood vessel to the patient area.

Images