Preoperative planning method and device for liver surgical operation based on artificial intelligence

Abstract

The invention relates to the technical field of medical image analysis, and discloses a preoperative planning method and device for liver surgery based on artificial intelligence, electronic equipment, a computer readable storage medium and a program product, which are used for solving the problem that a preoperative planning method in the prior art cannot cooperatively process various information in liver surgery planning. And the technical problem that the preoperative planning cannot be accurately and efficiently carried out is solved. Comprising the following steps: acquiring a CT image containing a liver part, and respectively inputting the CT image into a three-dimensional organ segmentation model and a three-dimensional blood vessel segmentation model for segmentation to obtain a liver mask and a blood vessel mask; splicing the liver mask and the blood vessel mask, and inputting the spliced liver mask and blood vessel mask into a liver segment and blood vessel naming model for naming to obtain a planar liver segment mask, a liver vein naming mask and a portal vein naming mask; calculating blood vessel space information and blood vessel size information related to the liver based on the hepatic vein naming mask and the portal vein naming mask, and constructing a blood vessel tree; and generating a drainage basin liver segment model according to the plane liver segment mask and the constructed blood vessel tree, and performing preoperative planning.

Description

Technical Field

[0001] The present invention relates to the technical field of medical image analysis, and in particular to a preoperative planning method, device, electronic device, computer storage medium, and computer program product for liver surgery based on artificial intelligence. Background Art

[0002] Preoperative evaluation and planning of liver surgery are important links to ensure the success of the surgery. However, due to the complex anatomical structure of the liver, there have been many challenges in preoperative evaluation and planning of liver surgery for a long time. If the traditional method relies on manual annotation and manual analysis, it not only takes a long time, but also has limited accuracy in the division of complex liver segments and the naming of blood vessel branches. Especially when dealing with complex cases, traditional techniques rely on the experience of doctors and it is difficult to provide a scientific basis for postoperative liver function protection. Therefore, with the development of medical imaging technology, methods for segmenting and identifying liver segments and blood vessels through image recognition technology and performing preoperative planning have gradually emerged, which can achieve higher processing efficiency and reduce the time of manual operation.

[0003] However, in the prior art, the preoperative planning methods used are usually limited to single tasks. Moreover, due to the highly individualized characteristics of the liver morphology and blood vessel distribution of different patients, the prior art lacks a systematic analysis of the spatial relationship of lesions and the influence of blood flow, fails to fully realize the synergistic effect between different modules, still lacks the ability to handle complex cases, and cannot meet the clinical requirements for efficient, accurate, and personalized surgical planning. Based on this, there is an urgent need for a liver surgery planning method that can synergistically process various information in surgical planning to accurately and efficiently meet the needs of preoperative planning for liver surgery. Summary of the Invention

[0004] The main object of the present invention is to solve the technical problem that the preoperative planning method in the prior art cannot synergistically process various information in liver surgery planning, resulting in the inability to accurately and efficiently perform preoperative planning for liver surgery.

[0005] The first aspect of the present invention provides a preoperative planning method for liver surgery based on artificial intelligence, including:

[0006] Obtain a CT image containing the liver part, and input the preprocessed CT image into a three-dimensional organ segmentation model and a three-dimensional blood vessel segmentation model respectively for segmentation to obtain a liver mask and a blood vessel mask;

[0007] After splicing the liver mask and the blood vessel mask, input them into a liver segment and blood vessel naming model for naming to obtain a planar liver segment mask, a hepatic vein naming mask, and a hepatic portal vein naming mask respectively;

[0008] Calculate the blood vessel spatial information and blood vessel size information related to the liver based on the hepatic vein naming mask and the portal vein naming mask, and construct a blood vessel tree;

[0009] Generate a watershed liver segment model according to the planar liver segment mask and the constructed blood vessel tree;

[0010] Perform preoperative planning based on the watershed liver segment model.

[0011] Optionally, in the first implementation manner of the first aspect of the present invention, the performing preoperative planning based on the watershed liver segment model includes:

[0012] Obtain the information of the liver lesion, and determine the lesion mask based on the information of the liver lesion;

[0013] Determine the spatial position of the liver lesion in the watershed liver segment model and the blood vessel tree branches supplying blood to the liver lesion according to the lesion mask;

[0014] Locate the clamping points of the blood vessel tree based on the spatial position information of the blood vessel tree branches;

[0015] According to the clamping points, combine the spatial structure of the blood vessel tree and the watershed liver segment model to determine the downstream area affected by the clamping points.

[0016] Optionally, in the second implementation manner of the first aspect of the present invention, calculating the relevant position of the liver lesion in the blood vessel tree based on the spatial information of the lesion mask and determining the blood vessel tree branches supplying blood to the liver lesion includes:

[0017] Calculate the Euclidean distance from the liver lesion to the spatial points included in the blood vessel tree based on the spatial information of the lesion mask;

[0018] Select the blood vessel tree trunk where the point with the closest Euclidean distance is located as the blood vessel tree branch supplying blood to the liver lesion.

[0019] Optionally, in the third implementation manner of the first aspect of the present invention, the blood vessel tree is a directed tree;

[0020] The determining the downstream watershed affected by the clamping points includes:

[0021] Based on the directed tree structure of the blood vessel tree, obtain all the subtree branches at the position of the clamping point, and determine the downstream watershed affected by the clamping point according to the positions of the subtree branches.

[0022] Optionally, in the fourth implementation manner of the first aspect of the present invention, the performing preoperative planning based on the watershed liver segment model further includes:

[0023] Add new clamping points to the blood vessel tree;

[0024] Combined with the structure of the vascular tree and the liver segment model of the drainage basin, determine the downstream drainage basin affected by the newly added clamping point.

[0025] Optionally, in the fifth implementation manner of the first aspect of the present invention, the liver segment vascular naming model includes three segmentation prediction heads, which are respectively used for planar liver segment recognition, hepatic vein vascular naming, and portal vein vascular naming;

[0026] The step of inputting the liver mask and the vascular mask into the liver segment vascular naming model for naming to obtain a planar liver segment mask, a hepatic vein naming mask, and a portal vein naming mask respectively includes:

[0027] Obtain the spatial vectors of the liver mask and the vascular mask, and splice the spatial vectors of the liver mask and the vascular mask in the cross-sectional direction of the CT image;

[0028] Input the spliced liver mask and vascular mask into the liver segment vascular naming model, and call different segmentation prediction heads for planar liver segment recognition, hepatic vein vascular naming, and portal vein vascular naming to obtain a planar liver segment mask, a hepatic vein naming mask, and a portal vein naming mask.

[0029] The second aspect of the present invention provides an artificial intelligence-based preoperative planning device for liver surgery, including:

[0030] A segmentation module, configured to obtain a CT image including a liver part, and input the preprocessed CT image into a three-dimensional organ segmentation model and a three-dimensional vascular segmentation model for segmentation respectively to obtain a liver mask and a vascular mask;

[0031] A naming module, configured to input the liver mask and the vascular mask into a liver segment vascular naming model for naming after splicing, and obtain a planar liver segment mask, a hepatic vein naming mask, and a portal vein naming mask respectively;

[0032] A vascular tree construction module, configured to calculate liver-related vascular spatial information and vascular size information based on the hepatic vein naming mask and the portal vein naming mask, and construct a vascular tree;

[0033] A model construction module, configured to generate a liver segment model of the drainage basin according to the planar liver segment mask and the constructed vascular tree;

[0034] A preoperative planning module, configured to perform preoperative planning based on the liver segment model of the drainage basin type.

[0035] In the third aspect of the present invention, a preoperative planning device for liver surgery based on artificial intelligence is provided, including: a memory and at least one processor, wherein instructions are stored in the memory; the at least one processor calls the instructions in the memory to enable the preoperative planning device for liver surgery based on artificial intelligence to execute the steps of the above-mentioned preoperative planning method for liver surgery based on artificial intelligence.

[0036] In the fourth aspect of the present invention, a computer-readable storage medium is provided, in which instructions are stored. When it runs on a computer, it enables the computer to execute the steps of the above-mentioned preoperative planning method for liver surgery based on artificial intelligence.

[0037] In the fifth aspect of the present invention, a computer program product is provided, including a computer program / instructions, characterized in that when the computer program / instructions are executed by a processor, the steps of the preoperative planning method for liver surgery based on artificial intelligence as described above are implemented.

[0038] In the technical solution provided by the present invention, a CT image containing the liver part is obtained, and the preprocessed CT image is respectively input into a three-dimensional organ segmentation model and a three-dimensional vascular segmentation model for segmentation to obtain a liver mask and a vascular mask; after splicing the liver mask and the vascular mask, they are input into a hepatic segment vascular naming model for naming to obtain a planar hepatic segment mask, a hepatic vein naming mask, and a hepatic portal vein naming mask respectively; based on the hepatic vein naming mask and the hepatic portal vein naming mask, the vascular spatial information and vascular dimension information related to the liver are calculated to construct a vascular tree; a watershed hepatic segment model is generated according to the planar hepatic segment mask and the constructed vascular tree; preoperative planning is performed based on the watershed hepatic segment model. This method can collaboratively process various information in liver surgery planning, comprehensively consider various information, and accurately and efficiently perform preoperative planning for liver surgery, providing a reference for surgical simulation and clinical surgical operations. A device, an electronic device, a computer-readable storage medium, and a computer program product provided by the present invention also solve the corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings:

[0040] Figure 1 It is a schematic flowchart of the first embodiment of the preoperative planning method for liver surgery based on artificial intelligence in the embodiments of the present invention;

[0041] Figure 2 It is a schematic processing architecture diagram of the hepatic segment and vascular naming part in the preoperative planning method for liver surgery based on artificial intelligence in the embodiments of the present invention;

[0042] Figure 3 It is a schematic diagram of the processing architecture for calculating the partial hepatic segments of the watershed in the preoperative planning method for liver surgery based on artificial intelligence in an embodiment of the present invention;

[0043] Figure 4 It is a schematic flowchart of the second embodiment of the preoperative planning method for liver surgery based on artificial intelligence in an embodiment of the present invention;

[0044] Figure 5 It is a schematic diagram of the processing architecture for determining the influencing part of the clamping point in the preoperative planning method for liver surgery based on artificial intelligence in an embodiment of the present invention;

[0045] Figure 6 It is a schematic diagram of an embodiment of the preoperative planning device for liver surgery based on artificial intelligence in an embodiment of the present invention;

[0046] Figure 7 It is a schematic diagram of an embodiment of the preoperative planning equipment for liver surgery based on artificial intelligence in an embodiment of the present invention;

[0047] Figure 8 It is a schematic diagram of the principle of a computer-readable medium in an embodiment of the present invention. Detailed implementation manners

[0048] Now, the exemplary embodiments of the present invention will be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. On the contrary, providing these exemplary embodiments enables the present invention to be more comprehensive and complete, and more conveniently conveys the inventive concept to those skilled in the art. Identical reference numerals in the figures denote identical or similar elements, components, or parts, and thus their repeated description will be omitted.

[0049] On the premise of conforming to the technical concept of the present invention, the features, structures, characteristics, or other details described in a specific embodiment do not exclude being combined in a suitable manner in one or more other embodiments.

[0050] In the description of specific embodiments, the features, structures, characteristics, or other details described in the present invention are for enabling those skilled in the art to fully understand the embodiments. However, it does not exclude that those skilled in the art can practice the technical solutions of the present invention without one or more of the specific features, structures, characteristics, or other details.

[0051] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps can be decomposed, and some operations / steps can be combined or partially combined, so the actual execution order may change according to actual conditions.

[0052] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities may be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0053] The term "and / or" or "and / or" includes all combinations of any one or more of the associated listed items.

[0054] See also Figures 1 - 3 The first embodiment of the preoperative planning method for liver surgery based on artificial intelligence in the embodiments of the present invention includes:

[0055] S101, acquiring a CT image including a liver part, and inputting the preprocessed CT image into a three-dimensional organ segmentation model and a three-dimensional blood vessel segmentation model for segmentation, respectively, to obtain a liver mask and a blood vessel mask;

[0056] It is understandable that the execution subject of the present invention may be an artificial intelligence-based liver surgery preoperative planning device, or a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking the server as the execution subject as an example.

[0057] See also Figure 2 After receiving the preoperative planning request, the server first obtains the patient information that needs preoperative planning and obtains the CT image containing the liver part; then the obtained CT image is preprocessed by normalization and segmentation. Through preprocessing, the image can be adjusted to be more suitable for the input of the image segmentation model and the subsequent liver segment vascular naming model, thereby improving the accuracy of segmentation recognition and naming.

[0058] The image segmentation model described in this embodiment is a three-dimensional organ segmentation model and a three-dimensional blood vessel segmentation model. The three-dimensional organ segmentation model is specifically used to identify the position of the target organ contained in the input CT image according to the information contained in the input CT image, and determine the three-dimensional organ mask based on the recognition result; the three-dimensional blood vessel segmentation model is specifically used to identify the type and position of the blood vessels contained in the input CT image according to the information contained in the input CT image, and determine the three-dimensional blood vessel mask.

[0059] In this embodiment, the target organ is the liver. After the preprocessed CT images are respectively input into the three-dimensional organ segmentation model and the three-dimensional vascular segmentation model, a liver mask and a vascular mask are obtained. Among them, the vascular mask includes masks of various blood vessels, such as the hepatic vein vessels, the portal vein vessels, the inferior vena cava vessels, and the abdominal artery vessels.

[0060] S102. After splicing the liver mask and the vascular mask, input them into the hepatic segment vascular naming model for naming, and respectively obtain a planar hepatic segment mask, a hepatic vein naming mask, and a portal vein naming mask.

[0061] Please continue to refer to Figure 2 , after obtaining the liver mask and the vascular mask, splice them in the direction of the cross-section of the original CT image, and input the spliced mask into the hepatic segment vascular naming model for naming recognition. Among them, the hepatic segment vascular naming model contains three segmentation prediction heads, which are respectively used for planar hepatic segment recognition, hepatic vein vessel naming recognition, and portal vein vessel naming recognition, so as to obtain a planar hepatic segment mask, a hepatic vein naming mask, and a portal vein naming mask.

[0062] S103. Calculate the vascular spatial information and vascular size information related to the liver based on the hepatic vein naming mask and the portal vein naming mask, and construct a vascular tree.

[0063] Please refer to Figure 3 , first, based on the hepatic vein naming mask and the portal vein naming mask, determine the vascular radius information and vascular spatial information of the hepatic vein vessels and the portal vein vessels, and locate their entrances according to the vascular radius information and vascular spatial information, and construct a vascular tree.

[0064] S104. Generate a watershed hepatic segment model according to the planar hepatic segment mask and the constructed vascular tree.

[0065] S105. Perform preoperative planning based on the watershed hepatic segment model.

[0066] Combined with the constructed vascular tree and the spatial distance and other information included in the planar hepatic segment mask, construct a watershed hepatic segment model, and perform preoperative planning for surgery based on the watershed hepatic segment model, such as hepatectomy or trisegmentectomy.

[0067] The method in the embodiment of the present invention can integrate artificial intelligence and medical image analysis technology to realize intelligent surgical planning that automatically completes hepatic segment and vascular naming recognition and lesion spatial relationship analysis.

[0068] Please refer to Figures 2 - 5 , the second embodiment of the preoperative planning method for liver surgery based on artificial intelligence in the embodiment of the present invention includes:

[0069] S201. Obtain the CT image containing the liver region, and input the preprocessed CT image into a three-dimensional organ segmentation model and a three-dimensional vascular segmentation model respectively for segmentation to obtain a liver mask and a vascular mask;

[0070] First, obtain the patient information that needs preoperative planning and the CT image containing the liver region. Among them, the CT image containing the liver region can be the portal venous phase image of thin-slice abdominal CT. Perform preprocessing such as normalization and slicing on the obtained CT image, and then use the preprocessed CT image to perform specific recognition and segmentation, so as to adjust the image to be more suitable for the input of the image segmentation model and the subsequent liver segment vascular naming model, thereby improving the accuracy of segmentation recognition and naming recognition.

[0071] In this embodiment, the image segmentation model includes two models, namely a three-dimensional organ segmentation model and a three-dimensional vascular segmentation model. Please refer to Figure 2 , after obtaining the preprocessed CT image, it will be input into the three-dimensional organ segmentation model and the three-dimensional vascular segmentation model respectively for organ and vascular recognition and segmentation to obtain a liver mask and a vascular mask. Among them, the liver mask can mark the voxels belonging to the liver region range, and the vascular mask can mark the voxels belonging to the blood vessels.

[0072] In this embodiment, the three-dimensional organ segmentation model and the three-dimensional vascular segmentation model can be pre-trained models constructed based on deep learning algorithms. Before performing specific recognition and segmentation tasks, an initial segmentation model is pre-constructed, and training data with organ and vascular position annotations is obtained for model training. Based on the training results, the model parameters are adjusted to obtain the trained three-dimensional organ segmentation model and three-dimensional vascular segmentation model.

[0073] Among them, in order for the three-dimensional vascular segmentation model to detect the third or even fourth-level branches of blood vessels, when constructing the three-dimensional vascular segmentation model, in addition to traditional segmentation tasks, a centerline regression task is specifically designed, and at the same time, the loss weight for small blood vessel branches is increased to improve the integrity of blood vessel detection. Specifically, when the three-dimensional vascular segmentation model outputs the mask of vascular segmentation, it also includes performing the regression prediction of the vascular centerline. Among them, the result of the regression prediction is the centerline regression value, and its data range is (0, 1). Based on the obtained centerline regression value, the centerline regression value is screened based on a pre-set centerline threshold to obtain an effective centerline regression prediction result, and the vascular segmentation result is supplemented based on the effective centerline regression prediction result to improve the accuracy and integrity of blood vessel detection.

[0074] In a specific implementation manner, when performing vascular annotation and segmentation based on a three-dimensional vascular segmentation model, it is necessary to distinguish different blood vessels. During annotation, a multi-value mask method is used for distinction. The obtained vascular segmentation results include a hepatic vein vascular mask, a portal vein vascular mask, an inferior vena cava vascular mask, and an abdominal artery vascular mask, where different types of blood vessels are represented by different labels.

[0075] S202. Based on the vector splicing method, splice the liver mask and the vascular mask in the cross-sectional direction to obtain an overall mask;

[0076] After obtaining the liver mask and the vascular mask in the foregoing steps, in the cross-sectional direction of the foregoing CT image, based on the vector splicing method, splice the liver mask and the vascular mask to obtain the mask of the overall target area. This is because the spatial correlation between liver segments and blood vessels is relatively important, and splicing them together can more accurately divide liver segments in subsequent steps.

[0077] Among them, when dividing the liver segments, it is specifically divided based on the basis of Couinaud liver segment division, specifically including dividing into 8 functional liver segments.

[0078] S203. Input the overall mask into the liver segment vascular naming model for naming to obtain a planar liver segment mask, a hepatic vein naming mask, and a portal vein naming mask respectively;

[0079] Please continue to refer to Figure 2 , input the spliced overall mask into the liver segment vascular naming model for naming recognition, recognize multiple liver segments in the overall mask to obtain a planar liver segment mask, and the naming of different blood vessel types. In this step, when performing naming recognition, it is no longer necessary to pay attention to the parts of the inferior vena cava mask and the abdominal artery mask, and only the hepatic vein naming mask and the portal vein naming mask need to be recognized.

[0080] In a specific implementation manner, it further includes, before splicing the liver mask and the vascular mask, removing the vascular masks in the vascular mask except for the hepatic vein mask and the portal vein mask for subsequent naming recognition.

[0081] Among them, the liver segment vascular naming model described in this embodiment is also a pre-trained model constructed based on a deep learning algorithm. The liver segment vascular naming model contains three segmentation prediction heads, which are respectively used for planar liver segment recognition, hepatic vein vascular naming, and portal vein vascular naming; thus enabling the model to make full use of the information of all masks during the feature extraction stage, and at the same time making separate predictions during the final prediction without interfering with each other.

[0082] S204. Calculate the vascular spatial information and vascular dimension information related to the liver based on the hepatic vein naming mask and the hepatic portal vein naming mask, and construct a vascular tree.

[0083] Please continue to refer to Figure 3 , first calculate the vascular spatial information and vascular dimension information related to the liver based on the hepatic vein naming mask and the hepatic portal vein naming mask output by the hepatic segment vascular naming model. Among them, the vascular dimension information can be the vascular radius. When performing specific calculations, the vascular radius calculation method in the prior art (such as the distance_from_edt algorithm in the scipy library) can be used. The spatial information refers to the spatial position of the blood vessel. For example, the entrance of the hepatic vein blood vessel is at the highest point of the cross-section in space; considering the radius at the same time, the entrances of the hepatic vein blood vessel and the hepatic portal vein blood vessel are generally at the position of the maximum radius of their vascular tree. By combining the radius and spatial information in this way, the entrances of the hepatic vein blood vessel and the hepatic portal vein blood vessel can be accurately located, and then a directed vascular tree of the liver can be constructed based on the blood vessel trend.

[0084] S205. Generate a watershed hepatic segment model according to the planar hepatic segment mask and the constructed vascular tree.

[0085] After obtaining the vascular tree, based on the direction and branch information recorded in the vascular tree and the hepatic portal vein naming mask, extract the tertiary vascular mask in the vascular tree, where the tertiary vascular mask includes 8 categories; calculate the Euclidean distance from the planar hepatic segment mask to the tertiary vascular mask respectively; divide the planar hepatic segment mask into 8 segments based on the Euclidean distance to obtain a watershed hepatic segment model. Subsequently, surgical planning can be performed according to the watershed hepatic segment model, such as surgical planning for hepatectomy or trisegmentectomy.

[0086] S206. Obtain the information of the liver lesion, and determine the lesion mask based on the information of the liver lesion.

[0087] S207. Determine the spatial position of the liver lesion in the watershed hepatic segment model and the branches of the vascular tree that supply blood to the liver lesion according to the lesion mask.

[0088] S208. Locate the clamping points of the vascular tree based on the spatial position information of the branches of the vascular tree.

[0089] S209. Determine the downstream area affected by the clamping points according to the clamping points, in combination with the spatial structure of the vascular tree and the watershed hepatic segment model.

[0090] Please continue to refer to Figure 5, in a specific embodiment, when performing a specific surgical plan, it includes dynamically simulating and calculating the downstream area affected by the intraoperative vascular clamping point and the position of the liver segment. For example, obtaining information about the liver lesion, and determining a lesion mask based on the information of the liver lesion; based on the lesion mask, the vascular tree personalized constructed for the target patient in the foregoing steps, and the generated watershed liver segment model, determining the spatial position of the liver lesion in the watershed liver segment model and the branches of the vascular tree supplying blood to the liver lesion, and based on the spatial position information of the branches of the vascular tree, locating the appropriate clamping point of the vascular tree, and simultaneously determining the downstream area affected by the clamping point. Among them, in the step of determining the branches of the vascular tree supplying blood to the liver lesion, the position information of the vascular tree can be obtained, and the Euclidean distance from the lesion mask to the position of the vascular tree (which can be a series of points in space) can be calculated, and the branch where the closest point is located is selected as the blood supply branch of the lesion.

[0091] Moreover, in a specific implementation scenario, it further includes: adding a new clamping point to the vascular tree; combining the structure of the vascular tree and the watershed liver segment model to determine the downstream watershed affected by the new clamping point; thereby facilitating surgical simulation.

[0092] The method in the embodiments of the present invention can integrate artificial intelligence and medical image analysis technology to achieve an intelligent surgical plan that automatically completes the naming and recognition of liver segments and blood vessels, the analysis of the spatial relationship of lesions, and the dynamic watershed calculation. Specifically, it can not only pay attention to the small blood vessel branches in the complex liver anatomical structure and the individual anatomical differences of patients, but also dynamically realize the function of real-time evaluating the impact of vascular clamping on the downstream watershed; in this way, it can improve the accuracy and robustness of the preoperative planning method, effectively solve the problems of insufficient accuracy, efficiency, and personalization in the prior art, and provide a more scientific, safe, and accurate surgical plan for patients.

[0093] The above describes the preoperative planning method for liver surgery based on artificial intelligence in the embodiments of the present invention. Next, the preoperative planning device for liver surgery based on artificial intelligence in the embodiments of the present invention will be described. Please refer to Figure 6 , an embodiment of the preoperative planning device for liver surgery based on artificial intelligence in the embodiments of the present invention includes:

[0094] The segmentation module 601 is used to obtain a CT image including the liver part, and input the preprocessed CT image into a three-dimensional organ segmentation model and a three-dimensional vascular segmentation model for segmentation respectively to obtain a liver mask and a vascular mask;

[0095] The naming module 602 is used to splice the liver mask and the vascular mask and input them into a liver segment and vascular naming model for naming to obtain a planar liver segment mask, a hepatic vein naming mask, and a hepatic portal vein naming mask respectively;

[0096] A vascular tree construction module 603, configured to calculate vascular space information and vascular dimension information related to the liver based on the hepatic vein naming mask and the portal vein naming mask, and construct a vascular tree;

[0097] A model construction module 604, configured to generate a watershed liver segment model according to the planar liver segment mask and the constructed vascular tree;

[0098] A preoperative planning module 605, configured to perform preoperative planning based on the watershed liver segment model.

[0099] The device in the embodiment of the present invention can integrate artificial intelligence and medical image analysis technology to realize intelligent surgical planning for automatically completing liver segment and vascular naming recognition and analyzing the spatial relationship of lesions.

[0100] In another embodiment of the present application, the preoperative planning module 605 includes:

[0101] A lesion position determination unit, configured to obtain information about a liver lesion and determine a lesion mask based on the information about the liver lesion;

[0102] A vascular tree branch search unit, configured to determine the spatial position of the liver lesion in the watershed liver segment model and the vascular tree branches supplying blood to the liver lesion according to the lesion mask;

[0103] A clamping point positioning unit, configured to locate the clamping points of the vascular tree based on the spatial position information of the vascular tree branches;

[0104] An affected area calculation unit, configured to determine the downstream area affected by the clamping points according to the clamping points, in combination with the spatial structure of the vascular tree and the watershed liver segment model.

[0105] In another embodiment of the present application, the vascular tree branch search unit is specifically configured to:

[0106] Calculate the Euclidean distance from the liver lesion to the spatial points included in the vascular tree based on the spatial information of the lesion mask;

[0107] Select the vascular tree trunk where the point with the closest Euclidean distance is located as the vascular tree branch supplying blood to the liver lesion.

[0108] In another embodiment of the present application, the vascular tree is a directed tree; the affected area calculation unit is specifically further configured to: based on the directed tree structure of the vascular tree, obtain all subtree branches at the position where the clamping point is located, and determine the downstream watershed affected by the clamping point according to the positions of the subtree branches.

[0109] In another embodiment of the present application, the impact area calculation unit is further used to: add a new pinch-off point on the vascular tree; and determine the downstream watershed affected by the new pinch-off point in combination with the structure of the vascular tree and the watershed liver segment model.

[0110] In another embodiment of the present application, the liver segment vessel naming model includes three segmentation prediction heads, which are respectively used for planar liver segment recognition, hepatic vein naming segmentation, and hepatic portal vein naming segmentation;

[0111] The naming module 602 is specifically used to: obtain the spatial vectors of the liver mask and the blood vessel mask, and splice the spatial vectors of the liver mask and the blood vessel mask according to the cross-sectional direction of the CT image;

[0112] The spliced ​​liver mask and the blood vessel mask are input into the liver segment blood vessel naming model, and different segmentation prediction heads are called to perform planar liver segment recognition, hepatic vein naming segmentation and portal vein naming segmentation to obtain a planar liver segment mask, a hepatic vein naming mask and a portal vein naming mask.

[0113] In another embodiment of the present application, the model building module 604 is specifically used to: extract a third-level vessel mask in the vessel tree based on the hepatic portal vein naming mask, wherein the third-level vessel mask includes 8 categories;

[0114] respectively calculating the Euclidean distances between the planar liver segment mask and the three-level blood vessel mask;

[0115] The planar liver segment mask is divided into 8 segments based on the Euclidean distance to obtain a watershed liver segment model.

[0116] In another embodiment of the present application, in the step of inputting the preprocessed CT image into a three-dimensional organ segmentation network and a three-dimensional blood vessel segmentation network for segmentation, respectively, to obtain a liver mask and a blood vessel mask, the blood vessel mask is a multi-value mask, and the blood vessel mask includes a hepatic vein mask, a portal vein mask, an inferior vena cava mask, and an abdominal artery mask; the naming module 602 is specifically used to: before splicing the liver mask and the blood vessel mask and inputting them into the liver segment blood vessel naming model for naming, remove the blood vessel masks other than the hepatic vein mask and the portal vein mask from the blood vessel mask.

[0117] The device provided in the embodiments of the present invention can integrate artificial intelligence and medical image analysis technology to achieve intelligent surgical planning that automatically completes the naming and recognition of liver segments and blood vessels, the analysis of the spatial relationship of lesions, and the dynamic watershed calculation. Specifically, it can not only pay attention to the small blood vessel branches in the complex liver anatomical structure and the individual anatomical differences of patients, but also dynamically realize the function of real-time evaluating the impact of blood vessel clamping on the downstream watershed; in this way, it can improve the accuracy and robustness of the preoperative planning device, effectively solve the problems of insufficient accuracy, efficiency, and personalization in the prior art, and provide a more scientific, safe, and accurate surgical plan for patients.

[0118] Based on the same inventive concept, the embodiments of this specification also provide a preoperative planning system for liver surgery based on artificial intelligence. The corresponding description of this system can refer to the above embodiments and will not be elaborated here.

[0119] Based on the same inventive concept, the embodiments of this specification also provide an electronic device for preoperative planning of liver surgery based on artificial intelligence. The electronic device for preoperative planning of liver surgery based on artificial intelligence in the embodiments of the present invention will be described in detail from the perspective of hardware processing below.

[0120] Figure 7 It is a schematic structural diagram of an electronic device provided in the embodiments of this specification. The following refers to Figure 7 to describe the electronic device 700 according to this embodiment of the present invention. Figure 7 The displayed electronic device 700 is only an example and should not bring any limitations to the functions and usage scope of the embodiments of the present invention.

[0121] As Figure 7 shown, the electronic device 700 is presented in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710), a display unit 740, etc.

[0122] Among them, the storage unit stores program codes, and the program codes can be executed by the processing unit 710, so that the processing unit 710 executes the steps according to various exemplary embodiments of the present invention described in the above processing method part of this specification. For example, the processing unit 710 can execute steps such as Figure 1 or Figure 4 shown.

[0123] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202, and may further include a read-only storage unit (ROM) 7203.

[0124] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205. Such program modules 7205 include, but are not limited to: an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment.

[0125] The bus 730 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus structures.

[0126] The electronic device 700 may also communicate with one or more external devices 100 (such as a keyboard, a pointing device, a Bluetooth device, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or may communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (such as a router, a modem, etc.). Such communication may be carried out through the input / output (I / O) interface 750. Moreover, the electronic device 700 may also communicate with one or more networks (such as a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through the network adapter 760. The network adapter 760 may communicate with other modules of the electronic device 700 through the bus 730. It should be understood that although Figure 7 not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0127] Through the description of the above embodiments, those skilled in the art can easily understand that the exemplary embodiments described in the present invention can be implemented by software, or can be implemented by the way of software in combination with necessary hardware. Therefore, the technical solutions according to the embodiments of the present invention can be embodied in the form of a software product, and the software product can be stored in a computer-readable storage medium (which may be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which may be a personal computer, a server, or a network device, etc.) to execute the above method according to the present invention. When the computer program is executed by a data processing device, the computer-readable medium can implement the above method of the present invention, that is: as Figure 1 or Figure 4 shown in the figure.

[0128] The present invention also provides a computer program product, including computer programs / instructions, which when executed by a processor, implement the artificial intelligence-based preoperative planning method for liver surgery as described in the above embodiments, that is: as Figure 1 or Figure 4 the method shown.

[0129] Figure 8 is a schematic diagram of the principle of a computer-readable medium provided by an embodiment of this specification.

[0130] Implement Figure 1 or Figure 4 The computer program for implementing the method shown can be stored on one or more computer-readable media. The computer-readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0131] The computer-readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries the readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The readable storage medium can also be any readable medium other than the readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination of the above.

[0132] The program code for performing the operations of the present invention can be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and also including conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, executed as a stand-alone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., by connecting through the Internet using an Internet service provider).

[0133] In summary, the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that general-purpose data processing devices such as microprocessors or digital signal processors (DSPs) can be used in practice to implement some or all of the functions of some or all of the components in the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (e.g., a computer program and a computer program product) for performing some or all of the methods described herein. Such a program for implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0134] The specific embodiments described above have further elaborated on the purpose, technical solutions, and beneficial effects of the present invention. It should be understood that the present invention is not inherently related to any specific computer, virtual device, or electronic device, and various general-purpose devices can also implement the present invention. The above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

[0135] Each embodiment in this specification is described in a progressive manner. The same or similar parts among the embodiments can be referred to each other, and the key points of each embodiment are the differences from other embodiments.

[0136] If the technical solution of this application involves personal information, the product using the technical solution of this application has clearly informed the personal information processing rules and obtained the individual's voluntary consent before processing the personal information. If the technical solution of this application involves sensitive personal information, the product using the technical solution of this application has obtained the individual's separate consent before processing the sensitive personal information, and at the same time meets the "explicit consent" requirement. For example, on personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If the individual voluntarily enters the collection scope, it is deemed that he or she agrees to the collection of his or her personal information; or on the device that processes personal information, the personal information processing rules are notified by obvious signs / information, and the individual's authorization is obtained through pop-up information or by asking the individual to upload his or her personal information; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the type of personal information processed.

[0137] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for preoperative planning of liver surgery based on artificial intelligence, characterized in that: include: Acquire a CT image containing a liver part, and input the preprocessed CT image into a three-dimensional organ segmentation model and a three-dimensional blood vessel segmentation model for segmentation, respectively, to obtain a liver mask and a blood vessel mask; The liver mask and the blood vessel mask are spliced ​​and input into the liver segment blood vessel naming model for naming, thereby obtaining a planar liver segment mask, a hepatic vein naming mask and a hepatic portal vein naming mask respectively; Calculating the vascular space information and vascular size information related to the liver based on the hepatic vein naming mask and the hepatic portal vein naming mask, and constructing a vascular tree; generating a watershed liver segment model according to the planar liver segment mask and the constructed vascular tree; Preoperative planning was performed based on the watershed-type liver segment model.

2. The method for preoperative planning of liver surgery based on artificial intelligence according to claim 1, characterized in that: The preoperative planning based on the watershed type liver segment model includes: Acquiring information about liver lesions, and determining a lesion mask based on the information about the liver lesions; Determine the spatial position of the liver lesion in the watershed liver segment model and the vascular tree branch supplying blood to the liver lesion according to the lesion mask; Based on the spatial position information of the vascular tree branches, locating the pinch-off point of the vascular tree; According to the pinch-off point, in combination with the spatial structure of the vascular tree and the liver segment model of the watershed, the downstream area affected by the pinch-off point is determined.

3. The method for preoperative planning of liver surgery based on artificial intelligence according to claim 2, characterized in that: The calculating the relative position of the liver lesion in the vascular tree based on the spatial information of the lesion mask and determining the vascular tree branch supplying blood to the liver lesion comprises: Calculating the Euclidean distance from the liver lesion to the spatial point included in the vascular tree based on the spatial information of the lesion mask; The vascular tree branch where the point with the shortest Euclidean distance is located is selected as the vascular tree branch supplying blood to the liver lesion.

4. The method for preoperative planning of liver surgery based on artificial intelligence according to claim 2, characterized in that: The blood vessel tree is a directed tree; The downstream watershed affected by the pinch-off point is determined to include: Based on the directed tree structure of the blood vessel tree, all subtree branches under the location of the pinch-off point are obtained, and the downstream watershed affected by the pinch-off point is determined according to the location of the subtree branches.

5. The method for preoperative planning of liver surgery based on artificial intelligence according to claim 2, characterized in that: The preoperative planning based on the watershed type liver segment model further includes: Adding a newly added pinch-off point on the vascular tree; The downstream watershed affected by the newly added pinch-off point is determined by combining the structure of the vascular tree and the watershed liver segment model.

6. The method for preoperative planning of liver surgery based on artificial intelligence according to any one of claims 1 to 5, characterized in that: The liver segment vessel naming model includes three segmentation prediction heads, which are used for plane liver segment recognition, hepatic vein vessel naming and hepatic portal vein vessel naming respectively; The splicing of the liver mask and the blood vessel mask and inputting them into the liver segment blood vessel naming model for naming, respectively obtaining a planar liver segment mask, a hepatic vein naming mask and a hepatic portal vein naming mask, comprises: Acquire the spatial vectors of the liver mask and the blood vessel mask, and splice the spatial vectors of the liver mask and the blood vessel mask according to the cross-sectional direction of the CT image; The spliced ​​liver mask and the blood vessel mask are input into the liver segment blood vessel naming model, and different segmentation prediction heads are called to perform planar liver segment recognition, hepatic vein blood vessel naming and portal vein blood vessel naming to obtain a planar liver segment mask, a hepatic vein naming mask and a portal vein naming mask.

7. An artificial intelligence-based preoperative planning device for liver surgery, characterized in that: The artificial intelligence-based preoperative planning device for liver surgery comprises: A segmentation module, used for acquiring a CT image containing a liver part, and inputting the preprocessed CT image into a three-dimensional organ segmentation model and a three-dimensional blood vessel segmentation model for segmentation, to obtain a liver mask and a blood vessel mask; A naming module, used for splicing the liver mask and the blood vessel mask and inputting them into the liver segment blood vessel naming model for naming, thereby obtaining a planar liver segment mask, a hepatic vein naming mask and a hepatic portal vein naming mask respectively; A vascular tree construction module, used to calculate the vascular space information and vascular size information related to the liver based on the hepatic vein naming mask and the hepatic portal vein naming mask, and to construct a vascular tree; A model construction module, used for generating a watershed liver segment model according to the planar liver segment mask and the constructed vascular tree; A preoperative planning module is used for preoperative planning based on the watershed type liver segment model.

8. An artificial intelligence-based preoperative planning device for liver surgery, characterized in that: The artificial intelligence-based preoperative planning device for liver surgery includes: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory to enable the artificial intelligence-based liver surgery preoperative planning device to perform the steps of the artificial intelligence-based liver surgery preoperative planning method as described in any one of claims 1-6.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the program / instructions are executed by a processor, the steps of the artificial intelligence-based preoperative planning method for liver surgery as described in any one of claims 1-6 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the artificial intelligence-based preoperative planning method for liver surgery as described in any one of claims 1 to 6 are implemented.

Images