A dynamic three-dimensional reconstruction method and system for liver resection

Abstract

The application relates to a dynamic three-dimensional reconstruction method and system for liver resection, which comprises the following steps: obtaining initial mesh data based on preoperative CT images; obtaining two-dimensional projection images of a surgical area by using intraoperative CT to obtain target mesh data; constructing surface models by using the target mesh data and the initial mesh data respectively and performing rigid registration, and demarcating a model surface part with a larger deformation region as a to-be-reconstructed space; performing non-rigid deformation on the initial mesh region in the to-be-reconstructed space to reconstruct a high-resolution surface model of the to-be-reconstructed space; and replacing the corresponding part of the preoperative full-liver tissue accurate three-dimensional surface model with the generated high-resolution surface model of the to-be-reconstructed space. Compared with the prior art, the application solves the problems of high time complexity, high computing power requirement and incapability of dynamic monitoring of a local region in the prior art, realizes rapid reconstruction of a liver tissue surface of a surgical area in operation, and achieves the purpose of dynamic monitoring and auxiliary accurate resection in operation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of three-dimensional reconstruction, in particular to a dynamic three-dimensional reconstruction method and system for liver resection. BACKGROUND

[0002] By using three-dimensional reconstruction technology, a clear three-dimensional model of the whole liver tissue can be established by using CT / MRI medical images, and further visualization of cancer foci in the liver tissue of cancer patients can be performed. Before liver resection of primary liver cancer patients, three-dimensional reconstruction technology can be used to help doctors to determine the optimal surgical boundary, so as to achieve the purpose of resecting the lesion along the path with the least damage and the greatest benefit.

[0003] However, in liver resection, on the one hand, various processing methods will cause different deformations of the liver tissue, making it difficult to accurately control the optimal surgical resection boundary. The current three-dimensional reconstruction technology based on ultrasound and CT images has the problem of high time complexity, which cannot quickly reconstruct the complex ducts in the liver (such as the hepatic vein, portal vein and bile duct system) during the operation, and it is difficult to achieve the purpose of dynamic monitoring and accurate resection during the operation.

[0004] A three-dimensional point cloud surface reconstruction method and device are disclosed in Chinese Patent CN106960470A, which smoothes and / or refines the grid data to make the reconstructed surface of the spatial object smoother, thereby obtaining a more realistic three-dimensional object model. However, this invention is inefficient and cannot provide timely and effective assistance to doctors in the operation scene. Chinese Patent CN106846465A discloses a CT three-dimensional reconstruction method and system, which is based on distributed technology and calculates the projection matrix and iterative attenuation component of each pixel point in the two-dimensional projection image through multiple lower computers, and then updates the current iteration state of the three-dimensional body data to be reconstructed. The current iteration state of the three-dimensional body data to be reconstructed after iteration is taken as the three-dimensional reconstruction result of the CT for the two-dimensional projection image. This invention solves the problems of large calculation amount, low calculation efficiency and poor reusability of the calculation result, and ensures the quality of the three-dimensional reconstruction image. However, this invention requires high computing power and needs to upgrade the hardware environment of the hospital before deployment; the update of the reconstructed three-dimensional body data needs multiple iterations and global update, which cannot locate the operation area during the operation and cannot achieve the purpose of dynamic positioning and real-time assistance to the operation area of the doctor along with the operation process. In the prior art, the three-dimensional reconstruction method or device for liver resection has high cost and high computing power requirement. If it is deployed in a hospital, the hardware environment of the hospital needs to be upgraded, and the efficiency is low. The reaction speed is slow for each reconstruction, which makes it difficult to provide timely and effective assistance to doctors in the operation scene. Moreover, it cannot dynamically model the operation area of the doctor along with the operation process, so the clinical effect is not good. SUMMARY

[0005] The present application aims to overcome the defects of the prior art and provide a dynamic three-dimensional reconstruction method and system for liver resection, based on a high-precision adaptive interpolation algorithm, by grid processing the contour data of the target tissue in the CT image, using the preoperative patient's full liver tissue precise three-dimensional curved surface model as a reference, automatically positioning the operating area of the surgeon under intraoperative CT assistance, and quickly reconstructing the different deformation curves of the liver tissue caused by various processing methods at high resolution, solving the problems of high time complexity, high computing power requirement and inability to dynamically monitor the local area in the prior art, realizing the rapid reconstruction of the liver tissue curved surface in the operating area, and achieving the purpose of dynamic monitoring and assisting accurate resection during the operation.

[0006] The purpose of the present application can be achieved by the following technical solutions:

[0007] A dynamic three-dimensional reconstruction method for liver resection, comprising the following steps:

[0008] Obtain preoperative CT images, grid process the contour data of the full liver tissue in the preoperative CT images to obtain initial grid data;

[0009] Use intraoperative CT to obtain two-dimensional projection images of the operating area at least at two rotation angles, and perform three-dimensional grid processing on the contour data of the full liver tissue in the two-dimensional projection images to obtain target grid data;

[0010] Use the target grid data and the initial grid data to construct curved surface models and perform rigid registration, and delineate the model curved surface part with larger deformation area as the to-be-reconstructed space;

[0011] Based on the initial grid data, the initial grid region in the to-be-reconstructed space is determined, and based on the target grid data, the target grid region in the to-be-reconstructed space is determined, the initial grid region in the to-be-reconstructed space is deformed non-rigidly, and a high-resolution curved surface model of the to-be-reconstructed space is reconstructed;

[0012] Obtain a preoperative full liver tissue precise three-dimensional curved surface model, use the generated high-resolution curved surface model of the to-be-reconstructed space to replace the corresponding part of the preoperative full liver tissue precise three-dimensional curved surface model, and perform visualization.

[0013] Further, the accuracy of the preoperative CT images is not lower than the accuracy of the two-dimensional projection images obtained by the intraoperative CT.

[0014] Further, the obtaining of the preoperative CT images and the grid processing of the contour data of the full liver tissue in the preoperative CT images to obtain the initial grid data are specifically:

[0015] The whole liver tissue is reconstructed in three dimensions by using CT before operation, and the outline data of each level of tissue in the whole liver tissue is converted into initial point cloud data P with different marks; the initial point cloud data of each level of tissue is triangulated respectively by using a Delaunay triangulation algorithm, and the triangulated initial mesh data of each level of tissue is obtained; wherein the initial point cloud data of the i-th level of tissue is:

[0016]

[0017] N i The number of initial point cloud data of the i-th level of tissue.

[0018] Further, the two-dimensional projection images of the surgical region are obtained by using the intraoperative CT at at least two rotation angles, the three-dimensional meshing processing is performed on the outline data of the whole liver tissue in the two-dimensional projection images, and the target mesh data is obtained, which is specifically:

[0019] The two-dimensional projection images of the surgical region are obtained by using the intraoperative CT at at least two rotation angles, the threshold segmentation method and the edge detection algorithm are used on the two-dimensional projection images, the outline data of each level of tissue of the whole liver tissue is obtained, and the outline data of each level of tissue is converted into target point cloud data D according to the scanning parameters of the intraoperative CT; the three-dimensional meshing processing is performed on the target point cloud data of each level of tissue respectively, and the triangulated target mesh data of each level of tissue is obtained; wherein the target point cloud data of the i-th level of tissue is:

[0020]

[0021] M i The number of target point cloud data of the i-th level of tissue.

[0022] Further, the curved surface model is constructed by using the target mesh data and the initial mesh data respectively, and the rigid registration is performed, and the model curved surface part with a larger deformation region is divided into a to-be-reconstructed space, which is specifically:

[0023] The rigid registration between the target mesh data and the initial mesh data of each level of tissue is performed by using the ICP algorithm, and the point pairs are obtained by mapping between the target mesh data and the initial mesh data; the Euclidean distance between the point pairs is taken as the objective function; the organization with the minimum objective function in each level of tissue is taken as the reference, and the target mesh data and the initial mesh data of other levels of tissue are added, so that the region with larger deformation in the liver region is obtained, and the region is divided into a to-be-reconstructed space; wherein the to-be-reconstructed space contains the initial mesh data p i of the i-th level of tissue and the target mesh data d i of the i-th level of tissue:

[0024]

[0025]

[0026] n i is the number of initial mesh data of the i-th level organization, m i is the number of target mesh data of the i-th level organization, and n i > m i .

[0027] Further, the initial mesh points are obtained based on the initial mesh data in the initial mesh region, and the target mesh points are obtained based on the target mesh data in the target mesh region, and the non-rigid deformation of the initial mesh region in the space to be reconstructed is performed to reconstruct the high-resolution curved surface model of the space to be reconstructed, and the method specifically comprises the following steps:

[0028] The target mesh points in the space to be reconstructed are one-to-one mapped to the initial mesh points, and the mapping relationship of all the points is saved as a mapping set F satisfying the one-to-one mapping requirement:

[0029]

[0030] wherein, and satisfy

[0031] The initial mesh points in the mapping set F having the mapping relationship are taken as key points, all the initial mesh points in the space to be reconstructed are interpolated to the target mesh region through inverse mapping, and new target point cloud data are formed in the target mesh region:

[0032] and

[0033] The new target point cloud data after the interpolation expansion in the space to be reconstructed are subjected to three-dimensional meshing processing, so as to satisfy the condition that the common edges of adjacent meshes remain boundary continuous, and the high-resolution curved surface model of the space to be reconstructed is reconstructed.

[0034] A dynamic three-dimensional reconstruction system for liver resection, comprising:

[0035] A preoperative model meshing processing module is configured to acquire a preoperative CT image, and perform meshing processing on contour data of whole liver tissue in the preoperative CT image to obtain initial mesh data.

[0036] An intraoperative image acquisition module is configured to acquire two-dimensional projection images of a surgical region obtained by an intraoperative CT at at least two rotation angles.

[0037] An intraoperative image meshing processing module is configured to perform three-dimensional meshing processing on contour data of whole liver tissue in the two-dimensional projection images to obtain target mesh data.

[0038] The whole liver tissue rigid registration module is used to construct surface models using target mesh data and initial mesh data respectively and perform rigid registration.

[0039] The space to be reconstructed module is used to delineate the curved surface of the model with a large deformation area into the space to be reconstructed.

[0040] The high-resolution interpolation module for the space to be reconstructed determines the initial mesh region within the space to be reconstructed based on the initial mesh data and the target mesh region within the space to be reconstructed based on the target mesh data, and is used to perform non-rigid deformation on the initial mesh region within the space to be reconstructed.

[0041] The high-resolution surface reconstruction module for deformed regions is used to reconstruct high-resolution surface models of the space to be reconstructed.

[0042] The dynamic reconstruction result visualization module is used to replace the corresponding part of the preoperative whole liver tissue precise three-dimensional surface model with the generated high-resolution surface model of the space to be reconstructed, and then visualize it.

[0043] Furthermore, the specific execution content of the preoperative model meshing processing module is as follows:

[0044] Preoperatively, 3D reconstruction of the entire liver tissue was performed using CT, and the contour data of each level of tissue in the whole liver tissue were converted into initial point cloud data P with different labels. The Delaunay triangulation algorithm was used to triangulate the initial point cloud data corresponding to each level of tissue, obtaining the triangulated initial mesh data of each level of tissue; wherein, the initial point cloud data of the i-th level tissue is:

[0045]

[0046] N i The number of initial point cloud data for the i-th level organization.

[0047] Furthermore, the specific execution content of the intraoperative image acquisition module is as follows:

[0048] Intraoperative CT scans were performed at at least two rotation angles to obtain two-dimensional projection images of the surgical area.

[0049] The specific execution content of the intraoperative image meshing processing module is as follows:

[0050] Thresholding segmentation and edge detection algorithms were used on the two-dimensional projection image to obtain the contour data of each level of the whole liver tissue. Based on the intraoperative CT scanning parameters, the contour data of each level of tissue was converted into target point cloud data D. The target point cloud data of each level of tissue were then subjected to three-dimensional meshing to obtain triangularized target mesh data for each level of tissue. The target point cloud data of the i-th level tissue is as follows:

[0051]

[0052] M i The number of target point cloud data of the i-th level organization.

[0053] Further, the specific implementation of the whole liver tissue rigid registration module is as follows:

[0054] The ICP algorithm is used to perform rigid registration between the target mesh data and the initial mesh data of each level organization, and point pairs are obtained by mapping between the target mesh data and the initial mesh data;

[0055] The specific implementation of the to-be-reconstructed space division module is as follows:

[0056] The Euclidean distance between the point pairs is taken as an objective function, and the organization with the minimum objective function in each level organization is taken as a reference, and the target mesh data and the initial mesh data of other level organizations are added to obtain a region with large deformation in the liver region, and the region is divided into a to-be-reconstructed space; wherein the to-be-reconstructed space contains the initial mesh data p i of the i-th level organization and the target mesh data d i of the i-th level organization:

[0057]

[0058]

[0059] n i The number of initial mesh data of the i-th level organization, m i The number of target mesh data of the i-th level organization, and n i >>m i .

[0060] Further, the initial mesh points are obtained based on the initial mesh data in the initial mesh region, the target mesh points are obtained based on the target mesh data in the target mesh region, and the specific implementation of the to-be-reconstructed space high-resolution interpolation module is as follows:

[0061] The target mesh points in the to-be-reconstructed space are one-to-one mapped to the initial mesh points, and the mapping relationship of all points is saved as a mapping set F satisfying the injective requirement:

[0062]

[0063] Wherein, And satisfies

[0064] Take the initial grid point with mapping relationship in the mapping set F as the key point, and insert all initial grid points in the space to be reconstructed to the target grid region through inverse mapping interpolation, to form new target point cloud data in the target grid region:

[0065] And

[0066] The specific execution content of the high-resolution surface reconstruction module of the deformation region is:

[0067] The new target point cloud data in the space to be reconstructed after interpolation expansion is subjected to three-dimensional gridding processing, so as to meet the condition that the common edge of adjacent grids remains boundary continuous, and the high-resolution surface model of the space to be reconstructed is reconstructed.

[0068] Compared with the prior art, the present application has the following beneficial effects:

[0069] (1) The present application uses the preoperative whole liver tissue precise three-dimensional surface model of the patient as a reference, ensuring the high precision of the three-dimensional surface model during the operation, and realizing personalized modeling for each patient, which provides strong support for personalized precise resection of liver resection;

[0070] (2) In the three-dimensional modeling of the present application, the contour data of each level of organ tissue is converted into point cloud data, which is not all voxel data of complete CT images, and the automatic division of the space to be reconstructed further reduces the calculation amount of three-dimensional surface modeling, so that the present application greatly saves computing power and improves computing efficiency, so that the entire software system meets the deployment conditions of hospitals at all levels;

[0071] (3) The automatic division of the space to be reconstructed in the present application represents the automatic monitoring of the deformation region of the organ tissue under the complex conditions during the operation, and combined with the navigation technology in the prior art, it can completely realize the purpose of assisting the doctor to complete the path resection of the lesion along the minimum damage and maximum benefit. BRIEF DESCRIPTION OF DRAWINGS

[0072] Figure 1 It is a flow chart of the dynamic three-dimensional reconstruction method;

[0073] Figure 2 It is an architecture diagram of the dynamic three-dimensional reconstruction system;

[0074] Figure 3 It is a flow chart of the dynamic three-dimensional reconstruction method of the liver blood vessel system in the embodiment. DETAILED DESCRIPTION

[0075] The application will be described in detail below with reference to the drawings and specific embodiments. The embodiments are implemented on the premise of the technical solutions of the application, and detailed implementation modes and specific operation processes are given, but the protection scope of the application is not limited to the following embodiments.

[0076] In the drawings, components of the same structure are denoted by the same reference numerals, and components similar in structure or function are denoted by similar reference numerals. The size and thickness of each component shown in the drawings are arbitrarily shown, and the application does not limit the size and thickness of each component. In order to make the drawing clearer, the components are appropriately exaggerated in some places in the drawing.

[0077] Embodiment 1

[0078] A dynamic three-dimensional reconstruction method for liver resection, as shown in Figure 1 includes the following steps:

[0079] 1) Obtain preoperative CT images, and perform meshing processing on the contour data of the whole liver tissue in the preoperative CT images to obtain initial mesh data;

[0080] In preoperative use of enhanced CT, the whole liver tissue (including each liver segment, hepatic vein, portal vein and bile duct system, tumor lesion, etc.) is three-dimensionally reconstructed, and the contour data of each level of tissue in the whole liver tissue is converted into initial point cloud data P with different color identifiers. The initial point cloud data of each level of tissue is triangulated using a Delaunay triangulation algorithm to obtain high-resolution triangulated initial mesh data of each level of tissue. The initial point cloud data of the i-th level of tissue is:

[0081]

[0082] N i is the number of initial point cloud data of the i-th level of tissue.

[0083] 2) Use intraoperative CT to obtain two-dimensional projection images of the surgical region at least at two rotation angles, and perform three-dimensional meshing processing on the contour data of the whole liver tissue in the two-dimensional projection images to obtain target mesh data;

[0084] The intraoperative CT is used to perform rapid scanning at least at two rotation angles to obtain two-dimensional projection images of the surgical region, also known as CT slice images. The accuracy of the intraoperative CT slice images can be lower than that of the preoperative CT images. For example, if the voxel size of the preoperative CT images is 1mm*1mm*1mm, the voxel size of the intraoperative CT slice images can be 2mm*2mm*2mm, but the scanning range of the intraoperative CT needs to cover the entire liver region.

[0085] Threshold segmentation method and edge detection algorithm are used on CT slice images to quickly obtain the contour data of each level of liver tissue, and the contour data of each level of tissue is converted into target point cloud data D according to the scanning parameters of the intraoperative CT; the three-dimensional gridding processing is performed on the target point cloud data of each level of tissue to obtain the low-resolution triangulated target mesh data of each level of tissue; wherein the target point cloud data of the i-th level of tissue is:

[0086]

[0087] M i is the number of target point cloud data of the i-th level of tissue.

[0088] 3) The surface model is constructed using the target mesh data and the initial mesh data, and rigid registration is performed, and the model surface part with larger deformation area is delineated as the space to be reconstructed;

[0089] ICP algorithm is used for rigid registration between low-resolution target mesh data and high-resolution initial mesh data of each level of tissue, and point pairs are obtained by mapping between target mesh data and initial mesh data, and the Euclidean distance between point pairs is used as the objective function; the organization with the minimum objective function in each level of tissue is taken as the reference, and the target mesh data and the initial mesh data of other levels of tissue are added to obtain the larger deformation area in the liver region, and the area is divided into the space to be reconstructed; wherein the initial mesh data p i of the i-th level of tissue and the target mesh data d i of the i-th level of tissue are contained in the space to be reconstructed.

[0090]

[0091]

[0092] n i is the number of initial mesh data of the i-th level of tissue, m i is the number of target mesh data of the i-th level of tissue, and n i >>m i .

[0093] 4) Based on the initial mesh data, the initial mesh region in the space to be reconstructed is determined, based on the target mesh data, the target mesh region in the space to be reconstructed is determined, and the initial mesh region in the space to be reconstructed is deformed non-rigidly to reconstruct the high-resolution surface model of the space to be reconstructed;

[0094] The initial mesh points are obtained based on the initial mesh data in the initial mesh region, and the target mesh points are obtained based on the target mesh data in the target mesh region. The target mesh points in the space to be reconstructed are one-to-one mapped to the initial mesh points, and the mapping relationship of all points is saved as a mapping set F that meets the one-to-one requirement:

[0095]

[0096] wherein, and satisfy

[0097] With the initial grid points in the mapping set F having a mapping relationship as the key points, all initial grid points in the space to be reconstructed are interpolated by inverse mapping to the target grid region, and high-resolution new target point cloud data is formed in the target grid region:

[0098] and

[0099] The high-resolution new target point cloud data in the space to be reconstructed after interpolation expansion is subjected to three-dimensional gridding processing, and each level of organization is rendered to meet the condition that the common edge of adjacent grids remains boundary continuous, thereby reconstructing a high-resolution curved surface model of the space to be reconstructed.

[0100] 5) Obtain a preoperative precise three-dimensional curved surface model of the whole liver tissue, and the preoperative precise three-dimensional curved surface model of the whole liver tissue is formed by a real CT image. The generated high-resolution curved surface model of the space to be reconstructed is used to replace the corresponding part of the preoperative precise three-dimensional curved surface model of the whole liver tissue, and visualization is performed, and the intraoperative three-dimensional curved surface model of the whole liver tissue is dynamically updated, the view is automatically positioned, and visualization of the deformation area is realized.

[0101] Embodiment 2:

[0102] A dynamic three-dimensional reconstruction system for liver resection, as shown in Figure 2 , comprising:

[0103] ① A preoperative model gridding processing module, configured to obtain a preoperative CT image, and perform gridding processing on the contour data of the whole liver tissue in the preoperative CT image to obtain initial grid data;

[0104] Specifically, the whole liver tissue (including each liver segment, hepatic vein, portal vein and bile duct system, tumor lesion, etc.) is three-dimensionally reconstructed by using enhanced CT before surgery, and the contour data of each level of tissue in the whole liver tissue is converted into initial point cloud data P with different color identifiers; the initial point cloud data of each level of tissue is triangularized by using a Delaunay triangulation algorithm, and high-resolution triangularized initial grid data of each level of tissue is obtained; wherein the initial point cloud data of the i-th level of tissue is:

[0105]

[0106] N iThe number of initial point cloud data of the i-th level organization.

[0107] ② An intraoperative image acquisition module, configured to acquire two-dimensional projection images of a surgical region obtained by the intraoperative CT at at least two rotation angles;

[0108] Specifically, the intraoperative CT is used to perform a rapid plain scan at at least two rotation angles to obtain two-dimensional projection images of the surgical region, also known as CT slice images. The accuracy of the intraoperative CT slice images can be lower than that of the preoperative CT images. For example, if the voxel size of the preoperative CT images is 1mm*1mm*1mm, the voxel size of the intraoperative CT slice images can be 2mm*2mm*2mm, but the scanning range of the intraoperative CT needs to cover the entire liver region.

[0109] ③ An intraoperative image gridding processing module, configured to perform three-dimensional gridding processing on the contour data of the whole liver tissue in the two-dimensional projection images to obtain target grid data;

[0110] Specifically, the threshold segmentation method and the edge detection algorithm are used for the CT slice images to quickly obtain the contour data of the organizations at all levels of the whole liver tissue, and the contour data of the organizations at all levels is converted into target point cloud data D according to the scanning parameters of the intraoperative CT. The target point cloud data of each level of organization is respectively subjected to three-dimensional gridding processing to obtain low-resolution triangulated target grid data of each level of organization. The target point cloud data of the i-th level organization is:

[0111]

[0112] M i The number of target point cloud data of the i-th level organization.

[0113] ④ A whole liver tissue rigid registration module, configured to construct a surface model using the target grid data and the initial grid data and perform rigid registration;

[0114] Specifically, the ICP algorithm is used to perform rigid registration between the low-resolution target grid data and the high-resolution initial grid data at all levels of organization, and the point pairs are obtained by mapping between the target grid data and the initial grid data.

[0115] ⑤ A to-be-reconstructed space division module, configured to divide a model surface part with a larger deformation region into a to-be-reconstructed space;

[0116] Specifically, the Euclidean distance between the point pairs is taken as the objective function, and the organization with the minimum objective function in each level of organization is taken as the reference. The target grid data and the initial grid data of other levels of organization are added to obtain a larger deformation region in the liver region, and the region is divided into a to-be-reconstructed space. The to-be-reconstructed space contains the initial grid data p i of the i-th level organization and the target grid data di :

[0117]

[0118]

[0119] n i is the number of initial mesh data of the i-th level organization, m i is the number of target mesh data of the i-th level organization, and n i > m i .

[0120] 6. A high-resolution interpolation module for a space to be reconstructed, which determines an initial mesh region in the space to be reconstructed based on the initial mesh data and determines a target mesh region in the space to be reconstructed based on the target mesh data, and is configured to perform non-rigid deformation on the initial mesh region in the space to be reconstructed;

[0121] Specifically, initial mesh points are obtained based on the initial mesh data in the initial mesh region, and target mesh points are obtained based on the target mesh data in the target mesh region. The target mesh points in the space to be reconstructed are one-to-one mapped to the initial mesh points, and the mapping relationship of all points is saved as a mapping set F satisfying the one-to-one mapping requirement:

[0122]

[0123] wherein, and satisfies

[0124] The initial mesh points in the mapping set F having the mapping relationship are taken as key points, all the initial mesh points in the space to be reconstructed are interpolated to the target mesh region through inverse mapping, and new high-resolution target point cloud data are formed in the target mesh region:

[0125] and

[0126] 7. A high-resolution surface reconstruction module for the space to be reconstructed, which is configured to perform three-dimensional meshing on the new high-resolution target point cloud data in the space to be reconstructed after interpolation and expansion, and render each level of organization to meet the condition that the common edges of adjacent meshes remain boundary continuous, thereby reconstructing a high-resolution surface model of the space to be reconstructed.

[0127] Specifically, the new high-resolution target point cloud data in the space to be reconstructed after interpolation and expansion are subjected to three-dimensional meshing, and each level of organization is rendered to meet the condition that the common edges of adjacent meshes remain boundary continuous, thereby reconstructing a high-resolution surface model of the space to be reconstructed.

[0128] 8. A dynamic reconstruction result visualization module for replacing the corresponding part of the preoperative full liver tissue precision three-dimensional surface model with the generated high-resolution surface model of the space to be reconstructed, and performing visualization, dynamically updating the intraoperative full liver tissue three-dimensional surface model, automatically positioning the view, and realizing visualization of the deformation region.

[0129] Embodiment 3

[0130] This embodiment takes a liver vascular system dynamic three-dimensional reconstruction method for liver resection as the implementation background, and the flowchart is as shown in Figure 3 The method can be performed by the dynamic three-dimensional reconstruction system provided by the present application, which can be realized by software and / or hardware and can be integrated into a computer or the like, and specifically includes the following steps:

[0131] S1. Obtain preoperative high-precision CT images, perform meshing processing on the contour data of the liver region vascular system in the images, obtain high-resolution initial mesh data, and classify the various blood vessels;

[0132] The CT scanning can obtain all the voxel information of the scanning region, but the size of the selected voxel when output can be determined according to actual needs. In order to ensure the high resolution of the preoperative data, the voxel size of 1mm*1mm*1mm is taken as an example here. The CT image data is saved in a three-dimensional array of Num*H*W, where Num is the number of image data frames, H is the image data height, and W is the image data width. After the doctor obtains the slice images, the blood vessel contours are drawn and described. The contour points can be directly exported and form point cloud data P, by which the initial point cloud data of various blood vessels can be triangulated using the Delaunay triangulation algorithm to obtain high-resolution triangulation initial mesh data of various tissues.

[0133] The initial point cloud data of the i-th category of blood vessels can be represented as:

[0134]

[0135] N i The number of initial point cloud data of the i-th category of blood vessels.

[0136] S2. Obtain two-dimensional projection images of the surgical region at least at two rotation angles using intraoperative CT, and quickly obtain the contour data of various blood vessels using threshold segmentation and edge detection algorithms;

[0137] In the method, the intraoperative CT is quickly scanned at at least two rotation angles and the scanning range needs to cover the whole liver region, because the directions of different blood vessels are different, a single direction will lose a large amount of spatial information of the same blood vessel, and the blood vessels in the liver region not affected by the operation are extremely important for subsequent rigid registration. In addition, the CT slice image output can be less accurate than the preoperative CT image. In the embodiment, the intraoperative CT slice image has a voxel size of 2mm*2mm*2mm.

[0138] In the method, the threshold segmentation method used for the CT slice image needs to be set by an experienced doctor in advance. The use of the edge detection algorithm can avoid a large amount of interference data caused by inaccurate threshold range setting, and then the profile data of various blood vessels can be quickly obtained through calculation of connectivity and outlier detection, and the profile data is converted into target point cloud data D according to the CT scanning parameters.

[0139] S3. The profile data of the blood vessel system in the liver region in the intraoperative low-resolution CT slice image is subjected to three-dimensional gridding processing to obtain low-resolution target grid data.

[0140] The three-dimensional gridding processing method is the same as above, and the Delaunay triangulation algorithm can be used to perform three-dimensional gridding processing on the target point cloud data corresponding to various blood vessels respectively to obtain low-resolution triangulation target grid data of various blood vessels, wherein the target point cloud data of the ith blood vessel is:

[0141]

[0142] M i is the number of target point cloud data of the ith blood vessel.

[0143] S4. The target grid data and the initial grid data are used to construct surface models and perform rigid registration, and the region with large deformation is defined as the to-be-reconstructed space.

[0144] In the method, the rigid registration between the target grid data of various blood vessels and the initial grid data is performed by using the ICP algorithm, and the Euclidean distance between the point pairs is used as the objective function.

[0145] In order to prevent the profile data of various blood vessels obtained intraoperatively from being inconsistent with the categories before the operation, the target grid data of various blood vessels intraoperatively and the initial grid data before the operation are subjected to rigid registration respectively. If the blood vessels in the liver blood vessel system are divided into I categories, the step will perform I*I times of rigid registration in parallel.

[0146] The automatic division of the space to be reconstructed is: taking the corresponding blood vessel with the minimum objective function in the I*I times of rigid registration as the reference, adding the target grid data of other blood vessels and the initial grid data, then calculating the region with larger deformation in the liver region to obtain the space to be reconstructed;

[0147] Wherein the initial grid data p of the i-th blood vessel is contained in the space to be reconstructed i And the target grid data d of the i-th blood vessel i :

[0148]

[0149]

[0150] n i The initial point cloud quantity of the i-th blood vessel, m i The target point cloud quantity of the i-th blood vessel, and n i >>m i ;

[0151] Wherein, the region with larger deformation in the liver region is obtained by calculating the region with poor matching accuracy after rigid registration. Due to the nature that the space to be reconstructed can be larger than the actual deformation region, the whole space can be uniformly divided, and then the matching degrees of each region are calculated in parallel. The regions with lower matching degrees are all divided into the space to be reconstructed.

[0152] S5. Non-rigid deformation is performed on the initial grid region in the space to be reconstructed, so that the target grid points in the region are one-to-one mapped to the initial grid points, and a high-resolution blood vessel surface model of the space to be reconstructed is reconstructed;

[0153] Wherein, the target grid points in the space to be reconstructed need to be one-to-one mapped to the initial grid points, and the mapping relationship of all points is saved as a mapping set F satisfying the injectivity requirement:

[0154]

[0155] Wherein, And Satisfy

[0156] Wherein, the high-resolution interpolation of the space to be reconstructed needs to take the initial grid points with mapping relationship in the mapping set as key points, interpolate all initial grid points in the space to be reconstructed to the target grid region through inverse mapping, and the new target point cloud data in the space to be reconstructed after inverse mapping is:

[0157] And

[0158] When reconstructing the high-resolution curved surface model of the space to be reconstructed, only the new target point cloud data of the space to be reconstructed after interpolation expansion is subjected to triangular meshing processing, and the condition that the common edges of adjacent triangular meshed grids remain boundary continuous is met.

[0159] S6. Using the generated high-resolution curved surface model of the space to be reconstructed, the corresponding part of the preoperative precise three-dimensional curved surface model of the hepatic region blood vessel system is replaced, and visualization is performed.

[0160] The present application has the following improvements:

[0161] Based on the initial mesh data obtained from the preoperative high-precision CT image, the CT slice image of the surgical region is obtained in real time during the operation to obtain the target mesh data, in the whole liver tissue rigid registration, the high-resolution three-dimensional data (i.e. the initial mesh data) of the patient before the operation is used as a reference, which ensures the basis of personalized modeling of the patient, and is also a necessary condition for subsequent reconstruction of the space to be reconstructed.

[0162] In the whole liver tissue rigid registration, the feature that only part of the liver tissue will deform at the same time under the intraoperative condition is used, and the rigid registration of each level of tissue is performed, the Euclidean distance between the points can be used as the objective function, and the liver tissue after deformation is obtained by taking the tissue with the minimum objective function in each level of tissue as the reference, which is different from the traditional rigid registration.

[0163] In the automatic division of the space to be reconstructed, the region with large deformation in the liver region is obtained by calculating the region with poor matching accuracy after rigid registration, and due to the nature that the space to be reconstructed can be larger than the actual deformed region, the whole space can be uniformly divided, and the matching degree of each region can be calculated in parallel, which avoids the additional computing power requirement caused by greedy calculation.

[0164] In the high-resolution interpolation calculation in the space to be reconstructed, the target mesh points are mapped to the initial mesh points one by one, and then the initial mesh points are taken as key points, and the initial mesh points (based on the preoperative high-precision image) are inversely mapped and interpolated to the target mesh region, which ensures the accuracy of the reconstructed image in the overall structure; when the new target point cloud data is subjected to triangular meshing processing, the boundary continuity of the triangular mesh is used as a constraint, which ensures the smoothness of the reconstructed curved surface.

[0165] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes without creative labor according to the concept of the present application. Therefore, any technical solution obtained by logical analysis, reasoning or limited experiment based on the prior art according to the concept of the present application shall be within the protection scope determined by the claims.

Claims

1. A dynamic three-dimensional reconstruction method for liver resection, characterized in that, The method comprises the following steps: obtaining a preoperative CT image, performing meshing processing on the contour data of the whole liver tissue in the preoperative CT image to obtain initial mesh data; obtaining two-dimensional projection images of the surgical region at least two rotation angles using intraoperative CT, performing three-dimensional meshing processing on the contour data of the whole liver tissue in the two-dimensional projection images to obtain target mesh data; constructing a surface model using the target mesh data and the initial mesh data respectively and performing rigid registration, and demarcating a model surface part with a larger deformation region as a to-be-reconstructed space; determining an initial mesh region in the to-be-reconstructed space based on the initial mesh data, determining a target mesh region in the to-be-reconstructed space based on the target mesh data, performing non-rigid deformation on the initial mesh region in the to-be-reconstructed space, and reconstructing a high-resolution surface model of the to-be-reconstructed space; obtaining a precise three-dimensional surface model of the whole liver tissue before operation, using the generated high-resolution surface model of the to-be-reconstructed space to replace the corresponding part of the precise three-dimensional surface model of the whole liver tissue before operation, and performing visualization; the obtaining of the preoperative CT image and the meshing processing on the contour data of the whole liver tissue in the preoperative CT image to obtain the initial mesh data specifically comprises: three-dimensional reconstruction of the whole liver tissue by using CT before operation, and conversion of the contour data of each level of tissue in the whole liver tissue into initial point cloud data P with different identifiers; the initial point cloud data of each level of tissue is triangulated respectively using a Delaunay triangulation algorithm to obtain the triangulated initial mesh data of each level of tissue; wherein the initial point cloud data of the i-th level of tissue is: N i N is the number of initial point cloud data for the i-th level organization; the obtaining of the two-dimensional projection images of the surgical region at least two rotation angles using intraoperative CT, and the three-dimensional meshing processing on the contour data of the whole liver tissue in the two-dimensional projection images to obtain the target mesh data specifically comprises: scanning the surgical region at least two rotation angles using intraoperative CT, obtaining the two-dimensional projection images, using a threshold segmentation method and an edge detection algorithm on the two-dimensional projection images to obtain the contour data of each level of tissue of the whole liver tissue, and converting the contour data of each level of tissue into target point cloud data D according to the scanning parameters of the intraoperative CT; the target point cloud data of each level of tissue is triangulated respectively to obtain the triangulated target mesh data of each level of tissue; wherein the target point cloud data of the i-th level of tissue is: M i N is the number of target point cloud data for the i-th level organization; the construction of a surface model using the target mesh data and the initial mesh data respectively and the rigid registration, and the demarcation of a model surface part with a larger deformation region as a to-be-reconstructed space specifically comprises: The ICP algorithm is used for rigid registration between the target grid data and the initial grid data of each level organization, and a point pair is obtained by mapping between the target grid data and the initial grid data, and the Euclidean distance between the point pair is taken as a target function; the organization with the minimum target function in each level organization is taken as a reference, and the target grid data and the initial grid data of other level organizations are added to obtain a region with larger deformation in the liver region, and the region is divided into a to-be-reconstructed space; wherein the to-be-reconstructed space contains the initial grid data p i of the i-level organization and the target grid data d i of the i-level organization. n i is the number of initial grid data of the i-th level organization, m i is the number of target grid data of the i-th level organization, and n i > m i ; obtaining initial mesh points based on the initial mesh data in the initial mesh region, and obtaining target mesh points based on the target mesh data in the target mesh region, and the non-rigid deformation of the initial mesh region in the to-be-reconstructed space and the reconstruction of a high-resolution surface model of the to-be-reconstructed space specifically comprises: mapping the target mesh points in the to-be-reconstructed space to the initial mesh points one by one, and saving the mapping relationship of all points as a mapping set F satisfying the injective requirement: wherein and satisfies Taking the initial grid points with mapping relationship in the mapping set F as key points, all initial grid points in the space to be reconstructed are interpolated to the target grid region by inverse mapping, and new target point cloud data is formed in the target grid region: and The new target point cloud data after interpolation expansion in the space to be reconstructed is subjected to three-dimensional gridding processing, so that it meets the condition that the common edges of adjacent grids remain boundary continuous, and a high-resolution curved surface model of the space to be reconstructed is reconstructed.

2. A dynamic three-dimensional reconstruction system for liver resection, characterized in that The dynamic three-dimensional reconstruction method for liver resection according to the method of claim 1, comprising: A preoperative model gridding processing module is configured to acquire preoperative CT images, perform gridding processing on contour data of whole liver tissue in the preoperative CT images, and obtain initial grid data; An intraoperative image acquisition module is configured to acquire two-dimensional projection images of a surgical region obtained by an intraoperative CT at at least two rotation angles; An intraoperative image gridding processing module is configured to perform three-dimensional gridding processing on contour data of whole liver tissue in the two-dimensional projection images, and obtain target grid data; A whole liver tissue rigid registration module is configured to construct curved surface models using the target grid data and the initial grid data, respectively, and perform rigid registration; A space to be reconstructed division module is configured to divide a model curved surface part with a larger deformation region into a space to be reconstructed; A space to be reconstructed high-resolution interpolation module is configured to determine an initial grid region in the space to be reconstructed based on the initial grid data, determine a target grid region in the space to be reconstructed based on the target grid data, and perform non-rigid deformation on the initial grid region in the space to be reconstructed; A deformation region high-resolution curved surface reconstruction module is configured to reconstruct a high-resolution curved surface model of the space to be reconstructed; A dynamic reconstruction result visualization module is configured to replace a corresponding part of a preoperative whole liver tissue precise three-dimensional curved surface model with the generated high-resolution curved surface model of the space to be reconstructed, and perform visualization.

3. The dynamic three-dimensional reconstruction system for hepatectomy according to claim 2, wherein The specific execution content of the preoperative model gridding processing module is: Three-dimensional reconstruction of whole liver tissue is performed using a CT before surgery, and contour data of each level of tissue in the whole liver tissue is converted into initial point cloud data P with different identifiers; the initial point cloud data of each level of tissue is triangulated using a Delaunay triangulation algorithm to obtain triangulated initial grid data of each level of tissue; wherein the initial point cloud data of the i-th level of tissue is: N i N is the number of initial point cloud data for the i-th level organization.

4. The dynamic three-dimensional reconstruction system for hepatectomy according to claim 2, wherein The specific execution content of the intraoperative image acquisition module is: The intraoperative CT is scanned at at least two rotation angles to obtain two-dimensional projection images of the surgical region; The specific execution content of the intraoperative image gridding processing module is: Threshold segmentation and edge detection algorithms are used on the two-dimensional projection images to obtain contour data of each level of tissue of the whole liver tissue, and the contour data of each level of tissue is converted into target point cloud data D according to the scanning parameters of the intraoperative CT; the target point cloud data of each level of tissue is subjected to three-dimensional gridding processing to obtain triangulated target grid data of each level of tissue; wherein the target point cloud data of the i-th level of tissue is: M i The number of target point cloud data for the i-th level organization.

5. The dynamic three-dimensional reconstruction system for hepatectomy according to claim 2, wherein, The specific execution content of the whole liver tissue rigid registration module is: The ICP algorithm is used for rigid registration between the target grid data and the initial grid data at each level of organization, and a point pair is obtained through mapping between the target grid data and the initial grid data; The specific execution content of the to-be-reconstructed space division module is: The point-to-point Euclidean distance and are taken as the objective function, and the organization with the minimum objective function in each level is taken as the reference, the target grid data of other organizations in each level and the initial grid data are added to obtain a region with large deformation in the liver region, and the region is divided into a to-be-reconstructed space; wherein the to-be-reconstructed space contains the initial grid data p i of the i-level organization and the target grid data d i of the i-level organization. n i is the number of initial grid data of the i-th level organization, m i is the number of target grid data of the i-th level organization, and n i >>m i .

6. The dynamic three-dimensional reconstruction system for hepatectomy according to claim 5, wherein An initial grid point is obtained based on the initial grid data in the initial grid region, and a target grid point is obtained based on the target grid data in the target grid region, and the specific execution content of the to-be-reconstructed space high-resolution interpolation module is: The target grid points in the to-be-reconstructed space are one-to-one mapped to the initial grid points, and the mapping relationship of all points is saved as a mapping set F satisfying the one-to-one mapping requirement: wherein and satisfies Taking the initial grid points having the mapping relationship in the mapping set F as key points, all the initial grid points in the to-be-reconstructed space are inversely mapped to the target grid region to form new target point cloud data in the target grid region: and The specific execution content of the deformation region high-resolution surface reconstruction module is: The new target point cloud data in the to-be-reconstructed space after the interpolation expansion is subjected to three-dimensional gridding processing, so as to meet the condition that the public edges of adjacent grids remain boundary continuous, and a high-resolution surface model of the to-be-reconstructed space is reconstructed.

Images