Liver tissue and vessel deformation prediction method, device, equipment, medium and product before liver tumor resection

Abstract

The invention discloses a liver tissue and vessel deformation prediction method, device, equipment, medium and product before liver tumor resection, and relates to the field of medical image processing, the method comprises the following steps: obtaining preoperative liver iconography data and postoperative liver iconography data of the liver tumor resection of a patient; constructing preoperative and postoperative liver three-dimensional models; constructing preoperative and postoperative liver finite element models; performing simulation analysis on the preoperative and postoperative liver finite element models to obtain two simulation results; judging whether the two are consistent; if yes, pairing is carried out, and a database is constructed; constructing a data set based on the database; constructing a neural network model; training a neural network model based on the data set; and predicting the displacement and stress of the cutting position in the liver tumor cutting operation based on the trained neural network model. In the planning process of the liver tumor resection operation, the displacement conditions of surrounding tissues and blood vessels after tumor resection can be predicted, and a basis is provided for operation planning of doctors.

Description

Technical Field

[0001] The present application relates to the field of medical image processing technology, and in particular to a method, device, equipment, medium and product for predicting liver tissue and vascular deformation before liver tumor resection. Background Art

[0002] Liver tumor resection surgery is one of the main treatments for various common benign and malignant liver diseases. However, due to the diversity of the location, size and shape of liver tumors, as well as the complexity of liver blood vessels, there are often problems such as inaccurate vascular positioning, high surgical difficulty, and postoperative complications during surgery. In addition, the texture of liver tissue is relatively soft, and the characteristics of its mechanical behavior are equivalent to the superposition of the properties of multiple types of tissues. These tissues work together and their properties are close to those of composite materials. During the clinical intervention of liver disease, organs are subjected to a variety of force loads, such as shear, tension, compression, or a combination of these. Based on the above, it is difficult to quantify the prognostic effect of expected interventional surgery based solely on the preoperative physician's experience.

[0003] Finite element simulation provides a powerful analytical tool for liver tumor resection surgery. It can provide in-depth analysis of the mechanical properties of liver tissue, optimize surgical strategies, and reduce potential risks by building a three-dimensional model of the liver and simulating the various mechanical stresses it bears during surgery, such as shear force, tensile force, and compressive force. The premise for reliable finite element simulation results is that the constitutive model used must take into account the various stiffnesses and different loading conditions of liver tissue. Due to the complexity of the liver organ structure, such as the influence of porosity in liquid (blood and bile) and thin-walled tissue, and the slow load application speed and small strain dissipation during surgical simulation, liver tissue can be regarded as a hyperelastic material, so the most realistic liver model is a viscous-porous-hyperelastic material. At present, the models for material properties in general commercial software for simulation purposes are relatively mature. Compared with the assignment of material property parameters, the complexity of numerical processing methods and the high time-consuming numerical simulation process are part of the reasons why this method is difficult to be applied on a large scale in liver surgery navigation, especially when patients need surgery in a short time. In addition, numerical simulation involves a lot of scientific calculations, which usually require high-configuration computer hardware support, such as GPU acceleration, to improve computing efficiency; the selection and optimization of numerical processing methods will also directly affect the simulation time. In order to reduce the consumption of simulation calculations, engineering usually uses GPU or other parallel computing resources to process multiple computing tasks at the same time, optimize numerical methods, and reasonably initialize data structures and parameters to reduce unnecessary calculation steps, etc., to reduce the time consumption of numerical simulation and improve computing efficiency. In recent years, deep learning has significant advantages in processing complex nonlinear patterns, is more efficient in high-dimensional data processing and feature extraction, and has powerful end-to-end learning capabilities. Deep learning can not only automatically learn complex features from large amounts of data, but also capture high-level structures between data, showing significant performance improvements over traditional machine learning methods. Summary of the invention

[0004] The purpose of this application is to provide a method, device, equipment, medium and product for predicting liver tissue and vascular deformation before liver tumor resection, which can predict the characteristics of the tumor and the displacement of surrounding tissues and blood vessels during the planning process of liver tumor resection surgery.

[0005] To achieve the above objectives, this application provides the following solutions:

[0006] In a first aspect, the present application provides a method for predicting liver tissue and vascular deformation before liver tumor resection, the method for predicting liver tissue and vascular deformation before liver tumor resection comprising:

[0007] Obtain the patient's preoperative and postoperative liver imaging data of liver tumor resection;

[0008] Constructing a preoperative three-dimensional liver model based on the preoperative liver imaging data; the preoperative three-dimensional liver model includes: a three-dimensional liver tissue model, a three-dimensional tumor model, and a three-dimensional vascular system model;

[0009] constructing a postoperative three-dimensional liver model based on the postoperative liver imaging data;

[0010] constructing a preoperative liver finite element model based on the preoperative liver three-dimensional model;

[0011] constructing a postoperative liver finite element model based on the postoperative liver three-dimensional model;

[0012] Performing simulation analysis on the preoperative liver finite element model to obtain a first simulation result; the first simulation result includes: cutting depth, applied load information, and stress-strain distribution of liver tissue and blood vessels during the cutting process;

[0013] Performing simulation analysis on the post-operative liver finite element model to obtain a second simulation result;

[0014] Determining whether the first simulation result is consistent with the second simulation result;

[0015] If they are consistent, pairing the first simulation result with the second simulation result;

[0016] Building a database based on the paired first simulation result and the second simulation result;

[0017] constructing a data set based on the database;

[0018] Build a neural network model;

[0019] Training a neural network model based on the data set;

[0020] The tissue / vascular displacement and stress distribution induced by different cutting positions during liver tumor resection surgery were predicted based on the trained neural network model.

[0021] Optionally, constructing a preoperative three-dimensional liver model based on the preoperative liver imaging data specifically comprises the following steps:

[0022] Acquire the geometric information of the patient's liver tissue according to the preoperative liver imaging data; the geometric information of the liver tissue includes: the overall shape and size of the liver tissue, and the shape and size of the lesion area;

[0023] Analyzing the imaging data of the patient's liver based on the geometric information of the patient's liver tissue to obtain a preliminary segmentation result of the liver tissue and the tumor tissue;

[0024] Post-processing the preliminary segmentation results using a global optimization strategy;

[0025] Convert the initial segmentation result after post-processing into a binary image;

[0026] Converting the binary image into three-dimensional voxels; each voxel represents a marker of a corresponding position in space, wherein the marker is a liver or a tumor;

[0027] Use 3D visualization software to load 3D voxel data and generate 3D liver tissue models and 3D tumor models;

[0028] Acquire liver vascular imaging data and segment liver vascular images;

[0029] Extracting the centerline of the blood vessel from the segmented blood vessel image to form a blood vessel branching map;

[0030] Establish a vascular network diagram based on the vascular branch diagram, and calculate four evaluation criteria for each vascular connection; the evaluation criteria include: image intensity, vascular diameter, angle, and distance from the vascular root;

[0031] Based on the evaluation criteria, determine whether the blood vessels in the blood vessel network diagram belong to different blood vessels, and obtain a determination result;

[0032] Based on the determination result, the blood vessel branches are marked, the blood vessel branches not connected to the blood vessel root are removed, and the intersection positions are extracted to identify the blood vessel intersection points;

[0033] For each vascular intersection, the vascular voxels around the vascular intersection were extracted, and the intersecting vascular segments were considered as two intersecting cylinders;

[0034] The RANSAC algorithm is used to extract samples from the blood vessel skeleton and optimize the center line of the cylinder;

[0035] Mark the optimized vascular branches according to the blood flow direction;

[0036] Generate a topological map of blood vessels based on the labeling results.

[0037] Optionally, the neural network model includes:

[0038] Input layer, first fully connected layer, second fully connected layer, third fully connected layer, fourth fully connected layer and output layer;

[0039] The input layer, the first fully connected layer, the second fully connected layer, the third fully connected layer, the fourth fully connected layer and the output layer are connected in sequence.

[0040] Optionally, the loss function used in training the neural network model is:

[0041] Loss = Ldata +L physicsU +L physicsT ;

[0042] Among them, L data is the data loss, L physicsU is the loss of strain energy, L physicsT is the stress loss.

[0043] Optionally, constructing a data set based on the database specifically includes:

[0044] Randomly select a number of instances from the database;

[0045] The examples are divided into a training set and a validation set according to a ratio of 7.5:2.5.

[0046] Optionally, constructing a preoperative liver finite element model based on the preoperative liver three-dimensional model specifically comprises the following steps:

[0047] The first-order and second-order Ogden models were used to describe the liver tissue material properties;

[0048] Meshing the three-dimensional liver model;

[0049] Apply boundary conditions and loads to the meshed 3D liver model.

[0050] In a second aspect, the present application provides a device for predicting liver tissue and vascular deformation before liver tumor resection, the device for predicting liver tissue and vascular deformation before liver tumor resection comprising:

[0051] An imaging data acquisition module, used to acquire preoperative liver imaging data and postoperative liver imaging data of a patient undergoing liver tumor resection;

[0052] A first three-dimensional model construction module is used to construct a preoperative three-dimensional liver model based on the preoperative liver imaging data; the preoperative three-dimensional liver model includes: a three-dimensional liver tissue model, a three-dimensional tumor model, and a three-dimensional vascular system model;

[0053] A second three-dimensional model construction module is used to construct a postoperative three-dimensional liver model based on the postoperative liver imaging data;

[0054] A first finite element model construction module is used to construct a preoperative liver finite element model based on the preoperative liver three-dimensional model;

[0055] A second finite element model construction module is used to construct a postoperative liver finite element model based on the postoperative liver three-dimensional model;

[0056] A first simulation analysis module is used to perform simulation analysis on the preoperative liver finite element model to obtain a first simulation result; the first simulation result includes: cutting depth, applied load information, and stress-strain distribution of liver tissue and blood vessels during the cutting process;

[0057] A second simulation analysis module is used to perform simulation analysis on the post-operative liver finite element model to obtain a second simulation result;

[0058] A judging module, used for judging whether the first simulation result is consistent with the second simulation result;

[0059] A pairing module, used for pairing the first simulation result with the second simulation result when they are consistent;

[0060] A database construction module, used to construct a database based on the paired first simulation result and the second simulation result;

[0061] A data set construction module, used to construct a data set based on the database;

[0062] Neural network model building module, used to build a neural network model;

[0063] A training module, used for training a neural network model based on the data set;

[0064] The prediction module is used to predict the tissue / vascular displacement and stress distribution induced by different cutting positions during liver tumor resection surgery based on the trained neural network model.

[0065] In a third aspect, the present application provides a computer device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of any of the above-mentioned methods for predicting liver tissue and vascular deformation before liver tumor resection.

[0066] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-mentioned methods for predicting liver tissue and vascular deformation before liver tumor resection.

[0067] In a fifth aspect, the present application provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of any of the above-mentioned methods for predicting liver tissue and vascular deformation before liver tumor resection.

[0068] According to the specific embodiments provided in this application, this application discloses the following technical effects:

[0069] The present application provides a method, device, equipment, medium and product for predicting liver tissue and vascular deformation before liver tumor resection. Through the paired database construction method provided by the present invention, based on batch data analysis and model training, doctors can obtain detailed information on the displacement and deformation of normal liver tissue and vascular system after tumor tissue resection before surgery, assisting doctors in planning surgical plans, especially the location, size and relationship of lesions with surrounding tissues, so as to determine the pathological boundaries of the target disease and the necessary resection range; ensure the necessary functional liver volume and the necessary retention range; determine the vascular structure, function and remaining functional liver volume of the reserved liver, determine the best liver resection method, and preoperatively. Navigation can help doctors avoid important blood vessels, reduce the risk of massive bleeding during surgery, and improve the patient's quality of life after surgery. Since the damage to normal liver tissue during surgery is reduced, the probability of postoperative complications such as bleeding, liver failure, biliary insufficiency, and infection in patients is greatly reduced, allowing patients to recover faster and have a higher quality of life after surgery. The surgical prediction method provided by the present invention can implement a variety of surgical plans through an interactive interface before surgery and obtain surgical results in real time. This method can quickly plan the most appropriate surgical path, allowing doctors to find a path to reach the tumor site in the shortest time while avoiding blood vessels, thereby reducing surgical time and improving surgical efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0071] Figure 1 This is a flow chart of a method for predicting liver tissue and vascular deformation before liver tumor resection in one embodiment of the present application;

[0072] Figure 2 This is a rendering of the reading of the liver CT original image, and the segmentation of tissue and vascular system in one embodiment of the present application;

[0073] Figure 3 This is a liver vascular system segmentation effect diagram in one embodiment of the present application;

[0074] Figure 4 A schematic diagram of a physical information based neural network model architecture used in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0077] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0078] Figure 1 FIG. 1 is a flow chart of a method for predicting liver tissue and vascular deformation before liver tumor resection in an embodiment of the present application, see Figure 1 , the method in this application includes:

[0079] Step 101: Obtain preoperative liver imaging data and postoperative liver imaging data of a patient undergoing liver tumor resection.

[0080] The imaging data may be a CT image.

[0081] Step 102: constructing a preoperative three-dimensional liver model based on the preoperative liver imaging data; the preoperative three-dimensional liver model includes: a three-dimensional liver tissue model, a three-dimensional tumor model, and a three-dimensional vascular system model.

[0082] Specifically, CT imaging data is used to obtain geometric information of the patient's liver tissue. The geometric information includes the overall shape and size of the liver tissue, the shape and size of the lesion area (tumor), and the liver tumor and surrounding important tissues are extracted by an image processing algorithm. The image processing algorithm is an automatic liver and tumor segmentation algorithm based on deep learning and global optimization refinement. The trained deep learning model is used to analyze the CT image of the patient's liver to obtain preliminary segmentation results of the liver tissue and tumor tissue. A global optimization strategy (conditional random field-CRF, based on the preliminary segmentation results, CRF optimizes the segmentation boundary by considering the relationship between pixels, reduces noise and false positives, and improves the smoothness and consistency of the segmentation) is used to post-process the preliminary segmentation results to improve the segmentation accuracy. The segmentation results are then converted into binary images (liver and tumor areas are marked as 1, and background is 0), and then the binary images are converted into three-dimensional voxels, each voxel represents a label (liver or tumor) at the corresponding position in space, and finally a three-dimensional visualization software (3D Slicer) is used to load the voxel data and generate / export a three-dimensional model. For details, see Figure 2 At this point, the three-dimensional model of liver tissue and the three-dimensional model of tumor have been constructed.

[0083] The following describes the process of building a 3D model of the vascular system:

[0084] First, based on the CT image of liver blood vessels, the liver blood vessel image is segmented, and then the blood vessel network is extracted. The specific steps for extracting the blood vessel network are: first, the center line of the blood vessel is extracted from the segmented blood vessel image to form a blood vessel branch map; second, the blood vessel network map is established, and the four evaluation criteria for each blood vessel connection, image intensity, blood vessel diameter, angle and distance from the blood vessel root, are calculated to determine whether they may belong to different blood vessels; finally, the blood vessel branches are marked, and the blood vessel branches that are not connected to the blood vessel root are removed, and the intersection positions are extracted to identify the blood vessel intersections. Next, the blood vessel branches near the blood vessel intersections are modeled and marked by the two-line RANSAC algorithm. The specific steps are: first, for each intersection, the surrounding blood vessel voxels are extracted, and the intersecting blood vessel segments are regarded as two intersecting cylinders; next, the RANSAC algorithm is used to extract samples from the blood vessel skeleton and optimize the center line of the cylinder to adapt to the surrounding blood vessel shape; finally, these branches are marked according to the blood flow direction. In the process of separating the main blood vessel trunk and branches, Murray's law is combined to ensure that the geometric characteristics of the blood vessel are consistent with the physiological principles. Finally, a topological map of the blood vessels is generated based on the marking results to show the connection relationship and structural characteristics of the blood vessels, as shown in the following figure. Figure 3 As shown, Figure 3 Part (a) is the vascular skeleton structure, part (b) is the corresponding vascular branch, lines of different colors represent different branches, and part (c) is the vascular element. Each vascular element is represented by a point cloud, and the color of the point cloud is the same as the color of the corresponding vascular branch in part (b).

[0085] Step 103: constructing a postoperative three-dimensional liver model based on the postoperative liver imaging data.

[0086] The construction of the postoperative three-dimensional liver model is the same as that before the operation and will not be repeated here.

[0087] Step 104: constructing a preoperative liver finite element model based on the preoperative liver three-dimensional model.

[0088] The specific steps include:

[0089] (1) Establishment of a three-dimensional finite element model of liver tissue: The finite element analysis performed in the present invention involves high nonlinearity of material properties, which is a natural characteristic of the liver under load conditions. The present invention uses the commercial software Abaqus to perform finite element simulation on the model because it has a strong ability to handle nonlinear problems. In addition, standard hyperelastic models such as Ogden, Mooney Rivlin, and Yeoh have been implemented in the software. The significant advantage of using this software is that the entire finite element simulation starting from the preprocessing steps (including meshing, loading, material allocation, and post-processing) can be programmed in Python, and it also provides the flexibility to select the load conditions imposed on the organ.

[0090] (2) Define material properties: The first-order and second-order Ogden models are used to describe the material properties of liver tissue. The model can provide the best fit of the material stress-strain curve. The model material parameters define the strain energy function of the material based on the Ogden formula. The evaluation system provided by the Abaqus software package is used to evaluate whether the provided material parameters are stable under specific strain states.

[0091] The liver is considered to be isotropic, hyperelastic and incompressible. The mechanical response of this type of material is described by the strain energy function U, from which the constitutive behavior of the material can be derived. In the present invention, the average Young's modulus of the liver is set to 10 kPa and the Poisson's ratio is 0.5. In order to simplify the analysis, the first-order Ogden model is used as the constitutive equation:

[0092]

[0093] Among them, μ and α are material constants, λ1, λ2, and λ3 are the principal stretches in three directions. The stretches in the three principal directions refer to the strains of the material in the three mutually perpendicular principal stress directions. λ1 is along the direction of the longest axis of the material, corresponding to the maximum principal stretch. If the material is a cuboid, this direction is usually along the length of the cuboid; λ2 is along the direction of the middle axis of the material, corresponding to the middle principal stretch. In cuboid materials, this direction is usually along the width of the cuboid; λ3 is along the direction of the shortest axis of the material, corresponding to the minimum principal stretch. In cuboid materials, this direction is usually along the height of the cuboid. The nominal stress T of the first-order Ogden model of the uniaxial test in the Abaqus software is given by the following formula:

[0094]

[0095] (3) Mesh division: The surface of the model is composed of a limited number of triangular facets, so the tetrahedral mesh selected by the present invention will not affect the model surface configuration and better fit the complex geometric shape. After the mesh sensitivity test, the model is meshed. The vascular system such as blood vessels in the liver has a complex topological structure and plays a vital role in the liver's response to external stimuli. Therefore, the present invention also incorporates the vascular system and tumor tissue into the finite element model. Since the hepatic vascular system is composed of many complex parts, we use an automatic extraction algorithm, a hepatic vascular segmentation algorithm based on Twin-Line RANSAC and Murray's law, that is, first extract the vascular image through image processing, then fit the vascular path through twin-line RANSAC, identify the intersection, verify the rationality of the vascular branch through Murray's law, optimize the results, generate a high-precision vascular tree structure, and obtain a three-dimensional model of the hepatic vascular system.

[0096] The simplified vascular structure was subtracted from the main liver tissue, and a shell element with a thickness of 2 mm was assigned to its contact surface with the liver tissue. The tumor tissue was represented separately with the actual model result segmented. Then the parenchyma, blood vessels, and tumor were meshed simultaneously to ensure that their common surfaces were consistent. Based on the Ogden model, μ = 0.0196 MPa and α = 10.3043 were assigned. For tumor tissue, areas with higher cell density in tumor tissue usually have higher elastic moduli, and areas with collagen-rich extracellular matrix show higher stiffness. In general, benign liver tumors (such as hepatic hemangiomas, cysts, etc.) usually present relatively low-density images, and their HU values ​​are generally between 0 and 20 HU. In contrast, malignant liver tumors (such as hepatocellular carcinoma, metastatic tumors, etc.) usually present relatively high-density images, and their HU values ​​may be between 20 and 80 HU, or even higher. However, the HU value cannot determine the nature of the tumor alone. Due to the lack of reliable data on the mechanical properties of cancer tissue, the same parameters as those of parenchyma were used. However, in general, tumors are known to be stiffer than their surrounding liver tissues.

[0097] (4) Applying boundary conditions and loads: Boundary conditions are applied to the meshed three-dimensional liver model. The human liver is in contact with several organs, which makes it difficult to find anatomically realistic model boundary conditions. The method in the present invention is based on relevant literature reports. Two different positions of the liver are considered to be fixed and have no displacement. The present invention adopts a method of fixing the grid nodes that are in contact with or close to the inferior vena cava. Another area that can also be included in the boundary conditions is the point located on the falciform ligament, which separates the left and right lobes and connects to the diaphragm and abdominal wall. The method of the present invention is that only the lymph nodes near the vena cava are restricted in all degrees of freedom, and the remaining lymph nodes are free, for example:

[0098] (a) Apply a displacement constraint to the bottom of the liver to fix the liver;

[0099] (b) applying displacement load to the lesion area to simulate the surgical procedure;

[0100] (c) define contact pairs and set contact properties (such as friction coefficient, contact type, etc.) to simulate the interaction between the scalpel and tissue;

[0101] (d) Force load is applied to the area where the scalpel acts to simulate the force applied by the doctor on the tissue during tissue removal.

[0102] Step 105: constructing a postoperative liver finite element model based on the postoperative liver three-dimensional model.

[0103] The method for constructing the postoperative liver finite element model is the same as that before the operation and will not be repeated here.

[0104] Step 106: Perform simulation analysis on the preoperative liver finite element model to obtain a first simulation result. The first simulation result includes: cutting depth, applied load information, and stress-strain distribution of liver tissue and blood vessels during the cutting process.

[0105] This step is the finite element solution simulation. The finite element solution is performed using Abaqus software to obtain the stress-strain distribution of liver lesion tissue during the cutting process. The nonlinear problem is analyzed using the implicit method, and a static general step that can handle linear, nonlinear and quasi-static problems is selected. This step does not consider the inertial effect and ignores the time dependence in material effects (such as creep and viscoelasticity), but it can be used for hyperelastic problems. In addition, in nonlinear analysis, it is crucial that the finite element solver divides each step into multiple time increments.

[0106] The present invention uses Abaqus to perform finite element simulation on the preoperative and postoperative models of the patient. The preoperative model is used to simulate the surgical process. According to the location and range of the tumor extracted from the imaging, combined with the guidance of experienced clinicians, the surgical incision and resection range in the simulated surgery are determined, and the displacement of the blood vessels is predicted after the incision is made and sutured. The finite element simulation is performed on the postoperative model of the patient, and the simulation results are used as the gold standard to verify the correctness of the simulation results of the preoperative model.

[0107] Step 107: Perform simulation analysis on the post-operative liver finite element model to obtain a second simulation result.

[0108] Step 108: Determine whether the first simulation result is consistent with the second simulation result.

[0109] Step 109: If they are consistent, pair the first simulation result and the second simulation result.

[0110] Step 110: construct a database based on the paired first simulation result and the second simulation result.

[0111] The specific steps include:

[0112] (1) Construction of platform modules

[0113] The database platform consists of PCs (such as servers) and smart terminals, providing interfaces for data uploading, processing and analysis. Doctors upload patients' liver imaging data and diagnostic reports through these terminals.

[0114] (2) Identity verification

[0115] The doctor submits an access request through the database platform, which will be reviewed by the system. After the review is passed, the system will further verify the doctor's identity based on the identity information in the access request. After the identity verification is successful, the system will feedback the review pass information and allow the doctor to formally access the database platform. At this time, the platform module will establish a secure communication link with the requester's request device. This communication link will ensure the security and privacy protection of data interaction. After the communication link is established, the requester accesses the platform and enters the data interaction interface.

[0116] (3) Data upload and interaction

[0117] Doctors upload patients' pre- and post-operative liver imaging data and diagnostic reports through the terminal.

[0118] (4) Finite element analysis

[0119] Engineers perform batch analysis and processing of the patient's preoperative and postoperative imaging data based on the data uploaded by the doctor, and use the finite element simulation results of the patient's postoperative model as the gold standard to verify the correctness of the results obtained by the preoperative model during the simulated surgery. After batch analysis of the data, if the simulation results after the simulated surgery match the simulation results of the patient's actual postoperative surgery, the preoperative and postoperative paired data and simulation results are uploaded to the database; if the paired data simulation results are inconsistent, the engineer needs to optimize the simulated surgery process in combination with the advice of experienced doctors and the preoperative diagnosis report, and continue to analyze according to steps 1-4.

[0120] Step 111: construct a data set based on the database.

[0121] The results calculated by Abaqus in the database obtained in the previous step are used as the training set of the deep learning network. The specific method is: based on the three-dimensional liver model and the results of finite element analysis (including cutting depth, applied load information, and stress-strain information at each node), a data set is constructed as the input information of the deep learning algorithm. The data set includes input data and corresponding output labels. The input data is the location, size and shape of the liver tumor, as well as the conditions of the surrounding tissues and blood vessels. The output labels are the optimal cutting path and cutting depth. The database established using the paired data before and after the operation improves the accuracy of the finite element results obtained by the preoperative simulation operation, and provides a more accurate training set for the training of the neural network driven by physical information. In actual use, the data sets are allocated to the training and validation subsets according to a specific ratio (75% / 25%).

[0122] Step 112: Construct a neural network model.

[0123] Based on the database in the previous step, the preoperative and postoperative simulation results of liver tumor resection surgery performed on a three-dimensional model using Abaqus finite element analysis software were used as the data set for training physical information neural networks (PINNs), and a neural network architecture that can quickly predict the postoperative effect of liver surgery was obtained. For details, see Figure 4 The network structure includes multiple hidden layers, and the number of neurons in each layer can be adjusted according to the accuracy requirements of the data and the hardware used for training. An exemplary network structure may be as follows:

[0124] Input layer: Input dimension: 9

[0125] The first fully connected layer: contains 1000 neurons, activation function ReLU, output dimension 1000

[0126] The second fully connected layer: contains 1000 neurons, activation function ReLU, output dimension 1000

[0127] The third fully connected layer: contains 1000 neurons, activation function ReLU, output dimension 1000

[0128] The fourth fully connected layer: contains 1000 neurons, activation function ReLU, output dimension 1000

[0129] Output layer: contains 4 neurons, activation function ReLU, output dimension 4.

[0130] Step 113: Train a neural network model based on the data set.

[0131] The constructed data set is input into the deep learning network for training. During the training process, the network will learn the mapping relationship between the input data and the output label, thereby optimizing the weights and parameters of the network. During the training process, optimization methods such as back propagation algorithm and gradient descent algorithm can be used to improve the performance of the network.

[0132] The loss function used in the training process is as follows:

[0133] Loss = L data +L physicsU +L physicsT ;

[0134] Among them, L data is the data loss, L physicsU is the loss of strain energy, L physicsT is the stress loss.

[0135] Step 114: Predict tissue / vascular displacement and stress distribution induced by different cutting positions during liver tumor cutting surgery based on the trained neural network model.

[0136] Through the above steps, the present invention provides a method for simulating liver tumor cutting surgery based on physical information neural network, which is implemented by constructing a data set and training the network. The network is driven by the joint simulation data and physical information, learns complex nonlinear patterns from a large amount of data, and embeds physical equations into the training process of the neural network to ensure that the prediction results conform to the established physical laws. This method can quickly and accurately predict the displacement and stress of the cutting position in liver tumor cutting surgery. Compared with the simple finite element simulation method, it can quickly provide doctors with displacement information of the cutting position to assist in surgical planning and implementation.

[0137] The surgical plan is finally determined, and based on the prediction results of the physical information neural network, navigation information for liver tumor resection is provided to doctors, including surgical path, resection depth, etc., to guide doctors to accurately and safely remove the tumor with as little damage to normal tissues and blood vessels as possible during the operation.

[0138] Based on the same inventive concept, the embodiment of the present application also provides a device for predicting the deformation of liver tissue and blood vessels before liver tumor resection as mentioned above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the embodiments of one or more devices for predicting the deformation of liver tissue and blood vessels before liver tumor resection provided below can refer to the limitations of the method for predicting the deformation of liver tissue and blood vessels before liver tumor resection above, and will not be repeated here.

[0139] In an exemplary embodiment, a device for predicting liver tissue and vascular deformation before liver tumor resection is provided, comprising:

[0140] The liver tissue and vascular deformation prediction device before liver tumor resection includes:

[0141] An imaging data acquisition module, used to acquire preoperative liver imaging data and postoperative liver imaging data of a patient undergoing liver tumor resection;

[0142] A first three-dimensional model construction module is used to construct a preoperative three-dimensional liver model based on the preoperative liver imaging data; the preoperative three-dimensional liver model includes: a three-dimensional liver tissue model, a three-dimensional tumor model, and a three-dimensional vascular system model;

[0143] A second three-dimensional model construction module is used to construct a postoperative three-dimensional liver model based on the postoperative liver imaging data;

[0144] A first finite element model construction module is used to construct a preoperative liver finite element model based on the preoperative liver three-dimensional model;

[0145] A second finite element model construction module is used to construct a postoperative liver finite element model based on the postoperative liver three-dimensional model;

[0146] A first simulation analysis module is used to perform simulation analysis on the preoperative liver finite element model to obtain a first simulation result; the first simulation result includes: cutting depth, applied load information, and stress-strain distribution of liver tissue and blood vessels during the cutting process;

[0147] A second simulation analysis module is used to perform simulation analysis on the post-operative liver finite element model to obtain a second simulation result;

[0148] A judging module, used for judging whether the first simulation result is consistent with the second simulation result;

[0149] A pairing module, used for pairing the first simulation result with the second simulation result when they are consistent;

[0150] A database construction module, used to construct a database based on the paired first simulation result and the second simulation result;

[0151] A data set construction module, used to construct a data set based on the database;

[0152] Neural network model building module, used to build a neural network model;

[0153] A training module, used for training a neural network model based on the data set;

[0154] The prediction module is used to predict the tissue / vascular displacement and stress distribution induced by different cutting positions during liver tumor resection surgery based on the trained neural network model.

[0155] In an exemplary embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 5 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store liver tissue and vascular deformation prediction data before liver tumor resection. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for predicting liver tissue and vascular deformation before liver tumor resection is implemented.

[0156] Those skilled in the art will understand that Figure 5 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In an exemplary embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0158] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0159] In an exemplary embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the above method embodiments are implemented.

[0160] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0162] The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a distributed database based on blockchain, etc., but is not limited thereto. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but is not limited thereto.

[0163] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0164] This article uses specific examples to illustrate the principles and implementation methods of this application. The description of the above embodiments is only used to help understand the method and core ideas of this application. At the same time, for those skilled in the art, according to the ideas of this application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A method for predicting liver tissue and vascular deformation before liver tumor resection, characterized in that: The method for predicting liver tissue and vascular deformation before liver tumor resection includes: Obtain the patient's preoperative and postoperative liver imaging data of liver tumor resection; Constructing a preoperative three-dimensional liver model based on the preoperative liver imaging data; the preoperative three-dimensional liver model includes: a three-dimensional liver tissue model, a three-dimensional tumor model, and a three-dimensional vascular system model; constructing a postoperative three-dimensional liver model based on the postoperative liver imaging data; constructing a preoperative liver finite element model based on the preoperative liver three-dimensional model; constructing a postoperative liver finite element model based on the postoperative liver three-dimensional model; Performing simulation analysis on the preoperative liver finite element model to obtain a first simulation result; the first simulation result includes: cutting depth, applied load information, and stress-strain distribution of liver tissue and blood vessels during the cutting process; Performing simulation analysis on the post-operative liver finite element model to obtain a second simulation result; Determining whether the first simulation result is consistent with the second simulation result; If they are consistent, pairing the first simulation result with the second simulation result; Building a database based on the paired first simulation result and the second simulation result; constructing a data set based on the database; Build a neural network model; Training a neural network model based on the data set; The tissue / vascular displacement and stress distribution induced by different cutting positions during liver tumor resection surgery were predicted based on the trained neural network model.

2. The method for predicting liver tissue and vascular deformation before liver tumor resection according to claim 1, characterized in that: Constructing a preoperative three-dimensional liver model based on the preoperative liver imaging data specifically includes the following steps: Acquire the geometric information of the patient's liver tissue according to the preoperative liver imaging data; the geometric information of the liver tissue includes: the overall shape and size of the liver tissue, and the shape and size of the lesion area; Analyzing the imaging data of the patient's liver based on the geometric information of the patient's liver tissue to obtain a preliminary segmentation result of the liver tissue and the tumor tissue; Post-processing the preliminary segmentation results using a global optimization strategy; Convert the initial segmentation result after post-processing into a binary image; Converting the binary image into three-dimensional voxels; each voxel represents a marker of a corresponding position in space, wherein the marker is a liver or a tumor; Use 3D visualization software to load 3D voxel data and generate 3D liver tissue models and 3D tumor models; Acquire liver vascular imaging data and segment liver vascular images; Extracting the centerline of the blood vessel from the segmented blood vessel image to form a blood vessel branching map; Establish a vascular network diagram based on the vascular branch diagram, and calculate four evaluation criteria for each vascular connection; the evaluation criteria include: image intensity, vascular diameter, angle, and distance from the vascular root; Based on the evaluation criteria, determine whether the blood vessels in the blood vessel network diagram belong to different blood vessels, and obtain a determination result; Based on the determination result, the blood vessel branches are marked, the blood vessel branches not connected to the blood vessel root are removed, and the intersection positions are extracted to identify the blood vessel intersection points; For each vascular intersection, the vascular voxels around the vascular intersection were extracted, and the intersecting vascular segments were considered as two intersecting cylinders; The RANSAC algorithm is used to extract samples from the blood vessel skeleton and optimize the center line of the cylinder; Mark the optimized vascular branches according to the blood flow direction; Generate a topological map of blood vessels based on the labeling results.

3. The method for predicting liver tissue and vascular deformation before liver tumor resection according to claim 1, characterized in that: The neural network model includes: Input layer, first fully connected layer, second fully connected layer, third fully connected layer, fourth fully connected layer and output layer; The input layer, the first fully connected layer, the second fully connected layer, the third fully connected layer, the fourth fully connected layer and the output layer are connected in sequence.

4. The method for predicting liver tissue and vascular deformation before liver tumor resection according to claim 1, characterized in that: The loss function used in training the neural network model is: Loss=L data +L physicsU +L physicsT ; Among them, L data is the data loss, L physicsU is the loss of strain energy, L physicsT is the stress loss.

5. The method for predicting liver tissue and vascular deformation before liver tumor resection according to claim 1, characterized in that: Constructing a data set based on the database specifically includes: Randomly select a number of instances from the database; The examples are divided into a training set and a validation set according to a ratio of 7.5:2.

5.

6. The method for predicting liver tissue and vascular deformation before liver tumor resection according to claim 1, characterized in that: Constructing a preoperative liver finite element model based on the preoperative liver three-dimensional model specifically includes the following steps: The first-order and second-order Ogden models were used to describe the liver tissue material properties; Meshing the three-dimensional liver model; Apply boundary conditions and loads to the meshed 3D liver model.

7. A device for predicting liver tissue and vascular deformation before liver tumor resection, characterized in that: The liver tissue and vascular deformation prediction device before liver tumor resection includes: An imaging data acquisition module, used to acquire preoperative liver imaging data and postoperative liver imaging data of a patient undergoing liver tumor resection; A first three-dimensional model construction module is used to construct a preoperative three-dimensional liver model based on the preoperative liver imaging data; the preoperative three-dimensional liver model includes: a three-dimensional liver tissue model, a three-dimensional tumor model, and a three-dimensional vascular system model; A second three-dimensional model construction module is used to construct a postoperative three-dimensional liver model based on the postoperative liver imaging data; A first finite element model construction module is used to construct a preoperative liver finite element model based on the preoperative liver three-dimensional model; A second finite element model construction module is used to construct a postoperative liver finite element model based on the postoperative liver three-dimensional model; A first simulation analysis module is used to perform simulation analysis on the preoperative liver finite element model to obtain a first simulation result; the first simulation result includes: cutting depth, applied load information, and stress-strain distribution of liver tissue and blood vessels during the cutting process; A second simulation analysis module is used to perform simulation analysis on the post-operative liver finite element model to obtain a second simulation result; A judging module, used for judging whether the first simulation result is consistent with the second simulation result; A pairing module, used for pairing the first simulation result with the second simulation result when they are consistent; A database construction module, used to construct a database based on the paired first simulation result and the second simulation result; A data set construction module, used to construct a data set based on the database; Neural network model building module, used to build a neural network model; A training module, used for training a neural network model based on the data set; The prediction module is used to predict the tissue / vascular displacement and stress distribution induced by different cutting positions during liver tumor resection surgery based on the trained neural network model.

8. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the method for predicting liver tissue and vascular deformation before liver tumor resection according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for predicting liver tissue and vascular deformation before liver tumor resection according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for predicting liver tissue and vascular deformation before liver tumor resection according to any one of claims 1 to 6 are implemented.

Images