Cardiac soft tissue model correction method and device based on physical information neural network

Abstract

The application discloses a heart soft tissue model correction method and device based on a physical information neural network, and comprises the following steps: acquiring preoperative images to construct a personalized heart biomechanical model and an active motion priori library; constructing a double-flow coupled physical information neural network, which comprises an active deformation prediction branch for predicting heart autonomous beats and a passive residual correction branch for correcting deformations caused by external contact forces; defining a loss function comprising a data matching loss and a biomechanical control equation loss, and training the network; receiving in-surgery collected cardiac phase data and sparse surface displacement data, inputting the data into the trained network, and outputting a global three-dimensional deformation field to update a preoperative three-dimensional model. The application effectively decouples the active and passive mixed deformation of the heart through a double-flow coupled architecture, and can infer the full-field volume deformation by only using sparse surface observation points by embedding physical control equations as strong constraints, thereby overcoming the data sparsity problem.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and medical image processing technology, specifically to a method and apparatus for correcting cardiac soft tissue models based on physical information neural networks. Background Technology

[0002] Augmented reality navigation systems play a crucial role in cardiac surgery, especially minimally invasive cardiac surgery. These systems can overlay occluded internal cardiac anatomical structures, such as coronary arteries and valves, onto the surgeon's field of vision as three-dimensional images, thereby improving the precision and safety of surgical procedures. However, high-precision augmented reality navigation systems face a key technical bottleneck: real-time deformation correction of cardiac soft tissues.

[0003] As a dynamic organ, the heart exhibits highly complex deformation. On one hand, the heart's own rhythmic beating generates periodic, large-amplitude active deformation. On the other hand, during surgery, surgical instruments (such as fixators and traction devices) contact, compress, and pull on cardiac tissue, inducing nonlinear passive deformation. These two types of deformation are closely coupled in time and space, resulting in significant geometric deviations between the preoperative three-dimensional anatomical model constructed based on static medical images (such as computed tomography (CT) or magnetic resonance imaging (MRI)) and the real-time cardiac morphology observed during surgery, thus reducing the reliability of navigation information.

[0004] To address these issues, existing technologies primarily employ two approaches. The first is a computer vision-based approach, which updates the geometric position of the model surface by tracking feature points on the heart's surface. The limitation of this approach is its inability to infer volumetric deformation of internal heart structures, and its robustness decreases significantly in the presence of interference such as blood or smoke within the surgical field. The second approach is a finite element analysis method based on biomechanical models. While this method offers advantages in physical accuracy, solving the governing equations describing the nonlinear hyperelastic behavior of soft tissue requires enormous computational resources; the time required for a single deformation calculation is typically on the order of minutes or even hours, far from meeting the stringent real-time requirements of surgical navigation (typically requiring latency of less than 40 milliseconds).

[0005] In recent years, Physics-Informed Neural Networks (PINNs) have been proposed to accelerate the solution of physical equations. This technique constrains the output of the neural network to conform to physical laws by using the residuals of the physical control equations as part of the loss function. However, directly applying existing PINN techniques to cardiac surgery navigation scenarios still faces challenges. First, existing models typically treat all deformations as a single physical process, failing to effectively distinguish and decouple active deformations caused by the heart's spontaneous pulsation from passive deformations caused by external instruments. This leads to physical ambiguity when predicting mixed deformations. Second, in minimally invasive surgery, the field of view obtained through endoscopy is limited, only observing local areas of the heart's surface. This results in highly sparse visual feature points available to drive the model. For traditional deep learning models that rely on dense boundary conditions, using such sparse surface displacement data to solve for the internal volume deformation of the entire heart is an extremely ill-conditioned inverse problem, making network convergence difficult or producing predictions that do not conform to physical laws. Therefore, providing a technical solution that can utilize sparse surface data to decouple and accurately correct the active and passive mixed deformation of the heart in real time is a technical problem that urgently needs to be solved in the current field. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide a method and device for correcting cardiac soft tissue model based on physical information neural network, which addresses the problems of insufficient real-time performance, inability to decouple active and passive deformation, and difficulty in utilizing sparse observation data in the existing technology when processing the deformation correction of cardiac soft tissue model in augmented reality surgical navigation system.

[0007] To address the aforementioned technical problems, embodiments of the present invention provide a method for correcting cardiac soft tissue models based on physical information neural networks, comprising the following steps:

[0008] Step 1: Based on the patient's preoperative time-series imaging data, construct a personalized biomechanical model of the heart and extract cardiac pulsation displacement field data to construct an active motion manifold space.

[0009] Step two: Construct a dual-flow coupled physical information neural network, which receives the spatial three-dimensional coordinates of the grid nodes of the cardiac biomechanical model, cardiac phase variables, and sparse boundary displacement vectors as inputs. The network includes: an active deformation prediction branch, used to predict the reference displacement field constrained by the active motion manifold space based on the spatial three-dimensional coordinates of the grid nodes and the cardiac phase variables, so as to conform to the physiological pulsation pattern of the heart; and a passive residual correction branch, used to solve the passive residual displacement field caused by the external contact force based on the spatial three-dimensional coordinates of the grid nodes, the sparse boundary displacement vector, and the reference displacement field; and superimpose the reference displacement field and the passive residual displacement field to obtain the total displacement field.

[0010] Step 3: Define a total loss function including data matching loss and physical control equation loss to train the network; the data matching loss calculates the error of the total displacement field predicted by the network at sparse observation points; the physical control equation loss calculates the residual after substituting the total displacement field into the soft tissue mechanical equilibrium equation, so as to achieve self-supervised learning under physical constraints.

[0011] Step 4: Receive ECG signal data and endoscopic video stream data collected during the operation, and determine the cardiac phase variable and sparse boundary displacement vector through signal processing and visual tracking;

[0012] Step 5: Input the cardiac phase variable and sparse boundary displacement vector into the trained network, perform forward propagation calculation on all grid vertices of the personalized biomechanical model to obtain the total displacement field, and use the total displacement field to update the three-dimensional morphology of the personalized biomechanical model.

[0013] Preferably, in step one, the construction of the active motion manifold space specifically includes: extracting the natural pulsation displacement field data of the heart in multiple consecutive cardiac cycles from the preoperative temporal image data, and performing dimensionality reduction processing on the natural pulsation displacement field data using principal component analysis or variational autoencoder, so as to obtain more stable statistical features by eliminating single-cycle random perturbations and extracting low-dimensional manifolds that characterize the spatiotemporal features of cardiac physiological motion, thereby constructing the low-dimensional active motion manifold space.

[0014] Preferably, in step two, the output of the active deformation prediction branch is subject to the regularization constraint of the active motion manifold space in the following way: the output layer of the active deformation prediction branch outputs a set of coordinate coefficients under the basis of the active motion manifold space, and the reference displacement field is obtained by multiplying the coordinate coefficients with the basis vectors of the active motion manifold space and summing them.

[0015] Furthermore, in step two, the network structure of the passive residual correction branch includes a graph neural network layer, which employs an anisotropic inductive graph message passing mechanism. During message passing, the anisotropic inductive graph message passing mechanism integrates myocardial fiber orientation field information obtained from the personalized biomechanical model. By defining an anisotropic graph convolution operator, when aggregating neighborhood node features, the aggregation weights are dynamically calculated based on the dot product relationship between the edge vectors connecting neighborhood nodes and the local myocardial fiber orientation vectors.

[0016] Furthermore, in step three, the soft tissue mechanical equilibrium equation is a static equilibrium equation, and the relationship between the total displacement field and the stress tensor is described by the Holzapfel-Ogden constitutive model, wherein the stress tensor is derived from the strain tensor calculated from the total displacement field.

[0017] Optionally, the training in step three includes an offline pre-training step; the offline pre-training step specifically involves: using the personalized biomechanical model to perform numerical simulation, by randomly applying virtual external forces of different positions, magnitudes and directions to the surface of the heart model, and combining them with randomly sampled cardiac phases to generate synthetic training samples for pre-training the dual-flow coupled physical information neural network.

[0018] Preferably, in step four, a visual tracking algorithm based on optical flow or sparse feature point matching is run on the endoscopic video stream to track natural texture points in the visible area of ​​the heart surface in real time, and the three-dimensional displacement vector of the natural texture points is calculated to form the sparse boundary displacement vector.

[0019] Optionally, the cardiac phase variable is a normalized value obtained by real-time intraoperative electrocardiogram signal after R-wave detection and period normalization processing, used to indicate the current systolic or diastolic phase of the heart.

[0020] Furthermore, it also includes the step of projecting the three-dimensional morphology of the updated personalized biomechanical model from step five onto the endoscopic image plane to generate an augmented reality navigation image.

[0021] To address the aforementioned technical problems, embodiments of the present invention also provide a cardiac soft tissue model correction device based on a physical information neural network, comprising:

[0022] One or more processors; and

[0023] A memory that stores computer instructions;

[0024] The one or more processors are configured to execute the steps of the above-described method for correcting cardiac soft tissue models based on physical information neural networks when executing the computer instructions.

[0025] Compared with the prior art, the present invention has the following advantages:

[0026] This invention constructs a dual-flow coupled physical information neural network architecture and introduces active motion manifold space constraints to decouple the active pulsating deformation of the heart from the passive contact deformation caused by instruments. This solves the problem of traditional methods struggling to handle mixed deformations, improving the accuracy and physical interpretability of the correction. Simultaneously, by embedding the biomechanical control equations describing myocardial tissue characteristics as strong constraints into the loss function of the neural network, the network can infer the volumetric deformation of the entire heart's interior based on only a very small number of observation points, overcoming the technical bottleneck of highly sparse observation data in minimally invasive surgical scenarios. Furthermore, this invention transforms the solution process from complex nonlinear finite element iterative calculations into efficient neural network forward inference, significantly reducing the time required for a single deformation calculation from minutes to milliseconds. This meets the real-time requirements of surgical navigation and provides a feasible technical approach for achieving high-fidelity augmented reality surgical navigation. Attached Figure Description

[0027] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0028] Figure 1 This is a flowchart illustrating a method for correcting a cardiac soft tissue model based on a physical information neural network, as provided in an embodiment of the present invention.

[0029] Figure 2 This is a schematic diagram of the structure of the dual-stream coupled physical information neural network in an embodiment of the present invention.

[0030] Figure 3 This is a schematic diagram illustrating the application scenario of real-time model correction and augmented reality navigation image rendering in an embodiment of the present invention.

[0031] Figure 4 This is a structural block diagram of a cardiac soft tissue model correction device provided in an embodiment of the present invention.

[0032] Figure 5 This is a schematic diagram of the directional field distribution of myocardial fibers in an embodiment of the present invention.

[0033] Figure 6 This is a convergence curve of the physical constraint loss function in an embodiment of the present invention.

[0034] Figure 7 This is a comparison diagram of the inference of the three-dimensional deformation field of sparse observation points in an embodiment of the present invention.

[0035] Figure 8 This is a comparison chart of calculation time in embodiments of the present invention. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of the present invention clearer, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0037] Example 1

[0038] This invention provides a method for correcting a cardiac soft tissue model based on a physical information neural network. (Refer to...) Figure 1 This method is primarily applied in augmented reality surgical navigation systems, aiming to correct the deformation of a pre-constructed 3D cardiac model in real time and accurately to match the complex deformations caused by cardiac pulsation and external contact forces during surgery. The specific process of this method includes the following steps:

[0039] Step S100: Construct a personalized cardiac biomechanical model and an active motion prior library. This step is completed offline before surgery. Its purpose is to provide a digital model and prior knowledge containing patient-specific anatomical information, mechanical properties, and motion patterns for subsequent neural network construction and training. Specifically, this step may include several sub-steps. First, acquire temporal multimodal imaging data of the patient's heart, such as ECG-gated four-dimensional computed tomography (4D-CT) sequences or cardiac cine-MRI sequences. This data can capture the dynamic changes of the heart throughout a complete cardiac cycle. Second, perform professional medical image segmentation and three-dimensional reconstruction processing on the acquired image data to generate a high-fidelity tetrahedral volume grid model of the heart. On this model, key anatomical structures such as coronary arteries, valvular annulus, and ventricular wall can be further accurately labeled. Subsequently, in order to establish an active motion prior library, extract the natural pulsating displacement field data of the heart during multiple consecutive cardiac cycles from the aforementioned temporal image sequences using an image registration algorithm. Using data from multiple consecutive cardiac cycles can effectively overcome minor rhythmic variations (such as disturbances caused by respiratory movements) and imaging noise interference that may exist in a single cycle, thus obtaining more stable and robust statistical characteristics. Due to the extremely high dimensionality of this displacement field data, to extract its inherent regularity, dimensionality reduction algorithms such as Principal Component Analysis (PCA) or Variational Autoencoder (VAE) are preferably used to process the high-dimensional temporal displacement field data. Through this processing, a low-dimensional manifold characterizing the spatiotemporal features of cardiac physiological motion can be extracted, thereby constructing a low-dimensional "active motion manifold space." This space can efficiently and accurately describe most of the morphological changes generated by cardiac pulsation using a linear combination of a few basis vectors. Furthermore, physical properties are assigned to this mesh model to establish a hyperelastic constitutive model describing the mechanical behavior of myocardial tissue. Preferably, the Holzapfel-Ogden constitutive model, which can accurately characterize the anisotropic nonlinear mechanical properties of myocardial fibers, is used. By repeatedly adjusting the material parameters in the model in the simulation environment, such as the stiffness coefficient in different directions and the directional field distribution of myocardial fibers, the deformation response of the model under virtual drive approximates the physiological performance observed from real images, thereby completing the calibration of a personalized biomechanical model.

[0040] Step S200: Construct a dual-stream coupled physical information neural network (DC-PINN). (Refer to...) Figure 2This step constructs a deep neural network architecture specifically designed to decouple and predict the mixed active and passive deformations of the heart. The input (10) of this network includes three-dimensional spatial coordinates x (i.e., the coordinates of any grid vertex in the heart model), the cardiac phase variable phi(t) representing the heart's beating state, and the sparse boundary displacement vector U_obs obtained from the intraoperative visual system. Among them, the cardiac phase variable phi(t) is a scalar normalized from 0 to 1, obtained by R-wave detection and period normalization processing of the electrocardiogram (ECG) signal acquired in real time during the operation, and is used to accurately indicate the current systolic or diastolic phase of the heart. The sparse boundary displacement vector U_obs contains the three-dimensional displacement observations of several visible feature points on the heart surface obtained by the endoscopic visual feature tracking algorithm. This dual-stream coupled physical information neural network architecture includes an active deformation prediction branch 20 and a passive residual correction branch 30, wherein the outputs of the two branches are added together to obtain the total displacement field (50).

[0041] Specifically, the active deformation prediction branch 20 receives spatial coordinates x and cardiac phase variable phi(t) as input. Its function is to predict the reference displacement field u_active(x,phi) generated by the spontaneous heartbeat under a specific cardiac phase. To ensure the physiological authenticity of the prediction results, the output of this branch is subject to strong regularization constraints of the "active motion manifold space" constructed in step S100. For example, the output layer of this branch does not directly output the displacement vector, but outputs a set of coordinate coefficients under the basis of the manifold space, and then obtains the final displacement field by multiplying and summing with the basis vectors, thereby ensuring that the generated deformation field always conforms to the inherent motion pattern of the physiological heartbeat.

[0042] The passive residual correction branch 30 receives spatial coordinates x, sparse boundary displacement vector U_obs, and the reference displacement field u_active output by the active branch as input. It is used to solve for the additional deformation caused by external contact forces such as surgical instruments, based on the active deformation, and outputs the passive residual displacement field u_passive(x, U_obs, u_active). To more effectively process unstructured mesh data representing the geometry of the heart, a graph neural network (GNN) layer is preferably introduced into the network structure of this branch. Furthermore, to accurately simulate the anisotropic mechanical properties of myocardial tissue, this GNN layer employs an anisotropic inductive graph message passing mechanism. When constructing the adjacency relationships of the graph and defining the message passing function, myocardial fiber orientation field information obtained from a personalized cardiac biomechanical model is explicitly incorporated. Figure 5As shown, the orientation field of myocardial fibers exhibits a spiral distribution within the ventricular wall, with the fiber angles continuously changing from the endocardium to the epicardium. Specifically, an anisotropic graph convolution operator is defined. When aggregating features of neighboring nodes, the aggregation weights are no longer isotropic but dynamically calculated based on the dot product between the edge vectors connecting neighboring nodes and the local myocardial fiber orientation vectors. This design allows the network structure itself to encode and utilize the physical prior knowledge that forces are primarily transmitted along the myocardial fiber direction, thus enabling inferences that are more consistent with physical laws even under extremely sparse data conditions.

[0043] Step S300: Define the physical constraint loss function and train the network. To enable the network to learn from sparse data and ensure the physical plausibility of its predictions, a composite loss function is constructed. The network's total loss function L_total is composed of two weighted parts: the data matching loss L_data and the physical governing equation loss L_pde, i.e., L = w_data * L_data + w_pde * L_pde, where w_data and w_pde are adjustable weight coefficients. The data matching loss L_data is used to ensure the accuracy of the network's predictions at the observation points. It is calculated by comparing the predicted displacement of the total displacement field u_total = u_active + u_passive at the sparse observation point with the sparse boundary displacement vector U_obs observed during the operation, and calculating the mean square error between the two.

[0044] The physical control equation loss L_pde is used to achieve self-supervised learning and ensure physical realism. It is calculated by substituting the total displacement field u_total output by the network into the static equilibrium equation describing the mechanical behavior of the heart's soft tissue. This equation can be expressed as div(sigma) + f = 0, where sigma is the Piola-Kirchhoff stress tensor of the first kind, and f is the body force (usually negligible in heart deformation problems). This equation holds under the quasi-static assumption of neglecting inertia terms. The stress tensor sigma is derived from the nonlinear functions of the deformation gradient tensor and strain tensor calculated from the displacement field u_total, based on the Holzapfel-Ogden constitutive relation selected in step S100. The residuals of this equation are calculated at a large number of sampling points (i.e., collocation points) within the model and used as the loss term L_pde. The goal of network training is to find a set of network weight parameters that minimizes the total loss function L using optimization algorithms such as gradient descent. Figure 6As shown, with the increase of training rounds, both the data matching loss L_data and the physical equation loss L_pde exhibit a rapid decreasing trend and eventually converge. This indicates that the network can simultaneously learn the distribution pattern of the observed data and the constraint relationship of the physical equation. The process of minimizing L_pde essentially drives the displacement field output by the network to automatically satisfy physical constraints such as mass conservation, momentum conservation, and myocardial incompressibility throughout the entire computational domain. It should be noted that although this embodiment uses the Holzapfel-Ogden constitutive model as an example to describe the mechanical properties of myocardial tissue, this is not the only limitation of the present invention. Any constitutive model that can reasonably characterize the anisotropic and nonlinear hyperelastic mechanical behavior of myocardium (such as the modified Mooney-Rivlin model, Guccione model, etc.) can be applied to the present invention and embedded as a physical constraint into the loss function of the neural network.

[0045] In terms of training strategy, an offline pre-training mode is adopted as the main approach. Before surgery, large-scale numerical simulations are performed using the biomechanical model constructed in step S100. By randomly applying virtual external forces of different locations, magnitudes, and directions to the surface of the heart model, and combining this with randomly sampled cardiac phases, a large number of synthetic training samples containing various potential stress states are generated for sufficient pre-training of the dual-flow coupled physical information neural network. Optionally, during the preparation phase before surgery, several cycles of free-beating heart video data acquired briefly during surgery can be used to quickly fine-tune or calibrate the active deformation prediction branch in the network online to eliminate systematic biases that may be caused by differences between preoperative images and actual intraoperative positioning.

[0046] Step S400: Receive intraoperative signals and extract cardiac phase and sparse boundary displacement. (Refer to...) Figure 3 This step is performed in real time during the surgery. At the start of the surgery, an initial rigid registration is performed to align the preoperatively constructed personalized cardiac biomechanical model with the initial position and orientation of the heart in the intraoperative endoscopic coordinate system, thus establishing a unified reference coordinate system. During the surgery, the system simultaneously acquires the intraoperative endoscopic video stream 61 and the patient's electrocardiogram (ECG) signal 62. By running a visual tracking algorithm based on optical flow or sparse feature point matching, 10 to 20 natural texture points or artificial marker points within the visible area of ​​the heart surface are tracked in real time from the video stream 61, and the displacement vectors of these points in three-dimensional space are calculated to form a sparse boundary displacement vector U_obs; here, the natural texture points refer to image feature points that can be directly extracted and identified from medical images due to the inherent features of the heart tissue surface (such as microvascular networks, natural tissue texture, etc.). Simultaneously, the cardiac phase phi(t) at the current moment is extracted in real time from the ECG signal 62.

[0047] In step S500, forward inference calculations are performed to update the 3D morphology of the model for augmented reality navigation. The current cardiac phase phi(t) and the tracked sparse displacement U_obs are input into the pre-trained dual-stream coupled physical information neural network model 70. The model, through a single efficient forward propagation calculation on a graphics processing unit (GPU), outputs a complete 3D displacement field u_total = u_active + u_passive covering the entire heart volume grid within milliseconds (e.g., less than 40 milliseconds). Then, the calculated complete 3D displacement field u_total is used to drive the position update of all mesh vertices of the preoperative high-precision 3D model, generating a deformed corrected model consistent with the actual intraoperative morphology of the heart. Finally, the corrected 3D model, including its internal coronary arteries, valves, and other structures, is projected onto the endoscopic image plane and displayed in a semi-transparent or wireframe mode overlaid on the real surgical field of view, thereby achieving precise registration between the virtual anatomical structure and the real scene, providing the surgeon with real-time augmented reality navigation information 80.

[0048] To verify the effectiveness of the method of the present invention, Figure 7 This paper demonstrates a comparison between the method of this invention and a purely data-driven method for deformation field inference under sparse observation conditions. Figure 7 As shown, with only sparse observation points on the surface, the DC-PINN method of this invention can accurately infer the internal volume deformation that highly matches the ground truth, while the prediction results of purely data-driven methods show significant deviations due to the lack of physical constraints. Furthermore, Figure 8 This demonstrates a comparison of the time consumption of different methods in deformation calculation. For example... Figure 8 As shown, the method of the present invention reduces the calculation time of a single deformation to the millisecond level (about 25 milliseconds), which has a significant speed advantage compared with the traditional finite element method (FEM) and the GPU-accelerated FEM method. Moreover, its calculation time remains almost constant as the mesh size increases, which fully meets the real-time requirements of surgical navigation.

[0049] Example 2

[0050] This invention also provides a cardiac soft tissue model correction device. (Refer to...) Figure 4 The device may be a dedicated computing unit integrated into a surgical navigation system, or a server configured with high-performance computing hardware. The device includes one or more processors 410, and a memory 420 storing computer-executable instructions that can run on the one or more processors 410. When executed, the instructions cause the one or more processors 410 to perform the method described in Embodiment 1.

[0051] Specifically, the instructions stored in memory 420 can be organized into multiple functional modules, including:

[0052] The model building module is used to perform the function described in step S100, that is, to build a personalized cardiac biomechanical model and active motion prior library that includes precise anatomical structure, material mechanical properties and voluntary pulsation patterns based on the patient's medical imaging data before surgery.

[0053] The network construction and training module is used to perform the functions described in steps S200 and S300, namely, to construct a dual-stream coupled physical information neural network architecture, define a loss function that includes data matching terms and physical constraint terms, and perform offline pre-training and optional online fine-tuning of the network using simulation data or real data.

[0054] The real-time correction module is used to perform the functions described in steps S400 and S500. During the operation, the module receives endoscopic video and electrocardiogram signals in real time, extracts sparse displacement observations and cardiac phases from them, inputs them into a trained neural network model for fast forward inference, obtains a full-field three-dimensional displacement field, and uses the displacement field to update the geometry of the preoperative three-dimensional model. Finally, the corrected model is rendered and superimposed on the surgical field of view to achieve augmented reality navigation.

[0055] Through the above embodiments, this invention successfully decouples the active pulsating deformation of the heart from the passive contact deformation caused by instruments, improving the accuracy and physical interpretability of the correction. Simultaneously, by embedding the biomechanical control equations as strong constraints into the neural network, the network can infer the volumetric deformation of the entire heart's interior in accordance with physical laws using only a very small number of observation points on the surface. Furthermore, this invention transforms the solution process into efficient neural network forward inference, significantly reducing the time required for a single deformation calculation to the millisecond level, meeting the real-time requirements of surgical navigation.

[0056] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for correcting a cardiac soft tissue model based on a physical information neural network, characterized in that, Includes the following steps: Step 1: Based on the patient's preoperative time-series imaging data, construct a personalized biomechanical model of the heart and extract cardiac pulsation displacement field data to construct an active motion manifold space. Step 2: Construct a dual-flow coupled physical information neural network, which receives the spatial three-dimensional coordinates, cardiac phase variables, and sparse boundary displacement vectors of the personalized biomechanical model grid nodes as input; The dual-flow coupled physical information neural network includes: an active deformation prediction branch, used to predict the reference displacement field constrained by the active motion manifold space based on the spatial three-dimensional coordinates of the grid nodes and the cardiac phase variable, so as to conform to the physiological pulsation pattern of the heart; And a passive residual correction branch, used to solve the passive residual displacement field caused by the external contact force based on the spatial three-dimensional coordinates of the grid nodes, the sparse boundary displacement vector and the reference displacement field; and superimpose the reference displacement field and the passive residual displacement field to obtain the total displacement field; Step 3: Define a total loss function including data matching loss and physical control equation loss to train the network; the data matching loss calculates the error of the total displacement field predicted by the network at sparse observation points; the physical control equation loss calculates the residual after substituting the total displacement field into the soft tissue mechanical equilibrium equation, so as to achieve self-supervised learning under physical constraints. Step 4: Receive ECG signal data and endoscopic video stream data collected during the operation, and determine the cardiac phase variable and sparse boundary displacement vector through signal processing and visual tracking; Step 5: Input the cardiac phase variable and sparse boundary displacement vector into the trained network, perform forward propagation calculation on all grid vertices of the personalized biomechanical model to obtain the total displacement field, and use the total displacement field to update the three-dimensional morphology of the personalized biomechanical model.

2. The method according to claim 1, characterized in that, In step one, the construction of the active motion manifold space specifically includes: extracting cardiac pulsation displacement field data of the heart in multiple consecutive cardiac cycles from the preoperative temporal image data, and performing dimensionality reduction processing on the cardiac pulsation displacement field data using principal component analysis or variational autoencoder to extract a low-dimensional manifold characterizing the spatiotemporal features of cardiac physiological motion, thereby constructing the low-dimensional active motion manifold space.

3. The method according to claim 1, characterized in that, In step two, the network structure of the passive residual correction branch includes a graph neural network layer, which employs an anisotropic inductive graph message passing mechanism. During message passing, this mechanism integrates myocardial fiber orientation field information obtained from the personalized biomechanical model. By defining an anisotropic graph convolution operator, the aggregation weights are dynamically calculated based on the dot product relationship between the edge vectors connecting neighboring nodes and the local myocardial fiber orientation vectors when aggregating features of neighboring nodes.

4. The method according to claim 1, characterized in that, In step three, the soft tissue mechanical equilibrium equation is a static equilibrium equation, and the relationship between the total displacement field and the stress tensor is described by the Holzapfel-Ogden constitutive model. The stress tensor is derived from the strain tensor calculated from the total displacement field.

5. The method according to claim 1, characterized in that, The training in step three includes an offline pre-training step; the offline pre-training step specifically involves: using the personalized biomechanical model to perform numerical simulation, by randomly applying virtual external forces of different positions, magnitudes and directions to the surface of the heart model, and combining them with randomly sampled cardiac phases to generate synthetic training samples for pre-training the dual-stream coupled physical information neural network.

6. The method according to claim 1, characterized in that, In step four, a visual tracking algorithm based on optical flow or sparse feature point matching is run on the endoscopic video stream data to track natural texture points in the visible area of ​​the heart surface in real time, and calculate the three-dimensional displacement vector of the natural texture points to form the sparse boundary displacement vector.

7. The method according to claim 1, characterized in that, In step two, the output of the active deformation prediction branch is subject to the regularization constraint of the active motion manifold space in the following way: the output layer of the active deformation prediction branch outputs a set of coordinate coefficients under the basis of the active motion manifold space, and the reference displacement field is obtained by multiplying the coordinate coefficients with the basis vectors of the active motion manifold space and summing them.

8. The method according to claim 1, characterized in that, The cardiac phase variable is a normalized value obtained by R-wave detection and period normalization of the electrocardiogram signal data, used to indicate the current systolic or diastolic phase of the heart.

9. The method according to claim 1, characterized in that, It also includes the step of projecting the three-dimensional shape of the updated personalized biomechanical model from step five onto the endoscopic image plane to generate an augmented reality navigation image.

10. A cardiac soft tissue model correction device based on a physical information neural network, characterized in that, include: One or more processors; as well as A memory that stores computer instructions; The one or more processors are configured to perform the method of any one of claims 1 to 9 when executing the computer instructions.

Images