Surgical data dynamic registration method and system based on physical simulation and multi-modal constraint

Abstract

The application provides a surgery data dynamic registration method and system based on physical simulation and multi-modal constraint, and the method comprises the following steps: reconstructing a three-dimensional model of a target organ based on preoperative images and labeling feature points; predicting intraoperative deformation through physical simulation to obtain a prediction model; collecting surface point clouds at the beginning of surgery for rigid registration; constructing multi-modal constraint by fusing electromagnetic positioning and visual data in real time during surgery; driving model non-rigid deformation by using position dynamics method to realize dynamic registration. The application can compensate soft tissue deformation with high precision and physical reasonableness, has real-time response capability, does not need to implant markers, is suitable for various soft tissue surgery scenes, and improves the accuracy and safety of surgery navigation.

Description

Technical Field

[0001] This invention relates to the fields of medical image processing and surgical navigation technology, specifically to a method and system for dynamic registration of surgical data based on physical simulation and multimodal constraints. Background Technology

[0002] Image-guided surgery (IGS) is one of the core technologies of modern precision surgery. By precisely matching the patient's preoperative medical images (such as CT and MRI) with the patient's actual anatomical location during surgery, it provides surgeons with navigation information that goes beyond direct visual observation, thereby significantly improving the safety and precision of the surgery.

[0003] In a typical surgical procedure, surgeons first perform 3D reconstruction and surgical planning based on high-resolution preoperative images, determining the location and boundaries of the lesion target area, key blood vessels, nerves, and other important structures. However, in actual surgery, there are often significant differences between the preoperative static planning model and the dynamic, deformed tissues and organs during the operation. These differences mainly stem from: (1) Changes in the patient's body position lead to changes in the relative spatial position of organs; (2) Non-rigid deformation of soft tissue caused by gravity, pneumoperitoneum pressure, instrument traction or tissue removal during the operation; (3) Periodic dynamic changes brought about by physiological movements such as breathing and heartbeat.

[0004] To address the above challenges, existing technologies have proposed various registration methods, but all of them have significant limitations: 1. Rigid registration method: This method can only handle overall translation and rotation, and cannot simulate or compensate for the complex non-rigid deformation of soft tissues. Its application effect is limited in soft tissue surgery such as abdominal and cranial surgeries.

[0005] 2. Registration based on artificial markers: This method involves implanting markers that are visible in images within the patient's body, and then tracking these markers during the procedure for registration. While this method improves accuracy, it is an invasive procedure, increasing patient trauma, infection risk, and surgical complexity.

[0006] 3. Non-rigid registration based on single-modal data: For example, registration relying solely on intraoperative ultrasound or intraoperative surface scan point clouds. These methods are limited by the inherent drawbacks of single-sensor data (e.g., high noise and low resolution in ultrasound images; lack of internal structural information in surface point clouds). When data quality is poor or features are missing, the accuracy and robustness of registration drop sharply.

[0007] 4. Deformation models lacking physical constraints: Although some non-rigid registration algorithms are flexible, they do not incorporate the biomechanical properties of tissues, which may lead to physically unreasonable registration results and mislead surgical navigation.

[0008] In summary, existing technologies struggle to address the preoperative-intraoperative data mismatch caused by non-rigid soft tissue deformation in a non-invasive, real-time, and highly accurate manner. Therefore, there is an urgent need for a dynamic registration method that can integrate multi-source information, conforms to physical laws, and meets clinical real-time requirements, thereby propelling image-guided surgery towards greater precision and intelligence. Summary of the Invention

[0009] To address the shortcomings of existing technologies, the purpose of this invention is to provide a method and system for dynamic registration of surgical data based on physical simulation and multimodal constraints.

[0010] A method for dynamic registration of surgical data based on physical simulation and multimodal constraints, provided by the present invention, includes the following steps: Step S1: Based on the patient's preoperative medical imaging data, reconstruct a three-dimensional model of the target organ, and mark key anatomical feature points on the three-dimensional model to obtain a set of preoperative anatomical feature points; Step S2: Import the three-dimensional model into the soft tissue simulation platform, assign it biomechanical parameters and apply intraoperative physiological load conditions, and obtain the deformation prediction model of the three-dimensional model under the load and the corresponding set of predicted anatomical feature points through physical simulation calculation; Step S3: At the start of the operation, collect the surface point cloud of the target organ during the operation, perform rigid registration with the corresponding surface point cloud of the deformation prediction model to obtain rigid transformation parameters, and apply the transformation to the deformation prediction model. Step S4: During the operation, multimodal intraoperative data, including sparse 3D keypoint data and dense 2D visual data, are collected in real time; based on the deformation prediction model that has been rigidly registered, multiple types of constraints are constructed according to the multimodal intraoperative data; the position dynamics method is used to solve the multiple types of constraints in real time, driving the deformation prediction model to undergo non-rigid deformation, thereby achieving dynamic registration with the actual intraoperative state.

[0011] Preferably, in step S1, the reconstruction of the three-dimensional model of the target organ specifically includes: using a medical base model, after generalization and transfer learning through downstream medical tasks, performing semantic segmentation on the preoperative medical image data to extract the outline of the target organ; and generating a triangular mesh surface model through voxelization and surface reconstruction algorithms.

[0012] Preferably, in step S2, the biomechanical parameters include tissue density, Young's modulus, and Poisson's ratio; the intraoperative physiological load conditions include gravity, intra-abdominal pressure, and positional constraints; and the physical simulation is solved using the finite element method or positional dynamics method.

[0013] Preferably, in step S3, the acquisition of the surface point cloud of the target organ during the procedure is achieved by sliding a handheld probe equipped with an electromagnetic positioning sensor on the organ surface for sampling; the rigid registration adopts the iterative nearest point algorithm or its improved algorithm.

[0014] Preferably, in step S4, the sparse three-dimensional key point data is obtained by acquiring the three-dimensional coordinates of markers at key anatomical locations in real time through an electromagnetic positioning system; the dense two-dimensional visual data is obtained by performing real-time semantic segmentation and feature point tracking on the intraoperative video stream.

[0015] Preferably, in step S4, the multi-type constraint conditions include at least one of the following: - Distance preservation constraints are used to maintain the local shape of tissues; - Hard constraints are used to force the model's key points to precisely coincide with the electromagnetic positioning points; - Soft constraints are used to align the projection of the model surface with the visually observed organ contours; - Volume preservation constraints are used to prevent the model from undergoing excessive non-physical deformation.

[0016] Preferably, in step S4, the real-time solution using the position dynamics method includes: predicting the vertex's position at each time step; projecting the vertex to a position that satisfies all constraints through iterative solution; and updating the vertex's position and velocity.

[0017] Preferably, the method further includes step S5, registration result output and visualization: the dynamically registered preoperative model, the planned surgical path and key anatomical structure information are superimposed and displayed on the intraoperative video image to form an augmented reality navigation view.

[0018] Preferably, step S5 further includes calculating and displaying the registration error index of the key locations.

[0019] A dynamic registration system for surgical data based on physical simulation and multimodal constraints, provided by the present invention, includes the following steps: Module M1: Based on the patient's preoperative medical imaging data, reconstruct a three-dimensional model of the target organ and mark key anatomical feature points on the three-dimensional model to obtain a set of preoperative anatomical feature points; Module M2: Import the three-dimensional model into the soft tissue simulation platform, assign it biomechanical parameters and apply intraoperative physiological load conditions, and obtain the deformation prediction model of the three-dimensional model under the load and the corresponding set of predicted anatomical feature points through physical simulation calculation; Module M3: At the start of surgery, the surface point cloud of the target organ is acquired during the operation, and rigidly registered with the corresponding surface point cloud of the deformation prediction model to obtain rigid transformation parameters, and the transformation is applied to the deformation prediction model. Module M4: During the operation, multimodal intraoperative data, including sparse 3D keypoint data and dense 2D visual data, are collected in real time; based on the deformation prediction model that has been rigidly registered, multiple types of constraints are constructed according to the multimodal intraoperative data; using the position dynamics method, the multi-type constraints are solved in real time, driving the deformation prediction model to undergo non-rigid deformation, thereby achieving dynamic registration with the actual intraoperative state.

[0020] Compared with the prior art, the present invention has the following beneficial effects: 1. Achieve high-precision and physically reasonable dynamic registration: By predicting intraoperative deformation through biomechanical simulation and optimizing it with multimodal data constraints, the registration results can achieve sub-centimeter accuracy while conforming to the real physical characteristics of the tissue, effectively overcoming the non-physiological deformation problem that is easily caused by simple image registration.

[0021] 2. Real-time response and multimodal robustness: Employs a high-efficiency position dynamics (PBD) solver to support real-time deformation calculation and updates during surgery (30... (60Hz). By integrating multi-source information such as electromagnetic positioning and visual contours, it can maintain registration stability even when some data is limited, ensuring continuous and reliable navigation.

[0022] 3. No need for implanted markers, reducing surgical invasiveness: Registration is based solely on intraoperative natural anatomical features and imaging data, avoiding the additional trauma, infection risk, and procedural burden caused by implanted artificial markers, making it more clinically friendly and safer.

[0023] 4. The method is highly versatile and easy to expand its application: The technical framework described does not depend on specific organs or surgical types and can be applied to various soft tissue surgical scenarios such as liver, kidney, and brain tissue, and has good clinical adaptability and promotion value. Attached Figure Description

[0024] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a flowchart of the dynamic registration method for surgical data based on physical simulation and multimodal constraints in an embodiment of the present invention; Figure 2 This is a flowchart of the intraoperative real-time multimodal non-rigid registration process in an embodiment of the present invention. Detailed Implementation

[0025] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0026] This invention provides a method for dynamic registration of intraoperative-preoperative data based on physical simulation and multimodal constraints. Its core lies in combining preoperative reconstruction, physical simulation pre-deformation, multimodal data acquisition and fusion, and real-time dynamic solution to form a closed-loop dynamic registration system. Specifically, this method includes the following steps: Step S1: Preoperative 3D model reconstruction This step aims to generate a high-precision, structured 3D organ model from the patient's preoperative medical images, which can be used for surgical planning and navigation.

[0027] 1.1 Data Acquisition and Preprocessing: Obtain DICOM sequence data of the patient's preoperative abdominal CT or MRI, and perform standardized preprocessing on the image data, including grayscale normalization, isotropic resampling (e.g., unifying the voxel size to 1mm×1mm×1mm), and noise reduction, to improve the accuracy and stability of subsequent segmentation.

[0028] 1.2 Deep Learning-Based 3D Reconstruction: Semantic segmentation: A medical-based model is used, which, after generalization and transfer learning through downstream medical tasks, automatically segments the preprocessed images. In a preferred embodiment, the medical-based model is a foundational model pre-trained on large-scale medical data, including but not limited to convolutional neural networks, Transformers, and Vision Transformers (ViT). This model, through training, can accurately segment target organs and their internal vascular systems. The output is binary or semantically labeled data with the same size as the original image.

[0029] Surface mesh generation: For the segmented binary data, morphological operations are first used to smooth it, removing small islands and holes. Then, the Marching Cubes algorithm or the Poisson surface reconstruction algorithm is used to extract the contour surfaces of the target, generating an initial triangular mesh surface model. For tubular structures such as vascular systems, a skeletonization combined with radial basis functions can be used for reconstruction.

[0030] Mesh optimization: The reconstructed 3D model undergoes mesh optimization, including: using the Laplacian smoothing algorithm to reduce staircase artifacts; employing the edge collapse algorithm to simplify the mesh, controlling the number of mesh faces while preserving features to improve subsequent computational efficiency; and performing topology checks and repairs to ensure the mesh is manifold.

[0031] 1.3 Feature point annotation and model preparation: On the optimized 3D model, key anatomical feature points are annotated by doctors or using automated algorithms. For example, for a liver model, the bifurcation points of hepatic veins, the attachment points of the falciform ligament, and obvious concave and convex points on the organ surface can be annotated. These points constitute the preoperative anatomical feature point set P_pre={p1, p2, ..., p n}

[0032] Finally, a standard 3D model file containing geometric, topological, and semantic information is generated for use in subsequent steps.

[0033] Step S2: Intraoperative deformation prediction based on physical simulation This step uses computer simulation to predict the deformation of the target organ from the preoperative scanning position to the intraoperative position and after loading, providing a better initial estimate for real-time registration. Specifically, it includes: 2.1 Simulation Environment Setup and Parameter Settings: Import the triangular mesh model obtained in step S1 into the open-source biomechanical simulation framework SOFA.

[0034] Model discretization: A tetrahedral mesh generator (such as Gmsh) is used to convert the surface triangular mesh into a volume mesh (tetrahedral mesh) to suit finite element analysis.

[0035] Material property assignment: A linear elastic or hyperelastic material model is set for the target organ tissue. Key biomechanical parameters are set with reference to publicly available literature and experimental data. For example, the density of liver tissue is ρ≈1060kg / m³, Young's modulus E≈5-15kPa (which can be finely adjusted according to individual patient differences), and Poisson's ratio ν≈0.45 (approximately incompressible).

[0036] Boundary conditions and load application: Fixed constraints: Simulate the connection points of major fixed structures such as the liver and inferior vena cava, and completely constrain the displacement of the mesh vertices in these areas.

[0037] Gravity load: Apply standard gravitational acceleration g=9.81m / s², with the direction set according to the actual intraoperative position (e.g., supine position).

[0038] Intra-abdominal pressure: A normal pressure P_abdominal is uniformly applied to the surface of the model to simulate the pressure caused by pneumoperitoneum (usually 12-15 mmHg, about 1.6-2.0 kPa).

[0039] 2.2 Deformation Simulation Solution: Explicit finite element method or quasi-static solver is used for simulation calculation. Due to the ample computation time available in the preoperative period, high-precision simulation can be performed. After solving, the displacement field of all vertices is obtained. This displacement field is applied to the original preoperative model to obtain the intraoperative prediction model M_pred. Simultaneously, each point in the preoperative feature point set P_pre is interpolated based on the vertex displacement of its corresponding tetrahedron to obtain the corresponding prediction feature point set P_pred={p1', p2', ..., p...}. n '}.

[0040] The M_pred output in this step has approximated the deformation caused by body shape and basic load, making it more spatially similar to the initial state of the organ during surgery. This significantly reduces the deformation that needs to be compensated for in subsequent real-time registration, and improves the convergence speed and final accuracy of registration.

[0041] Step S3: Preoperative-intraoperative initial rigid registration This step rapidly establishes the spatial correspondence between the preoperative imaging coordinate system and the intraoperative world coordinate system at the start of the surgery. Specifically, it includes: 3.1 Intraoperative point cloud acquisition: An electromagnetic positioning device integrated into the surgical navigation system is used. The surgeon holds a probe equipped with a 6-DOF electromagnetic sensor. After pneumoperitoneum is established and the surgical field is exposed, the surgeon gently slides the probe along different directions on the surface of the target organ (such as the liver) for approximately 10-15 seconds. The electromagnetic positioning system records the three-dimensional coordinates of the probe tip in real time at a frequency of f=20Hz. After filtering (such as Kalman filtering) to remove hand tremor noise, a sparse point cloud Q_intra={q1, q2, ..., q3} is formed on the surface of the organ during surgery. m (World coordinate system).

[0042] 3.2 Rigid Registration Calculation: From the surface of the prediction model M_pred, point cloud Q_pre is generated by randomly sampling or extracting vertices corresponding to the probe sliding region.

[0043] Registration is performed using the Iterative Closest Point (ICP) algorithm. 1. Initialization: Using the alignment of the centroids of the two point clouds as the initial transformation, initialize the rotation matrix R0 and the translation vector t0; 2. Iteration: In each iteration, for each point in Q_intra, find the nearest neighbor in Q_pre and calculate the normal distance to the triangular facet containing that point.

[0044] 3. Solution: By minimizing the sum of squared distances from a point to a surface, the optimal rotation matrix R and translation vector t are obtained using the least squares method, such that the registration error E = Σ‖q i -(R·p i Minimize +t)‖²; apply transformations to update the preoperative point cloud positions.

[0045] 4. Convergence judgment: Stop when the root mean square error change between two consecutive iterations is less than the threshold ε=0.1mm or the maximum number of iterations (e.g., 50 times) is reached.

[0046] The obtained {R, t} is applied to the entire prediction model M_pred and its feature point set P_pred to complete the initial spatial alignment.

[0047] Step S4: Intraoperative real-time multimodal non-rigid registration This is the core of the invention: it operates continuously during surgery, dynamically compensating for complex deformations caused by instrument manipulation, respiration, etc. (See reference...) Figure 2 As shown, it specifically includes: 4.1 Real-time acquisition and processing of multimodal data: Electromagnetic positioning data (sparse, high-precision 3D constraints): 2-4 miniature passive electromagnetic induction coils are placed as markers at key anatomical locations of the target organ (e.g., near the feature points marked in step S1). The navigation system calculates the 3D coordinates of these markers in real time at a frequency of f1=30Hz, forming a constraint point set L_mag={l1, l2, ..., l...} k (Sampling frequency f1Hz).

[0048] Laparoscopic visual data (dense, two-dimensional constrained): High-definition laparoscopy outputs video stream at a frame rate of f2=25fps.

[0049] Real-time semantic segmentation: A lightweight deep learning model is used to segment each frame of the image in real time, and the pixel-level mask of the target organ is output with GPU acceleration.

[0050] Feature point tracking: Within the segmented region, for the surface feature points defined in step S1 that are visible from the current viewpoint, cross-frame tracking is performed using optical flow or feature matching algorithms to obtain their two-dimensional pixel coordinate sequence V_2d={v1(t), v2(t), ..., v j (t)}.

[0051] 4.2 Real-time deformation solution based on position dynamics: System Modeling: The prediction model M_pred, after rigid registration in step S3, is represented as a particle system in positional dynamics. The vertices of the model are the particles, and the edges and faces of the tetrahedral mesh are used to define internal constraints.

[0052] Construction of multiple types of constraints: 1. Based on Position-Based Dynamics (PBD), the simulation internal parameter constraints control the restoring force of the model under tension / bending through the parameter stiffness. This indicates that the parameter `poisson_ratio` controls the degree of lateral expansion during the compression of the model being simulated. This indicates that the parameter liver_volume_stiffness controls the ability of the simulated model to resist volume changes. express.

[0053] 2. Hard constraint (electromagnetic marker point): for each electromagnetic marker l k The time t is the nearest vertex p on its corresponding model surface. k Apply positional constraints C_mag, C_mag(p) k )=p k -l k =0, this constraint weight is set to w_mag=1.0, and will be strictly satisfied during the solution iteration to ensure the absolute accuracy of the key points. For points with C_mag constraint... After coordinate constraints, the updates are as follows:

[0054] 3. Soft Constraints (Visual Projection): For the visible vertices of the model surface from the current camera's perspective, project them onto the image plane using the known intrinsic and extrinsic parameters of the laparoscopic camera. Calculate the shortest two-dimensional distance d between the projected points and the organ contour mask obtained from real-time segmentation. Define the visual constraint energy C_vis=Σd²(proj(p i The constraint C_vis is a weight (e.g., 0.3) that dynamically adjusts based on image segmentation confidence. This constraint guides the overall contour of the model to align with visual observation. Points constrained by C_vis... After coordinate constraints, the updates are as follows:

[0055] PBD solution loop: The solver runs at a frequency synchronized with the visual frame rate (e.g., 25Hz). Within each time step Δt, the core solution based on pyPBD consists of, in addition to its own constraint parameter characteristics, hard constraints C_mag based on the preoperative and intraoperative 3D coordinate positions of electromagnetic markers and soft constraints C_vis based on the preoperative and intraoperative 2D positions of soft markers. This is solved using position dynamics PBD, followed by coordinate correction for the corresponding points. ) After each frame is solved, the deformed model M_warped(t) is output, which is the preoperative model that is dynamically registered with the current intraoperative actual state.

[0056] Step S5: Registration Result Output and Visualization The registration results are seamlessly integrated into the surgical navigation interface, providing doctors with intuitive augmented reality guidance.

[0057] The system renders the real-time deformed 3D model M_warped(t) according to the camera pose and blends it with the real-time laparoscopic video footage in a semi-transparent overlay or outline manner.

[0058] The preoperatively planned surgical path (such as ablation needle path) and risk areas (such as tumor boundaries and major blood vessels) are also displayed simultaneously.

[0059] On one side of the interface, the estimated registration error of key areas (such as the root mean square error of electromagnetic markers, which can usually be kept within 2mm) is displayed in real time, providing doctors with confidence in the accuracy.

[0060] The entire system runs on a workstation equipped with a high-performance GPU, and the total latency from data acquisition to result visualization is controlled within 100ms, meeting the requirements for real-time interaction.

[0061] By combining the above steps, this invention achieves high-precision, high-real-time, physically reasonable intraoperative dynamic registration without the need for implanted markers, significantly improving the reliability and effectiveness of image-guided surgery.

[0062] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0063] In the description of this application, it should be understood that the terms "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application.

[0064] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A method for dynamic registration of surgical data based on physical simulation and multimodal constraints, characterized in that, Includes the following steps: Step S1: Based on the patient's preoperative medical imaging data, reconstruct a three-dimensional model of the target organ, and mark key anatomical feature points on the three-dimensional model to obtain a set of preoperative anatomical feature points; Step S2: Import the three-dimensional model into the soft tissue simulation platform, assign it biomechanical parameters and apply intraoperative physiological load conditions, and obtain the deformation prediction model of the three-dimensional model under the load and the corresponding set of predicted anatomical feature points through physical simulation calculation; Step S3: At the start of the operation, collect the surface point cloud of the target organ during the operation, perform rigid registration with the corresponding surface point cloud of the deformation prediction model to obtain rigid transformation parameters, and apply the transformation to the deformation prediction model. Step S4: During the operation, multimodal intraoperative data, including sparse 3D keypoint data and dense 2D visual data, are collected in real time; based on the deformation prediction model after rigid registration, multiple types of constraints are constructed according to the multimodal intraoperative data. The position dynamics method is used to solve the problem in real time under the multi-type constraint conditions, driving the deformation prediction model to undergo non-rigid deformation, thereby achieving dynamic registration with the actual intraoperative state.

2. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to claim 1, characterized in that, In step S1, the reconstruction of the three-dimensional model of the target organ specifically includes: using a medical base model, after generalization and transfer learning through downstream medical tasks, performing semantic segmentation on the preoperative medical image data to extract the outline of the target organ; and generating a triangular mesh surface model through voxelization and surface reconstruction algorithms.

3. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to claim 1, characterized in that, In step S2, the biomechanical parameters include tissue density, Young's modulus, and Poisson's ratio; the intraoperative physiological load conditions include gravity, intra-abdominal pressure, and positional constraints; and the physical simulation is solved using the finite element method or positional dynamics method.

4. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to claim 1, characterized in that, In step S3, the acquisition of surface point cloud of the target organ during the procedure is achieved by sliding a handheld probe equipped with an electromagnetic positioning sensor on the organ surface for sampling; the rigid registration adopts the iterative nearest point algorithm or its improved algorithm.

5. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to claim 1, characterized in that, In step S4, the sparse three-dimensional key point data is obtained by acquiring the three-dimensional coordinates of the markers at the key anatomical locations in real time through an electromagnetic positioning system. The dense two-dimensional visual data is obtained by performing real-time semantic segmentation and feature point tracking on the intraoperative video stream.

6. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to claim 1, characterized in that, In step S4, the multi-type constraint conditions include at least one of the following: - Distance preservation constraints are used to maintain the local shape of tissues; - Hard constraints are used to force the model's key points to precisely coincide with the electromagnetic positioning points; - Soft constraints are used to align the projection of the model surface with the visually observed organ contours; - Volume preservation constraints are used to prevent the model from undergoing excessive non-physical deformation.

7. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to claim 1, characterized in that, In step S4, the real-time solution using the position dynamics method includes: predicting the vertex position at each time step; projecting the vertex to a position that satisfies all constraints through iterative solution; and updating the vertex position and velocity.

8. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to any one of claims 1 to 7, characterized in that, It also includes step S5, registration result output and visualization: the preoperative model after dynamic registration, the planned surgical path and key anatomical structure information are superimposed and displayed on the intraoperative video image to form an augmented reality navigation view.

9. The method for dynamic registration of surgical data based on physical simulation and multimodal constraints according to claim 8, characterized in that, Step S5 also includes calculating and displaying the registration error index for key locations.

10. A dynamic registration system for surgical data based on physical simulation and multimodal constraints, characterized in that, Includes the following steps: Module M1: Based on the patient's preoperative medical imaging data, reconstruct a three-dimensional model of the target organ and mark key anatomical feature points on the three-dimensional model to obtain a set of preoperative anatomical feature points; Module M2: Import the three-dimensional model into the soft tissue simulation platform, assign it biomechanical parameters and apply intraoperative physiological load conditions, and obtain the deformation prediction model of the three-dimensional model under the load and the corresponding set of predicted anatomical feature points through physical simulation calculation; Module M3: At the start of surgery, the surface point cloud of the target organ is acquired during the operation, and rigidly registered with the corresponding surface point cloud of the deformation prediction model to obtain rigid transformation parameters, and the transformation is applied to the deformation prediction model. Module M4: During the operation, multimodal intraoperative data, including sparse 3D keypoint data and dense 2D visual data, are collected in real time; based on the deformation prediction model after rigid registration, multiple types of constraints are constructed according to the multimodal intraoperative data. The position dynamics method is used to solve the problem in real time under the multi-type constraint conditions, driving the deformation prediction model to undergo non-rigid deformation, thereby achieving dynamic registration with the actual intraoperative state.

Images