A multi-view visual navigation platform based on digital twinning of vessels and instruments

Abstract

The application discloses a multi-view visual navigation platform based on blood vessel and instrument digital twinning, belongs to the technical field of medical instruments, and is suitable for intraluminal vascular intervention surgery. The platform realizes accurate navigation through the cooperation of five core modules: a blood vessel digital twinning construction module integrates multiple modal images and sensing data of a patient, generates an individualized three-dimensional model containing anatomical structures, functional parameters and a virtual center line; a 2D / 3D image registration module realizes real-time high-precision alignment of a preoperative three-dimensional model and an intraoperative two-dimensional image through feature point matching, PnP rough positioning and center line constraint fine positioning; an instrument digital twinning construction module calculates the pose and curvature of an instrument based on multi-modal sensor data, and reconstructs a three-dimensional digital twinning in real time; a multi-view visual navigation interface unifies a coordinate system through marker calibration, provides multi-view interactive views through a Qt framework, and fuses and displays multi-source data; and a simulation analysis and interactive feedback module outputs offset detection, risk early warning and multi-dimensional feedback based on physical simulation and reinforcement learning. The application solves the defects of traditional surgery guide wire dependence and non-intuitive image, supports guide wire-free accurate operation, adapts to various blood vessels, intraluminal environments and operation terminals, and improves the safety and accuracy of surgery.

Description

Technical Field

[0001] This invention belongs to the field of medical device technology, specifically relating to a navigation system for endovascular interventional surgery, and particularly to a multi-view visualization navigation platform based on digital twins of blood vessels and instruments. Background Technology

[0002] With the development of interventional vascular technology, it has become an important means of treating cardiovascular and peripheral vascular diseases. However, traditional interventional surgery heavily relies on two-dimensional X-ray fluoroscopy and guidewire guidance, which has problems such as guidewire deviation, unintuitive imaging, and unclear instrument position and shape. As a result, current interventional vascular surgery can only provide palliative treatment and cannot eradicate lesions. Although preoperative CT, MRI and other three-dimensional images can provide rich anatomical information, how to accurately combine them with the real-time situation during the operation and track the precise three-dimensional shape of flexible instruments in the vascular lumen in real time remains a challenge for current technology.

[0003] Currently, there is a lack of navigation systems that can perform unified, real-time, and high-precision digital mapping of the vascular environment and instrument morphology, which limits the accuracy and safety of surgery and makes radical vascular interventional surgery impossible. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a multi-view visualization navigation platform based on digital twins of blood vessels and medical devices. By constructing a patient-individualized digital twin of blood vessels and a real-time digital twin of medical devices, multi-view fusion display is performed in a unified visualization interface, providing doctors with "see-through" navigation capabilities that surpass traditional two-dimensional images.

[0005] To achieve the above objectives, the present invention adopts the following technical solution, including: The vascular digital twin construction module is used to integrate patient multimodal imaging data and perceptual system data. Through physical modeling and data-driven methods, it generates an individualized vascular 3D digital twin model that includes vascular anatomy, functional parameters and virtual centerline. The model is stored in a tree-like hierarchical structure. The 2D / 3D image registration module achieves real-time dynamic high-precision registration between the digital twin of the blood vessel and the intraoperative two-dimensional image through feature point extraction and matching, PnP coarse localization and optimized fine localization with the blood vessel centerline as the core constraint, and outputs a pose matrix. The device digital twin construction module calculates the spatial pose and local curvature of the device based on data collected by the multimodal sensors at the tip of the implanted device. After coordinate system transformation, it generates and updates the three-dimensional digital twin model of the intravascular device in real time. The multi-view visualization navigation interface module achieves multi-coordinate system unification through marker-based electromagnetic-CT calibration, reconstructs the three-dimensional scene using geometric methods, and provides interactive multi-view views through the Qt framework, integrating and displaying digital twins of blood vessels, intraoperative two-dimensional images, and digital twins of instruments; The simulation analysis and interactive feedback module constructs a virtual surgical scene based on the digital twin of the blood vessel and the instrument. It calculates the interaction parameters between the instrument and the blood vessel through physical simulation, detects path deviation, predicts penetration risk, and generates operation suggestions and multi-dimensional feedback information.

[0006] Furthermore, the multimodal image data of the vascular digital twin construction module includes preoperative CTA and MRA data, and the sensor system data includes sensor and physiological monitoring equipment data; data preprocessing uses U-Net, V-Net deep learning models or thresholding and region growing methods to segment blood vessels and extract vessel walls, lumens, plaques, and thrombus structures; three-dimensional reconstruction uses isosurface or voxel reconstruction methods to output lightweight model files in obj or .vtk format.

[0007] Furthermore, the feature point extraction and matching process of the 2D / 3D image registration module includes: extracting significant feature points such as vascular bifurcation and calcification points from the preoperative vascular digital twin and projecting them onto the 2D plane; extracting the vascular contour of the DSA image through Canny edge detection + Hough transform during the operation, and achieving preoperative-intraoperative feature point matching through SIFT or ORB algorithms.

[0008] Furthermore, the multimodal sensors of the instrument digital twin construction module include a miniature electromagnetic sensor and a fiber Bragg grating sensor, which are used to sense the instrument's six-degree-of-freedom pose and local micro-strain and bending curvature, respectively; the shape reconstruction adopts a parametric spline curve model, with the electromagnetic sensor coordinates as control points and the curvature measured by the fiber Bragg grating sensor as the optimization constraint, and a smooth and continuous instrument centerline is solved through a nonlinear optimization algorithm.

[0009] Furthermore, the multi-view visualization navigation interface module includes a main operation view, a three-dimensional global view, an intraluminal cross-sectional view, and an auxiliary information panel. The main operation view is used to display the fusion of 2D images and 3D blood vessel contours. The three-dimensional global view is used to display the overall spatial relationship between the device and the blood vessel. The intraluminal cross-sectional view is used to display the distance between the device and the blood vessel wall. It uses OpenGL or Vulkan for accelerated graphics rendering and supports blood vessel transparency adjustment, lesion area highlighting, blood vessel centerline visualization, and automatic tracking of the device tip.

[0010] Furthermore, the simulation analysis and interactive feedback module uses position dynamics or the finite element method to construct a physical simulation engine, which calculates the force, deformation, and contact pressure between the device and the blood vessel wall in real time; it generates operation suggestions for adjusting the angle and propulsion speed based on a reinforcement learning model; the multi-dimensional feedback includes: visual feedback that displays the pressure distribution with color mapping, auditory cues that trigger offset / risk, and tactile feedback that simulates resistance with a force feedback device.

[0011] Furthermore, the multi-coordinate system is implemented as follows: during preoperative CT scanning, a marker kit containing at least three non-collinear points is placed on the patient's body surface; during surgery, the electromagnetic navigation system tracks the markers, and the optimal rigid body transformation is solved to obtain the transformation matrix from the electromagnetic coordinate system to the CT coordinate system. This enables spatial alignment of digital twins of blood vessels, digital twins of instruments, and intraoperative two-dimensional images.

[0012] Furthermore, the platform can be connected to a vascular interventional surgical robot system, providing navigation data through digital twins of blood vessels and instruments, and supporting semi-automatic operation, remote control operation, and remote surgical operation.

[0013] Furthermore, the platform can be adapted to various vascular types such as the femoral-popliteal artery, aorta, carotid artery, cardiovascular system, and cerebrovascular system, as well as various intravascular environments such as occlusive diseases, dilated diseases, plaques, thrombosis, and dissections, and various operating terminals such as flexible robots, vascular tunneling machines, laser fibers, drug-eluting hydrogel nozzles, and stent release systems.

[0014] Beneficial effects: 1. This invention constructs an integrated "anatomical-functional" digital twin of blood vessels and a real-time updated digital twin of instruments, realizing a comprehensive digital mapping of the surgical environment and providing doctors with intuitive and accurate multi-view three-dimensional navigation information; 2. This invention adopts a two-level registration strategy of "PnP coarse registration + centerline constraint fine registration" to overcome the influence of patient movement and position changes during surgery, and ensure that the digital twin and the real anatomical position are consistent in real time with high precision. 3. This invention is based on the multimodal fusion technology of "electromagnetic sensing + fiber optic grating sensing" to display the precise three-dimensional shape and position of flexible devices in blood vessels in real time, solving the core problem of unclear spatial relationship of devices under traditional two-dimensional fluoroscopy. 4. The multi-view visualization interactive interface of this invention, combined with accelerated graphics rendering, significantly improves the intuitiveness and accuracy of surgical operations, and is expected to shorten the learning curve for doctors, increase the success rate of surgery, and reduce dependence on radiation. 5. This invention supports assisted guidewire movement or guidewire-free movement, solving the inherent defects of traditional guidewire guidance such as eccentric movement, soft escape, anatomical resistance, and three-dimensional disorientation, enabling intravascular instruments to move strictly along the vascular centerline; 6. This invention has strong migration adaptability and can be flexibly adapted to various vascular types such as the femoral popliteal artery, aorta, carotid artery, cardiovascular system, and cerebrovascular system, as well as various intravascular environments such as occlusive diseases, dilated diseases, plaques, thrombosis, and dissections, and various operating terminals such as flexible robots, vascular tunneling machines, and laser fibers, making it applicable to a wide range of scenarios. Attached Figure Description

[0015] Figure 1 This is a schematic diagram of the process of this invention.

[0016] Figure 2 This is a flowchart of the 2D / 3D image registration process of the present invention.

[0017] Figure 3 This is a schematic diagram of the multi-view visual navigation interface of the present invention.

[0018] Figure 4 This is a flowchart of the digital twin shape reconstruction process of the device of the present invention.

[0019] Figure 5 This is a flowchart of the simulation analysis and interactive feedback module of the present invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0021] like Figure 1 As shown, this embodiment discloses a multi-view visualization navigation platform based on digital twins of blood vessels and medical devices. Its core lies in constructing and applying digital twins of blood vessels and medical devices. The platform is specifically implemented through the following technical modules: 1. Vascular Digital Twin Construction Module: Integrating multimodal imaging data such as preoperative CT and MRI scans of patients, and using physical and data-driven methods such as 3D reconstruction, finite element modeling, and computational fluid dynamics, it generates personalized 3D digital twins of blood vessels containing anatomical and functional information. Its data processing flow is as follows: 1.1 Data input: Preoperative CTA (CT angiography), MRA (magnetic resonance angiography) and other multimodal data of the patient.

[0022] 1.2 Data Preprocessing: This includes denoising, enhancement, and vessel segmentation (using deep learning models such as U-Net or V-Net, or traditional thresholding and region growing methods) to extract structures such as vessel walls, lumens, plaques, and thrombi. 1.3 Three-dimensional reconstruction: The vascular surface model is extracted and generated based on isosurface or voxel reconstruction methods.

[0023] 1.4 Centerline extraction: The centerline of the blood vessel is extracted using topology refinement or fast travel method, and serves as the benchmark for path planning and registration.

[0024] 1.5 Hemodynamic simulation: Based on computational fluid dynamics simulation, blood flow velocity and pressure distribution are simulated, and narrow areas are marked.

[0025] 1.6 Structural Mechanics Modeling: The finite element method is used to simulate the elasticity and deformation of the blood vessel wall, providing mechanical parameters for the simulation module.

[0026] 1.7 Data Structure: A tree-like hierarchical structure is used for storage. The root node represents the vascular system, and the child nodes represent the branch vessels. Each node contains a geometric model, centerline, physical attributes, lesion labels, etc.

[0027] 1.8 Output: Outputs lightweight 3D model files in .obj or .vtk format for use by the registration and visualization modules.

[0028] 2.2D / 3D Image Registration Module: This module enables dynamic linking of the digital twin of blood vessels with the real world; specific process... include: 2.1 Feature point extraction and matching: Based on the 3D model obtained from Module 1, significant feature points such as vascular bifurcation and calcification points are extracted and projected onto a 2D plane (simulating the DSA view). During the operation, based on the real-time DSA image, the vascular contour is extracted by Canny edge detection + Hough transform, and then the corresponding 2D points are found by SIFT or ORB feature matching.

[0029] 2.2 Coarse Registration: Based on the preoperatively reconstructed 3D vascular model obtained in Module 1, multiple significant feature points are selected. During the operation, feature point identification and segmentation are performed on the 2D X-ray image to obtain the corresponding coordinates of the feature points marked on the 3D model in the 2D image. Using the PnP algorithm, rapid 2D and 3D coarse registration is performed on the corresponding feature points in the 2D image and the 3D model to calculate the initial pose of the actual 3D vascular body relative to the current DSA during the operation.

[0030] 2.3 Fine Registration: Using the centerline of the 3D vascular model extracted preoperatively in Module 1 as the registration benchmark, 2D vascular images are captured in real time during the operation using DSA equipment, and the centerline of the vascular vessels in the images is extracted simultaneously. Using the two centerlines as core constraints, an optimization algorithm is used, with the result of coarse registration as the initial value of the algorithm, to accurately estimate the spatial pose of the actual vascular vessels, achieving fine registration from 2D images to 3D models.

[0031] 2.4 Registered Pose Matrix: Output the registered pose matrix to modules 3 and 5 to provide spatial alignment for visualization and simulation.

[0032] 3. Multi-source data fusion and visualization navigation interface: This module is the "presentation center" of the navigation platform. Its core function is to fuse and render digital twins of blood vessels and instruments from different coordinate systems, as well as intraoperative 2D images, under a unified spatiotemporal reference, and present them to the surgeon through an intuitive multi-view interactive interface. The specific process is as follows: 3.1 Marker-based Electromagnetic-CT Reference Calibration: A registration algorithm based on rigid markers is used to establish the coordinate system. The specific process is as follows: During preoperative CT scanning, a marker kit containing at least three non-collinear points is placed on the patient's body surface; during surgery, the same set of markers is tracked by the electromagnetic navigation system. By solving for the optimal rigid body transformation between the two sets of three-dimensional point sets, the accurate transformation matrix mapping the electromagnetic navigation coordinate system to the preoperative CT image coordinate system is calculated. .

[0033] 3.2 Scene fusion: Utilizing precise transformation matrices The instrument shape reconstruction module outputs a 3D model of the device in the electromagnetic coordinate system in real time, which is then uniformly converted to the CT coordinate system of the vascular digital twin. Simultaneously, the real-time pose matrix output by the 2D / 3D image registration module is used to visually project and align the vascular digital twin with the current fluoroscopic image (2DDSA). At this point, the device, vascular model, and 2D image achieve a precise spatial correspondence within the same 3D virtual scene.

[0034] 3.3 Multi-view Visualization and Interactive Interface Based on Qt: Develop a graphical user interface based on the Qt framework, providing multi-view (such as front view, side view, 3D rendered image), scalable, rotatable, and interactive navigation views. 3.3.1 Main Operation View: Displays a fused overlay of 2DDSA images and the outline of a three-dimensional vascular model, and is the main instrument navigation view.

[0035] 3.3.2 Three-dimensional global view: In the form of three-dimensional rendering, the complete shape and spatial position of the digital twin of the device in the blood vessel lumen are displayed from all angles.

[0036] 3.3.3 Intraluminal sectional view: A sectional view of the vascular cavity is generated along the direction of instrument travel, which visually shows the relative distance between the instrument and the vessel wall.

[0037] 3.3.4 Auxiliary Information Panel: Displays real-time information such as offset distance, angle, and warnings provided by the simulation feedback module.

[0038] 3.3.5 Rendering and Interaction Optimization: Utilizing OpenGL or Vulkan for accelerated graphics rendering ensures smooth interaction with complex 3D models. Key interactive functions are provided, such as vessel transparency adjustment, lesion area highlighting, visualization of the vessel centerline (path planning), instrument tip tracking, and automatic visual following.

[0039] 4. Intravascular Instrument Shape Reconstruction Module: This module is the core of constructing digital twins of instruments. It is responsible for reconstructing the three-dimensional geometry and spatial posture of flexible interventional instruments (such as catheters, guidewires, and flexible robots) in the vascular lumen in real time and with high precision under a unified world coordinate system.

[0040] 4.1 Multimodal sensing and data acquisition: A miniature sensing unit array is integrated at a predetermined spatial interval at the tip of interventional devices (such as catheters, guidewires, and flexible robots). This includes miniature electromagnetic sensors for sensing the six-degree-of-freedom (6DoF) position and orientation (attitude) of the device in an electromagnetic field established by an external field generator, and fiber Bragg grating sensors for measuring the local micro-strain and bending curvature of the device.

[0041] 4.2 Electromagnetic coordinate calculation: The system receives the raw signals from the electromagnetic sensors and calculates the three-dimensional coordinates and orientation quaternions of each electromagnetic sensor in the electromagnetic navigation system coordinate system in real time through a pre-calibrated magnetic field-position mapping model.

[0042] 4.3 Fine Shape Reconstruction: The calculated discrete sensor spatial position points are used as control points. A parametric spline curve model is used for fitting, with the sensor coordinates as constraints. The fitting process is optimized using local curvature data measured by fiber optic sensors to ensure that the reconstructed curve is smooth and continuous, and accurately reflects the physical bending state of the instrument, especially the shape between sensor intervals.

[0043] 4.4 Coordinate System Unification: The reconstructed instrument curves are transformed from the electromagnetic coordinate system to the preoperative CT image coordinate system (i.e., the coordinate system of the vascular digital twin) by using the fixed transformation matrix Tem→CT obtained by “marker-based electromagnetic-CT calibration” (see module 3.1).

[0044] 4.5 Digital Twin Entity Generation: Using the reconstructed center spline curve as the axis, and based on the instrument's preset physical diameter, a digital twin model of the instrument with a three-dimensional voxel or surface mesh representation is constructed in real time through geometric lofting or tubular mesh generation algorithms.

[0045] 4.6 Output and System Integration: The output of this module is a 3D geometric model of the device, updated over time and located in the same coordinate system as the digital twin of the blood vessel. This model is pushed in real time to: a multi-view visualization navigation interface: overlaid with the registered blood vessel model, intuitively presenting the precise position and shape of the device within the blood vessel lumen. Simulation Analysis and Interactive Feedback Module: serving as input for physical simulation, used to calculate device-blood vessel interaction, predict path deviation, and collision risk.

[0046] 5. Simulation Analysis and Interactive Feedback Module: This module performs physical simulation and decision support within the digital twin environment. The workflow is as follows: 5.1 Simulation Environment Construction: Based on digital twins of blood vessels (including geometric and physical properties) and digital twins of instruments (including shape, stiffness, and motion state), a virtual surgical scene is constructed.

[0047] 5.2 Physics Simulation Engine: Employs positional dynamics or the finite element method to simulate the contact, friction, and bending motion between instruments and blood vessels. This allows for real-time calculation of the force, deformation, and contact pressure between the device and the blood vessel wall.

[0048] 5.3 Path offset detection: Real-time calculation of the distance and angular deviation between the instrument and the centerline of the blood vessel.

[0049] 5.4 Penetration risk prediction: If the contact pressure exceeds the yield strength threshold of the blood vessel wall, an early warning will be triggered.

[0050] 5.5 Operation suggestion generation: Based on the reinforcement learning model, suggestions are given for adjusting the angle, speed, etc.

[0051] 5.6 Visual Feedback: Display stress distribution on the interface using color mapping (e.g., red → high risk). 5.7 Auditory feedback: When the deviation exceeds the limit or approaches the blood vessel wall, a warning sound is emitted.

[0052] 5.8 Tactile Feedback: Connect to isomorphic or heteromorphic master-slave teleoperated instruments to simulate resistance through force feedback devices.

[0053] Functionality implemented: 1. On the navigation screen, the registration and overlay results of the preoperative three-dimensional vascular digital twin and the intraoperative two-dimensional image are displayed in a multi-view window format.

[0054] 2. On the navigation screen, the three-dimensional shape of the digital twin of the intravascular device and its precise position within the registered digital twin of the blood vessel are displayed in real time in the form of a multi-view window.

[0055] 3. On the navigation screen, the centerline of the blood vessel is provided as the recommended travel path in the form of a multi-view window; information such as the offset distance between the device and the centerline of the blood vessel, the offset angle, the predicted travel result, the correction operation suggestions, and the warning of the risk of blood vessel wall penetration are provided.

[0056] The system in this embodiment mainly includes a data acquisition and processing server, an electromagnetic navigation and positioning system, an image display workstation, and interventional instruments. First, the vascular digital twin modeling module uses a deep learning vascular segmentation network (such as nnUNet) to automatically segment the CTA sequence and generate a vascular mask; then, it uses an isosurface algorithm to reconstruct a three-dimensional mesh model.

[0057] Centerline extraction employs topology refinement and preliminarily estimates hemodynamic parameters based on the Poiseuille flow formula, storing the results as a JSON-structured attribute file. Secondly, the device digital twin module constructs a high-precision digital twin of the interventional device using multi-sensor fusion technology.

[0058] Its core process is as follows: Real-time signals from miniature electromagnetic sensors and fiber Bragg grating sensors integrated into the flexible segment of the instrument are acquired; electromagnetic sensor data is used to calculate the spatial pose of discrete points, and fiber optic sensor data is used to measure local curvature and strain.

[0059] Subsequently, using the position of the electromagnetic sensor as the control point and the measured curvature as the key constraint, a nonlinear optimization problem was constructed. The objective function aimed to minimize the positional error between the reconstructed curve and the control point, as well as the difference between the reconstructed curve and the measured curvature. The Levenberg-Marquardt algorithm was used to solve the problem, resulting in a smooth and continuous three-dimensional B-spline curve of the instrument centerline.

[0060] Finally, using this centerline as the axis, a tubular three-dimensional mesh model is generated based on the physical diameter of the instrument. Then, using a pre-calibrated transformation matrix, the model is transformed from the electromagnetic navigation coordinate system to a unified preoperative CT coordinate system, thus completing the construction and spatial alignment of the instrument's digital twin.

[0061] like Figure 2 As shown, during the operation, the system uses the PnP algorithm to perform rapid coarse registration based on the correspondence between preoperative 3D model feature points and intraoperative DSA images to obtain the initial pose. Subsequently, using this pose as the initial value, a dual objective function is constructed that integrates "3D-2D centerline distance constraints" and "image similarity constraints based on normalized gradient mutual information," and an optimization algorithm is used for iterative solution to achieve sub-millimeter-level fine registration. During the operation, dynamic tracking is achieved through motion prediction and local optimization, and finally, a high-precision registration pose matrix is ​​continuously output, providing a unified spatial alignment reference for the visualization and simulation modules.

[0062] like Figure 3 As shown, in the visualization interface, the software developed using Qt renders the registered 3D vascular model, real-time 2D DSA images, and reconstructed instrument shapes in the same scene using multiple viewpoints, providing doctors with multiple observation windows such as the main viewpoint, intracavitary viewpoint, and 3D global viewpoint.

[0063] like Figure 4As shown, the electromagnetic sensor at the head of the device emits signals, which are received by the field generator. The positioning engine calculates the spatial coordinates of the sensor. Simultaneously, the fiber optic demodulator acquires the strain distribution along the length of the device in real time and converts it into local curvature. The device digital twin module uses the electromagnetic sensor pose as control points and the curvature measured by FBG as a strong constraint to construct and solve a nonlinear optimization problem. Finally, the continuous three-dimensional shape of the device is reconstructed through algorithms such as spline curve interpolation. A transformation matrix is ​​applied to update and overlay the shape in real time in the vascular digital twin.

[0064] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-view visualization navigation platform based on digital twins of blood vessels and medical devices, characterized in that, include: The vascular digital twin construction module is used to integrate patient multimodal imaging data and perceptual system data. Through physical modeling and data-driven methods, it generates an individualized vascular 3D digital twin model that includes vascular anatomy, functional parameters and virtual centerline. The model is stored in a tree-like hierarchical structure. The 2D / 3D image registration module achieves real-time dynamic high-precision registration between the digital twin of the blood vessel and the intraoperative two-dimensional image through feature point extraction and matching, PnP coarse localization and optimized fine localization with the blood vessel centerline as the core constraint, and outputs a pose matrix. The device digital twin construction module calculates the spatial pose and local curvature of the device based on data collected by the multimodal sensors at the tip of the implanted device. After coordinate system transformation, it generates and updates the three-dimensional digital twin model of the intravascular device in real time. The multi-view visualization navigation interface module achieves multi-coordinate system unification through marker-based electromagnetic-CT calibration, reconstructs the three-dimensional scene using geometric methods, and provides interactive multi-view views through the Qt framework, integrating and displaying digital twins of blood vessels, intraoperative two-dimensional images, and digital twins of instruments; The simulation analysis and interactive feedback module constructs a virtual surgical scene based on the digital twin of the blood vessel and the instrument. It calculates the interaction parameters between the instrument and the blood vessel through physical simulation, detects path deviation, predicts penetration risk, and generates operation suggestions and multi-dimensional feedback information.

2. The navigation platform according to claim 1, characterized in that, The multimodal imaging data of the vascular digital twin construction module includes preoperative CTA and MRA data, and the sensor system data includes sensor and physiological monitoring equipment data. Data preprocessing uses U-Net and V-Net deep learning models or thresholding and region growing methods to segment blood vessels and extract vessel walls, lumens, plaques, and thrombus structures. Three-dimensional reconstruction uses isosurface or voxel reconstruction methods to output lightweight model files in obj or .vtk format.

3. The navigation platform according to claim 1, characterized in that, The feature point extraction and matching process of the 2D / 3D image registration module includes: extracting significant feature points such as vascular bifurcation and calcification points from the preoperative vascular digital twin and projecting them onto the 2D plane; extracting the vascular contour of the DSA image through Canny edge detection + Hough transform during the operation, and achieving preoperative-intraoperative feature point matching through SIFT or ORB algorithms.

4. The navigation platform according to claim 1, characterized in that, The multimodal sensors in the digital twin construction module of the instrument include miniature electromagnetic sensors and fiber Bragg grating sensors, which are used to sense the six degrees of freedom pose of the instrument and local micro-strain and bending curvature, respectively. The shape reconstruction adopts a parametric spline curve model, with the coordinates of the electromagnetic sensors as control points and the curvature measured by the fiber Bragg grating sensors as optimization constraints. A smooth and continuous instrument centerline is solved through a nonlinear optimization algorithm.

5. The navigation platform according to claim 1, characterized in that, The multi-view visualization navigation interface module includes a main operation view, a 3D global view, an intraluminal cross-sectional view, and an auxiliary information panel. The main operation view is used to display the fusion of 2D images and 3D blood vessel contours. The 3D global view is used to display the overall spatial relationship between the device and the blood vessel. The intraluminal cross-sectional view is used to display the distance between the device and the blood vessel wall. It uses OpenGL or Vulkan for accelerated graphics rendering and supports blood vessel transparency adjustment, lesion area highlighting, blood vessel centerline visualization, and automatic tracking of the device tip.

6. The navigation platform according to claim 1, characterized in that, The simulation analysis and interactive feedback module uses position dynamics or finite element method to build a physical simulation engine to calculate the force, deformation and contact pressure between the device and the blood vessel wall in real time. Based on a reinforcement learning model, operational suggestions for adjusting the angle and propulsion speed are generated. Multi-dimensional feedback includes: visual feedback such as color mapping to display pressure distribution, auditory cues for offset / risk triggers, and tactile feedback such as force feedback devices to simulate resistance.

7. The navigation platform according to claim 1, characterized in that, The multi-coordinate system is implemented as follows: During preoperative CT scanning, a marker kit containing at least three non-collinear points is placed on the patient's body surface; during surgery, the electromagnetic navigation system tracks the markers, and the optimal rigid body transformation is solved to obtain the transformation matrix from the electromagnetic coordinate system to the CT coordinate system. This enables spatial alignment of digital twins of blood vessels, digital twins of instruments, and intraoperative two-dimensional images.

8. The navigation platform according to claim 1, characterized in that, The platform can be connected to a vascular interventional surgical robot system, providing navigation data through digital twins of blood vessels and instruments, and supporting semi-automatic operation, remote control operation, and remote surgical operation.

9. The navigation platform according to claim 1, characterized in that, The platform can be adapted to various vascular types such as the femoral-popliteal artery, aorta, carotid artery, cardiovascular system, and cerebrovascular system, as well as various intravascular environments such as occlusive diseases, dilated diseases, plaques, thrombi, and dissections, and various operating terminals such as flexible robots, vascular tunneling machines, laser fibers, drug-eluting hydrogel nozzles, and stent release systems.

Images