Interventional surgery navigation method based on multi-modal image fusion

Abstract

The invention relates to the field of cardiology department operations through vascular intervention, and discloses an interventional operation navigation system based on multi-modal image fusion, which comprises a preoperative three-dimensional reconstruction and planning module, which is mainly used for generating a three-dimensional model of a heart through preoperative image data, and planning the position and section of an intraoperative ICE ultrasonic probe based on the model, the intraoperative multi-modal fusion navigation module is mainly used for fusing a three-dimensional model generated before an operation with an intraoperative real-time image and providing accurate navigation support for a doctor; and the dynamic feedback optimization module is mainly used for updating a dynamic track in real time and performing safety protection. According to the invention, through the preoperative three-dimensional reconstruction and virtual path planning module, in combination with the improved 3D U-Net network and the high-precision segmentation algorithm, a three-dimensional model containing key anatomical markers is generated, 20 groups of ICE probe alternative paths are planned, seamless connection of preoperative three-dimensional anatomical information and intraoperative real-time images is realized, doctors can quickly call pre-planned paths, and the accuracy of the ICE probe anatomy is improved. Adjusting time during operation is shortened, and operation efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of cardiology surgery with transvascular intervention, and in particular to an interventional surgery navigation method based on multimodal image fusion. Background Art

[0002] As the incidence of cardiovascular diseases increases year by year, the application of transvascular interventional surgery in cardiology has gradually become popular in clinical practice. For some complex cardiovascular diseases, such as valvular disease, atrial fibrillation, left atrial appendage occlusion, etc., interventional treatment has been widely adopted as a relatively safe and effective treatment option. However, in these interventional treatments, the precise positioning and real-time monitoring of imaging technology are still the key factors to ensure the success of the operation. Existing imaging technologies include preoperative computed tomography (CTA) and magnetic resonance imaging (MRI), intraoperative digital subtraction angiography (DSA) and intracardiac ultrasound (ICE), etc. Although these technologies are widely used in clinical practice, they still face the following problems and shortcomings during actual surgery:

[0003] 1. Preoperative and intraoperative images are relatively independent: Preoperative images (such as CTA / MRI) are typically used to provide information about the patient's anatomical structure to assist physicians in preoperative planning, while intraoperative images (such as DSA / ICE) are used to monitor and guide interventional procedures in real time, helping physicians make real-time operational decisions. However, preoperative and intraoperative images are often relatively independent and lack effective fusion. As a result, preoperative three-dimensional anatomical information and intraoperative real-time dynamic information cannot be combined, making it impossible for physicians to quickly locate the target area according to preoperative planning, impacting surgical efficiency and accuracy.

[0004] 2. Disconnection between preoperative imaging and real-time surgical procedures: Due to the lack of effective integration and registration of preoperative and intraoperative images, surgeons often rely solely on 2D real-time imaging during surgery, without comprehensive reference to the detailed 3D anatomy provided preoperatively. This disconnect results in an incomplete understanding of the patient's anatomy during surgery, increasing the difficulty and risk of surgery. Especially in complex lesions, the crucial information provided by preoperative planning fails to effectively translate into intraoperative guidance.

[0005] 3. Difficulty in ultrasound section positioning: Intracardiac ultrasound (ICE) provides real-time, high-resolution images during surgery, helping physicians accurately observe the structure and pathology of the heart and vascular cavities. However, ICE ultrasound is complex to perform, and images are typically two-dimensional. Within the complex three-dimensional space, physicians often need a long time to find the ideal ultrasound section. This difficulty in section positioning not only increases procedure time but may also affect the accuracy and safety of ultrasound-guided interventional procedures.

[0006] 4. Lack of intraoperative navigation support: Traditional surgical navigation systems typically only provide two-dimensional real-time imaging data and cannot effectively integrate preoperative three-dimensional reconstruction information, resulting in a lack of sufficient spatial positioning reference during surgery. Although DSA and ICE images can help doctors observe the real-time status of blood vessels, their two-dimensional nature limits understanding of complex structures and operational guidance.

[0007] Existing technologies have certain deficiencies in the integration of preoperative and intraoperative images, real-time image navigation, and ultrasound section positioning. These issues not only affect the efficiency of interventional surgery, but also increase surgical risks and operational difficulty for doctors. Summary of the Invention

[0008] In order to remedy the above deficiencies, the present invention provides an interventional surgery navigation method based on multimodal image fusion, aiming to improve the problem of "very limited working operations" mentioned in the prior art.

[0009] To achieve the above objectives, the present invention adopts the following technical solution: an interventional surgery navigation system based on multimodal image fusion, comprising:

[0010] Preoperative 3D reconstruction and planning module: This module generates a 3D model of the heart using preoperative imaging data and plans the position and section of the ICE ultrasound probe during surgery based on this model.

[0011] Intraoperative multimodal fusion navigation module: This module integrates the preoperative 3D model with real-time intraoperative images to provide precise navigation support for doctors.

[0012] Dynamic feedback optimization module: mainly updates dynamic trajectories in real time and provides safety protection;

[0013] The preoperative three-dimensional reconstruction and planning module, the intraoperative multimodal fusion navigation module and the dynamic feedback optimization module are connected through network communication.

[0014] As a further description of the above technical solution:

[0015] The preoperative 3D reconstruction and planning module includes an image acquisition unit, a preprocessing unit, a heart segmentation unit, a 3D modeling unit, and a virtual planning unit. The image acquisition unit mainly imports the patient's CTA / MRI DICOM data, where the layer thickness is ≤0.5mm. The preprocessing unit uses non-local mean filtering to reduce noise and enhances vascular contrast based on histogram equalization.

[0016] As a further description of the above technical solution:

[0017] The cardiac segmentation unit uses an improved 3D U-Net network to automatically segment cardiac anatomical structures, including but not limited to the left atrium, pulmonary veins, and coronary arteries, with a segmentation accuracy of 94.2%. The 3D modeling unit generates an editable mesh model using the MarchingCubes algorithm and supports manual correction of segmentation errors.

[0018] As a further description of the above technical solution:

[0019] The virtual planning unit mainly marks the six standard section positions of the ICE probe in the model, including but not limited to the mid-esophageal two-chamber cardiac section and the gastric fundus short-axis section, and also generates a probe path library.

[0020] As a further description of the above technical solution:

[0021] The intraoperative multimodal fusion navigation module includes an image registration unit, a fusion display unit and a navigation guidance unit. The image registration unit mainly extracts vascular skeleton feature points in the DSA image, and aligns the DSA image with the three-dimensional model through an improved ICP algorithm, with a registration error of ≤1.2mm. The fusion display unit mainly superimposes the three-dimensional model on the DSA image in a semi-transparent mode, and also displays the ICE ultrasound section in a picture-in-picture format and compares it side by side with the virtual section. The navigation guidance unit mainly calculates the Euclidean distance between the ICE probe position and the virtual planning point in real time, and indicates the probe adjustment direction through an augmented reality arrow.

[0022] As a further description of the above technical solution:

[0023] The dynamic feedback optimization module includes a motion compensation unit and a risk warning unit. The motion compensation unit tracks the heart beat trajectory based on the optical flow method and dynamically updates the three-dimensional model position. In addition, respiratory motion compensation is synchronized with the abdominal belt sensor signal. The motion compensation unit mainly marks high-risk areas with heat maps in the fusion interface and triggers an audible and visual alarm when the catheter approaches the risk area.

[0024] As a further description of the above technical solution:

[0025] An interventional surgery navigation method based on multimodal image fusion, characterized by comprising:

[0026] S1. Perform 3D cardiac reconstruction based on preoperative CTA / MRI images to generate an editable model with anatomical landmarks.

[0027] S2. Virtually planning the target position of the ICE probe and the corresponding ideal ultrasound section in the model;

[0028] S3. performing spatiotemporal registration of the model with the intraoperative DSA image to achieve multimodal fusion display;

[0029] S4. Navigate the ICE probe to the virtual planned position based on the fused image and compare the actual / virtual ultrasound sections in real time.

[0030] As a further description of the above technical solution:

[0031] The three-dimensional reconstruction in S1 includes using a 3D U-Net network with an attention mechanism to segment cardiac tissue and perform topological checks on the segmentation results, automatically repairing holes and islands. The virtual planning in S2 includes: generating a path library for the ICE probe based on anatomical landmarks. The path library contains the six-degree-of-freedom posture parameters of the probe, and the path library supports selective call during surgery.

[0032] As a further description of the above technical solution:

[0033] The registration method in S3 is to extract the vascular branch points in the DSA image as feature points, and use the improved ICP algorithm for non-rigid registration. S4 also includes real-time calculation of the deviation value between the actual probe position and the virtual position. When the deviation exceeds the set threshold, the correction direction is prompted by the AR arrow.

[0034] As a further description of the above technical solution:

[0035] The invention further includes providing a multi-view mode in the fusion interface, specifically including a three-dimensional model perspective view, a DSA real-time image view, an ICE ultrasound cross-section view, and a probe path deviation trend view. In addition, the method is applicable to at least one of transcatheter aortic valve replacement, left atrial appendage occlusion, and radiofrequency ablation of atrial fibrillation. The present invention has the following beneficial effects:

[0036] 1. In Example 1 of the present invention, a preoperative 3D reconstruction and virtual path planning module, combined with an improved 3DU-Net network and a high-precision segmentation algorithm, generates a 3D model containing key anatomical landmarks and plans 20 sets of ICE probe alternative paths, achieving seamless integration of preoperative 3D anatomical information with intraoperative real-time images. Doctors can quickly call pre-planned paths, reducing intraoperative adjustment time and improving surgical efficiency. The precise overlay of the 3D model and DSA images in the fusion display interface provides an intuitive spatial positioning reference, reducing the risk of misjudgment due to the limitations of 2D images. Finally, by compensating for dynamic cardiac motion in real time, navigation deviations caused by anatomical displacement are reduced, and the high-risk area warning function significantly reduces the incidence of intraoperative complications.

[0037] 2. In the second embodiment of the present invention, based on the pre-planned standard section position of the ICE probe and combined with augmented reality arrow guidance, rapid probe navigation is achieved, which shortens the ICE section positioning time from 18.2±3.5 minutes to 6.8±1.2 minutes, significantly improving the fluency of the operation. The precise navigation reduces the dependence on DSA fluoroscopy, and the dynamic fusion display optimizes the catheter operation path, which reduces the DSA fluoroscopy time from 32.4±5.1 minutes to 17.6±2.8 minutes, significantly reducing the radiation dose for patients and medical staff. The real-time early warning function triggers a red alarm when the catheter approaches the visceral pericardium, and provides avoidance suggestions to ensure the safety of the operation.

[0038] 3. In the third embodiment of the present invention, the matching degree between the occluder and the opening is dynamically calculated through the three-dimensional model of the left atrial appendage, and the matching status is displayed in real time to reduce the risk of vascular damage caused by repeated adjustments. The short-axis and long-axis sections pre-planned by the ICE probe are used to accurately evaluate the position and adhesion effect of the occluder, thereby improving the long-term treatment effect. In addition, the alternative paths in the path library adapt to different anatomical variations, and the position of the catheter tip is tracked in combination with the electromagnetic positioning system, so that it can still achieve precise navigation under complex left atrial appendage morphology, expanding the scope of surgical indications. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 Schematic diagram of the system framework of an interventional surgery navigation method based on multimodal image fusion in the present invention;

[0040] Figure 2 Schematic diagram of a three-dimensional reconstruction model of a heart for an interventional surgery navigation method based on multimodal image fusion in the present invention;

[0041] Figure 3 A schematic diagram of the fusion of a three-dimensional reconstructed heart model and real-time DSA images in an interventional surgery navigation method based on multimodal image fusion in the present invention;

[0042] Figure 4 A schematic diagram of a heart three-dimensional reconstruction model, DSA images, and ICE ultrasound image multimodal fusion in an interventional surgery navigation method based on multimodal image fusion in the present invention;

[0043] Figure 5 A schematic diagram of an embodiment of an interventional surgery navigation method based on multimodal image fusion in the present invention deployed on a dedicated software and hardware platform and deployed in a hospital;

[0044] Figure 6 Schematic diagram of the framework of a preoperative 3D reconstruction and planning module of an interventional surgery navigation method based on multimodal image fusion in the present invention;

[0045] Figure 7Schematic diagram of the framework of an intraoperative multimodal fusion navigation module of an interventional surgery navigation method based on multimodal image fusion in the present invention;

[0046] Figure 8 Schematic diagram of a dynamic feedback optimization module of an interventional surgery navigation method based on multimodal image fusion in the present invention.

[0047] Legend:

[0048] 1. Preoperative 3D reconstruction and planning module; 2. Intraoperative multimodal fusion navigation module; 3. Dynamic feedback optimization module; 101. Image acquisition unit; 102. Preprocessing unit; 103. Heart segmentation unit; 104. 3D modeling unit; 105. Virtual planning unit; 201. Image registration unit; 202. Fusion display unit; 203. Navigation guidance unit; 301. Motion compensation unit; 302. Risk warning unit. DETAILED DESCRIPTION

[0049] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0050] Example 1

[0051] Reference Figure 1 - Figure 8 The present invention provides an interventional surgery navigation system based on multimodal image fusion, which is characterized by comprising a preoperative three-dimensional reconstruction and planning module 1, an intraoperative multimodal fusion navigation module 2, and a dynamic feedback optimization module 3. The preoperative three-dimensional reconstruction and planning module 1, the intraoperative multimodal fusion navigation module 2, and the dynamic feedback optimization module 3 are connected through network communication.

[0052] The preoperative 3D reconstruction and planning module 1 mainly generates a 3D model of the heart through preoperative image data, and plans the position and section of the ICE ultrasound probe during the operation based on the model. Specifically, the preoperative 3D reconstruction and planning module 1 includes an image acquisition unit 101, a preprocessing unit 102, a heart segmentation unit 103, a 3D modeling unit 104 and a virtual planning unit 105. The image acquisition unit 101 mainly imports the patient's CTA / MRI DICOM data, where the layer thickness is ≤0.5mm. The preprocessing unit 102 uses non-local mean filtering to reduce noise and enhances vascular contrast based on histogram equalization. The heart segmentation unit 103 uses an improved 3D U-Net network to automatically segment the heart's anatomical structures, including but not limited to the left atrium, pulmonary veins, and coronary arteries, with a segmentation accuracy of 94.2%. The 3D modeling unit 104 uses Marching The Cubes algorithm generates an editable mesh model and supports manual correction of segmentation errors. The virtual planning unit 105 mainly marks the six standard section positions of the ICE probe in the model, including but not limited to the mid-esophageal two-chamber cardiac view and the short-axis view of the gastric fundus. In addition, a probe path library is generated, which contains 20 sets of alternative paths to accommodate different anatomical variations.

[0053] The intraoperative multimodal fusion navigation module 2 mainly fuses the three-dimensional model generated before the operation with the real-time image during the operation to provide doctors with accurate navigation support. Specifically, the intraoperative multimodal fusion navigation module 2 includes an image registration unit 201, a fusion display unit 202 and a navigation guidance unit 203. The image registration unit 201 mainly extracts the vascular skeleton feature points in the DSA image, adopts the SIFT algorithm, and aligns the DSA image with the three-dimensional model through the improved ICP algorithm, with a registration error of ≤1.2mm. The fusion display unit 202 mainly superimposes the three-dimensional model on the DSA image in a semi-transparent mode with adjustable transparency, and also displays the ICE ultrasound section in picture-in-picture format and compares it side by side with the virtual section. The navigation guidance unit 203 mainly calculates the Euclidean distance between the ICE probe position and the virtual planning point in real time, and indicates the probe adjustment direction through augmented reality arrows, specifically left, right, forward and backward.

[0054] The dynamic feedback optimization module 3 mainly updates the dynamic trajectory in real time and performs safety protection. Specifically, the dynamic feedback optimization module 3 includes a motion compensation unit 301 and a risk warning unit 302. The motion compensation unit 301 tracks the heart beat trajectory based on the optical flow method and dynamically updates the three-dimensional model position. In addition, respiratory motion compensation is synchronized through the abdominal belt sensor signal. The motion compensation unit 301 mainly uses a heat map to mark high-risk areas in the fusion interface, such as within 5mm of the coronary artery trunk. When the catheter approaches the risk area, an audible and visual alarm is triggered, and the threshold value can be set in an interval.

[0055] Based on the above embodiment, the present invention further provides an interventional surgery navigation method based on multimodal image fusion, which specifically includes the following steps:

[0056] S1. Perform 3D cardiac reconstruction on preoperative CTA / MRI images to generate an editable model with anatomical landmarks. This 3D reconstruction includes cardiac tissue segmentation using a 3D U-Net network with an attention mechanism, as well as topological verification of the segmentation results to automatically repair holes and islands.

[0057] S2. Virtually plan the target position of the ICE probe and the corresponding ideal ultrasound section within the model. This virtual planning includes generating a path library for the ICE probe based on anatomical landmarks. The path library contains the probe's six-degree-of-freedom pose parameters and supports selective intraoperative recall.

[0058] S3. Temporally and spatially register the model with the intraoperative DSA images to achieve multimodal fusion display. The registration method extracts vascular branch points from the DSA images as feature points and uses an improved ICP algorithm for non-rigid registration.

[0059] S4. Based on the fused image, the ICE probe is navigated to the virtual planned position and the actual / virtual ultrasound sections are compared in real time. The system also calculates the deviation between the actual probe position and the virtual position in real time. When the deviation exceeds the set threshold, generally 2 mm by default, an AR arrow prompts the correction direction.

[0060] It further includes providing multiple view modes in the fusion interface, including a three-dimensional model perspective view, a DSA real-time image view, an ICE ultrasound section view, and a probe path deviation trend view. The three-dimensional model perspective view supports rotation at any angle, the DSA real-time image view can superimpose the vascular skeleton line, and the ICE ultrasound section view has a virtual section comparison box.

[0061] In addition, this method is deployed on a dedicated hardware platform, including an image processing unit equipped with an NVIDIA A100 GPU, a touch-screen navigation display supporting 4K resolution, and a multi-channel DICOM data acquisition interface. It needs to be compatible with Siemens or Philips DSA devices, and it also needs to use the ROS2 architecture to synchronize data between DSA and ICE devices, and track the position of the ICE catheter tip through an electromagnetic positioning system, such as Figure 5 shown.

[0062] Example 2

[0063] Atrial Fibrillation Ablation Navigation Practice Based on Example 1

[0064] Preoperative preparation: First, the patient's cardiac CTA data is input, with a slice thickness of 0.625 mm and a matrix of 512 × 512. 3D U-Net is then used to segment the left atrium and pulmonary veins, generating a 3D model with anatomical landmarks. The ICE probe's four standard sections are then planned within the model, including but not limited to the mid-esophageal two-chamber view and the bilateral superior pulmonary vein view.

[0065] During the intraoperative phase: First, real-time DSA images are acquired at a rate of 15 frames per second and registered with the 3D model. The ICE probe is then navigated to the planned position, and the spatial relationship between the ablation catheter and the pulmonary vein orifice is displayed in real time. When the catheter tip is less than 3 mm from the visceral pericardium, the fusion interface triggers a red alert.

[0066] The experimental results are as follows:

[0067]

[0068] Experimental results: ICE section positioning time and DSA fluoroscopy time were reduced by 62.6% and 45.7% respectively, and the surgical success rate increased from 83% to 96%. Through real-time warning function and multimodal fusion display, the potential risks during surgery were effectively reduced, the DSA fluoroscopy time was shortened, and the radiation exposure risk of patients and medical staff was reduced.

[0069] Example 3

[0070] Left atrial appendage occlusion navigation practice based on Example 1

[0071] Preoperative preparation: Reconstruct a three-dimensional model of the left atrial appendage (LAA) using CTA data, measure the diameter of the anchoring zone with an accuracy of ±0.3 mm, and then plan two key sections of the ICE probe in the right atrium, including the short-axis section and long-axis section of the LAA opening.

[0072] Intraoperative stage:

[0073] The occluder delivery sheath is superimposed on the 3D model, and the matching degree between the occluder release position and the left atrial appendage opening is dynamically calculated, with green matching greater than 90% and red matching less than 70%. Two key sections of the ICE probe in the right atrium are planned, including the short-axis section and long-axis section of the left atrial appendage opening.

[0074] Technical effect: The success rate of one-time occluder release increased from 78% to 94%, and the incidence of residual shunt decreased from 12% to 3%.

[0075] Finally, it should be noted that the above are only preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent replacements for some of the technical features therein. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. An interventional surgery navigation system based on multimodal image fusion, characterized by: include: Preoperative 3D reconstruction and planning module (1): mainly generates a 3D model of the heart through preoperative imaging data, and plans the position and section of the ICE ultrasound probe during surgery based on the model; Intraoperative multimodal fusion navigation module (2): mainly integrates the preoperative 3D model with the real-time intraoperative image to provide doctors with accurate navigation support; Dynamic feedback optimization module (3) mainly updates dynamic trajectories in real time and performs safety protection; The preoperative three-dimensional reconstruction and planning module (1), the intraoperative multimodal fusion navigation module (2) and the dynamic feedback optimization module (3) are connected via network communication.

2. The interventional surgery navigation system based on multimodal image fusion according to claim 1, characterized in that: The preoperative three-dimensional reconstruction and planning module (1) includes an image acquisition unit (101), a preprocessing unit (102), a heart segmentation unit (103), a three-dimensional modeling unit (104) and a virtual planning unit (105). The image acquisition unit (101) mainly imports DICOM data of patient CTA / MRI, wherein the layer thickness is ≤0.5 mm. The preprocessing unit (102) uses non-local mean filtering to reduce noise and enhances vascular contrast based on histogram equalization.

3. The interventional surgery navigation system based on multimodal image fusion according to claim 2, characterized in that: The heart segmentation unit (103) uses an improved 3D U-Net network to automatically segment the heart anatomical structure, including but not limited to the left atrium, pulmonary veins, and coronary arteries, with a segmentation accuracy of 94.2%. The three-dimensional modeling unit (104) generates an editable mesh model through a Marching Cubes algorithm and supports manual correction of segmentation errors.

4. The interventional surgery navigation system based on multimodal image fusion according to claim 3, characterized in that: The virtual planning unit (105) is mainly used to mark the six standard sections of the ICE probe in the model, including but not limited to the mid-esophageal bicaval section and the gastric fundus short axis section, and to generate a probe path library.

5. The interventional surgery navigation system based on multimodal image fusion according to claim 1, characterized in that: The intraoperative multimodal fusion navigation module (2) includes an image registration unit (201), a fusion display unit (202) and a navigation guidance unit (203). The image registration unit (201) mainly extracts vascular skeleton feature points in the DSA image and registers the DSA image with the three-dimensional model through an improved ICP algorithm, with a registration error of ≤1.2mm. The fusion display unit (202) mainly superimposes the three-dimensional model on the DSA image in a semi-transparent mode, and displays the ICE ultrasound section in a picture-in-picture format and compares it with the virtual section side by side. The navigation guidance unit (203) mainly calculates the Euclidean distance between the ICE probe position and the virtual planning point in real time, and indicates the probe adjustment direction through an augmented reality arrow.

6. The interventional surgery navigation system based on multimodal image fusion according to claim 1, characterized in that: The dynamic feedback optimization module (3) includes a motion compensation unit (301) and a risk warning unit (302). The motion compensation unit (301) tracks the heart beat trajectory based on the optical flow method and dynamically updates the three-dimensional model position. In addition, respiratory motion compensation is synchronized with the abdominal belt sensor signal. The motion compensation unit (301) mainly marks high-risk areas with a heat map in the fusion interface, and triggers an audible and visual alarm when the catheter approaches the risk area.

7. An interventional surgery navigation method based on multimodal image fusion according to claims 1-6, characterized in that: include: S1. Perform 3D cardiac reconstruction based on preoperative CTA / MRI images to generate an editable model with anatomical landmarks. S2. Virtually planning the target position of the ICE probe and the corresponding ideal ultrasound section in the model; S3. performing spatiotemporal registration of the model with the intraoperative DSA image to achieve multimodal fusion display; S4. Navigate the ICE probe to the virtual planned position based on the fused image and compare the actual / virtual ultrasound sections in real time.

8. The method for interventional surgery navigation based on multimodal image fusion according to claim 7, characterized in that: The three-dimensional reconstruction in S1 includes using a 3D U-Net network with an attention mechanism to segment cardiac tissue and perform topological checks on the segmentation results, automatically repairing holes and islands. The virtual planning in S2 includes: generating a path library for the ICE probe based on anatomical landmarks. The path library contains the probe's six-degree-of-freedom posture parameters, and the path library supports selective call-up during surgery.

9. The method for interventional surgery navigation based on multimodal image fusion according to claim 7, characterized in that: The registration method in S3 is to extract the vascular branch points in the DSA image as feature points, and use the improved ICP algorithm for non-rigid registration. S4 also includes real-time calculation of the deviation value between the actual probe position and the virtual position. When the deviation exceeds the set threshold, the correction direction is prompted by the AR arrow.

10. The method for interventional surgery navigation based on multimodal image fusion according to claim 7, characterized in that: It further includes providing a multi-view mode in the fusion interface, specifically including a three-dimensional model perspective view, a DSA real-time image view, an ICE ultrasound cross-section view, and a probe path deviation trend view. In addition, the method is applicable to at least one of transcatheter aortic valve replacement, left atrial appendage occlusion, and atrial fibrillation radiofrequency ablation.

Images