An intelligent navigation system for atrial septal puncture based on virtual-real fusion and electromagnetic positioning

Abstract

The application discloses an intelligent navigation system for atrial septostomy based on virtual-real fusion and electromagnetic positioning, and relates to the field of medical image processing and interventional operation navigation.The system comprises an image preprocessing and modeling module, a virtual-real mapping and posture correction module, a path planning and risk avoidance calculation module, a navigation deviation feedback module, a path self-adaptive re-planning module and a visual output module; through preoperative multi-modal cardiac image modeling and intraoperative electromagnetic positioning real-time fusion, a virtual-real unified cardiac structure model is constructed; based on puncture needle tip positioning signals and risk structure information, an optimal puncture path is dynamically generated and a deviation vector is real-timely fed back; path cycle self-adaptive updating is realized in combination with an electrocardiosignal; and finally, puncture navigation information is presented in the form of three-dimensional augmented reality; the application can significantly improve the accuracy and safety of atrial septostomy and has a good clinical application prospect.

Description

Technical Field

[0001] This invention relates to the field of medical image processing and interventional surgical navigation, specifically to an intelligent navigation system for atrial septal puncture based on virtual-real fusion and electromagnetic positioning. Background Technology

[0002] Transseptal puncture is a high-risk, critical step in structural cardiac interventional procedures (such as mitral valve clipping and left atrial appendage occlusion). The accuracy of the puncture directly impacts the incidence of postoperative complications and can even be life-threatening. Currently, this procedure primarily relies on the operator's experience and judgment under two-dimensional ultrasound guidance, inserting a needle from the right atrial septum into the left atrium. However, the limited spatial representation of two-dimensional imaging, its high anatomical variability, and the frequent dynamic changes in the heart during the procedure can easily lead to puncture errors, including accidental entry into the aorta, esophagus, or pericardial cavity. In severe cases, this can cause complications such as cardiac rupture and massive hemorrhage.

[0003] While existing navigation-assisted methods such as 3D ultrasound and CT fusion guidance can improve spatial cognition to some extent, they still cannot achieve high-precision prediction and correction of the puncture path due to imaging delays, time-consuming reconstruction calculations, and the lack of real-time intraoperative positioning feedback. Especially in special pathological structures such as atrial septal aneurysms, scar tissue, or postoperative reconstructed atria, the puncture window is narrow and the boundaries are blurred, making it difficult for traditional methods to accurately anchor the puncture target. Clinically, there is still a conservative attitude towards their reliability. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent navigation system for room-wide puncture surgery based on virtual-real fusion and electromagnetic positioning, so as to overcome the shortcomings of the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution: an intelligent navigation system for inter-room puncture surgery based on virtual-real fusion and electromagnetic positioning, comprising:

[0006] Image preprocessing and modeling module: Acquire preoperative three-dimensional cardiac image data of patients, perform structural segmentation based on atlas registration, extract anatomical feature points of the atrial septum, left atrium, right atrium and key adjacent tissues, and construct a preoperative virtual heart model; Virtual-real mapping and posture correction module: Spatial mapping and fusion of the preoperative virtual heart model and the puncture instrument position information collected in real time by the intraoperative electromagnetic positioning system, correcting the model drift caused by changes in patient posture, and obtaining a virtual-real fused heart structure model; Path planning and risk avoidance calculation module: In the virtual-real fusion heart structure model, based on the electromagnetic position signal of the puncture needle tip, the shortest safe path between it and the virtual left atrial target point is calculated, and combined with the puncture angle constraint and the risk area rejection field, the optimal puncture navigation path is generated. Navigation offset feedback module: Based on the optimal puncture navigation path, outputs the real-time puncture offset vector and feeds it back to the operator's terminal to indicate the current puncture deviation direction and correction amount; Path adaptive replanning module: When the real-time offset vector of the puncture exceeds the preset error threshold, the navigation path is updated by combining the historical sequence of the electromagnetic positioning point trajectory with the cardiac cycle phase synchronization information; Visualization output module: Outputs the final puncture window positioning result and overlays it onto the preoperative virtual heart model in a 3D augmented reality form on the operator's display interface to assist the operator in performing the puncture operation.

[0007] Preferably, the calculation of the shortest safe path between the puncture needle tip and the virtual left atrial target point based on the electromagnetic position signal of the puncture needle tip includes the following steps: Based on the real-time electromagnetic position signal of the puncture needle tip, a needle tip spatial state vector is constructed in the virtual-real fusion heart structure model, and a locally reachable spatial domain is generated by combining the atrial septum surface mesh. Within the locally reachable spatial domain, a set of candidate paths is established based on the location coordinates of the virtual left atrial target point using a weighted graph search algorithm, wherein the path weights are jointly determined by tissue thickness parameters, path curvature constraints, and distance factors to adjacent risk structures. A multi-objective cost function evaluation is performed on the candidate path set, and the path with the minimum cost is selected as the initial safe path. The initial safe path is checked for continuity and punctureability, and the shortest safe path that satisfies the minimum path length and maximum safety margin constraints is output.

[0008] Preferably, the optimal puncture navigation path is generated by combining the puncture angle constraint and the risk zone repulsion field, including the following steps: Based on the shortest safe path, the initial direction vector of the puncture needle tip at the starting point of the path is extracted, and a puncture angle constraint model is established in combination with the surface normal vector of the interatrial septum to limit the puncture direction to within a preset angle range. In the virtual-real fusion cardiac structure model, a continuous spatial risk zone rejection field is constructed for the aorta, pericardial cavity and esophagus. The rejection strength is inversely proportional to the distance from the puncture path node to the risk structure. The puncture angle constraint model and the risk zone rejection field are jointly introduced into the path optimization function to iteratively correct the shortest safe path and form a set of candidate navigation paths. The path that simultaneously satisfies angular stability and minimizes risk rejection cost is selected from the candidate navigation path set as the optimal puncture navigation path.

[0009] Preferably, the real-time puncture offset vector is output based on the optimal puncture navigation path, including the following steps: Obtain the electromagnetic position information of the current puncture needle tip of the puncture instrument and represent it as a real-time position point vector in the virtual-real fusion heart structure model; The real-time location point vector is orthogonally projected onto the curve of the optimal puncture navigation path, and the spatial coordinates and path tangent vector of the corresponding projection point are extracted. An offset vector is constructed based on the three-dimensional vector difference between the actual position of the puncture needle tip and the projection point, where the vector magnitude represents the distance deviation and the direction represents the offset direction.

[0010] Preferably, updating the navigation path includes the following steps: Record the electromagnetic positioning points of the puncture instrument at multiple moments during the heartbeat cycle to construct a historical sequence of the needle tip's movement trajectory; Based on synchronously acquired electrocardiogram signals, each location point is labeled with its corresponding cardiac cycle phase, and the cycle is standardized using the R wave as a reference point. A puncture needle tip motion model within the cardiac cycle is established using the historical sequence of the needle tip motion trajectory, and a weighted fitting method is used to predict the expected needle tip position distribution under the target phase. The predicted results are spatially compared with the original optimal puncture navigation path, and a new navigation path that satisfies phase stability is generated when the deviation exceeds a set threshold.

[0011] Preferably, the image preprocessing and modeling module is used to perform rigid and elastic registration of the three-dimensional cardiac images acquired by the patient before surgery based on cardiac atlas registration, and to perform semantic segmentation of the atrial septum, left atrium, right atrium, pulmonary vein inlet, aorta and esophagus structure through a deep learning segmentation network in order to extract key anatomical feature points and construct a preoperative virtual heart model.

[0012] Preferably, the virtual-real mapping and posture correction module establishes a preoperative model reference anatomical coordinate system and uses a minimum mean square error point cloud registration algorithm, combined with the real-time tracking of the puncture instrument position information by the electromagnetic positioning system, to achieve coordinate docking between the preoperative model and the intraoperative space; and performs real-time dynamic correction of model drift based on the posture drift vector field compensation mechanism to construct a virtual-real fusion cardiac structure model.

[0013] Preferably, when the magnitude of the offset vector exceeds a preset spatial error threshold, the offset vector is color-coded and rendered with a three-dimensional directional arrow, and then overlaid and displayed on the display interface.

[0014] The technical effects and advantages provided by the present invention in the above technical solution are as follows: 1. This invention, by constructing an integrated navigation system that combines preoperative image modeling, intraoperative electromagnetic positioning, intelligent path planning, and 3D augmented reality visualization, achieves for the first time enhanced spatial perception and dynamic path guidance throughout the entire atrial septal puncture procedure. Compared to traditional methods relying on two-dimensional images and the surgeon's experience, this invention can construct an individualized virtual model based on multimodal cardiac images and integrate the 3D position information of the puncture instruments in real time, providing high-precision and highly responsive navigation support, fundamentally improving the accuracy and safety of puncture positioning.

[0015] 2. This invention introduces an electrocardiogram synchronization mechanism and a puncture offset feedback mechanism, enabling phase-adaptive updates of the navigation path under dynamic changes in the cardiac cycle. Furthermore, it uses 3D augmented reality to intuitively overlay and display information such as the puncture window, path direction, and risk areas in the surgeon's field of vision, effectively improving the visualization and intervention response capabilities of the intraoperative procedure. The overall solution possesses the advantages of intelligent operation, precise navigation, and intuitive display, making it suitable for puncture assistance scenarios in various structural heart disease interventional surgeries and possessing significant clinical application value. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0017] Figure 1 This is a flowchart of the system modules of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] For examples, please refer to Figure 1 As shown in the figure, the intelligent navigation system for interstitial puncture based on virtual-real fusion and electromagnetic positioning described in this embodiment includes: Image preprocessing and modeling module: Acquire preoperative three-dimensional cardiac image data of patients, perform structural segmentation based on atlas registration, extract anatomical feature points of the atrial septum, left atrium, right atrium and key adjacent tissues, and construct a preoperative virtual heart model.

[0020] In this embodiment of the invention, the image preprocessing and modeling module is used to perform high-precision modeling of the patient's cardiac structure in the preoperative stage, providing basic anatomical support for subsequent virtual-real fusion navigation, path planning, and intraoperative visualization. This module mainly includes core steps such as 3D image acquisition, atlas registration, structural segmentation, and feature extraction, and its processing flow is as follows: Preferably, the three-dimensional image data can be acquired using multimodal medical imaging technologies, including but not limited to contrast-enhanced cardiac CT (CTA), cardiac MRI (CMR), or three-dimensional transesophageal ultrasound (3D TEE). Among these, CTA images have high spatial resolution and are suitable for capturing the boundaries of tissues such as the atrial septum, left atrium, and right atrium; CMR, on the other hand, has advantages in distinguishing between soft and hard tissues and can provide more accurate tissue contrast information.

[0021] To ensure modeling quality, it is recommended that the acquired images have a voxel resolution of at least 0.5 mm and cover the entire cardiac structure area, ensuring that the atrial septum and adjacent risk structures (such as the aorta, pulmonary veins, and esophagus) are fully represented.

[0022] After image acquisition, the system calls upon the built-in cardiac structural anatomy atlas and employs a combination of rigid and elastic deformation registration to achieve spatial alignment between individual images and the standard model. Specifically, the following steps are included: Rigid registration stage: Using the heart center, the direction of the heart axis and the long axis as references, the patient's cardiac image is roughly matched to the standard atlas coordinate system through affine transformation; Elastic registration stage: Based on B-spline or diffeomorphic deformation models, nonlinear deformation optimization is performed on local areas (such as the atrial septum and atrial wall) to ensure precise alignment of fine structures; Registration evaluation: The registration effect is automatically evaluated by using the Dice coefficient and the Hausdorff distance to ensure that the average overlap accuracy is higher than 90%.

[0023] After registration, a deep learning segmentation network (such as 3D U-Net or nnU-Net) is used to perform structural recognition and semantic segmentation on the cardiac image data. This network is pre-trained on a large-scale cardiac image dataset and can adapt to different image modalities. The segmentation results include the following key structures: the atrial septum, left atrial cavity and anterior wall of the left atrium, right atrial cavity and atrial appendage, the origin of the aorta, the pulmonary vein inlet, and the relative position of the esophagus, among other important adjacent anatomical structures. Subsequently, key anatomical feature points are extracted based on the segmentation results, including: The central point of the atrial septum (as the default puncture window candidate), the target area of ​​the left atrium (such as the confluence of the pulmonary veins), the tissue thickness distribution map (used for subsequent path avoidance calculations), and the interatrial distance and relative angle (used for path angle optimization) are extracted. Feature point extraction can be completed using algorithms such as edge detection, centerline extraction, and regional morphology analysis. The output is a structured feature list, which is used for subsequent path generation and 3D augmented reality modeling.

[0024] Finally, the system performs mesh reconstruction on the registered 3D image data and extracted anatomical structures to construct a preoperative virtual heart model. This model supports: realistic surface rendering, multi-structure semi-transparent display, and spatial anchoring functions that interface with the intraoperative electromagnetic positioning coordinate system. As a reference benchmark for virtual-real fusion, this model is continuously updated during the operation and drives the dynamic planning and navigation visualization of the puncture path.

[0025] Virtual-real mapping and posture correction module: Spatial mapping and fusion of the preoperative virtual heart model and the puncture instrument position information collected in real time by the intraoperative electromagnetic positioning system, correcting the model drift caused by changes in patient posture, and obtaining a virtual-real fused heart structure model.

[0026] In this embodiment of the invention, the virtual-real mapping and posture correction module is used to perform real-time spatial mapping and fusion of the preoperatively constructed virtual heart structure model and the puncture instrument position information collected by the intraoperative electromagnetic positioning system. The core objective of this module is to achieve precise alignment between the intraoperative dynamic scene and the preoperative static model, compensating for model offsets caused by factors such as patient position, respiratory movements, and tissue elastic deformation, thereby constructing a clinically valuable "virtual-real fusion" heart structure model, providing a reliable reference for subsequent puncture path navigation and surgeon operation interface. This module mainly includes the following three sub-functions: preoperative model coordinate system standardization, real-time acquisition of intraoperative electromagnetic data, and posture correction algorithm processing.

[0027] Before performing virtual-real fusion, the system first standardizes the spatial coordinate system of the preoperative virtual heart model. Specifically, this includes: Establish a reference anatomical coordinate system: with the long axis of the heart (from the apex of the heart to the center of the left atrium) as the Z-axis direction and the normal vector of the interatrial septum plane as the Y-axis direction, determine the three-dimensional right-handed coordinate system; Model rigidity normalization: Through scale standardization and rotation alignment, the virtual model is uniformly transformed into this anatomical coordinate system as an intraoperative fusion benchmark; Feature point spatial labeling: Record the position vectors of key anatomical feature points (such as the center of the atrial septum, the target point of the left atrium, etc.) extracted before surgery in the standard coordinate system for intraoperative matching.

[0028] This invention employs an intraoperative three-dimensional positioning system based on the principle of electromagnetic induction (such as products from Medtronic, NDI, or Biosense) to track the spatial position and orientation of puncture instruments (puncture needles, sheaths, guidewires, etc.) within the body in real time during surgery. This positioning system boasts sub-millimeter spatial accuracy and a sampling frequency of up to 40 Hz.

[0029] To improve the accuracy of spatial fusion, the system introduces the following processing methods: Multi-channel filtering and fusion: The original positioning points are processed by Kalman filtering to remove transient noise caused by electromagnetic interference; Attitude Quaternion Solution: The directional vector of the puncture instrument is interpolated and the attitude is smoothed using quaternion representation to avoid the singularity problem in Euler angle representation; Static calibration of reference points: During the procedure, a reference positioning sensor is placed at the atrial inlet or the root of the sheath as a stable anchor point to calibrate the overall system drift.

[0030] To achieve real-time fusion of preoperative models and intraoperative electromagnetic data, this invention proposes a fusion strategy based on rigid registration and attitude drift compensation, mainly including: In the initial stage, point cloud registration is performed using at least three sets of spatially corresponding points (such as the tip of the sheath, the center of the interatrial septum, and the bend of the guidewire), and the optimal rigid transformation matrix is ​​calculated using the minimum mean square error (RMSD) algorithm. The resulting rotation matrix and translation vector map the preoperative virtual model to the intraoperative positioning coordinate system. Intraoperative real-time calculation of the displacement difference between the model reference point and the electromagnetic anchor point; Based on time-series attitude changes, a drift vector field is constructed to dynamically fine-tune the virtual model; If the drift exceeds the threshold (e.g., 2 mm or 3 degrees), the model is triggered to re-register in real time to ensure accuracy. The final result is a virtual-real fusion cardiac structure model that matches the patient's current position and is aligned with the real-time position of the device. Its update frequency can reach more than 30 frames per second, ensuring that the system has clinical real-time response capabilities.

[0031] It should be noted that the virtual-real fusion model output by this module is transmitted to the path planning module and the 3D augmented reality visualization module through a unified data bus interface, enabling data sharing and state synchronization between modules. The system supports the visualization of fusion errors using heatmaps for surgeons' reference and intervention.

[0032] Path planning and risk avoidance calculation module: In the virtual-real fusion heart structure model, based on the electromagnetic position signal of the puncture needle tip, the shortest safe path between it and the virtual left atrial target point is calculated, and combined with the puncture angle constraint and the risk area rejection field, the optimal puncture navigation path is generated.

[0033] In this embodiment of the invention, to achieve the shortest safe path planning from the puncture needle tip to the virtual left atrial target, it is necessary to construct the needle tip spatial state based on electromagnetic signal data in a virtual-real fusion cardiac structure model, extract the reachable spatial region, and perform path calculation and verification under various physiological and anatomical constraints. Specifically, the steps include: Based on the real-time electromagnetic position signal of the puncture needle tip, its position coordinates and direction vector in three-dimensional space are extracted, and a local coordinate system is established in the virtual-real fusion heart structure model with this point as the center. Let the current spatial position of the needle tip be point P0, and its unit direction vector be D0, then the spatial state vector of the needle tip is defined as [P0, D0].

[0034] Using this state vector as a reference, a local three-dimensional mesh region with a radius of r (e.g., 20 mm) centered at P0 is constructed by combining the mesh surface nodes of the atrial septum, which is then defined as a locally accessible spatial domain. Within this spatial domain, only surface points where the angle between the puncture direction and the normal to the atrial septum is less than 45 degrees are retained to ensure the permeability of subsequent paths.

[0035] Within the aforementioned locally reachable spatial domain, a three-dimensional weighted graph is constructed based on the target point location coordinates T (i.e., the center of the preset puncture target area in the virtual left atrium). Each node in the graph represents a reachable point on the grid surface, and the edges represent puncture path segments.

[0036] To assess organizational security, the edge weight W is composed of the following three types of factors: Tissue thickness parameter H(x): represents the thickness of the atrium tissue. The smaller the thickness, the higher the puncture risk, and the higher the weight assigned. Path curvature constraint C(x): defined as the cosine of the angle between the directions of the preceding and following path segments; the larger the angle, the higher the weight. The proximity risk structure distance factor R(x) is defined as the inverse function of the distance from the point to the nearest high-risk structure (such as the aorta or pericardial cavity). The closer the distance, the higher the weight.

[0037] The total edge weight is then: W = α·H(x) + β·C(x) + γ·R(x), where α, β, and γ are weight coefficients, preferably set to [0.4, 0.3, 0.3]. This is based on Dijkstra's algorithm or A*. The algorithm uses a graph search method to generate a set of several candidate paths leading to the target T.

[0038] For each path in the candidate path set, a multi-objective cost function is evaluated. The sum of the cost function F and the integral values ​​of the three types of factors mentioned above are used to measure the safety and feasibility of the path. The path with the smallest F value is defined as the initial safe path. This path is the globally shortest cost path in the local spatial domain.

[0039] To ensure the path is practically feasible, the following two checks are performed on the initial safe path: Continuity check: Traverse the path node sequence to confirm that the path direction changes smoothly and the angle between unit vectors does not exceed 30 degrees. Punctureability check: The thickness of the room grid through which the path passes must be greater than 2 mm, and the angle between the path and the normal of the room grid must not exceed 60 degrees.

[0040] If the above test is passed, the path is defined as the shortest safe path that satisfies the minimum path length and maximum safety margin constraints.

[0041] Based on the completion of the shortest safe path planning, this invention, to further improve the stability and safety of intraoperative puncture navigation, introduces a puncture angle constraint model and a risk structure repulsion field model to optimize and correct the path, generating the optimal puncture navigation path. Specifically, it includes the following steps: First, extract the needle tip direction vector D0 at the starting point of the shortest safe path, and obtain the normal vector N0 of the corresponding atrial septum surface unit. The system defines the puncture angle θ as the angle between D0 and N0.

[0042] When constructing the puncture angle constraint model, an acceptable range for θ values ​​is defined as 20 to 45 degrees. If a segment of the path has a puncture angle exceeding this range, a penalty term is applied to the optimization function to drive the path towards an acceptable angle. This angle constraint model is defined as: A(x) = 0, when... ; ,when λ is the penalty coefficient, typically 5.

[0043] In the virtual-real fusion cardiac structure model, a continuous spatial repulsion field is constructed based on the three-dimensional anatomical boundaries of key anatomical structures around the atrial septum (including the aorta, pericardial cavity, and esophagus).

[0044] The repulsive field strength R(x) is defined as: ; where d(x) is the shortest Euclidean distance from path point x to the nearest risk structure, δ is the maximum repulsion strength (e.g., 100), and ε is a minimal constant to prevent division by zero errors (e.g., 0.01).

[0045] During the path search process, the risk repulsion field uses the field strength as the path energy penalty term to avoid the path getting too close to high-risk anatomical structures.

[0046] The puncture angle constraint model A(x) and the risk rejection field R(x) are introduced into the path optimization function F', and combined with the original path cost function F to form a composite energy function: F'=F+θ1·A(x)+θ2·R(x); Where θ1 and θ2 are the model fusion weights, with preferred values ​​of θ1=0.6 and θ2=0.8. Based on this composite energy function, the shortest path is iteratively perturbed using heuristic optimization algorithms (such as simulated annealing or ant colony optimization) to generate multiple feasible navigation paths.

[0047] The candidate path set above is scored, and paths that meet the following two criteria are selected first: Angle stability index S1: The variation range of the puncture angle throughout the entire path is less than 20 degrees; Risk exclusion cost minimization index S2: Minimize the overall path R(x) integral value.

[0048] If multiple paths satisfy the above two indicators, the one with the smallest F' value is selected as the optimal puncture navigation path.

[0049] Navigation offset feedback module: Based on the optimal puncture navigation path, outputs the real-time puncture offset vector and feeds it back to the operator's terminal to indicate the current puncture deviation direction and correction amount.

[0050] In the implementation of this invention, to assist the surgeon in real-time perception of the spatial deviation between the puncture path and the navigation path, thereby improving the accuracy and safety of the puncture operation, the puncture offset vector is calculated in real-time based on the spatial mapping relationship between the electromagnetic positioning information of the puncture needle tip and the optimal puncture navigation path. The direction of this offset vector indicates the direction in which the current puncture needle tip deviates from the navigation path, and the vector length represents the distance error. This error is then visually fed back to the surgeon's interface. Specifically, the steps include the following: The position of the tip of the puncture instrument (such as a puncture needle, catheter, or sheath) is continuously tracked by an electromagnetic positioning system deployed during the operation. The positioning data is collected in real time in the form of three-dimensional spatial coordinates and recorded as the current position of the puncture needle tip, Preal=(xr,yr,zr).

[0051] The system has completed the spatial registration of the virtual-real fusion cardiac structure model and the electromagnetic positioning coordinate system before the operation. Therefore, the above coordinates can be directly mapped to the unified spatial coordinate system in the fusion model.

[0052] The position of this point is represented as a three-dimensional vector NPreal, which serves as the real-time state of the needle tip in space during subsequent calculations.

[0053] The optimal puncture navigation path has been generated in the preceding steps and is denoted as path curve C(s), where s represents the arc length parameter on the path. This path is stored as a discrete point sequence, containing a series of ordered path nodes {P1, P2, ..., Pn}, and can be continuously expressed as a path curve through linear interpolation or cubic spline curves.

[0054] For the current needle tip position NPreal, the minimum Euclidean distance criterion is used to orthogonally project it onto the path curve C(s), and the path point NPproj that satisfies the following conditions is searched: The foot of the perpendicular from a point to a path is an actual point or interpolation point on the path; The line connecting NPproj and NPreal is perpendicular to the tangent vector of the path at that point.

[0055] The spatial coordinates of the projection point NPproj are recorded for subsequent difference calculations. Simultaneously, the unit tangent vector NTproj is calculated at this point on the path, i.e., the path tangent vector. , where s0 is the path parameter value corresponding to the projection point. This tangent vector represents the current theoretical puncture direction and serves as a reference for subsequent offset direction comparisons.

[0056] An offset vector NDoffset is constructed using the spatial difference between the actual position vector NPreal of the puncture needle tip and its orthogonal projection vector NPpro on the navigation path: the offset vector is defined as: NDoffset = NPreal NPproj; This vector's three-dimensional direction represents the specific direction in which the current puncture path deviates from the navigation path, and its magnitude... This indicates the spatial offset distance, in millimeters (mm).

[0057] In this invention, if the offset distance exceeds a preset clinically acceptable threshold (e.g., 3 mm), a navigation path update or real-time alarm mechanism is triggered. This threshold can be set by clinical experts and adjusted based on empirical statistics or individual anatomical characteristics.

[0058] In addition, to guide the operator to correct the puncture action in the correct direction, the angle between the offset vector and the path tangent vector can be calculated to prompt the operator to adjust the puncture angle or direction.

[0059] To enhance the intuitiveness of clinical operations, the offset vector NDoffset is mapped onto the surgeon's 3D display terminal and superimposed on the virtual-real fusion cardiac structure model. The offset vector is presented in the form of a 3D directional arrow: Starting point: The actual location of the puncture needle tip; Point of view: The direction of the projection point on the navigation path; Color: The color changes according to the offset distance, for example, green (<2 mm), yellow (2~3 mm), red (>3 mm); Length: Scaled proportionally to the magnitude of the offset vector to visually reflect the size of the deviation.

[0060] In addition, the offset vector data is synchronously transmitted to the surgeon's interactive control panel through the data interface, presenting the real-time offset in numerical form, and providing auditory or tactile feedback interfaces (such as vibration prompts) to prompt the surgeon to correct the path or reduce the angle deviation.

[0061] Path adaptive replanning module: When the real-time offset vector of the puncture exceeds the preset error threshold, the navigation path is updated by combining the historical sequence of the electromagnetic positioning point trajectory with the phase synchronization information of the cardiac cycle.

[0062] To address the issue of navigation path stability caused by intraoperative displacement of the puncture instrument due to cardiac pulsation, this invention proposes a dynamic navigation path update method that combines the historical sequence of the electromagnetic positioning point trajectory of the puncture needle tip with ECG cycle phase synchronization information. This method utilizes the fusion of multi-time-phase spatial location information and ECG time stamps to establish a phase-dependent spatial model, enabling the prediction and reconstruction of puncture path deviation trends under different cardiac cycles. Specifically, it includes the following steps: During the procedure, an electromagnetic sensor attached to the tip of the puncture instrument continuously outputs its position data in three-dimensional space. The system samples the needle tip position at fixed time intervals (e.g., every 25 milliseconds), continuously recording its dynamic changes during cardiac pulsation. Each positioning point is represented in the form of a three-dimensional vector: Pi=(xi,yi,zi),ti∈[t0,tn]; where ti represents the timestamp, and Pi is the spatial position at the corresponding time point.

[0063] The collected time-point sequences are organized into a time-ordered set to form the historical sequence of needle tip movement trajectory T={(t1,P1),(t2,P2),...,(tn,Pn)}, which reflects the spatial movement trend of the puncture instrument within multiple cardiac cycles.

[0064] Electrocardiogram (ECG) signals are synchronously acquired at high frequencies (e.g., 1 kHz) using surface electrodes, recording the changes in electrical signal waveforms throughout a complete cardiac cycle. The system uses the R wave as a reference marker for the cardiac cycle, automatically detecting the time point tRk of each R wave occurrence, and defining the time interval between two consecutive R waves as a complete cardiac cycle.

[0065] For each electromagnetic positioning point Pi, calculate its corresponding cardiac cycle number k, and map its relative position within the cycle to a normalized cardiac phase based on the start and end times of that cycle. i∈[0,1], where: This method maps all electromagnetic positioning points to the same periodic time scale, providing a phase-aligned data basis for subsequent fitting and modeling.

[0066] The set of positioning points with labeled phase information {( The heart rate is segmented according to the phase from 0 to 1, and weighted least squares fitting is performed on all points in each phase interval (e.g., every 0.05 phase step).

[0067] The following method is used to fit the x, y, and z coordinate components as a function of phase. Changing function: x( )=a0+a1 +a2 2+...+an nx; y( )=b0+b1 +b2 2+...+bn n; z( )=c0+c1 +c2 2+...+cn n; Where n is the order of the polynomial, preferably 3 or 4; the weighting function is assigned inversely proportional to the offset of each point from the reference path, so as to reduce the influence of abnormal deviation points on the fitting.

[0068] After fitting, the system can then determine the phase of any given cardiac cycle. t (e.g., the phase at the current time point) predicts the spatial position EP of the puncture needle tip. t=(x( t),y( t),z( The predicted location reflects the ideal spatial position that the puncture instrument should reach under the current cardiac motion state.

[0069] The system will predict the needle tip position EP under the current phase. The reference path point Pref, which is in the same phase as the original optimal puncture navigation path, t performs spatial comparison.

[0070] Calculate the three-dimensional Euclidean distance deviation between the two: If the deviation exceeds the set threshold δmax, for example, 3.5 mm, the system determines that the current path has a risk of spatial instability in the state of cardiac synchronization.

[0071] At this point, the system triggers a path reconstruction process, using the fitted point sequence from the fitting results as the new reference path data source. It then re-executes the optimal puncture navigation path planning process (such as re-constraining the puncture angle and updating the risk rejection field calculation), generating a phase-adaptive navigation path that satisfies the current phase stability constraints. This updated path will serve as the navigation benchmark under the current cardiac phase and will proceed to the next offset vector calculation and path tracking process.

[0072] Visualization output module: Outputs the final puncture window positioning result and overlays it onto the preoperative virtual heart model in a 3D augmented reality form on the operator's display interface to assist the operator in performing the puncture operation.

[0073] In the implementation of this invention, to improve the surgeon's perception of the spatial relationship between the puncture path and key cardiac anatomical structures, and to reduce reliance on subjective judgment of two-dimensional images, a three-dimensional augmented reality approach is used to visualize the final puncture window positioning result, and this information is presented in a spatial overlay form on the surgeon's display interface. The core function of this module is to combine the virtual puncture path with real intraoperative positioning data to construct an intuitive and highly interactive three-dimensional visual navigation interface. Its technical implementation includes the following aspects: The final puncture window localization result is output by the aforementioned path optimization module, including: The coordinates of the puncture entry point (i.e., the position where the puncture needle will soon touch the interatrial septum) are Pentry; The coordinates of the puncture target (i.e., the target area located in the left atrium) are Ptarget; Path direction vector EDpath=Ptarget Pentry; The results of the puncture angle constraint range and the risk structure distance assessment.

[0074] The positioning results are recorded in a structured data format and bound to cardiac phase markers for dynamic updating and synchronization control in augmented reality displays.

[0075] The preoperative virtual cardiac model has been spatially aligned using atlas registration and an electromagnetic positioning system, and is constructed within a unified world coordinate system. The augmented reality display module employs a 3D rendering space consistent with this coordinate system, ensuring that the positional relationships between all puncture paths and anatomical structures are realistically comparable.

[0076] If the surgeon uses a wearable augmented reality display device (such as optical vision AR glasses), the model space is dynamically registered using the surgeon's eye view as the observation reference; if a fixed three-dimensional display terminal is used, the intraoperative calibration device provides reference coordinates for anchoring, thereby achieving a stable and controllable spatial display reference.

[0077] The system builds an augmented reality interface based on a 3D rendering engine (such as OpenGL or Unity platform) and overlays the following information onto the preoperative virtual heart model for real-time rendering: Puncture path rendering: The puncture path is represented by a colored semi-transparent cylinder; the starting point connects to the puncture entry point Pentry, and the ending point points to the target point Ptarget; the path is surrounded by a transparent safety cone to indicate the acceptable puncture angle range.

[0078] Puncture window area marking: The puncture entry area is marked on the corresponding area of ​​the atrial septum surface with a bright circle or a flat shadow; if the puncture point is located in atrial septum scar tissue or a suspected weak area, it will automatically change color to indicate (such as red or yellow).

[0079] Key structures are transparently visualized: Structures such as the left atrium, right atrium, atrial septum, aorta, and esophagus in the virtual model are displayed with different levels of transparency; users can rotate the model, adjust the viewpoint, and turn structural layers on or off using gestures or controllers.

[0080] Real-time position and offset indication: If the position of the puncture needle tip is obtained in real time through the electromagnetic positioning system, its actual position is marked with an arrow or a light dot; when the deviation from the optimal path exceeds the preset threshold, the offset vector is displayed on the model to prompt the surgeon to make corrections.

[0081] To improve the convenience of intraoperative procedures and navigation accuracy, this invention provides the following human-computer interaction functions: Voice control is used to switch display modes, lock paths, or recalibrate models; The graphical panel is used to display numerical information such as path angle, distance to the risk area, and puncture depth; The operator can use gestures to select the path prediction results under different phases and observe the effect of the cardiac cycle on path stability. If the model and instrument are misaligned, the system can trigger automatic calibration or pop up a prompt requiring re-registration.

[0082] Through the aforementioned visualization output module, the operator can accurately identify the puncture site and grasp the puncture direction in a spatially intuitive and real-time interactive 3D navigation environment, and dynamically adjust the operation process according to augmented reality prompts, effectively improving the accuracy, safety, and intraoperative responsiveness of atrial septal puncture.

[0083] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A room-wide puncture intelligent navigation system based on virtual-real fusion and electromagnetic positioning, characterized in that: include: Image preprocessing and modeling module: Acquire preoperative three-dimensional cardiac image data of patients, perform structural segmentation based on atlas registration, extract anatomical feature points of the atrial septum, left atrium, right atrium and key adjacent tissues, and construct a preoperative virtual heart model; Virtual-real mapping and posture correction module: Spatial mapping and fusion of the preoperative virtual heart model and the puncture instrument position information collected in real time by the intraoperative electromagnetic positioning system, correcting the model drift caused by changes in patient posture, and obtaining a virtual-real fused heart structure model; Path planning and risk avoidance calculation module: In the virtual-real fusion heart structure model, based on the electromagnetic position signal of the puncture needle tip, the shortest safe path between it and the virtual left atrial target point is calculated, and combined with the puncture angle constraint and the risk area rejection field, the optimal puncture navigation path is generated. Navigation offset feedback module: Based on the optimal puncture navigation path, outputs the real-time puncture offset vector and feeds it back to the operator's terminal to indicate the current puncture deviation direction and correction amount; Path adaptive replanning module: When the real-time offset vector of the puncture exceeds the preset error threshold, the navigation path is updated by combining the historical sequence of the electromagnetic positioning point trajectory with the cardiac cycle phase synchronization information; Visualization output module: Outputs the final puncture window positioning result and overlays it onto the preoperative virtual heart model in a 3D augmented reality form on the operator's display interface to assist the operator in performing the puncture operation.

2. The intelligent navigation system for inter-room puncture surgery based on virtual-real fusion and electromagnetic positioning according to claim 1, characterized in that: The calculation of the shortest safe path between the puncture needle tip and the virtual left atrial target point based on the electromagnetic position signal of the puncture needle tip includes the following steps: Based on the real-time electromagnetic position signal of the puncture needle tip, a needle tip spatial state vector is constructed in the virtual-real fusion heart structure model, and a locally reachable spatial domain is generated by combining the atrial septum surface mesh. Within the locally reachable spatial domain, a set of candidate paths is established based on the location coordinates of the virtual left atrial target point using a weighted graph search algorithm, wherein the path weights are jointly determined by tissue thickness parameters, path curvature constraints, and distance factors to adjacent risk structures. A multi-objective cost function evaluation is performed on the candidate path set, and the path with the minimum cost is selected as the initial safe path. The initial safe path is checked for continuity and punctureability, and the shortest safe path that satisfies the minimum path length and maximum safety margin constraints is output.

3. The intelligent navigation system for room-wide puncture surgery based on virtual-real fusion and electromagnetic positioning according to claim 2, characterized in that: The optimal puncture navigation path is generated by combining the puncture angle constraint and the risk zone repulsion field, including the following steps: Based on the shortest safe path, the initial direction vector of the puncture needle tip at the starting point of the path is extracted, and a puncture angle constraint model is established in combination with the surface normal vector of the interatrial septum to limit the puncture direction to within a preset angle range. In the virtual-real fusion cardiac structure model, a continuous spatial risk zone rejection field is constructed for the aorta, pericardial cavity and esophagus. The rejection strength is inversely proportional to the distance from the puncture path node to the risk structure. The puncture angle constraint model and the risk zone rejection field are jointly introduced into the path optimization function to iteratively correct the shortest safe path and form a set of candidate navigation paths. The path that simultaneously satisfies angular stability and minimizes risk rejection cost is selected from the candidate navigation path set as the optimal puncture navigation path.

4. The intelligent navigation system for inter-room puncture surgery based on virtual-real fusion and electromagnetic positioning according to claim 3, characterized in that: Based on the optimal puncture navigation path, the real-time puncture offset vector is output, including the following steps: Obtain the electromagnetic position information of the current puncture needle tip of the puncture instrument and represent it as a real-time position point vector in the virtual-real fusion heart structure model; The real-time location point vector is orthogonally projected onto the curve of the optimal puncture navigation path, and the spatial coordinates and path tangent vector of the corresponding projection point are extracted. An offset vector is constructed based on the three-dimensional vector difference between the actual position of the puncture needle tip and the projection point, where the vector magnitude represents the distance deviation and the direction represents the offset direction.

5. The intelligent navigation system for room-wide puncture surgery based on virtual-real fusion and electromagnetic positioning according to claim 1, characterized in that: The updated navigation path includes the following steps: Record the electromagnetic positioning points of the puncture instrument at multiple moments during the heartbeat cycle to construct a historical sequence of the needle tip's movement trajectory; Based on synchronously acquired electrocardiogram signals, each location point is labeled with its corresponding cardiac cycle phase, and the cycle is standardized using the R wave as a reference point. A puncture needle tip motion model within the cardiac cycle is established using the historical sequence of the needle tip motion trajectory, and a weighted fitting method is used to predict the expected needle tip position distribution under the target phase. The predicted results are spatially compared with the original optimal puncture navigation path, and a new navigation path that satisfies phase stability is generated when the deviation exceeds a set threshold.

6. The intelligent navigation system for inter-room puncture surgery based on virtual-real fusion and electromagnetic positioning according to claim 1, characterized in that: The image preprocessing and modeling module is used to perform rigid and elastic registration of the three-dimensional cardiac images acquired by the patient before surgery based on cardiac atlas registration. It also uses a deep learning segmentation network to perform semantic segmentation of the atrial septum, left atrium, right atrium, pulmonary vein inlet, aorta and esophagus to extract key anatomical feature points and construct a preoperative virtual heart model.

7. The intelligent navigation system for inter-room puncture surgery based on virtual-real fusion and electromagnetic positioning according to claim 1, characterized in that: The virtual-real mapping and posture correction module establishes a preoperative model reference anatomical coordinate system and uses a minimum mean square error point cloud registration algorithm, combined with the real-time tracking of the puncture instrument position information by the electromagnetic positioning system, to achieve coordinate docking between the preoperative model and the intraoperative space; and performs real-time dynamic correction of model drift based on the posture drift vector field compensation mechanism to construct a virtual-real fusion cardiac structure model.

8. The intelligent navigation system for inter-room puncture surgery based on virtual-real fusion and electromagnetic positioning according to claim 4, characterized in that: When the magnitude of the offset vector exceeds a preset spatial error threshold, the offset vector is color-coded and rendered with a three-dimensional directional arrow, and then overlaid and displayed on the display interface.

Images