Surgical navigation system and method based on electromagnetic positioning and augmented reality

Abstract

The invention provides a surgical navigation system and method based on electromagnetic positioning and augmented reality, and the system comprises an image data three-dimensional reconstruction module which is used for importing preoperative medical image data into the system, and generating a visual model through a three-dimensional reconstruction algorithm; the pre-operation registration module is used for realizing space mapping of an image coordinate system and an operation coordinate system through a registration method based on mark points; the electromagnetic positioning and coordinate mapping module is used for tracking the pose of the surgical instrument in real time and mapping the coordinates of the instrument to the surgical coordinate system; the augmented reality and high-precision navigation module is used for fusing and displaying the registered virtual navigation image and a real operation scene; the electromagnetic positioning and coordinate mapping module and the augmented reality and high-precision navigation module form a surgical navigation platform integrating electromagnetic tracking and augmented reality display; by fusing the high-precision anti-shielding capability of electromagnetic positioning and the visual visualization effect of augmented reality, stable, accurate and visual intraoperative navigation can be realized.

Description

Technical Field

[0001] This invention relates to the fields of augmented reality technology and computer-aided surgical instruments, and in particular to a surgical navigation system and method based on electromagnetic positioning and augmented reality. Background Technology

[0002] In traditional surgical navigation systems, optical positioning technology is typically used to track the position of surgical instruments and map the positional information of the instruments onto preoperative medical images. Three-dimensional visualization technology is used to generate navigation images that are displayed on the screen. However, doctors need to repeatedly switch their gaze between the surgical area and the navigation screen, which interferes with the surgical process and affects the smoothness and accuracy of the operation.

[0003] Augmented reality (AR) technology precisely registers and overlays real-world scenes captured by optical systems with virtual navigation images, seamlessly integrating navigation information into the surgical field of vision. Optical positioning is susceptible to intraoperative obstructions and changes in lighting. Electromagnetic positioning is less affected by obstacles, facilitating the guidance of instruments within the surgical space; however, pure electromagnetic navigation lacks a clear and intuitive fusion of virtual and real visual effects. Seamlessly integrating high-precision electromagnetic positioning with immersive AR display, while ensuring data real-time performance and system usability, presents significant technical challenges.

[0004] Therefore, how to seamlessly integrate high-precision electromagnetic positioning with immersive AR display to build a surgical navigation platform that is both stable and reliable as well as intuitive and easy to use is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0005] This invention proposes a surgical navigation system and method based on electromagnetic positioning and augmented reality. By integrating the high-precision anti-occlusion capability of electromagnetic positioning with the intuitive visualization effect of augmented reality, it solves the problems of "virtual-real separation" in existing navigation systems, the susceptibility of optical tracking to occlusion, and the lack of intuitive visualization in electromagnetic navigation, thus achieving stable, accurate, and intuitive intraoperative navigation.

[0006] The present invention adopts the following technical solution.

[0007] A surgical navigation system based on electromagnetic positioning and augmented reality, comprising a surgical navigation platform integrating electromagnetic tracking and augmented reality display, which coordinates augmented reality devices and a high-precision navigation module through electromagnetic positioning and coordinate mapping; including:

[0008] The image data 3D reconstruction module is used to import preoperative medical image data (such as CT, MRI, etc.) into the system and use 3D reconstruction algorithms to generate high-precision vascular visualization models, providing a precise virtual anatomical basis for surgical navigation;

[0009] The preoperative registration module is used to achieve spatial mapping between the image coordinate system and the surgical coordinate system through a registration method based on marker points;

[0010] The electromagnetic positioning and coordinate mapping module is used to track the position and orientation of surgical instruments in real time through the electromagnetic positioning system and map the instrument coordinates to the surgical coordinate system.

[0011] Augmented reality and high-precision navigation module is used to fuse and display the registered virtual navigation image with the real surgical scene;

[0012] The electromagnetic positioning and coordinate mapping module and the augmented reality and high-precision navigation module work together to form an integrated electromagnetic tracking and augmented reality display surgical navigation platform.

[0013] The preoperative registration module executes an automated marker extraction algorithm based on image analysis, including the following:

[0014] Preprocessing and connected component analysis of medical images to identify potential marker regions;

[0015] A point cloud is generated for each connected region, and the main geometric orientation of the point cloud is extracted by principal component analysis.

[0016] The point cloud and the template point cloud are precisely registered using an iterative nearest-point algorithm, and a similarity metric is calculated to confirm the marked points.

[0017] The preoperative registration module uses the Dice coefficient as the similarity metric, and through experimental verification, connected components with a Dice coefficient higher than 0.75 are determined to be valid markers.

[0018] The automated marker extraction algorithm employs a coarse-fine registration strategy. By combining image preprocessing, connected component analysis, principal component analysis (PCA), and iterative nearest point (ICP) registration techniques, it can automatically and efficiently extract spatial markers used for surgical operations from medical images.

[0019] The electromagnetic positioning and coordinate mapping module establishes rigid body transformation relationships between multiple coordinate systems, including the electromagnetic space coordinate system, the surgical space coordinate system, and the instrument coordinate system, to realize real-time pose calculation of surgical instruments in the surgical space coordinate system.

[0020] The augmented reality and high-precision navigation module includes a marker incorporating Diamand codes. The marker includes a flat substrate, Diamand codes attached to the substrate, and a grooved protrusion located in the central region of the marker for fixing an electromagnetic sensor. The marking position of the marker includes surgical instruments.

[0021] When used, the marker uses its Diamand code as a visual reference. It boasts high recognition rate and anti-blurring properties, allowing for fast and stable recognition by the camera of AR devices such as HoloLens 2. The central groove of the marker is used to fix an electromagnetic sensor, serving as a spatial position reference. The electromagnetic sensor is tracked by an electromagnetic positioning system to provide its position within the surgical space. Three-dimensional coordinates in;

[0022] The augmented reality and high-precision navigation module identifies the Diamand code through its camera and uses the PnP algorithm to calculate the camera pose relative to the marker. Then, through the transformation chain of the virtual space coordinate system, the patient coordinate system, the camera coordinate system and the image coordinate system, it realizes the accurate projection of the virtual model on the augmented reality display device.

[0023] The electromagnetic positioning and coordinate mapping module uses the Aurora electromagnetic tracking system, along with the Aurora 6D reference standard sensor and a surgical platform with markers, to create a stable surgical space.

[0024] When the electromagnetic positioning and coordinate mapping module is working, its electromagnetic positioning coordinate transformation, the electromagnetic spatial coordinate system defined by the magnetic field generator is denoted as... The surgical space coordinate system established by the Aurora 6D reference sensor is denoted as... The calibrated standard probe coordinate system is defined as Reference standards are in The translation vector below is Orientation information is calculated using rotation matrices Row_x, Row_y, and Row_z around the corresponding coordinate axes, ultimately yielding... Transform to Rotation matrix Row:

[0025]

[0026] Within the mathematical framework of spatial mapping, we define For the surgical space coordinate system, This is an electromagnetic space coordinate system. For any point in ∈R³, which is in Corresponding points in The mapping relationship between ∈R³ is established through rigid body transformation, and its mathematical expression is as follows:

[0027]

[0028] Surgical instrument coordinate system To electromagnetic space coordinate system Spatial mapping relationships are achieved through rotation matrices. ∈R(3×3) and translation vector A complete description of ∈R(3×1) can be expressed mathematically as follows:

[0029]

[0030] Based on electromagnetic space coordinate system A unified reference framework is established to create a spatial pose mapping relationship between surgical instruments and reference sensors, defining... ∈R³ represents the local coordinate system of the surgical instrument. The coordinates of any point in space, ∈R³ indicates its position in the global coordinate system of the surgery. The corresponding mapped coordinates in the data are then from... arrive The composite spatial coordinate transformation can be expressed as the following rigid body transformation form:

[0031]

[0032] Wherein, rotation matrix =Row−1∙ Translation vector = −1∙( - Based on spatial pose mapping, the hand-guided navigation system calculates the position of surgical instruments in the surgical space coordinate system in real time. Precise pose coordinates in the instrument coordinate system; by establishing a complete rigid body transformation chain, the transformation from the instrument coordinate system is realized. To the surgical space coordinate system Continuous coordinate mapping;

[0033] To achieve coordinate compatibility and unification between the electromagnetic positioning system and augmented reality and virtual reality systems, the optimal rigid body transformation between point sets is calculated using the singular value decomposition algorithm to complete high-precision registration from image space to surgical space.

[0034] set up This is a set of three-dimensional coordinates of marker points extracted from preoperative CT images (VR / AR image space). This refers to the set of coordinates of corresponding physical marker points acquired intraoperatively using an electromagnetic positioning probe (operating space). The objective is to find an optimal rotation matrix R and translation vector t that minimizes the following objective function:

[0035]

[0036] The optimal transformation is calculated using the following steps (Kabsch algorithm):

[0037] Step (1) Point set centering: Calculate the centroid of the point set in the surgical space and the image space respectively. and And the point set is centralized:

[0038]

[0039] Step (2) Covariance Matrix Calculation: Calculate the covariance matrix H of the centered point set:

[0040]

[0041] Step (3) SVD decomposition: Perform singular value decomposition (SVD) on matrix H:

[0042]

[0043] The resulting R and t constitute the optimal rigid body transformation matrix T from the image space to the surgical space; the electromagnetic positioning system tracks the surgical instruments in the electromagnetic space in real time. coordinates in The image coordinate system VM, consistent with the 3D virtual model, is transformed using the following formula to obtain... :

[0044]

[0045] By ensuring the uniform compatibility of the coordinates of the Aurora electromagnetic tracking system with the virtual coordinate system of the VR / AR platform, it is guaranteed that the virtual instrument model can maintain accurate spatial consistency with the virtual anatomical structure, whether in the workstation VR interface or in the HoloLens AR field of view.

[0046] In the electromagnetic positioning and coordinate mapping module, the electromagnetic navigator and the workstation of the surgical navigation system communicate using the TCP / IP protocol; in the augmented reality and high-precision navigation module, the electromagnetic navigator of the surgical navigation system and the augmented reality display device transmit surgical instrument pose data in real time using the UDP protocol.

[0047] A surgical navigation method based on electromagnetic positioning and augmented reality, employing the aforementioned surgical navigation system based on electromagnetic positioning and augmented reality, is characterized by: the navigation method following a coherent logic from "preoperative preparation" to "intraoperative navigation," and the system hardware mainly including: an NDI Aurora electromagnetic positioning system, a Microsoft HoloLens 2 augmented reality head-mounted display device, a high-performance graphics workstation, a developed surgical platform, and dedicated markers with Diamand codes, as referenced. Figure 1 It includes the following steps;

[0048] Step 1: Preoperative preparation: Generate a three-dimensional virtual anatomical model based on preoperative medical imaging data of the surgical target;

[0049] Step 2, Automated Registration Step: Automatically extract marker points from the medical images and register them with physical marker points in the surgical space to establish a mapping relationship from the image space to the surgical space;

[0050] Step 3, Electromagnetic Tracking Step: The positional data of the surgical instruments are acquired in real time through the electromagnetic positioning system and mapped to the surgical coordinate system;

[0051] Step 4, Augmented Reality Navigation: Using augmented reality devices to identify markers in the real-world scene, the three-dimensional virtual anatomical model and the real-time positions of surgical instruments are superimposed on the surgeon's real surgical field for visual navigation.

[0052] In step two, a coarse-fine registration strategy is adopted, sequentially employing image preprocessing, connected component analysis, principal component analysis, and iterative nearest-point registration; refer to Figure 2 The specific implementation is as follows:

[0053] During the preoperative preparation phase, the patient's preoperative CT or MRI image data is imported into the system, and a vascular model is generated through a three-dimensional reconstruction algorithm to provide anatomical benchmarks for navigation.

[0054] Reference Figure 3 The system is based on an image analysis-based marker extraction algorithm: it performs preprocessing, connected component analysis, principal component analysis, and iterative nearest point registration on medical images, and automatically and accurately identifies and extracts the three-dimensional coordinates of markers for spatial registration from the medical image data corresponding to the surgery.

[0055] The image preprocessing specifically involves first loading and preprocessing the medical image, converting it into a grayscale image, and then using a median filtering algorithm to remove random noise from the image while preserving edge information, thus providing clear image data for subsequent analysis.

[0056] The connected component analysis specifically involves identifying potential markers in an image. Thresholding is performed on the image based on a preset grayscale range to separate markers from the background. An 8-neighborhood-based connected component labeling algorithm is used to ensure that adjacent pixels are considered to belong to the same region in the horizontal, vertical, and diagonal directions. The thresholding formula is as follows:

[0057]

[0058] Where f(x,y) represents the grayscale value of the filtered image. and These represent the minimum and maximum values ​​of the grayscale threshold, respectively. This bandpass thresholding design effectively separates marker points from the human body surface while preserving the integrity of the target area; after binarization, the marker area in the image is extracted as the foreground, and the background area is set to 0.

[0059] The principal component analysis (PCA) algorithm generates a corresponding point cloud for each connected region. PCA projects the original high-dimensional data into a low-dimensional subspace spanned by eigenvectors by calculating the eigenvalue decomposition of the covariance matrix. The mutually orthogonal eigenvectors are the principal component directions.

[0060] The iterative nearest point registration (ICP) specifically involves the following steps: after point cloud generation and orientation analysis, the algorithm accurately aligns suspected markers by comparing them with the template point cloud. Initial spatial transformation is used to roughly align the point cloud to be registered with the reference point cloud, providing good initial conditions for subsequent fine registration.

[0061] ICP is an optimal registration method based on least squares, mainly used to accurately align point clouds in two different coordinate systems. This invention provides good initial conditions for the ICP algorithm through coarse registration, and ensures that the point clouds are roughly aligned based on the principal direction extracted by PCA, which significantly reduces the risk of getting trapped in local optima, thus achieving high-precision and stable point cloud alignment. After the point clouds are aligned, the algorithm quantifies the similarity between point clouds by calculating the Dice coefficient. This invention identifies connected components with a Dice coefficient higher than 0.75 as potential markers.

[0062] In step two, principal component analysis (PCA) is used to extract the main geometric features of the point cloud, providing a foundation for subsequent point cloud alignment. The specific steps include: calculation of the covariance matrix, decomposition of eigenvalues, and selection of principal components.

[0063] The calculation of the covariance matrix is ​​as follows: a covariance matrix C is constructed based on the point cloud dataset. This matrix describes the statistical correlation between the point cloud in each dimension and reflects the synergy of its spatial distribution characteristics, expressed by the formula:

[0064]

[0065] The eigenvalue decomposition involves performing an eigenvalue decomposition operation on the covariance matrix C to obtain its eigenvalues. , , and the corresponding feature vectors , , The relationship between eigenvalues ​​and eigenvectors is given by the following formula:

[0066]

[0067] Among them, eigenvalues The eigenvector represents the contribution of the i-th principal component to the overall data variance. This indicates the directionality of the principal component in space;

[0068] The principal component selection is based on the order of the eigenvalues, selecting the first k eigenvectors as principal components to characterize the feature directions with the highest variance proportion in the data;

[0069] When the marker is axisymmetric, select the third principal component. As the main direction of the point cloud, it is used for subsequent alignment and registration.

[0070] In step three, a stable surgical space is constructed through an electromagnetic positioning system. Specifically, the operator uses an electromagnetic positioning probe to collect the coordinates of physical marker points formed by markers actually placed on the patient's body surface or surgical platform. The surgical navigation system uses a singular value decomposition algorithm to optimally match the set of virtual marker points extracted before surgery with the set of physical marker points collected during surgery, and solves the rigid body transformation matrix from image space to surgical space, thereby achieving high-precision alignment of the two spaces and establishing a mapping bridge between the virtual world and the real world.

[0071] Reference Figure 4 The surgical navigation system defines an electromagnetic spatial coordinate system. Surgical space coordinate system and instrument coordinate system The operator uses an electromagnetic probe to collect physical markers on the surgical platform. Coordinates in;

[0072] During the intraoperative real-time navigation phase, the geometric mapping between the surgical space and electromagnetic space developed in this invention is referenced... Figure 7 ;

[0073] During the intraoperative real-time navigation phase, the doctor wore an augmented reality head-mounted display device, HoloLens 2, which used its camera to identify... Figure 6 Diamand code markers shown;

[0074] The surgical navigation system uses the PnP algorithm to calculate the pose of the camera coordinate system (CAM) relative to the markers.

[0075] Based on the aforementioned registration results, the surgical navigation system establishes a system as follows: Figure 5 The complete coordinate transformation chain shown, namely VM→PT→CAM→IMG, enables the virtual vascular model to be stably and accurately superimposed on the real surgical field, achieving the effect reference. Figure 8 ;

[0076] During surgery, electromagnetic sensors mounted on surgical instruments transmit their pose data in real time via UDP protocol to the HoloLens 2 augmented reality head-mounted display worn by the operator, ensuring low-latency updates. This allows the surgical navigation system to dynamically render the real-time position and orientation of virtual instruments within the AR field of view, referencing... Figure 9 This allows doctors to directly observe the relationship between the instruments and the lesions. Simultaneously, the electromagnetic navigation system communicates with the workstation via TCP / IP protocol, ensuring reliable transmission of core control commands and data.

[0077] In surgical navigation methods, during augmented reality (AR) virtual-real fusion registration, marker detection and pose estimation are core components for achieving precise alignment between virtual objects and real-world scenes. The key lies in detecting Diamand code markers from images and estimating their position and pose relative to the camera. This is achieved by converting color images to grayscale to reduce computational complexity while preserving crucial feature information. Subsequently, binarization is used to extract the marker's boundary information. Finally, the ArUco marker library is used to detect Diamand codes and extract their corner coordinates. The PnP algorithm determines the marker's position and pose relative to the camera. The core idea of ​​the PnP algorithm is to solve for the camera's pose parameters by leveraging the known correspondence between 2D image points and 3D spatial points. Its mathematical foundation is the perspective projection model, which projects 3D spatial points onto a 2D image plane. By minimizing the reprojection error, the system can obtain the optimal camera pose parameters.

[0078] When transforming coordinate relationships in augmented reality, high-precision virtual-real fusion is achieved through the coordinated transformation of five key coordinate systems: the virtual space coordinate system VM defines the surgical planning model, the real world coordinate system RW establishes a global reference with the magnetic field generator as the origin, the patient coordinate system PT is established through a fixed reference sensor on the body surface, the camera coordinate system CAM is determined by the optical system of the AR device, and the image coordinate system IMG represents the final display image.

[0079] In the augmented reality precise registration process, the surgical navigation system first identifies optical markers in the real space through the camera and obtains the initial pose of the markers in the camera coordinate system (CAM). Combining the known coordinates of the markers in the real space coordinate system (RW), the patient coordinate system (PT), and the virtual space coordinate system (VM), the precise position of the camera in each coordinate system is derived. In subsequent operations, the changes in the camera's pose are tracked and compensated in real time through the built-in gyroscope. Based on the precisely calibrated intrinsic and extrinsic parameters of the camera, the surgical navigation system accurately projects the virtual model onto the display screen of the augmented reality head-mounted display device worn by the operator through the transformation chain of VM→PT→CAM→IMG, thus achieving virtual-real registration.

[0080] In electromagnetic positioning and coordinate mapping, in order to ensure the real-time performance and stability of data transmission between the electromagnetic navigator and the workstation, the system adopts the TCP / IP protocol to realize communication between the two, taking advantage of its connection-oriented and reliable transmission characteristics.

[0081] The augmented reality system displays the real-time location of the device. The system uses the UDP protocol to achieve real-time data transmission between the electromagnetic navigator and the Microsoft HoloLens 2, taking advantage of its connectionless and low-latency characteristics.

[0082] This invention proposes a surgical navigation system and method based on electromagnetic positioning and augmented reality. The system includes: a 3D image data reconstruction module, a preoperative registration module, an electromagnetic positioning and coordinate mapping module, and an augmented reality and high-precision navigation module. The method includes: generating a virtual anatomical model through 3D reconstruction; achieving high-precision registration between image space and surgical space using an automated marker extraction algorithm based on image analysis; tracking the pose of surgical instruments in real time through an electromagnetic positioning system; and achieving virtual-real fusion navigation by utilizing a marker structure combined with Diamand codes and multi-coordinate system collaborative transformation. This invention combines the anti-occlusion properties of electromagnetic positioning with the intuitiveness of augmented reality, solving problems such as field-of-view switching and easy occlusion in optical tracking in traditional navigation, providing doctors with a stable, accurate, and immersive surgical navigation experience.

[0083] Compared with existing technologies, the system and method described in this invention have the following beneficial technical effects: Deep integration of the electromagnetic positioning system and the augmented reality display system constructs a unified technical framework, achieving synergistic effects of "stable tracking" and "intuitive display"; High-precision registration: Through an automated marker extraction algorithm based on image analysis, integrating connected component analysis, PCA, and ICP registration technologies, high-precision registration from medical images to the surgical space is achieved, reducing human intervention errors; Spatial mapping and virtual-real fusion: A complete and continuous coordinate transformation chain is established from electromagnetic space to surgical space, and then to augmented reality space. Through an innovative Diamand code marker structure, high-precision and stable superposition of virtual navigation information and the real surgical scene is achieved; System robustness and real-time performance: The TCP / IP protocol ensures stable transmission of control and data streams between the workstation and the electromagnetic navigator, while the UDP protocol meets the low-latency, high-efficiency transmission of instrument pose data between the AR device and the system, ensuring the robustness and real-time navigation performance of the entire system in complex surgical environments. Attached Figure Description

[0084] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments:

[0085] Appendix Figure 1 This is a schematic diagram of the overall framework of the surgical navigation system and method based on electromagnetic positioning and augmented reality of the present invention;

[0086] Appendix Figure 2 This is a schematic diagram of the system flow of the surgical navigation system and method based on electromagnetic positioning and augmented reality of the present invention;

[0087] Appendix Figure 3 This is a schematic diagram of the automatic marker extraction algorithm flow of the surgical navigation system and method based on electromagnetic positioning and augmented reality of the present invention;

[0088] Appendix Figure 4 This is a schematic diagram illustrating the geometric mapping relationship between electromagnetic space and surgical space in the surgical navigation system and method based on electromagnetic positioning and augmented reality of the present invention.

[0089] Appendix Figure 5 This is a schematic diagram of the coordinate transformation relationships in the surgical navigation system of the present invention, which is based on electromagnetic positioning and augmented reality.

[0090] Appendix Figure 6 This is a schematic diagram of the Diamand code marker structure for the surgical navigation system and method based on electromagnetic positioning and augmented reality according to the present invention;

[0091] Appendix Figure 7 This is a schematic diagram of a virtual surgical scene for the surgical navigation system and method based on electromagnetic positioning and augmented reality according to the present invention;

[0092] Appendix Figure 8 This is a schematic diagram illustrating the augmented reality fusion of the surgical navigation system and method based on electromagnetic positioning and augmented reality according to the present invention.

[0093] Appendix Figure 9 This is a schematic diagram of the augmented reality instrument used in the surgical navigation system and method based on electromagnetic positioning and augmented reality of the present invention. Detailed Implementation

[0094] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0095] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0096] Reference Figure 1-9The present invention provides a surgical navigation system and method based on electromagnetic positioning and augmented reality. The system includes an image data three-dimensional reconstruction module, a preoperative registration module, an electromagnetic positioning and coordinate mapping module, and an augmented reality and high-precision navigation module.

[0097] In this embodiment, a coronary artery calcification rotational atherectomy is used as an example to demonstrate the complete workflow of the system. The system hardware mainly includes: the NDI Aurora electromagnetic positioning system, the Microsoft HoloLens 2 augmented reality head-mounted display, a high-performance graphics workstation, a developed surgical platform, and dedicated markers with Diamand codes, as shown in the reference. Figure 1 .

[0098] As shown in the figure, a surgical navigation system based on electromagnetic positioning and augmented reality is described. The system is a surgical navigation platform integrating electromagnetic tracking and augmented reality display, which coordinates augmented reality devices and a high-precision navigation module through electromagnetic positioning and coordinate mapping. The system includes:

[0099] The image data 3D reconstruction module is used to import preoperative medical image data (such as CT, MRI, etc.) into the system and use 3D reconstruction algorithms to generate high-precision vascular visualization models, providing a precise virtual anatomical basis for surgical navigation;

[0100] The preoperative registration module is used to achieve spatial mapping between the image coordinate system and the surgical coordinate system through a registration method based on marker points;

[0101] The electromagnetic positioning and coordinate mapping module is used to track the position and orientation of surgical instruments in real time through the electromagnetic positioning system and map the instrument coordinates to the surgical coordinate system.

[0102] Augmented reality and high-precision navigation module is used to fuse and display the registered virtual navigation image with the real surgical scene;

[0103] The electromagnetic positioning and coordinate mapping module and the augmented reality and high-precision navigation module work together to form an integrated electromagnetic tracking and augmented reality display surgical navigation platform.

[0104] The preoperative registration module executes an automated marker extraction algorithm based on image analysis, including the following:

[0105] Preprocessing and connected component analysis of medical images to identify potential marker regions;

[0106] A point cloud is generated for each connected region, and the main geometric orientation of the point cloud is extracted by principal component analysis.

[0107] The point cloud and the template point cloud are precisely registered using an iterative nearest-point algorithm, and a similarity metric is calculated to confirm the marked points.

[0108] The preoperative registration module uses the Dice coefficient as the similarity metric, and through experimental verification, connected components with a Dice coefficient higher than 0.75 are determined to be valid markers.

[0109] The automated marker extraction algorithm employs a coarse-fine registration strategy. By combining image preprocessing, connected component analysis, principal component analysis (PCA), and iterative nearest point (ICP) registration techniques, it can automatically and efficiently extract spatial markers used for surgical operations from medical images.

[0110] The electromagnetic positioning and coordinate mapping module establishes rigid body transformation relationships between multiple coordinate systems, including the electromagnetic space coordinate system, the surgical space coordinate system, and the instrument coordinate system, to realize real-time pose calculation of surgical instruments in the surgical space coordinate system.

[0111] The augmented reality and high-precision navigation module includes a marker incorporating Diamand codes. The marker includes a flat substrate, Diamand codes attached to the substrate, and a grooved protrusion located in the central region of the marker for fixing an electromagnetic sensor. The marking position of the marker includes surgical instruments.

[0112] When used, the marker uses its Diamand code as a visual reference. It boasts high recognition rate and anti-blurring properties, allowing for fast and stable recognition by the camera of AR devices such as HoloLens 2. The central groove of the marker is used to fix an electromagnetic sensor, serving as a spatial position reference. The electromagnetic sensor is tracked by an electromagnetic positioning system to provide its position within the surgical space. Three-dimensional coordinates in;

[0113] The augmented reality and high-precision navigation module identifies the Diamand code through its camera and uses the PnP algorithm to calculate the camera pose relative to the marker. Then, through the transformation chain of the virtual space coordinate system, the patient coordinate system, the camera coordinate system and the image coordinate system, it realizes the accurate projection of the virtual model on the augmented reality display device.

[0114] The electromagnetic positioning and coordinate mapping module uses the Aurora electromagnetic tracking system, along with the Aurora 6D reference standard sensor and a surgical platform with markers, to create a stable surgical space.

[0115] When the electromagnetic positioning and coordinate mapping module is working, its electromagnetic positioning coordinate transformation, the electromagnetic spatial coordinate system defined by the magnetic field generator is denoted as... The surgical space coordinate system established by the Aurora 6D reference sensor is denoted as... The calibrated standard probe coordinate system is defined as Reference standards are in The translation vector below is Orientation information is calculated using rotation matrices Row_x, Row_y, and Row_z around the corresponding coordinate axes, ultimately yielding... Transform to Rotation matrix Row:

[0116]

[0117] Within the mathematical framework of spatial mapping, we define For the surgical space coordinate system, This is an electromagnetic space coordinate system. For any point in ∈R³, which is in Corresponding points in The mapping relationship between ∈R³ is established through rigid body transformation, and its mathematical expression is as follows:

[0118]

[0119] Surgical instrument coordinate system To electromagnetic space coordinate system Spatial mapping relationships are achieved through rotation matrices. ∈R(3×3) and translation vector A complete description of ∈R(3×1) can be expressed mathematically as follows:

[0120]

[0121] Based on electromagnetic space coordinate system A unified reference framework is established to create a spatial pose mapping relationship between surgical instruments and reference sensors, defining... ∈R³ represents the local coordinate system of the surgical instrument. The coordinates of any point in space, ∈R³ indicates its position in the global coordinate system of the surgery. The corresponding mapped coordinates in the data are then from... arrive The composite spatial coordinate transformation can be expressed as the following rigid body transformation form:

[0122]

[0123] Wherein, rotation matrix =Row−1∙ Translation vector = −1∙( - Based on spatial pose mapping, the hand-guided navigation system calculates the position of surgical instruments in the surgical space coordinate system in real time. Precise pose coordinates in the instrument coordinate system; by establishing a complete rigid body transformation chain, the transformation from the instrument coordinate system is realized. To the surgical space coordinate system Continuous coordinate mapping;

[0124] To achieve coordinate compatibility and unification between the electromagnetic positioning system and augmented reality and virtual reality systems, the optimal rigid body transformation between point sets is calculated using the singular value decomposition algorithm to complete high-precision registration from image space to surgical space.

[0125] set up This is a set of three-dimensional coordinates of marker points extracted from preoperative CT images (VR / AR image space). This refers to the set of coordinates of corresponding physical marker points acquired intraoperatively using an electromagnetic positioning probe (operating space). The objective is to find an optimal rotation matrix R and translation vector t that minimizes the following objective function:

[0126]

[0127] The optimal transformation is calculated using the following steps (Kabsch algorithm):

[0128] Step (1) Point set centering: Calculate the centroid of the point set in the surgical space and the image space respectively. and And the point set is centralized:

[0129]

[0130] Step (2) Covariance Matrix Calculation: Calculate the covariance matrix H of the centered point set:

[0131]

[0132] Step (3) SVD decomposition: Perform singular value decomposition (SVD) on matrix H:

[0133]

[0134] The resulting R and t constitute the optimal rigid body transformation matrix T from the image space to the surgical space; the electromagnetic positioning system tracks the surgical instruments in the electromagnetic space in real time. coordinates in The image coordinate system VM, consistent with the 3D virtual model, is transformed using the following formula to obtain... :

[0135]

[0136] By ensuring the uniform compatibility of the coordinates of the Aurora electromagnetic tracking system with the virtual coordinate system of the VR / AR platform, it is guaranteed that the virtual instrument model can maintain accurate spatial consistency with the virtual anatomical structure, whether in the workstation VR interface or in the HoloLens AR field of view.

[0137] In the electromagnetic positioning and coordinate mapping module, the electromagnetic navigator and the workstation of the surgical navigation system communicate using the TCP / IP protocol; in the augmented reality and high-precision navigation module, the electromagnetic navigator of the surgical navigation system and the augmented reality display device transmit surgical instrument pose data in real time using the UDP protocol.

[0138] A surgical navigation method based on electromagnetic positioning and augmented reality, employing the aforementioned surgical navigation system based on electromagnetic positioning and augmented reality, is characterized by: the navigation method following a coherent logic from "preoperative preparation" to "intraoperative navigation," and the system hardware mainly including: an NDI Aurora electromagnetic positioning system, a Microsoft HoloLens 2 augmented reality head-mounted display device, a high-performance graphics workstation, a developed surgical platform, and dedicated markers with Diamand codes, as referenced. Figure 1 It includes the following steps;

[0139] Step 1: Preoperative preparation: Generate a three-dimensional virtual anatomical model based on preoperative medical imaging data of the surgical target;

[0140] Step 2, Automated Registration Step: Automatically extract marker points from the medical images and register them with physical marker points in the surgical space to establish a mapping relationship from the image space to the surgical space;

[0141] Step 3, Electromagnetic Tracking Step: The positional data of the surgical instruments are acquired in real time through the electromagnetic positioning system and mapped to the surgical coordinate system;

[0142] Step 4, Augmented Reality Navigation: Using augmented reality devices to identify markers in the real-world scene, the three-dimensional virtual anatomical model and the real-time positions of surgical instruments are superimposed on the surgeon's real surgical field for visual navigation.

[0143] In step two, a coarse-fine registration strategy is adopted, sequentially employing image preprocessing, connected component analysis, principal component analysis, and iterative nearest-point registration; refer to Figure 2 The specific implementation is as follows:

[0144] During the preoperative preparation phase, the patient's preoperative CT or MRI image data is imported into the system, and a vascular model is generated through a three-dimensional reconstruction algorithm to provide anatomical benchmarks for navigation.

[0145] Reference Figure 3The system is based on an image analysis-based marker extraction algorithm: it performs preprocessing, connected component analysis, principal component analysis, and iterative nearest point registration on medical images, and automatically and accurately identifies and extracts the three-dimensional coordinates of markers for spatial registration from the medical image data corresponding to the surgery.

[0146] The image preprocessing specifically involves first loading and preprocessing the medical image, converting it into a grayscale image, and then using a median filtering algorithm to remove random noise from the image while preserving edge information, thus providing clear image data for subsequent analysis.

[0147] The connected component analysis specifically involves identifying potential markers in an image. Thresholding is performed on the image based on a preset grayscale range to separate markers from the background. An 8-neighborhood-based connected component labeling algorithm is used to ensure that adjacent pixels are considered to belong to the same region in the horizontal, vertical, and diagonal directions. The thresholding formula is as follows:

[0148]

[0149] Where f(x,y) represents the grayscale value of the filtered image. and These represent the minimum and maximum values ​​of the grayscale threshold, respectively. This bandpass thresholding design effectively separates marker points from the human body surface while preserving the integrity of the target area; after binarization, the marker area in the image is extracted as the foreground, and the background area is set to 0.

[0150] The principal component analysis (PCA) algorithm generates a corresponding point cloud for each connected region. PCA projects the original high-dimensional data into a low-dimensional subspace spanned by eigenvectors by calculating the eigenvalue decomposition of the covariance matrix. The mutually orthogonal eigenvectors are the principal component directions.

[0151] The iterative nearest point registration (ICP) specifically involves the following steps: after point cloud generation and orientation analysis, the algorithm accurately aligns suspected markers by comparing them with the template point cloud. Initial spatial transformation is used to roughly align the point cloud to be registered with the reference point cloud, providing good initial conditions for subsequent fine registration.

[0152] ICP is an optimal registration method based on least squares, mainly used to accurately align point clouds in two different coordinate systems. This invention provides good initial conditions for the ICP algorithm through coarse registration, and ensures that the point clouds are roughly aligned based on the principal direction extracted by PCA, which significantly reduces the risk of getting trapped in local optima, thus achieving high-precision and stable point cloud alignment. After the point clouds are aligned, the algorithm quantifies the similarity between point clouds by calculating the Dice coefficient. This invention identifies connected components with a Dice coefficient higher than 0.75 as potential markers.

[0153] In step two, principal component analysis (PCA) is used to extract the main geometric features of the point cloud, providing a foundation for subsequent point cloud alignment. The specific steps include: calculation of the covariance matrix, decomposition of eigenvalues, and selection of principal components.

[0154] The calculation of the covariance matrix is ​​as follows: a covariance matrix C is constructed based on the point cloud dataset. This matrix describes the statistical correlation between the point cloud in each dimension and reflects the synergy of its spatial distribution characteristics, expressed by the formula:

[0155]

[0156] The eigenvalue decomposition involves performing an eigenvalue decomposition operation on the covariance matrix C to obtain its eigenvalues. , , and the corresponding feature vectors , , The relationship between eigenvalues ​​and eigenvectors is given by the following formula:

[0157]

[0158] Among them, eigenvalues The eigenvector represents the contribution of the i-th principal component to the overall data variance. This indicates the directionality of the principal component in space;

[0159] The principal component selection is based on the order of the eigenvalues, selecting the first k eigenvectors as principal components to characterize the feature directions with the highest variance proportion in the data;

[0160] When the marker is axisymmetric, select the third principal component. As the main direction of the point cloud, it is used for subsequent alignment and registration.

[0161] In step three, a stable surgical space is constructed through an electromagnetic positioning system. Specifically, the operator uses an electromagnetic positioning probe to collect the coordinates of physical marker points formed by markers actually placed on the patient's body surface or surgical platform. The surgical navigation system uses a singular value decomposition algorithm to optimally match the set of virtual marker points extracted before surgery with the set of physical marker points collected during surgery, and solves the rigid body transformation matrix from image space to surgical space, thereby achieving high-precision alignment of the two spaces and establishing a mapping bridge between the virtual world and the real world.

[0162] Reference Figure 4 The surgical navigation system defines an electromagnetic spatial coordinate system. Surgical space coordinate system and instrument coordinate system The operator uses an electromagnetic probe to collect physical markers on the surgical platform. Coordinates in;

[0163] During the intraoperative real-time navigation phase, the geometric mapping between the surgical space and electromagnetic space developed in this invention is referenced... Figure 7 ;

[0164] During the intraoperative real-time navigation phase, the doctor wore an augmented reality head-mounted display device, HoloLens 2, which used its camera to identify... Figure 6 Diamand code markers shown;

[0165] The surgical navigation system uses the PnP algorithm to calculate the pose of the camera coordinate system (CAM) relative to the markers.

[0166] Based on the aforementioned registration results, the surgical navigation system establishes a system as follows: Figure 5 The complete coordinate transformation chain shown, namely VM→PT→CAM→IMG, enables the virtual vascular model to be stably and accurately superimposed on the real surgical field, achieving the effect reference. Figure 8 ;

[0167] During surgery, electromagnetic sensors mounted on surgical instruments transmit their pose data in real time via UDP protocol to the HoloLens 2 augmented reality head-mounted display worn by the operator, ensuring low-latency updates. This allows the surgical navigation system to dynamically render the real-time position and orientation of virtual instruments within the AR field of view, referencing... Figure 9 This allows doctors to directly observe the relationship between the instruments and the lesions. Simultaneously, the electromagnetic navigation system communicates with the workstation via TCP / IP protocol, ensuring reliable transmission of core control commands and data.

[0168] In surgical navigation methods, during augmented reality (AR) virtual-real fusion registration, marker detection and pose estimation are core components for achieving precise alignment between virtual objects and real-world scenes. The key lies in detecting Diamand code markers from images and estimating their position and pose relative to the camera. This is achieved by converting color images to grayscale to reduce computational complexity while preserving crucial feature information. Subsequently, binarization is used to extract the marker's boundary information. Finally, the ArUco marker library is used to detect Diamand codes and extract their corner coordinates. The PnP algorithm determines the marker's position and pose relative to the camera. The core idea of ​​the PnP algorithm is to solve for the camera's pose parameters by leveraging the known correspondence between 2D image points and 3D spatial points. Its mathematical foundation is the perspective projection model, which projects 3D spatial points onto a 2D image plane. By minimizing the reprojection error, the system can obtain the optimal camera pose parameters.

[0169] When transforming coordinate relationships in augmented reality, high-precision virtual-real fusion is achieved through the coordinated transformation of five key coordinate systems: the virtual space coordinate system VM defines the surgical planning model, the real world coordinate system RW establishes a global reference with the magnetic field generator as the origin, the patient coordinate system PT is established through a fixed reference sensor on the body surface, the camera coordinate system CAM is determined by the optical system of the AR device, and the image coordinate system IMG represents the final display image.

[0170] In the augmented reality precise registration process, the surgical navigation system first identifies optical markers in the real space through the camera and obtains the initial pose of the markers in the camera coordinate system (CAM). Combining the known coordinates of the markers in the real space coordinate system (RW), the patient coordinate system (PT), and the virtual space coordinate system (VM), the precise position of the camera in each coordinate system is derived. In subsequent operations, the changes in the camera's pose are tracked and compensated in real time through the built-in gyroscope. Based on the precisely calibrated intrinsic and extrinsic parameters of the camera, the surgical navigation system accurately projects the virtual model onto the display screen of the augmented reality head-mounted display device worn by the operator through the transformation chain of VM→PT→CAM→IMG, thus achieving virtual-real registration.

[0171] In electromagnetic positioning and coordinate mapping, in order to ensure the real-time performance and stability of data transmission between the electromagnetic navigator and the workstation, the system adopts the TCP / IP protocol to realize communication between the two, taking advantage of its connection-oriented and reliable transmission characteristics.

[0172] The augmented reality system displays the real-time location of the device. The system uses the UDP protocol to achieve real-time data transmission between the electromagnetic navigator and the Microsoft HoloLens 2, taking advantage of its connectionless and low-latency characteristics.

[0173] In this example, the Iterative Closest Point (ICP) registration, after point cloud generation and orientation analysis, uses the algorithm to precisely align suspected markers by comparing them with the template point cloud. This invention first uses a preliminary spatial transformation to roughly align the point cloud to be registered with the reference point cloud, providing good initial conditions for subsequent fine-grained registration. ICP is an optimal registration method based on least squares, mainly used to precisely align point clouds in two different coordinate systems. This invention provides good initial conditions for the ICP algorithm through coarse registration, and ensures that the point clouds are roughly aligned based on the principal orientation extracted by PCA, significantly reducing the risk of getting trapped in local optima, thus achieving high-precision and stable point cloud alignment. After completing the point cloud alignment, the algorithm quantifies the similarity between point clouds by calculating the Dice coefficient. This invention identifies connected components with a Dice coefficient higher than 0.75 as potential markers.

[0174] In this example, the navigation system constructs a stable surgical space through an electromagnetic positioning system. The operator uses an electromagnetic positioning probe to collect the coordinates of physical markers actually placed on the patient's body surface or the surgical platform. The system uses a singular value decomposition algorithm to optimally match the set of virtual markers extracted before surgery with the set of physical markers collected during surgery, solving for the rigid body transformation matrix from image space to surgical space, thereby achieving high-precision alignment between the two spaces and establishing a mapping bridge between the virtual world and the real world.

[0175] Reference Figure 4 The system defines the electromagnetic space coordinate system S. w Surgical space coordinate system S o and instrument coordinate system S s The operator uses an electromagnetic probe to collect physical markers on the surgical platform at point S. o Coordinates in;

[0176] During the intraoperative real-time navigation phase, in this case, the developed geometric mapping between the surgical space and the electromagnetic space was referenced. Figure 7 ;

[0177] In this case, during the intraoperative real-time navigation phase, the doctor wore a HoloLens 2, whose camera recognized... Figure 6 Diamand code markers shown;

[0178] In this example, the system uses the PnP algorithm to calculate the pose of the camera coordinate system (CAM) relative to the marker.

[0179] In this example, based on the aforementioned registration results, the system establishes as follows: Figure 5 The complete coordinate transformation chain is shown (VM→PT→CAM→IMG). Ultimately, the virtual vascular model is stably and accurately superimposed onto the real surgical field, achieving the desired effect. Figure 8 ;

[0180] In this case, during surgery, electromagnetic sensors mounted on the instruments transmitted their pose data to the HoloLens 2 in real time via UDP protocol, ensuring low-latency updates. The system dynamically renders the real-time position and orientation of the virtual instruments within the AR field of view, referencing... Figure 9 This allows doctors to directly observe the relationship between the instruments and the lesions. Simultaneously, the electromagnetic navigation system communicates with the workstation via TCP / IP protocol, ensuring reliable transmission of core control commands and data.

[0181] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made by those skilled in the art to the technical solutions of the present invention without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.

Claims

1. A surgical navigation system based on electromagnetic positioning and augmented reality, characterized in that: The system is a surgical navigation platform that integrates electromagnetic tracking and augmented reality display, coordinating augmented reality devices and a high-precision navigation module through electromagnetic positioning and coordinate mapping; it includes: The image data 3D reconstruction module is used to import preoperative medical image data into the system and generate a visualization model using 3D reconstruction algorithms, providing a precise virtual anatomical basis for surgical navigation; The preoperative registration module is used to achieve spatial mapping between the image coordinate system and the surgical coordinate system through a registration method based on marker points; The electromagnetic positioning and coordinate mapping module is used to track the position and orientation of surgical instruments in real time through the electromagnetic positioning system and map the instrument coordinates to the surgical coordinate system. Augmented reality and high-precision navigation module is used to fuse and display the registered virtual navigation image with the real surgical scene; The electromagnetic positioning and coordinate mapping module and the augmented reality and high-precision navigation module form an integrated electromagnetic tracking and augmented reality display surgical navigation platform.

2. The surgical navigation system based on electromagnetic positioning and augmented reality according to claim 1, characterized in that: The preoperative registration module executes an automated marker extraction algorithm based on image analysis, including the following: Preprocessing and connected component analysis of medical images to identify potential marker regions; A point cloud is generated for each connected region, and the main geometric orientation of the point cloud is extracted by principal component analysis. The point cloud and the template point cloud are precisely registered using an iterative nearest-point algorithm, and a similarity metric is calculated to confirm the marked points. The similarity metric used in the preoperative registration module is the Dice coefficient. The automated marker extraction algorithm employs a coarse-fine registration strategy, combining image preprocessing, connected component analysis, principal component analysis (PCA), and iterative nearest point (ICP) registration techniques to extract spatial markers for surgical operations from medical images.

3. The surgical navigation system based on electromagnetic positioning and augmented reality according to claim 1, characterized in that: The electromagnetic positioning and coordinate mapping module establishes rigid body transformation relationships between multiple coordinate systems, including the electromagnetic space coordinate system, the surgical space coordinate system, and the instrument coordinate system, to realize real-time pose calculation of surgical instruments in the surgical space coordinate system. The augmented reality and high-precision navigation module includes a marker incorporating Diamand codes. The marker includes a flat substrate, Diamand codes attached to the substrate, and a grooved protrusion located in the central region of the marker for fixing an electromagnetic sensor. The marking position of the marker includes surgical instruments. When in use, the marker uses its Diamand code as a visual reference, which is then recognized by the AR device's camera. The central groove of the marker is used to fix an electromagnetic sensor, serving as a spatial position reference. The electromagnetic sensor is tracked by an electromagnetic positioning system to provide its position within the surgical space. Three-dimensional coordinates in; The augmented reality and high-precision navigation module identifies the Diamand code through its camera and uses the PnP algorithm to calculate the camera pose relative to the marker. Then, through the transformation chain of the virtual space coordinate system, the patient coordinate system, the camera coordinate system and the image coordinate system, it realizes the accurate projection of the virtual model on the augmented reality display device. The electromagnetic positioning and coordinate mapping module uses the Aurora electromagnetic tracking system, along with the Aurora 6D reference standard sensor and a surgical platform with markers, to create a stable surgical space.

4. A surgical navigation system based on electromagnetic positioning and augmented reality according to claim 3, characterized in that: When the electromagnetic positioning and coordinate mapping module is working, its electromagnetic positioning coordinate transformation, the electromagnetic spatial coordinate system defined by the magnetic field generator is denoted as... The surgical space coordinate system established by the Aurora 6D reference sensor is denoted as... The calibrated standard probe coordinate system is defined as Reference standards are in The translation vector below is Orientation information is calculated using rotation matrices Row_x, Row_y, and Row_z around the corresponding coordinate axes, ultimately yielding... Transform to Rotation matrix Row: Within the mathematical framework of spatial mapping, we define For the surgical space coordinate system, This is an electromagnetic space coordinate system. For any point in ∈R³, which is in Corresponding points in The mapping relationship between ∈R³ is established through rigid body transformation, and its mathematical expression is as follows: Surgical instrument coordinate system To electromagnetic space coordinate system Spatial mapping relationships are achieved through rotation matrices. ∈R(3×3) and translation vector A complete description of ∈R(3×1) can be expressed mathematically as follows: Based on electromagnetic space coordinate system A unified reference framework is established to create a spatial pose mapping relationship between surgical instruments and reference sensors, defining... ∈R³ represents the local coordinate system of the surgical instrument. The coordinates of any point in space, ∈R³ indicates its position in the global coordinate system of the surgery. The corresponding mapped coordinates in the data are then from... arrive The composite spatial coordinate transformation can be expressed as the following rigid body transformation form: Wherein, rotation matrix =Row−1∙ Translation vector = −1∙( - Based on spatial pose mapping, the hand-guided navigation system calculates the position of surgical instruments in the surgical space coordinate system in real time. Precise pose coordinates in the instrument coordinate system; by establishing a complete rigid body transformation chain, the transformation from the instrument coordinate system is realized. To the surgical space coordinate system Continuous coordinate mapping; The optimal rigid body transformation between point sets is calculated using the singular value decomposition algorithm, thereby achieving high-precision registration from image space to surgical space. set up This is a set of three-dimensional coordinates of marker points extracted from preoperative CT images. Let R be the set of coordinates of the corresponding physical markers acquired intraoperatively by an electromagnetic positioning probe; the objective is to find an optimal rotation matrix R and translation vector t that minimizes the following objective function: The optimal transformation is calculated through the following steps: Step (1) Point set centering: Calculate the centroid of the point set in the surgical space and the image space respectively. and And the point set is centralized: Step (2) Covariance Matrix Calculation: Calculate the covariance matrix H of the centered point set: Step (3) SVD decomposition: Perform singular value decomposition on matrix H: The resulting R and t constitute the optimal rigid body transformation matrix T from the image space to the surgical space; the electromagnetic positioning system tracks the surgical instruments in the electromagnetic space in real time. coordinates in The image coordinate system VM, consistent with the 3D virtual model, is transformed using the following formula to obtain... : By ensuring the uniform compatibility of the coordinates of the Aurora electromagnetic tracking system with the virtual coordinate system of the VR / AR platform, spatial consistency between the virtual instrument model and the virtual anatomical structure is guaranteed.

5. A surgical navigation system based on electromagnetic positioning and augmented reality according to claim 4, characterized in that: In the electromagnetic positioning and coordinate mapping module, the electromagnetic navigator and the workstation of the surgical navigation system communicate using the TCP / IP protocol; in the augmented reality and high-precision navigation module, the electromagnetic navigator of the surgical navigation system and the augmented reality display device transmit surgical instrument pose data in real time using the UDP protocol.

6. A surgical navigation method based on electromagnetic positioning and augmented reality, employing a surgical navigation system based on electromagnetic positioning and augmented reality as described in any one of claims 1-5, characterized in that: The navigation method includes the following steps; Step 1: Preoperative preparation: Generate a three-dimensional virtual anatomical model based on preoperative medical imaging data of the surgical target; Step 2, Automated Registration Step: Automatically extract marker points from the medical images and register them with physical marker points in the surgical space to establish a mapping relationship from the image space to the surgical space; Step 3, Electromagnetic Tracking Step: The positional data of the surgical instruments are acquired in real time through the electromagnetic positioning system and mapped to the surgical coordinate system; Step 4, Augmented Reality Navigation: Using augmented reality devices to identify markers in the real-world scene, the three-dimensional virtual anatomical model and the real-time positions of surgical instruments are superimposed on the surgeon's real surgical field for visual navigation.

7. The surgical navigation method based on electromagnetic positioning and augmented reality according to claim 6, characterized in that: In step two, a coarse-fine registration strategy is adopted, which sequentially employs image preprocessing, connected component analysis, principal component analysis, and iterative nearest-point registration; the specific implementation is as follows: During the preoperative preparation phase, the patient's preoperative CT or MRI image data is imported into the system, and a vascular model is generated through a three-dimensional reconstruction algorithm to provide anatomical benchmarks for navigation. The system uses an image analysis-based marker extraction algorithm: it performs preprocessing, connected component analysis, principal component analysis, and iterative nearest point registration on medical images to automatically and accurately identify and extract the three-dimensional coordinates of markers for spatial registration from the medical image data corresponding to the surgery. The image preprocessing specifically involves first loading and preprocessing the medical image, converting it into a grayscale image, and then using a median filtering algorithm to remove random noise from the image while preserving edge information, thus providing clear image data for subsequent analysis. The connected component analysis specifically involves identifying potential markers in an image. Thresholding is performed on the image based on a preset grayscale range to separate markers from the background. An 8-neighborhood-based connected component labeling algorithm is used to ensure that adjacent pixels are considered to belong to the same region in the horizontal, vertical, and diagonal directions. The thresholding formula is as follows: Where f(x,y) represents the grayscale value of the filtered image. and These represent the minimum and maximum values ​​of the grayscale threshold, respectively. This bandpass thresholding design effectively separates the marker points from the human body surface while preserving the integrity of the target area; after binarization, the marker area in the image is extracted as the foreground. The principal component analysis (PCA) algorithm generates a corresponding point cloud for each connected region. PCA projects the original high-dimensional data into a low-dimensional subspace spanned by eigenvectors by calculating the eigenvalue decomposition of the covariance matrix. The mutually orthogonal eigenvectors are the principal component directions. The iterative nearest point registration (ICP) specifically involves the following steps: after point cloud generation and orientation analysis, the algorithm accurately aligns suspected markers by comparing them with the template point cloud. It first performs a preliminary spatial transformation to roughly align the point cloud to be registered with the reference point cloud, providing initial conditions for subsequent registration.

8. The surgical navigation method based on electromagnetic positioning and augmented reality according to claim 7, characterized in that: In step two, principal component analysis (PCA) is used to extract the geometric features of the point cloud. The specific steps include: calculation of the covariance matrix, decomposition of eigenvalues, and selection of principal components. The calculation of the covariance matrix is ​​as follows: a covariance matrix C is constructed based on the point cloud dataset, expressed by the formula: The eigenvalue decomposition involves performing an eigenvalue decomposition operation on the covariance matrix C to obtain its eigenvalues. , , and the corresponding feature vectors , , The relationship between eigenvalues ​​and eigenvectors is given by the following formula: Among them, eigenvalues The eigenvector represents the contribution of the i-th principal component to the overall data variance. This indicates the directionality of the principal component in space; The principal component selection is based on the order of the eigenvalues, selecting the first k eigenvectors as principal components to characterize the feature directions with the highest variance proportion in the data; When the marker is axisymmetric, select the third principal component. As the main direction of the point cloud, it is used for subsequent alignment and registration.

9. The surgical navigation method based on electromagnetic positioning and augmented reality according to claim 6, characterized in that: In step three, a stable surgical space is constructed through an electromagnetic positioning system. Specifically, the operator uses an electromagnetic positioning probe to collect the coordinates of physical marker points formed by markers actually placed on the patient's body surface or surgical platform. The surgical navigation system uses a singular value decomposition algorithm to optimally match the set of virtual marker points extracted before surgery with the set of physical marker points collected during surgery, and solves the rigid body transformation matrix from image space to surgical space, thereby achieving high-precision alignment between the two spaces. Surgical navigation system defines an electromagnetic space coordinate system. Surgical space coordinate system and instrument coordinate system The operator uses an electromagnetic probe to collect physical markers on the surgical platform. Coordinates in; During the intraoperative real-time navigation phase, doctors wear augmented reality head-mounted displays and use their cameras to identify Diamand code markers. The surgical navigation system uses the PnP algorithm to calculate the pose of the camera coordinate system (CAM) relative to the markers. Based on the aforementioned registration results, the surgical navigation system establishes a complete coordinate transformation chain, namely VM→PT→CAM→IMG, which enables the virtual vascular model to be stably and accurately superimposed on the real surgical field. During surgery, electromagnetic sensors mounted on surgical instruments transmit their pose data in real time via UDP protocol to an augmented reality head-mounted display device worn by the operator, enabling the surgical navigation system to dynamically render the real-time position and posture of virtual instruments within the AR field of view.

10. The surgical navigation method based on electromagnetic positioning and augmented reality according to claim 6, characterized in that: In surgical navigation methods, during augmented reality virtual-real fusion registration, Diamand code markers are detected from images and their position and pose relative to the camera are estimated. Color images are converted to grayscale images through grayscale processing to reduce computational complexity and retain key feature information. Subsequently, the boundary information of the markers was extracted through binarization, and finally the Diamand code was detected and its corner coordinates were extracted using the ArUco marker library; the position and pose of the markers relative to the camera were determined by the PnP algorithm. When transforming coordinates in augmented reality, high-precision virtual-real fusion is achieved through the coordinated transformation of five coordinate systems: the virtual space coordinate system (VM) defines the surgical planning model, the real world coordinate system (RW) establishes a global reference with the magnetic field generator as the origin, the patient coordinate system (PT) is established through a fixed reference sensor on the body surface, the camera coordinate system (CAM) is determined by the optical system of the AR device, and the image coordinate system (IMG) represents the final displayed image. In the augmented reality precise registration process, the surgical navigation system first identifies optical markers in real space using a camera, obtaining the initial pose of the markers in the camera coordinate system (CAM). Combining the known coordinates of the markers in the real space coordinate system (RW), the patient coordinate system (PT), and the virtual space coordinate system (VM), the precise position of the camera in each coordinate system is derived. In subsequent operations, changes in the camera's pose are tracked and compensated in real time using a built-in gyroscope. Based on the precisely calibrated intrinsic and extrinsic parameters of the camera, the surgical navigation system accurately projects the virtual model onto the display screen of the augmented reality head-mounted display device worn by the operator through the transformation chain of VM→PT→CAM→IMG, achieving virtual-real registration.

Images