Surgical navigation system and method based on fusion of three-dimensional space registration and augmented reality

Abstract

The invention discloses a surgical navigation system and method based on three-dimensional space registration and augmented reality fusion, and relates to the technical field of medical equipment, and the surgical navigation system comprises a sterile adaptive AR helmet, a multi-source fusion positioning module, a medical level display module, an intelligent control host and a three-dimensional anchoring processing module. According to the invention, through the cooperation of the sterile adaptive AR helmet and the multi-source fusion positioning module, the accuracy and safety of the operation are remarkably improved, the medical level display module is combined with the real-time space registration of the three-dimensional anchoring processing module, the immersive 3D navigation of the DICOM image is realized, the view switching distraction is eliminated, the operation efficiency is optimized, and the operation accuracy and safety are improved. Intraoperative infection prevention and control are jointly enhanced through the intelligent control host and the sterile design, and the pollution risk is reduced. The dynamic scene adaptability and standardization compatibility of the system further expand the application potential of the system in complex scenes of pleuroperitoneal cavity minimally invasive surgery and orthopedic joint replacement, and provide reliable technical support for precise and minimally invasive surgery.

Description

Technical Field

[0001] The present invention relates to the technical field of medical equipment, and in particular to a surgical navigation system and method based on the fusion of three-dimensional space registration and augmented reality. Background Art

[0002] In modern surgery, accurate surgical navigation is crucial to improving surgical success rates and reducing patient trauma. However, traditional surgical navigation equipment has many problems in practical applications:

[0003] In sterile operating scenarios, the wearing structure of traditional equipment is poorly compatible with sterile caps and gowns, making it cumbersome to put on and take off, affecting surgical efficiency. The device casing temperature is poorly controlled, posing a risk of high-temperature burns. The fan heat dissipation design generates noise that interferes with the surgical environment. Insufficient cable and interface stability also poses safety risks.

[0004] Irrational power consumption control means the system maintains high power consumption even when not in use, resulting in energy waste. Furthermore, manual interaction is often required, increasing the risk of contamination during surgery and reducing surgical efficiency.

[0005] The spatial positioning accuracy and stability are insufficient. Traditional positioning methods are prone to positioning drift in dynamic surgical environments. The registration accuracy of patient medical images and surgical space is not high, and the posture of surgical instruments is not updated in a timely manner, which affects the doctor's accurate judgment of the position of surgical instruments.

[0006] Therefore, there is an urgent need for a surgical navigation system and method that can solve the above problems.

[0007] The defects of existing surgical navigation systems and methods are:

[0008] 1. Patent document CN115813555A discloses an ultrasound puncture navigation system, method and medium based on cross-modal image precise registration, "including: an image acquisition device, communicating with an image fusion device, for acquiring image data of the area where surgery is required; an ultrasound probe, communicating with the image fusion device, for acquiring ultrasound images of the area where surgery is required; a marker point, affixed to the area where surgery is required; a CT imaging device, communicating with the image fusion device, for acquiring CT images of the area where surgery is required; an image fusion device, for acquiring image data, ultrasound images and CT images of the area where surgery is required, and fusing them to obtain a composite image; a three-dimensional spatial coordinate system establishment module, for establishing a three-dimensional spatial coordinate system based on the composite image; a path planning module, for performing ultrasound puncture navigation path planning in the three-dimensional spatial coordinate system based on the pathological location and the surgery to be performed to obtain a planned path; and a radiofrequency puncture treatment needle, for performing puncture treatment according to the planned path." However, the device in the above document has a technical problem of relying on a single positioning technology and being easily disturbed by the surgical environment, resulting in large navigation deviations.

[0009] 2. Patent document CN115836914A discloses an endoscopic surgery navigation method and device based on video image enhanced reality, "The method comprises: in the preoperative CT image, the key parts of the surgery are segmented and the anatomical landmark points are selected; passive tracking tools are installed on the patient's surgery site and the endoscope respectively; the real patient is registered with the three-dimensional model using the anatomical landmark points with the calibrated probe; the parameters of the endoscope are calibrated under the navigation system; the conversion matrix of the patient's surgery site passive tracking tool coordinate system relative to the endoscope coordinate system is determined, the three-dimensional model of the preoperative segmented key parts of the surgery is projected and rendered using the pinhole model simulation imaging and processed using the distortion parameters to obtain a virtual image, and the virtual image is fused with the image collected by the endoscope in real time and displayed. Compared with the prior art, the virtual image is fused with the real image of the endoscope and presented on the same screen, which reduces the intraoperative judgment time of the doctor and reduces the difficulty of hand-eye coordination", but the device in the above document has the technical problems of device wearing discomfort, poor voice control noise resistance, slow interface adaptation, poor compatibility, leading to long preoperative preparation time and intraoperative process interruption;

[0010] 3. Patent document CN113349927A discloses a facial autologous fat transplantation surgery navigation system and virtual scale prompting method, "The system comprises: a dental mold registration block, a dental mold registration block, a dental mold connecting sleeve, a dental mold registration mark plate, a surgical instrument registration mark plate, and a mark drawing. The system can further comprise an intelligent terminal. The system is based on augmented reality technology, uses personalized non-invasive registration dental mold, and fuses and superimposes virtual information such as skin, blood vessels, bones, and preoperative planning with real information such as patient entities. Artificially implanted marker nails are not required before surgery, non-invasive registration of personalized dental mold is used to avoid additional trauma to patients caused by invasive implantation. Intraoperative fusion of the patient's three-dimensional model and preoperative surgical planning with patient entity information and real-time tracking of surgical instruments can provide real-time reminders for the pose and needle insertion distance of the syringe during needle insertion, so that doctors can more accurately complete the surgery", but the device in the above document is prone to misalignment in complex scenes, the positioning matrix is unstable, and the navigation drifts or fails. SUMMARY

[0011] The purpose of the present application is to provide a surgery navigation system and method based on three-dimensional space registration and augmented reality fusion to solve the technical problems raised in the background art.

[0012] To achieve the above-mentioned purpose, the present application provides the following technical scheme: a surgery navigation system based on three-dimensional space registration and augmented reality fusion, comprising a sterile AR helmet, a multi-source fusion positioning module, a medical-grade display module, an intelligent control host and a three-dimensional anchor processing module.

[0013] The sterile AR helmet, comprising:

[0014] The adjustable head circumference wearing structure is compatible with sterile caps and sterile clothes.

[0015] The helmet shell adopts natural convection heat dissipation without fan or low-noise fan design, and the surface temperature is ≤40℃.

[0016] The cable interface is integrated in the helmet end and supports quick removal during surgery.

[0017] The multi-source fusion positioning module integrates navigation cameras, SLAM cameras and AprilTag recognition units, and realizes static positioning error ≤3.5mm through joint calling, and when the device moves at a speed of ≤1m / s in a dynamic scene, the drift per minute is ≤0.8% of the moving distance.

[0018] The medical-grade display module has a monocular resolution of ≥20PPD, a display delay of ≤60ms, a SLAM virtual image MTP delay of ≤60ms, a navigation MTP delay of ≤70ms, and supports DICOM format image MPR and 2D / biocular 3D display mode.

[0019] The intelligent control host is electrically connected with an in-place detection sensor, which is arranged inside the sterile AR helmet. When not wearing, the navigation camera and SLAM camera are turned off and enter sleep mode, and a noise suppression voice interaction unit is integrated to support keyword wake-up.

[0020] The three-dimensional anchor processing module performs medical image and surgical space registration and updates the pose of surgical instruments in real time.

[0021] Preferably, the sterile AR helmet lining adopts a long-wearing pressure-free material, and the cable interface supports quick removal within 1 second.

[0022] Preferably, in the multi-source fusion positioning module: the navigation camera provides ≤1mm optical positioning data, the AprilTag recognition unit recognizes preset marker points, the SLAM camera dynamically constructs an environment map using an improved algorithm, and after three-source data fusion, the static 40-minute drift is ≤2.0mm, and the multi-source fusion positioning module interface is compatible with the Unity general platform SDK, supporting independent calling and joint calling of AprilTag / infrared light ball recognition mode.

[0023] Preferably, the medical-grade display module comprises:

[0024] Window width and window position dynamic adjustment function;

[0025] Mapping function of assigning colors according to tissue types;

[0026] Tissue transparency continuous adjustment function.

[0027] Preferably, the three-dimensional anchor processing module comprises:

[0028] The three-dimensional reconstruction unit adopts a hybrid technique of surface rendering and volume rendering, wherein the bone structure is suitable for surface rendering and the soft tissue is suitable for volume rendering.

[0029] The registration unit performs multi-modal mutual information registration, and the registration error of the brain is ≤1.5 mm, the registration error of the chest and abdomen is ≤3 mm, the accuracy is verified by using the fiducial marker point, and the registration error (TRE) is evaluated.

[0030] The spatial positioning unit tracks the position of the instrument in the virtual coordinate system through the infrared locator, and dynamically renders the virtual probe in the virtual coordinate system.

[0031] Preferably, the registration unit performs:

[0032] The patient AprilTag tracker coordinates are mapped to the virtual world coordinate system, and the spatial alignment is realized by solving the least squares solution of the projection matrix.

[0033] Preferably, the voice interaction unit of the intelligent control host realizes accurate keyword pickup in a noisy operating environment.

[0034] Preferably, the method of three-dimensional space registration and augmented reality fusion surgical navigation comprises the following steps:

[0035] S1, analyze the CT, MRI and PET images in DICOM format, and generate transverse, coronal and sagittal reconstruction images through the MPR algorithm;

[0036] S2, fuse the navigation camera optical positioning data, AprilTag identification preset marker point data and SLAM environment map data, and calculate the spatial positioning matrix;

[0037] S3, register the preoperative three-dimensional reconstruction model with the spatial position of the patient: align the multi-modal images by using the mutual information registration algorithm, and bind the virtual coordinate system through the patient AprilTag tracker;

[0038] S4, real-time tracking of the position of the surgical instrument: obtain the tip coordinates of the instrument through the infrared sensor, and convert to the local reference coordinate system by using the homogeneous transformation matrix, wherein the rotation matrix is generated by quaternion calculation;

[0039] S5, dynamically superimpose the instrument position data and the three-dimensional anchor image in the AR helmet display module, and the refresh delay is ≤70 ms;

[0040] S6, realize scene-based power consumption control through head-in-place detection: turn off the navigation camera and the SLAM camera to enter sleep in the non-wearing state.

[0041] Preferably, the S1 includes:

[0042] MPVR technology is used to adjust the reconstruction layer thickness to display curved organs, and MIP projection is used to enhance angiographic structures;

[0043] MPR: Generates 2D tomographic image planes in transverse, coronal, and sagittal planes by selecting a fixed point on the 3D image;

[0044] MPVR: Based on the two-dimensional tomographic image formed by MPR, the thickness of the reconstructed image is changed by selecting the distance between two parallel lines, thereby fully displaying the information of curved organs and removing background interference;

[0045] MIP: Suitable for blood vessels with high-density signals, capable of highlighting contrast-enhanced vascular structures

[0046] Preferably, the S4 includes:

[0047] Determine the local reference coordinate system parameters: the origin position (ox, oy, oz), and the posture is represented by the quaternion q = (q0, q1, q2, q3):

[0048] Convert the quaternion to a rotation matrix R:

[0049] ;

[0050] Construct the homogeneous transformation matrix T:

[0051] ;

[0052] The translation vector T=(ox,oy,oz)

[0053] The position of the tool in the coordinate system is (x, y, z), which is converted to homogeneous coordinate form and converted to the position in the local reference coordinate system through matrix multiplication. The formula is:

[0054] ;

[0055] ;

[0056] ;

[0057] Compute the local reference coordinates (u, v, w).

[0058] Compared with the prior art, the present invention has the following beneficial effects:

[0059] 1. This invention significantly improves surgical precision and safety through the collaboration of a sterile, adaptable AR helmet and a multi-source fusion positioning module. The medical-grade display module, combined with real-time spatial registration by a 3D anchoring processing module, enables immersive 3D navigation of DICOM images, eliminating distractions caused by field of view switching and optimizing surgical efficiency. Furthermore, the intelligent control host and sterile design enhance intraoperative infection prevention and control, reducing contamination risks. Its dynamic scenario adaptability and standardized compatibility further expand the system's potential for application in complex scenarios such as minimally invasive thoracic and abdominal surgery and orthopedic joint replacements, providing reliable technical support for precision and minimally invasive surgery.

[0060] 2. This invention achieves anatomically specific precision control of cranial registration errors of ≤1.5mm and thoracic and abdominal registration errors of ≤3mm through multi-source data fusion and an improved SLAM algorithm, combined with multimodal mutual information registration and verification with fiducial markers. It also features a 1-second quick-release interface for improved efficiency, an intelligent visualization structure for optimized tissue observation, and three-source redundant positioning to ensure reliable tracking. It is seamlessly compatible with existing hospital markers and orthopedic, neurosurgery, and laparoscopic procedures, providing a fully optimized physician experience with non-indentation wear, noise-resistant voice control, hybrid rendering of anatomical organs, and a low-latency virtual probe.

[0061] 3. This invention dynamically reconstructs multimodal images using MPR and MPVR technologies, combines MIP projection with enhanced visualization of curved vascular organs, integrates navigation camera optical positioning, AprilTag marker recognition, and SLAM environmental map data to construct a robust spatial positioning matrix, significantly improving anti-interference capabilities in complex surgical environments. It utilizes a mutual information registration algorithm to align multimodal images and binds the patient coordinate system via the AprilTag tracer, enabling rapid millimeter-level registration between the preoperative model and the intraoperative space. Quaternion-based real-time calculation of rotation matrices and homogeneous transformations efficiently maps the instrument pose captured by the infrared sensor to the local reference coordinate system, providing submillimeter tracking accuracy. Three-dimensional anchor images and instrument data are dynamically superimposed on the AR helmet, with a refresh delay of ≤70ms, ensuring a dizziness-free immersive navigation experience. Furthermore, the head-in-position detection intelligent start-stop hardware module optimizes system power consumption and extends battery life, ultimately achieving high-precision, low-latency, and highly stable surgical navigation, significantly improving operational safety and efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the overall flow of the system of the present invention;

[0063] Figure 2 This is a flow chart of the multi-source fusion positioning module of the present invention;

[0064] Figure 3 This is a schematic diagram of the three-dimensional anchoring processing module flow of the present invention;

[0065] Figure 4This is a schematic diagram of the medical power consumption control process of the present invention;

[0066] Figure 5 This is a schematic diagram of the medical display process of the present invention;

[0067] Figure 6 This is a schematic diagram of the three-dimensional structure of the sterile adaptable AR helmet of the present invention;

[0068] Figure 7 This is a schematic diagram of the front structure of the sterile-fit AR helmet of the present invention;

[0069] Figure 8 This is a schematic diagram of the side structure of the sterile-fit AR helmet of the present invention.

[0070] In the picture: 1. Sterile-fitting AR helmet; 2. Intelligent control host. DETAILED DESCRIPTION

[0071] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] In the description of the present invention, it should be noted that the terms "upper," "lower," "inner," "outer," "front end," "rear end," "both ends," "one end," "the other end," and the like, indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limiting the present invention. Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0073] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "provided with," "connected," etc., should be understood in a broad sense. For example, "connected" may refer to a fixed connection, a detachable connection, or an integral connection; it may refer to a mechanical connection or an electrical connection; it may refer to a direct connection or an indirect connection through an intermediate medium; it may refer to internal communication between two components. Those skilled in the art will be able to understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0074] Example 1: Please refer to Figure 1 、 Figure 2 、 Figure 5 、 Figure 6 、 Figure 7And Figure 8 An embodiment of the present application provides a surgical navigation system based on three-dimensional space registration and augmented reality fusion, comprising a sterile AR helmet 1, a multi-source fusion positioning module, a medical-grade display module, an intelligent control host 2 and a three-dimensional anchor processing module. The sterile AR helmet 1 comprises a wearable structure with adjustable head circumference and is compatible with sterile caps and sterile clothes. The helmet shell adopts natural convection cooling without fan or low-noise fan design, and the surface temperature is ≤40℃. The cable interface is integrated at the helmet end and supports quick removal during surgery. The multi-source fusion positioning module integrates navigation cameras, SLAM cameras and AprilTag recognition units, and realizes static positioning error ≤3.5mm through joint calling. When the device moves at a speed of ≤1m / s in a dynamic scene, the drift per minute is ≤0.8% of the moving distance. The medical-grade display module has a monocular resolution of ≥20PPD, a display delay of ≤60ms, a SLAM virtual image MTP delay of ≤60ms, and a navigation MTP delay of ≤70ms. It supports MPR and 2D / biocular 3D display modes of DICOM format images. The intelligent control host 2 is electrically connected with an in-position detection sensor, which is arranged inside the sterile AR helmet 1. In a non-wearing state, the navigation camera and the SLAM camera are turned off and enter a sleep mode, and a noise suppression voice interaction unit is integrated to support keyword wake-up. The three-dimensional anchor processing module performs medical image and surgical space registration and updates the surgical instrument pose in real time.

[0075] Further, the surgical precision and safety are improved. Through the navigation camera, the SLAM camera and the AprilTag recognition unit in the multi-source fusion positioning module, the static error is ≤3.5mm and the dynamic drift rate is ≤0.8% / minute, which ensures the stability of the intraoperative instrument pose tracking, reduces the risk of misoperation, and dynamically matches the medical image and the real surgical space through the three-dimensional anchor processing module to update the instrument pose in real time, assisting the doctor in accurately positioning the lesion and the key anatomical structure.

[0076] The surgical procedure and efficiency are optimized. The medical-grade display module directly projects the DICOM image to the doctor's field of view, eliminating the distraction operation of looking up at the screen and looking down at the surgery in traditional navigation, shortening the surgery time, and the cable integrated design and the quick removal interface at the helmet end facilitate emergency adjustment or temporary departure during surgery, avoiding interruption of the surgical procedure due to equipment problems.

[0077] The intraoperative sterile protection is strengthened. The helmet adjustable structure is compatible with sterile caps and sterile clothes, adopts natural convection cooling without fan or low-noise fan design, and the surface temperature is ≤40℃, avoiding pollution of the surgical area due to the heat dissipation holes of traditional electronic devices. The in-position detection sensor automatically turns off the camera and enters sleep mode, reducing device contact during non-use and reducing the risk of pollution.

[0078] Enhanced human-computer interaction experience: With a single-eye resolution of ≥20PPD, SLAM virtual image delay ≤60ms, and navigation delay ≤70ms, dizziness caused by visual smearing is avoided, ensuring comfort during long surgeries. Furthermore, with noise-suppressed voice interaction and keyword wake-up support, doctors can access images or command systems without touching the screen in a sterile environment.

[0079] Reduce system operation and maintenance costs: The fanless natural convection cooling or low-noise fan design extends device lifespan, and the intelligent sleep mechanism reduces ineffective power consumption. Suitable for continuous surgery scenarios, it supports DICOM format and MPR reconstruction, seamlessly connects to the hospital's PACS system, and avoids data conversion costs.

[0080] Expanding clinical application scenarios: Joint positioning of SLAM and AprilTag can maintain accuracy when instruments are blocked or tissues are displaced. It is suitable for surgeries that are significantly affected by thoracic and abdominal respiratory movements.

[0081] Example 2: Please refer to Figure 2 and Figure 4 , an embodiment provided by the present invention: the lining of the sterile adaptive AR helmet 1 is made of a material that does not leave marks after long-term wear, the cable interface supports quick removal within 1 second, and in the multi-source fusion positioning module: the navigation camera provides ≤1mm optical positioning data, the AprilTag recognition unit recognizes preset markers, the SLAM camera uses an improved algorithm to dynamically build an environmental map, and the static drift of 40 minutes after the three-source data is fused is ≤2.0mm, and the multi-source fusion positioning module interface is compatible with the Unity general platform SDK, supporting independent and joint calls of AprilTag / infrared light ball recognition methods, and a medical-grade display module, including: dynamic adjustment function of window width and window position, mapping function of color assignment according to tissue type, continuous adjustment function of tissue transparency, three-dimensional The anchor processing module includes: a 3D reconstruction unit that uses a hybrid surface rendering and volume rendering technology, with surface rendering being suitable for skeletal structures and volume rendering being suitable for soft tissues; a registration unit that performs multimodal mutual information registration, with a cranial registration error of ≤1.5mm and a thoracic and abdominal registration error of ≤3mm. The registration is verified using reference markers through target registration error (TRE) evaluation to improve accuracy; a spatial positioning unit that tracks the instrument's position through an infrared locator and dynamically renders a virtual probe in a virtual coordinate system; the registration unit maps the patient's AprilTag tracer coordinates to the virtual world coordinate system and achieves spatial alignment by solving the least squares solution of the projection matrix; and the voice interaction unit of the intelligent control host 2 achieves accurate keyword pickup in a noisy surgical environment.

[0082] Furthermore, surgical precision has been revolutionized: multi-source data fusion achieves a static drift of ≤2.0mm over 40 minutes, ensuring instrument tracking during prolonged surgeries. The improved SLAM algorithm dynamically constructs an environmental map, maintaining positioning continuity during tissue displacement. Furthermore, the cranial registration error is ≤1.5mm, and the chest and abdomen ≤3mm. Anatomical calibration deviation is reduced through TRE evaluation and verification using fiducial markers. AprilTag virtual coordinate system mapping uses a least-squares solution of the projection matrix to achieve an instrument-image space error of ≤1.5mm.

[0083] Intelligent tissue visualization: Dynamic adjustment of window width and window position can optimize image contrast in real time, pseudo-color mapping can automatically stain according to tissue type, and transparency adjustment can observe deep tumors through muscle layers;

[0084] Unity platform compatibility: supports seamless integration of existing hospital navigation markers, saving system switching time;

[0085] Optimized surgeon experience: The inner lining is made of non-marking material, which facilitates continuous wear for long periods of time without causing facial indentations. It also supports keyword recognition in noisy electrosurgical environments, avoiding contamination from manual manipulation during surgery. Hybrid rendering technology allows for clear bone contours and gradient transparency in soft tissue, giving virtual organs an anatomical texture.

[0086] Dynamic rendering of virtual probe: The instrument posture is synchronized with the virtual coordinate system in real time, reducing visual delay;

[0087] Three-source positioning redundancy design: When SLAM is blocked by blood stains, AprilTag / optical positioning automatically takes over;

[0088] System scalability upgrade: compatible with orthopedics, infrared light ball tracking bone drills, neurosurgery, AprilTag skull marking, laparoscopic surgery, SLAM construction of abdominal cavity maps, and DICOM images reconstructed by MPR, directly mapped to generate surgical navigation base maps, which helps shorten preoperative planning time.

[0089] Example 3: Please refer to Figure 3 The present invention provides an embodiment of a method for surgical navigation integrating three-dimensional space registration with augmented reality, and the steps are as follows:

[0090] S1. Parse CT, MRI, and PET images in DICOM format and generate transverse, coronal, and sagittal reconstruction images using the MPR algorithm.

[0091] S2, integrates the navigation camera's optical positioning data, AprilTag's preset marker data, and SLAM environment map data to calculate the spatial positioning matrix;

[0092] S3. Align the preoperative 3D reconstructed model with the intraoperative patient spatial position: Use the mutual information registration algorithm to align the multimodal images and bind the virtual coordinate system through the patient's AprilTag tracer;

[0093] S4. Real-time tracking of surgical instrument posture: The coordinates of the instrument tip are obtained through infrared sensors and converted to the local reference coordinate system using a homogeneous transformation matrix, where the rotation matrix is ​​generated by quaternion calculation;

[0094] S5. Dynamically superimpose device posture data and 3D anchor images on the AR helmet display module, with a refresh delay of ≤70ms;

[0095] S6. Scenario-based power consumption control is achieved through head presence detection: the navigation camera and SLAM camera are turned off and put into sleep mode when not wearing the head;

[0096] S1 includes: using MPVR technology to adjust the reconstruction layer thickness to display curved organs and enhancing angiographic structures through MIP projection;

[0097] S4 includes: determining the local reference coordinate system parameters: the origin position (ox, oy, oz), the posture is determined by the quaternion q = (q0, q1, q2, q 3） express:

[0098] Convert the quaternion to a rotation matrix R:

[0099] ;

[0100] Construct the homogeneous transformation matrix T:

[0101] ;

[0102] The translation vector T=(ox,oy,oz)

[0103] The position of the tool in the coordinate system is (x, y, z), which is converted to homogeneous coordinate form and converted to the position in the local reference coordinate system through matrix multiplication. The formula is:

[0104] ;

[0105] ;

[0106] ;

[0107] Calculate the local reference coordinates (u, v, w);

[0108] Furthermore, multimodal image reconstruction is precise: using MPR and MPVR technologies, the reconstruction layer thickness is dynamically adjusted to clearly display curved organ structures, and combined with MIP to enhance angiographic contrast, the recognition of key anatomical structures during surgery is improved;

[0109] High-precision spatial positioning integrates optical positioning, AprilTag marker recognition, and SLAM environmental map data to build a robust spatial positioning matrix, significantly improving anti-interference capabilities and adapting to complex surgical environments;

[0110] Fast and accurate intraoperative registration: Mutual information registration algorithms align multimodal images to resolve differences between different imaging modalities. AprilTag tracers are used to bind the patient coordinate system, achieving millimeter-level registration between the virtual model and the physical space.

[0111] Low-latency instrument tracking: The coordinates of the instrument tip are captured in real time by infrared sensors. The instrument position is mapped to a local reference coordinate system through mathematical transformations from quaternions to rotation matrices to homogeneous transformation matrices. The computational efficiency ensures accurate and real-time tracking.

[0112] Immersive AR navigation experience: The AR helmet dynamically overlays 3D anchor images and instrument posture, with a refresh delay of ≤70ms, helping doctors intuitively observe the spatial relationship between instruments and anatomical structures, reducing blind spots in operation.

[0113] Intelligent power consumption optimization: Automatically starts and stops the navigation camera and SLAM camera through head presence detection, reducing equipment standby power consumption and extending the battery life of the surgical system.

[0114] Working principle: Through the collaboration of sterile adaptive AR helmet 1 and multi-source fusion positioning module, the accuracy and safety of surgery are significantly improved. The medical-grade display module is combined with the real-time spatial registration of the three-dimensional anchoring processing module to realize immersive 3D navigation of DICOM images, eliminate distraction caused by field of view switching, optimize surgical efficiency, and then strengthen intraoperative infection prevention and control through the intelligent control host 2 and sterile design to reduce the risk of contamination; its dynamic scene adaptability and standardized compatibility further expand the application potential of the system in complex scenes of thoracic and abdominal minimally invasive surgery and orthopedic joint replacement, providing reliable technical support for precise and minimally invasive surgery. Through multi-source data fusion and improved SLAM algorithm, combined with multi-modal mutual information registration and verification with reference markers, anatomical-specific precision control of cranial registration error ≤1.5mm and thoracic and abdominal ≤3mm is achieved, and it is equipped with a 1-second quick-release interface to improve efficiency, intelligent visualization structure, optimized tissue observation, three-source redundant positioning to ensure reliable tracking, seamless compatibility with the hospital's existing markers and orthopedic, neurosurgery, and laparoscopic procedures, and comprehensive optimization of doctor experience, no indentation wearing, anti-noise The system uses audio manipulation, hybrid rendering of anatomical organs, and low-latency virtual probes. It dynamically reconstructs multimodal images using MPR and MPVR technologies, and combines MIP projection to enhance the visualization of curved vascular organs. It integrates navigation camera optical positioning, AprilTag landmark recognition, and SLAM environmental map data to construct a robust spatial positioning matrix, significantly improving interference resistance in complex surgical environments. It uses a mutual information registration algorithm to align multimodal images and binds the patient coordinate system via the AprilTag tracer, achieving millimeter-level rapid registration between the preoperative model and the intraoperative space. It uses quaternions to calculate rotation matrices and homogeneous transformations in real time, efficiently mapping the instrument pose captured by the infrared sensor to the local reference coordinate system, providing submillimeter tracking accuracy. It dynamically overlays 3D anchor images and instrument data on the AR headset with a refresh latency of ≤70ms, ensuring a dizziness-free immersive navigation experience. Furthermore, the head presence detection intelligent start-stop hardware module optimizes system power consumption and extends battery life, ultimately achieving high-precision, low-latency, and highly stable surgical navigation, significantly improving operational safety and efficiency.

[0115] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations that come within the meaning and range of equivalents of the claims be embraced therein.

[0116] Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A surgical navigation system based on the fusion of three-dimensional spatial registration and augmented reality, characterized by: It includes a sterile adaptive AR helmet (1), a multi-source fusion positioning module, a medical-grade display module, an intelligent control host (2) and a three-dimensional anchoring processing module; The sterile adaptable AR helmet (1) comprises: The wearing structure can be adjusted to the head circumference and is compatible with sterile caps and sterile gowns; Helmet shell: adopts fanless natural convection cooling or low-noise fan design, surface temperature ≤40℃; Cable interface: integrated into the helmet end, supporting rapid removal during surgery; The multi-source fusion positioning module integrates a navigation camera, a SLAM camera, and an AprilTag recognition unit, and achieves a static positioning error of ≤3.5mm through joint calls. In dynamic scenarios, when the device moves at a speed of ≤1m / s, the drift per minute is ≤0.8% of the moving distance. The medical-grade display module: monocular resolution ≥ 20PPD, display delay ≤ 60ms, SLAM virtual image MTP delay ≤ 60ms, navigation MTP delay ≤ 70ms, supports multi-planar reconstruction (MPR) of DICOM format images and 2D / binocular 3D display mode; The intelligent control host (2) is electrically connected to a presence detection sensor, and the presence detection sensor is arranged inside the sterile adaptive AR helmet (1). When the helmet is not worn, the navigation camera and the SLAM camera are turned off and enter a sleep mode. The intelligent control host (2) is integrated with a noise suppression voice interaction unit and supports keyword wake-up. The three-dimensional anchoring processing module performs registration of medical images and surgical space, and updates the posture of surgical instruments in real time.

2. The surgical navigation system based on 3D spatial registration and augmented reality fusion according to claim 1, characterized in that: The inner lining of the sterile adaptable AR helmet (1) is made of a material that does not leave marks when worn for a long time, and the cable interface supports quick removal within 1 second.

3. The surgical navigation system based on 3D space registration and augmented reality fusion according to claim 1, characterized in that: In the multi-source fusion positioning module: the navigation camera provides ≤1mm optical positioning data, the AprilTag recognition unit identifies preset markers, and the SLAM camera uses an improved algorithm to dynamically build an environmental map. After the three-source data is fused, the static drift within 40 minutes is ≤2.0mm. The multi-source fusion positioning module interface is compatible with the Unity general platform SDK and supports independent and joint calls of AprilTag / infrared light ball recognition methods.

4. The surgical navigation system based on 3D space registration and augmented reality fusion according to claim 1, characterized in that: The medical-grade display module comprises: Dynamic adjustment function of window width and window position; Mapping function to assign colors by tissue type; Continuous adjustment function for tissue transparency.

5. The surgical navigation system based on 3D space registration and augmented reality fusion according to claim 1, characterized in that: The three-dimensional anchoring processing module includes: 3D reconstruction unit: uses a hybrid technology of surface rendering and volume rendering, where surface rendering is suitable for bone structures and volume rendering is suitable for soft tissues; Registration unit: Performs multimodal mutual information registration, with a cranial registration error of ≤1.5mm and a thoracic and abdominal registration error of ≤3mm. Target registration error (TRE) evaluation is performed, and fiducial markers are used to verify registration accuracy. Spatial positioning unit: Tracks the instrument's position through an infrared locator and dynamically renders a virtual probe in a virtual coordinate system.

6. The surgical navigation system based on 3D space registration and augmented reality fusion according to claim 5, characterized in that: The registration unit performs: The coordinates of the patient's AprilTag tracer are mapped to the virtual world coordinate system, and spatial alignment is achieved by solving the least squares solution of the projection matrix.

7. The surgical navigation system based on 3D space registration and augmented reality fusion according to claim 1, characterized in that: The voice interaction unit of the intelligent control host (2) realizes accurate keyword pickup in a surgical noise environment.

8. A surgical navigation method based on 3D spatial registration and augmented reality fusion, applicable to the surgical navigation system based on 3D spatial registration and augmented reality fusion according to any one of claims 1 to 7, characterized in that: The steps of the surgical navigation method for integrating three-dimensional space registration with augmented reality are as follows: S1. Parse CT, MRI, and PET images in DICOM format and generate transverse, coronal, and sagittal reconstruction images using the MPR algorithm. S2, integrates the navigation camera's optical positioning data, AprilTag's preset marker data, and SLAM environment map data to calculate the spatial positioning matrix; S3. Align the preoperative 3D reconstructed model with the intraoperative patient spatial position: Use the mutual information registration algorithm to align the multimodal images and bind the virtual coordinate system through the patient's AprilTag tracer; S4. Real-time tracking of surgical instrument posture: The coordinates of the instrument tip are obtained through infrared sensors and converted to the local reference coordinate system using a homogeneous transformation matrix, where the rotation matrix is ​​generated by quaternion calculation; S5. Dynamically superimpose device posture data and 3D anchor images on the AR helmet display module, with a refresh delay of ≤70ms; S6. Scenario-based power consumption control is achieved through head presence detection: the navigation camera and SLAM camera are turned off and put into sleep mode when not being worn.

9. The surgical navigation method based on 3D space registration and augmented reality fusion according to claim 8, characterized in that: Said S1 comprises: Multiplanar volume reconstruction (MPVR) technology was used to adjust the reconstruction slice thickness to display curved organs, and maximum intensity projection (MIP) projection was used to enhance angiographic structures.

10. The surgical navigation method based on 3D space registration and augmented reality fusion according to claim 8, characterized in that: The S4 includes: Determine the local reference coordinate system parameters: origin position (ox, oy, oz), posture is determined by quaternion q = (q0, q1, q2, q 3） express: Convert the quaternion to a rotation matrix R: Construct the homogeneous transformation matrix T: The translation vector T=(ox,oy,oz) The position of the tool in the coordinate system is (x, y, z), which is converted to homogeneous coordinate form and converted to the position in the local reference coordinate system through matrix multiplication. The formula is: Compute the local reference coordinates (u, v, w).

Images