A method of surgical navigation registration based on marker detection

Abstract

The application discloses a kind of based on identification object detection surgical navigation registration method, comprising the following steps: data acquisition is carried out to the patient cranium of sticking mark point, and scanning obtains medical image data;Based on the medical image data set scanned, pre-processes image data;Based on the tracking registration method of computer vision, the positioned tracking and space registration of pasted identification object are carried out, and according to the registration result, auxiliary navigation is carried out;Based on the AR.js framework of Web, AR visualization platform system is constructed.The application adopts the above-mentioned one based on identification object detection surgical navigation registration method, and the positioned tracking of pasted identification object is carried out by ordinary monocular camera identification, and it is simple and convenient;Using the mode of space coordinate conversion, the space registration of virtual model to patient entity is realized, the accuracy is guaranteed while realizing surgical navigation process;Augmented reality visualization using the AR.js framework of web mode, without additional application program installed by user.

Description

Technical Field

[0001] This invention relates to the field of medical image processing technology, and in particular to a surgical navigation registration method based on marker detection. Background Technology

[0002] With the rapid development of technologies such as digital image processing, computer vision, network communication, and location tracking, computer-aided diagnosis and treatment has become an indispensable part of modern surgical techniques. Surgical navigation systems are an important application of computer-aided technology in surgery. By combining the patient's anatomical information with real-time positioning technology, they accurately locate lesions and plan surgical paths, thereby improving the safety and effectiveness of surgery. This rapid development has greatly promoted the advancement of modern surgical techniques. Previously, many surgeries relied on years of experience from surgeons, especially in craniocerebral surgery, where the complex intracranial structures significantly increased surgical risks. The emergence of surgical navigation systems has completely changed the traditional surgical model. With its high precision and minimally invasive characteristics, it has greatly improved the safety of surgical procedures and met people's urgent need for advanced medical technology.

[0003] However, current surgical navigation systems on the domestic and international markets generally suffer from problems such as bulky instruments, high prices, and inconvenient operation. In view of this, this invention proposes a surgical navigation registration method based on marker detection. This method is simple, convenient, highly accurate, and real-time, providing a more precise and efficient auxiliary tool for surgical procedures. Summary of the Invention

[0004] The purpose of this invention is to provide a surgical navigation registration method based on marker detection, overcoming the shortcomings of traditional surgical methods and solving the problems of requiring multiple large and cumbersome machines, tedious processes, and complex calculations in surgical navigation. This invention utilizes the ARToolkit software development kit, using a common monocular camera to identify markers for positioning and tracking. This method is not only simple and convenient with a basic device, but also achieves spatial registration from the virtual model to the patient entity through spatial coordinate transformation. While ensuring accuracy, it features low computational load and high real-time performance, enabling successful surgical navigation. The web-based AR.js framework is used for augmented reality visualization, eliminating the need for users to install additional applications, and offering cross-platform compatibility and wide applicability.

[0005] To achieve the above objectives, the present invention provides a surgical navigation registration method based on marker detection, comprising the following steps:

[0006] S1. Data is collected from the patient's cranium with the marked points, and medical imaging data is obtained by scanning.

[0007] S2. Preprocess the acquired medical image data, including: image segmentation, 3D reconstruction, access channel design, and 3D model rendering.

[0008] S3. A computer vision-based tracking and registration method is used to locate and spatially register the pasted markers, and to provide assisted navigation based on the registration results.

[0009] S4, a web-based AR.js framework, is used to build an AR visualization platform system.

[0010] Preferably, in step S1, the specific steps for acquiring medical image data are as follows:

[0011] S11. Select no fewer than three non-collinear physiological feature points on the patient's skull and attach X-ray-opaque markers to the selected physiological feature points.

[0012] S12. Scan the cranium with marked points to acquire multimodal two-dimensional image data output by medical imaging equipment, and save the scan results in the prescribed format; wherein, the multimodal two-dimensional image data includes CT, MRI, DR, CTA, and PET images; 3DSlicer saves them in DICOM format.

[0013] Preferably, in step S2, the acquired medical image data is preprocessed, and the specific process is as follows:

[0014] S21. Image segmentation: Based on 3DSlicer software, segment medical image data and extract cranial lesions, skull models, and pasted markers from the original image data.

[0015] The segmentation methods include automatic segmentation and manual segmentation; automatic segmentation includes thresholding, level set, and region growing methods; manual segmentation involves drawing specific regions within the reconstruction area of ​​each layer for segmentation.

[0016] S22. 3D Reconstruction: Based on the above segmentation results, 3D reconstruction is performed using the 3DSlicer 3D reconstruction method to obtain the corresponding 3D model; the 3D reconstruction method includes volume rendering and surface rendering; the 3D model is a cranial outline with marked points, and the lesions are located at the corresponding positions;

[0017] S23. Approach design, which is to design the optimal surgical route for craniocerebral neurosurgery, as follows:

[0018] S231. When designing the access route on the model, use the Lines measurement tool of 3DSlicer to perform multiple measurements to ensure that important functional areas and arteries and veins of the patient's brain are avoided during the puncture access route, thereby determining the starting point s and the ending point e of the access route, and thus obtaining the channel depth h of the lesion.

[0019] S232. Using the line connecting the puncture start point and the puncture end point as the central axis, use the region growing method to generate an access channel model with a radius of r, obtain the optimal surgical puncture route, and save it as a vtk format for export.

[0020] S24, 3D model rendering, the specific process is as follows:

[0021] S241. Import the VTK format data exported in the above process into the rendering software Blender and add texture features, selecting different textures for different tissues;

[0022] S242. Save the rendered model as an .obj file and export it to provide a model foundation for subsequent augmented reality system development.

[0023] Preferably, in step S3, a computer vision-based tracking and registration method is used to locate and spatially register the pasted markers, and assisted navigation is performed based on the registration results. The specific process is as follows:

[0024] S31. Create a marker to be identified. Take any asymmetrical image, perform image grayscale conversion, binarization and noise reduction processing to make the marker image recognizable and tracked.

[0025] S32. Print out two different markers, Marker1 and Marker2, with one placed around the patient or pasted on the skull, and the other pasted and fixed to one end of the surgical instrument.

[0026] S33. After calibrating the camera used, establish a camera coordinate system; place the virtual image model in a fixed position in the Marker1 coordinate system, represent the patient entity with the Marker1 coordinate system, and establish the actual world coordinate system of the patient entity based on the coordinate space of the stereo positioning device; represent the surgical instruments with the Marker2 coordinate system, and establish a surgical instrument coordinate system based on the positioning markers of the surgical instruments.

[0027] S34. The camera captures the video stream in real time and identifies the markers from it, then performs positioning and tracking to calculate the specific position and orientation of the markers relative to the camera.

[0028] S35. Based on the positions of Marker1 and Marker2, locate the patient entity and the tip of the surgical instrument, establish the spatial transformation relationship between the reference coordinate systems in step S33, and calculate the transformation matrix for registration from the virtual model to the patient entity.

[0029] S36. Capture real surgical scenes with a camera, calculate the transformation relationship between coordinate systems in ARToolKit in real time, obtain the 3D registration matrix, import the virtual cranial model into the video stream, and display the rendered 3D model in the corresponding position of the patient entity.

[0030] S37. By identifying the positions of Marker1 and Marker2, and based on the lesion model position determined in the Marker1 coordinate system, perform spatial coordinate transformation to achieve precise localization of the lesion in the real world; at the same time, calculate the position of the surgical instruments in the real world in real time based on the Marker2 coordinate system.

[0031] Let the actual coordinates of the surgical instrument be T(t1, t2, t3), and the actual coordinates of the center point of the lesion be O(o1, o2, o3). Then the distance from the surgical instrument to the lesion is as follows:

[0032]

[0033] This distance assistance information is displayed on the screen and the distance is updated in real time as the instrument Marker2 moves;

[0034] S38. Based on the position of the access channel designed in the Marker1 coordinate system, after spatial registration and coordinate transformation, it is displayed at the actual patient location. In the actual world coordinate system, let the coordinates of the starting point of the access channel be s(s1, s2, s3) and the coordinates of the ending point of the access channel be e(e1, e2, e3). The direction from the starting point to the ending point is the puncture access channel direction vector, as shown below:

[0035]

[0036] Standardize the direction vector to obtain The unit vector is shown below:

[0037]

[0038] The direction of the unit vector D remains unchanged regardless of the actual distance between the surgical instrument and the target point.

[0039] The vector from the surgical instrument coordinate T to the channel start point s is shown below:

[0040]

[0041] vector In direction vector The projected length on the surface is shown below:

[0042]

[0043] Wherein, the projection length d represents the position of the surgical instrument in the channel direction;

[0044] Let P(p1, p2, p3) be the projection point of the surgical instrument coordinates T onto the surgical path, as shown below:

[0045] P = s + d·D;

[0046]

[0047]

[0048]

[0049] The distance d from the surgical instrument to the access channel Dis As shown below

[0050]

[0051] By distance d Dis To determine whether the surgical instruments are within the access channel, if the distance is d... Dis If the radius is less than or equal to r, the surgical instrument is inside the access channel; otherwise, the surgical instrument is outside the channel.

[0052] To guide the surgeon in moving surgical instruments, the direction vector from the surgical instrument to the nearest point P on the surgical path is calculated, as shown below;

[0053]

[0054] Then normalize the direction vector to obtain Obtain the orientation of the access channel relative to the surgical instruments;

[0055] Based on the calculated direction vector and distance results, the distance d is displayed on the interface. Dis Text and directional arrow indicators provide auxiliary information, and yaw warning sounds are designed to provide real-time updates of auxiliary information to guide doctors in their operations.

[0056] Preferably, in step S4, an AR visualization system platform is built based on the web-based AR.js framework, and the specific steps are as follows:

[0057] S41. The web-based AR.js framework imports the saved rendered 3D model data in obj format into the AR visualization system.

[0058] S42. Implement the 3D scene loading part using Three.js and use mobile devices for augmented reality visualization;

[0059] S43. When the mobile device's camera recognizes the marker, it performs a coordinate transformation using a matrix to register and overlay the virtual model at the corresponding position.

[0060] S44. Observe that the marker points on the virtual model and the actual marker points are completely registered;

[0061] S45. Visualize the planned surgical route model on a mobile device, displaying the distance TO of the surgical instruments from the lesion and the distance d of the surgical instruments from the planned surgical route. Dis With directional arrows, follow the planned route for the surgery.

[0062] Therefore, this invention employs the aforementioned surgical navigation registration method based on marker detection, overcoming the shortcomings of traditional surgical procedures. It solves the problems of requiring multiple large and bulky machines, cumbersome processes, and complex calculations in the surgical navigation process. Utilizing the ARToolkit software development kit, it identifies and tracks markers using a regular monocular camera, which is not only simple and convenient with easy-to-use equipment, but also achieves spatial registration from the virtual model to the patient entity through spatial coordinate transformation. This ensures accuracy while minimizing computational load and providing high real-time performance for the surgical navigation process. Furthermore, it uses the web-based AR.js framework for augmented reality visualization, eliminating the need for users to install additional applications and offering cross-platform compatibility and wide applicability.

[0063] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0064] Figure 1 This is a flowchart of a surgical navigation registration method based on marker detection according to the present invention.

[0065] Figure 2 This is a flowchart of the spatial positioning and registration process of a surgical navigation registration method based on marker detection according to the present invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the embodiments of the present invention and are not intended to limit the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments in this application without creative effort are within the scope of protection of this application.

[0067] This invention proposes a surgical navigation registration method based on marker detection. First, a three-dimensional model is reconstructed from the image data obtained from the patient's cranial scan. Then, based on the ARToolkit framework, a camera is used to identify binary asymmetric rectangular markers placed around the patient or attached to the cranium, thereby locating the patient's position in the world coordinate system and realizing spatial registration technology for virtual-real overlay display. Finally, using coordinate transformation between spatial coordinate systems, the virtual three-dimensional lesion model is registered onto the patient and overlaid on the device for display. Visualization is achieved on mobile devices through the web-based AR.js framework, guiding and assisting doctors in performing surgery.

[0068] Example

[0069] like Figure 1 As shown, a surgical navigation registration method based on marker detection includes the following steps:

[0070] S1. Data is collected from the patient's cranium with the marked points, and medical imaging data is obtained by scanning.

[0071] S2. Preprocess the acquired medical image data, including: image segmentation, 3D reconstruction, access channel design, and 3D model rendering.

[0072] S3. A computer vision-based tracking and registration method is used to locate and spatially register the pasted markers, and to provide assisted navigation based on the registration results.

[0073] S4, a web-based AR.js framework, is used to build an AR visualization platform system.

[0074] To achieve augmented reality surgical navigation, this invention utilizes the ARToolKit augmented reality system software development kit to locate the patient's position in the world coordinate system, realizing spatial registration technology for virtual and real overlay display. To simultaneously locate the lesion and surgical instruments, a method of recognizing multiple markers is adopted, and the spatial distance between the markers is calculated simultaneously, that is, the distance from the tip of the surgical instrument to the lesion is calculated in real time. By referring to the virtual and real registration accuracy of the markers pasted on the surface of the skull, the registration accuracy is obtained, which assists in completing the surgical process.

[0075] S1. Data is collected from the patient's brain by pasting the markers, and medical imaging data is obtained by scanning.

[0076] S11. Select no fewer than three non-collinear physiological feature points on the patient's skull, and attach X-ray-blocking markers to the selected physiological feature points, such as the temples and the tip of the nose.

[0077] S12. Scan the marked cranium to acquire multimodal two-dimensional image data output by medical imaging equipment, and save the scan results in a specified format, such as the DICOM format used by 3DSlicer. The multimodal two-dimensional image data includes CT, MRI, DR, CTA, and PET images.

[0078] S2. Preprocess the acquired medical image data, including: image segmentation, 3D reconstruction, access channel design, and 3D model rendering.

[0079] S21. Image segmentation: Based on 3DSlicer software, medical image data is segmented to extract cranial lesions, skull models, and pasted markers from the original image data.

[0080] The segmentation methods can include automatic segmentation (threshold segmentation, etc.) and manual segmentation (layer domain painting). Automatic segmentation mainly uses methods such as thresholding, level set, and region growing. Manual segmentation mainly involves painting a specific (interested) region within the area to be reconstructed in each layer for segmentation.

[0081] S22. 3D Reconstruction: The segmentation results from step S21 are reconstructed using the 3DSlicer 3D reconstruction method to obtain the corresponding 3D model. The 3D reconstruction method includes volume rendering and surface rendering. The 3D model is a cranial outline with marked points, and lesions are located at their respective positions.

[0082] S23. Approach design, which is to design the optimal surgical route for craniocerebral neurosurgery, as follows:

[0083] S231. When designing the access route on the model, in order to avoid important functional areas of the patient's brain and arteries and veins during the puncture access, the Lines measurement tool of 3DSlicer is used to perform multiple measurements to determine the starting point s and the ending point e of the access route, and thus obtain the channel depth h of the lesion.

[0084] S232. Using the line connecting the puncture start point and the puncture end point as the central axis, use the region growing method to generate an access channel model with a radius of r, obtain the optimal surgical puncture route, and save it as a vtk format for export.

[0085] S24, 3D model rendering, the specific process is as follows:

[0086] S241. Import the previously exported VTK format data into the rendering software Blender and add texture features, selecting different textures for different tissues; for example, design the cranial outline as transparent, and design lesions and markers as different but eye-catching colors;

[0087] S242. Save the rendered model as an .obj file and export it to provide a model foundation for subsequent augmented reality system development.

[0088] S3. A computer vision-based tracking and registration method for locating and spatially registering markers.

[0089] like Figure 2 As shown, a computer vision-based tracking and registration method is used for the localization, tracking, and spatial registration of the markers. The specific process is as follows:

[0090] S31. Create a marker to be identified. Take any asymmetrical image, perform image grayscale conversion, binarization and noise reduction processing to make the marker image recognizable and tracked.

[0091] S32. Print out two different markers, Marker1 and Marker2, with one placed around the patient or pasted on the skull, and the other pasted and fixed to one end of the surgical instrument.

[0092] S33. After calibrating the camera used, establish a camera coordinate system; place the virtual image model in a fixed position in the Marker1 coordinate system, represent the patient entity with the Marker1 coordinate system, and establish the actual world coordinate system of the patient entity based on the coordinate space of the stereo positioning device; represent the surgical instruments with the Marker2 coordinate system, and establish a surgical instrument coordinate system based on the positioning markers of the surgical instruments.

[0093] S34. The camera captures the video stream in real time and identifies the markers from it, then performs positioning and tracking to calculate the specific position and orientation of the markers relative to the camera.

[0094] S35. Based on the positions of Marker1 and Marker2, locate the patient entity and the tip of the surgical instrument, establish the spatial transformation relationship between the reference coordinate systems in step S33, and calculate the transformation matrix for registration from the virtual model to the patient entity.

[0095] S36. Capture real surgical scenes with a camera, calculate the transformation relationship between coordinate systems in ARToolKit in real time, obtain the 3D registration matrix, import the virtual cranial model into the video stream, and display the rendered 3D model in the corresponding position of the patient entity.

[0096] S37. By identifying the positions of Marker1 and Marker2, and based on the lesion model position determined in the Marker1 coordinate system, perform spatial coordinate transformation to achieve precise localization of the lesion in the real world; at the same time, calculate the position of the surgical instruments in the real world in real time based on the Marker2 coordinate system.

[0097] Let the actual coordinates of the surgical instrument be T(t1, t2, t3), and the actual coordinates of the center point of the lesion be O(o1, o2, o3). Then the distance from the surgical instrument to the lesion is as follows:

[0098]

[0099] This distance assistance information is displayed on the screen and the distance is updated in real time as the instrument Marker2 moves;

[0100] S38. Based on the position of the access channel designed in the Marker1 coordinate system, after spatial registration and coordinate transformation, it is displayed at the actual patient location. In the actual world coordinate system, let the coordinates of the starting point of the access channel be s(s1, s2, s3) and the coordinates of the ending point of the access channel be e(e1, e2, e3). The direction from the starting point to the ending point is the puncture access channel direction vector, as shown below:

[0101]

[0102] Standardize the direction vector to obtain The unit vector is shown below:

[0103]

[0104] The direction of the unit vector D remains unchanged regardless of the actual distance between the surgical instrument and the target point.

[0105] The vector from the surgical instrument coordinate T to the channel start point s is shown below:

[0106]

[0107] vector In direction vector The projected length on the surface is shown below:

[0108]

[0109] Wherein, the projection length d represents the position of the surgical instrument in the channel direction;

[0110] Let P(p1, p2, p3) be the projection point of the surgical instrument coordinates T onto the surgical path, as shown below:

[0111] P = s + d·D;

[0112]

[0113]

[0114]

[0115] The distance d from the surgical instrument to the access channel Dis As shown below

[0116]

[0117] By distance d Dis To determine whether the surgical instruments are within the access channel, if the distance is d... Dis If the radius is less than or equal to r, the surgical instrument is inside the access channel; otherwise, the surgical instrument is outside the channel.

[0118] To guide the surgeon in moving surgical instruments, the direction vector from the surgical instrument to the nearest point P on the surgical path is calculated, as shown below;

[0119]

[0120] Then normalize the direction vector to obtain Obtain the orientation of the access channel relative to the surgical instruments;

[0121] Based on the calculated direction vector and distance results, the distance d is displayed on the interface. Dis Text and directional arrow indicators provide auxiliary information, and yaw warning sounds are designed to provide real-time updates of auxiliary information to guide doctors in their operations.

[0122] S4, a web-based AR.js framework, is used to build an AR visualization platform.

[0123] S41, AR augmented reality visualization, is built using the web-based AR.js framework, which imports the rendered 3D model data (in obj format) saved in S3 into the AR visualization system;

[0124] S42. Load 3D scenes using Three.js and visualize them using augmented reality on mobile devices;

[0125] S43. AR visualization can be implemented on devices that support web browsers, and can be viewed on multiple devices simultaneously;

[0126] S44. After the camera recognizes the marker, it will register and overlay the virtual model at the corresponding position through matrix coordinate transformation.

[0127] S45. After the virtual model markers and actual markers are fully registered, the planned surgical route model is visualized on the mobile device, displaying the distance TO of the surgical instruments from the lesion and the distance d of the surgical instruments from the planned surgical route. Dis With directional arrows, follow the planned route for the surgery.

[0128] Therefore, this invention employs the aforementioned surgical navigation registration method based on marker detection, overcoming the shortcomings of traditional surgical procedures. It solves the problems of requiring multiple large and cumbersome machines, tedious processes, and complex calculations in the surgical navigation process. Utilizing the ARToolkit augmented reality software development kit, it identifies and tracks markers using a regular monocular camera. This method is not only simple and convenient with easy-to-use equipment, but also achieves spatial registration from the virtual model to the patient entity through spatial coordinate transformation. While ensuring accuracy, it features low computational load and high real-time performance, thus realizing the surgical navigation process. The AR.js framework, used in a web-based manner, is used for augmented reality visualization, eliminating the need for users to install additional applications and possessing cross-platform compatibility and wide applicability.

[0129] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A surgical navigation registration method based on marker detection, characterized in that, Includes the following steps: S1. Data is collected from the patient's cranium with the marked points, and medical imaging data is obtained by scanning. S2. Preprocess the acquired medical image data, including: image segmentation, 3D reconstruction, access channel design, and 3D model rendering. S3. A computer vision-based tracking and registration method is used to locate and spatially register the pasted markers, and to provide assisted navigation based on the registration results. The specific process is as follows: S31. Create a marker to be identified. Take any asymmetrical image, perform image grayscale conversion, binarization and noise reduction processing to make the marker image recognizable and tracked. S32. Print out two different markers, Marker1 and Marker2, with one placed around the patient or pasted on the skull, and the other pasted and fixed to one end of the surgical instrument. S33. After calibrating the camera used, establish a camera coordinate system; place the virtual image model in a fixed position in the Marker1 coordinate system, represent the patient entity with the Marker1 coordinate system, and establish the actual world coordinate system of the patient entity based on the coordinate space of the stereo positioning device; represent the surgical instruments with the Marker2 coordinate system, and establish a surgical instrument coordinate system based on the positioning markers of the surgical instruments. S34. The camera captures the video stream in real time and identifies the markers from it, then performs positioning and tracking to calculate the specific position and orientation of the markers relative to the camera. S35. Based on the positions of Marker1 and Marker2, locate the patient entity and the tip of the surgical instrument, establish the spatial transformation relationship between the reference coordinate systems in step S33, and calculate the transformation matrix for registration from the virtual model to the patient entity. S36. Capture real surgical scenes with a camera, calculate the transformation relationship between coordinate systems in ARToolKit in real time, obtain the 3D registration matrix, import the virtual cranial model into the video stream, and display the rendered 3D model in the corresponding position of the patient entity. S37. By identifying the positions of Marker1 and Marker2, and based on the lesion model position determined in the Marker1 coordinate system, perform spatial coordinate transformation to achieve precise localization of the lesion in the real world; at the same time, calculate the position of the surgical instruments in the real world in real time based on the Marker2 coordinate system. Let the actual coordinates of the surgical instrument be T(t1, t2, t3), and the actual coordinates of the center point of the lesion be O(o1, o2, o3). Then the distance from the surgical instrument to the lesion is as follows: This distance assistance information is displayed on the screen and the distance is updated in real time as the instrument Marker2 moves; S38. Based on the position of the access channel designed in the Marker1 coordinate system, after spatial registration and coordinate transformation, it is displayed at the actual patient location. In the actual world coordinate system, let the coordinates of the starting point of the access channel be s(s1, s2, s3) and the coordinates of the ending point of the access channel be e(e1, e2, e3). The direction from the starting point to the ending point is the puncture access channel direction vector, as shown below: Standardize the direction vector to obtain The unit vector is shown below: The direction of the unit vector D remains unchanged regardless of the actual distance between the surgical instrument and the target point. The vector from the surgical instrument coordinate T to the channel start point s is shown below: vector In direction vector The projected length on the surface is shown below: Wherein, the projection length d represents the position of the surgical instrument in the channel direction; Let P(p1, p2, p3) be the projection point of the surgical instrument coordinates T onto the surgical path, as shown below: P = s + d·D; The distance d from the surgical instrument to the access channel Dis As shown below By distance d Dis To determine whether the surgical instruments are within the access channel, if the distance is d... Dis If the radius is less than or equal to r, the surgical instrument is inside the access channel; otherwise, the surgical instrument is outside the channel. To guide the surgeon in moving surgical instruments, the direction vector from the surgical instrument to the nearest point P on the surgical path is calculated, as shown below; Then normalize the direction vector to obtain Obtain the orientation of the access channel relative to the surgical instruments; Based on the calculated direction vector and distance results, the distance d is displayed on the interface. Dis Text and directional arrow indicators provide auxiliary information, yaw warning sounds are designed, and auxiliary information is updated in real time to guide doctors' operations. S4. Based on the web-based AR.js framework, build an AR visualization platform system. The specific steps are as follows: S41. The web-based AR.js framework imports the saved rendered 3D model data in obj format into the AR visualization system. S42. Implement the 3D scene loading part using Three.js and use mobile devices for augmented reality visualization; S43. When the mobile device's camera recognizes the marker, it performs a coordinate transformation using a matrix to register and overlay the virtual model at the corresponding position. S44. Observe that the marker points on the virtual model and the actual marker points are completely registered; S45. Visualize the planned surgical route model on a mobile device, displaying the distance TO of the surgical instruments from the lesion and the distance d of the surgical instruments from the planned surgical route. Dis With directional arrows, follow the planned route for the surgery.

2. The surgical navigation registration method based on marker detection according to claim 1, characterized in that, In step S1, the specific steps for acquiring medical image data are as follows: S11. Select no fewer than three non-collinear physiological feature points on the patient's skull and attach X-ray-opaque markers to the selected physiological feature points. S12. Scan the cranium with marked points to acquire multimodal two-dimensional image data output by medical imaging equipment, and save the scan results in the prescribed format; wherein, the multimodal two-dimensional image data includes CT, MRI, DR, CTA, and PET images; 3DSlicer saves them in DICOM format.

3. The surgical navigation registration method based on marker detection according to claim 1, characterized in that, In step S2, the acquired medical image data is preprocessed, and the specific process is as follows: S21. Image segmentation: Based on 3DSlicer software, segment medical image data and extract cranial lesions, skull models, and pasted markers from the original image data. The segmentation methods include automatic segmentation and manual segmentation; automatic segmentation includes thresholding, level set, and region growing methods; manual segmentation involves drawing specific regions within the reconstruction area of ​​each layer for segmentation. S22. 3D Reconstruction: Based on the above segmentation results, 3D reconstruction is performed using the 3DSlicer 3D reconstruction method to obtain the corresponding 3D model; the 3D reconstruction method includes volume rendering and surface rendering; the 3D model is a cranial outline with marked points, and the lesions are located at the corresponding positions; S23. Approach design, which is to design the optimal surgical route for craniocerebral neurosurgery, as follows: S231. When designing the access route on the model, use the Lines measurement tool of 3DSlicer to perform multiple measurements to ensure that important functional areas and arteries and veins of the patient's brain are avoided during the puncture access route, thereby determining the starting point s and the ending point e of the access route, and thus obtaining the channel depth h of the lesion. S232. Using the line connecting the puncture start point and the puncture end point as the central axis, use the region growing method to generate an access channel model with a radius of r, obtain the optimal surgical puncture route, and save it as a vtk format for export. S24, 3D model rendering, the specific process is as follows: S241. Import the VTK format data exported in the above process into the rendering software Blender and add texture features, selecting different textures for different tissues; S242. Save the rendered model as an .obj file and export it to provide a model foundation for subsequent augmented reality system development.

Images