A surgical training and navigation device based on head-mounted three-dimensional augmented reality equipment

Through head-mounted three-dimensional augmented reality display device and binocular camera tracking technology, the problems of perspective switching and depth perception in remote surgical guidance are solved, the accuracy and efficiency of surgical operations are improved, and the synchronous display of virtual and real models is realized. for surgical training and navigation.

CN111728697BActive Publication Date: 2024-11-05UNIV OF SCI & TECH OF CHINA
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010703957.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-21
Publication Date
2024-11-05
Estimated Expiration
2040-07-21

AI Technical Summary

Technical Problem

The existing remote surgical guidance system relies on video conferencing, resulting in doctors having to frequently switch viewing angles and surgical areas, affecting operational accuracy and efficiency, and lack of perception of depth direction.

Method used

Using head-mounted 3D augmented reality display device, combined with a binocular camera to track the three-dimensional coordinates of the scalpel in real time, and transmit data through the network, providing guidance for virtual scalpels, allowing doctors to see real and virtual models in the same field of view, improving surgery Positioning accuracy.

Benefits of technology

It solves the problem of perspective switching, improves the accuracy and efficiency of surgical operations, provides perception of depth direction, facilitates surgical training and navigation, and allows trainers to guide and record surgical paths in real time, improving the utilization rate of medical resources.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111728697B_ABST
    Figure CN111728697B_ABST
Patent Text Reader

Abstract

The present invention proposes a surgical training and navigation device based on a head-mounted three-dimensional augmented reality device, the device comprising: a local tracker module, a remote tracker module, a local binocular vision tracking module, a remote binocular vision tracking module, a local image processing module, a remote image processing module, a three-dimensional model scanning module, a three-dimensional model processing module, a three-dimensional path planning module and an augmented reality display module; the remote tracker module and the local tracker module have the same structure, respectively comprising a bracket and three-color LED lights located on the same plane; the local binocular vision tracking module is used to collect images of the local tracker module and transmit them to the local image processing module in real time; the local image processing module is used to extract the three-dimensional spatial coordinates of the LED lights of the local tracker module according to a color algorithm, and calculate and record the real-time trajectory of the local surgical knife tip according to the fixed spatial position relationship between the local tracker module and the surgical knife tip.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of medical devices, and in particular to a surgical training and surgical and navigation device based on a head-mounted three-dimensional augmented reality display device. Background Art

[0002] The uneven distribution of medical resources has always been a major challenge for the medical system in China and even in the world. The global distribution of medical accessibility and quality of care index (HAQ) has shown a disparity at the local level in many developing countries (such as China and India). The lack of well-trained surgical teams has seriously hindered the popularization of low-cost and high-quality surgical care in low- and middle-income countries. Even in developed countries, it is very challenging to provide efficient medical care quickly in resource-limited areas, especially in the event of disasters or accidents. Considering that human resources are one of the most important elements in the health system, appropriate human resource allocation strategies across regions can provide more effective medical assistance. Telemedicine can provide remote medical services and health care services to patients and ensure the continuity of medical services and the best use of medical resources in remote areas. In addition, telemedicine can provide a real-time, low-cost way to share remote medical resources for doctors who want to improve their surgical skills and patients who want to improve treatment. The main mode of surgical telemedicine is remote guidance, including intraoperative navigation and surgical training. The remote guidance system not only enables medical experts to guide surgical operations when on-site rescuers have little relevant experience, but also enables experienced surgeons to remotely train students in surgical operations in an interactive manner.

[0003] Since the first case of remote guidance of open heart surgery was performed via satellite video conferencing in the 1960s, various surgical remote guidance systems have emerged and have been tested in laparoscopic and open surgery. One of the most common forms of remote guidance is real-time video conferencing. When using a video conferencing system for remote guidance, experts provide guidance through voice. The accuracy of the guidance results mainly depends on the ability of the remote instructor to transmit voice instructions and the local operator to understand the voice instructions. The effect of this guidance method is not very ideal. Another commonly used remote guidance system provides guidance through video. Experts can mark the location on the real-time transmission of the surgical site image so that the local operator can obtain guidance information visually and auditorily. The main limitation of these video-based remote guidance systems is that they need to use a screen to display the surgical image and make annotations. During the remote guidance operation, the local operator must constantly switch the view between the surgical area and the screen to follow the remote annotations, so that the local doctor needs to spend a lot of effort to locate the position of the video guidance display in the surgical area.

[0004] In recent years, augmented reality technology, as a technology that can integrate virtual models into the real world, has been used in remote guidance. Through the display of augmented reality, local doctors can see the remote guidance and the on-site surgical area at the same time in the same field of view, overcoming the limitations of traditional screen displays. One type of these AR remote guidance systems displays the real surgical area information and virtual models on the screen together, so local doctors only need to observe the screen when operating. Another type of AR guidance system uses wearable augmented reality devices to display two-dimensional virtual models under the doctor's real visual perception. However, these two-dimensional display screens cannot provide doctors with depth perception.

[0005] Head-mounted 3D augmented reality display devices can display 3D virtual models in the form of holograms and integrate them into the real world, and have great potential in the field of surgical guidance and training. Head-mounted 3D augmented reality display devices currently on the market have longer battery life and brighter image display. The 3D remote communication function makes head-mounted 3D augmented reality display devices a suitable tool in many medical remote guidance scenarios, such as AR-guided surgery, anatomy and operation training. In surgical remote guidance, it is necessary to record the position of the expert's scalpel, and stereoscopic vision is often used for scalpel position tracking, especially in image-guided neurosurgery. Summary of the invention

[0006] In order to solve the above technical problems, the present invention provides a novel remote surgical guidance and surgical training device that integrates three-dimensional coordinate tracking and head-mounted three-dimensional augmented reality display devices. The three-dimensional coordinates of the scalpel can be tracked in real time using a binocular camera and the data can be transmitted to the head-mounted three-dimensional augmented reality display device in real time. This augmented reality device can be used for remote surgical guidance and surgical training. During surgical training, the virtual model is aligned and fused with the real model. The position of the virtual scalpel can be tracked and recorded in advance using a binocular vision system or tracked in real time at the remote end and transmitted to the local end. The head-mounted three-dimensional augmented reality display device receives the scalpel position information and uses it to guide the virtual scalpel, so that the trainees at the local end can operate according to the guidance of the virtual scalpel. When performing a remotely guided operation, first scan the three-dimensional model of the surgical site and transmit it to the remote end. The doctor at the remote end plans the scalpel path on the virtual model and transmits it to the head-mounted three-dimensional augmented reality display device to guide the virtual scalpel. The doctor at the local end can operate under the guidance of the virtual scalpel. The device can track the position of the local scalpel in real time and feed it back to the operator to ensure the accuracy of the operation.

[0007] The technical solution of the present invention is: a surgical training and navigation device based on a head-mounted three-dimensional augmented reality device, comprising:

[0008] Local tracker module, remote tracker module, local binocular vision tracking module, remote binocular vision tracking module, local image processing module, remote image processing module, 3D model scanning module, 3D model processing module, 3D path planning module and augmented reality display module;

[0009] The remote tracker module and the local tracker module have the same structure, and each includes a bracket and three-color LED lights located on the same plane. The remote tracker module and the local tracker module are respectively fixedly mounted on the scalpels at two locations;

[0010] The local binocular vision tracking module includes a binocular camera, which is used to collect images of the local tracker module and transmit them to the local image processing module in real time. The local image processing module is used to extract the three-dimensional spatial coordinates of the LED light of the local tracker module according to the color algorithm, and calculate and record the real-time trajectory of the local surgical knife tip according to the fixed spatial position relationship between the local tracker module and the surgical knife tip;

[0011] The three-dimensional model scanning module is connected to the three-dimensional model processing module and the augmented reality display module; the three-dimensional path planning module is located at the remote end.

[0012] Furthermore, the augmented reality display module is a developable three-dimensional head-mounted augmented reality display device that displays the virtual model in the form of a hologram.

[0013] Furthermore, the three-dimensional model scanning module is a three-dimensional scanner;

[0014] Furthermore, the remote tracker module includes a bracket, a first LED light, a second LED light, and a third LED light. The three LED lights are located on the same plane, and the three LED lights are of different colors. The remote tracker module and the local tracker module are respectively fixedly mounted on scalpels at two locations. The bracket extends three poles, each of which is equipped with an LED light, and the angles between every two poles are the same.

[0015] Furthermore, the three-dimensional model processing module and the three-dimensional path planning module are located on the aforementioned local and remote computers respectively.

[0016] According to another aspect of the present invention, a surgical training method based on a head-mounted three-dimensional augmented reality device is provided, comprising the following steps:

[0017] Step 1: The trainer conducts real-time teaching at the remote end. The same teaching phantom model is used locally and remotely. The virtual model and the real world are integrated in advance in the local head-mounted 3D augmented reality display device.

[0018] Step 2: After the local and remote binocular cameras are calibrated with internal and external parameters, the remote binocular camera takes a picture of the trainer's scalpel in real time and transmits it to the remote image processing module;

[0019] Step 3, the remote image processing module identifies the luminous point on the image, takes n frames of images to calculate the average value, then calculates the coordinates of the scalpel tip according to the spatial structure of the scalpel, and transmits the calculated scalpel tip coordinate data to the local trainee's three-dimensional augmented reality display device through the network;

[0020] Step 4: After the local head-mounted 3D augmented reality display device successfully receives the coordinate information of the scalpel tip sent remotely, it moves the position of the virtual scalpel according to the received coordinate information and draws a moving trajectory;

[0021] Step 5: The trainee at the local end sees the movement and movement trajectory of the virtual scalpel through the head-mounted 3D augmented reality display device, and locates the scalpel according to the position of the virtual scalpel. During the operation, the scalpel in the hand follows the trajectory of the virtual scalpel;

[0022] Step 6: The local image processing module tracks the scalpel in the trainee's hand in the same way, calculates the coordinates of the tip of the real scalpel, and transmits the calculated coordinate data to the three-dimensional augmented reality display device worn by the local trainee through the network;

[0023] Step 7. The head-mounted 3D augmented reality display device receives the position of the local trainee's scalpel, draws the trajectory of the local scalpel, and compares and calculates the positions of the trainer's and trainee's scalpels. When the deviation exceeds a threshold, an image and voice prompt are given. The trainee uses this as feedback to continuously adjust and improve positioning accuracy.

[0024] Furthermore, during surgical training, the remote trainer records the teaching model in advance, the trajectory of the remote trainer's scalpel is tracked and recorded, and local students use the pre-recorded model to train under the guidance of the virtual model of the head-mounted 3D augmented reality display device.

[0025] Furthermore, the head-mounted three-dimensional augmented reality display device can share the image, so that the remote instructor can see the operator's field of view and provide voice guidance based on the image transmitted back.

[0026] The present invention also proposes a surgical navigation method based on a head-mounted three-dimensional augmented reality device, comprising the following steps:

[0027] Step 1: First, the local end scans the 3D model of the surgical scene and performs processing optimization, and transmits it to the remote doctor through the network. The remote doctor plans the scalpel incision trajectory through 3D software, and transmits the 3D model with the planned scalpel path back to the local head-mounted 3D augmented reality display device. The local operator matches the returned virtual 3D model with the real world;

[0028] Step 2: The local operator sees the movement and trajectory of the virtual scalpel through a head-mounted 3D augmented reality display device, and locates the scalpel according to the position of the virtual scalpel. During the operation, the scalpel in the hand follows the trajectory of the virtual scalpel.

[0029] Step 3: The local image processing module tracks the scalpel in the operator's hand in the same way, calculates the coordinates of the tip of the real scalpel, and transmits the calculated coordinate data to the 3D augmented reality display device worn by the local operator through the network;

[0030] Step 4: The head-mounted 3D augmented reality display device receives the position of the local operator's scalpel, draws the trajectory of the local scalpel, and compares and calculates the positions of the trainer's and the operator's scalpels. When the deviation exceeds the threshold, an image and voice prompt are given. The operator uses this as feedback to continuously adjust and improve positioning accuracy.

[0031] The device of the present invention includes a local tracker module, a remote tracker module, a local binocular vision tracking module, a remote binocular vision tracking module, a local image processing module, a remote image processing module, a three-dimensional model scanning module, a three-dimensional model processing module, a three-dimensional path planning module and an augmented reality display module. When performing surgical training, the local trainee end and the remote trainer end use the same training phantom model, and the remote and local tracker modules are fixedly installed on the scalpels in two places, and are composed of three-color LED lights and their brackets distributed on the same plane. The binocular vision tracking module is two cameras, which can collect images of the tracker module and transmit them to the image processing module in real time. The image processing module uses a color algorithm to extract the three-dimensional spatial coordinates of the LED lights of the tracker module, and calculates and records the real-time trajectory of the surgical tip based on the fixed spatial position relationship between the tracker module and the surgical tip. The binocular vision tracking module at the remote end captures the image of the tracker in real time and transmits it to the remote image processing module for position calculation. The calculated position of the trainer's scalpel is transmitted to the head-mounted three-dimensional augmented reality display device in the augmented reality display module at the local end to guide the trainee to see the position of the virtual scalpel. The position of the local trainee's scalpel calculated by the binocular vision tracking module at the local end is transmitted to the local image processing module, the local scalpel position is calculated after image processing, and then the position information is transmitted to the head-mounted three-dimensional augmented reality display device for comparison with the position of the trainer's scalpel to assist the trainee in judging whether the operation is accurate. When performing surgical guidance, the three-dimensional model scanning module, i.e., the three-dimensional scanner, scans the three-dimensional model of the local surgical scene, imports the model into the three-dimensional model processing module for three-dimensional model optimization and cutting processing, and transmits the processed three-dimensional model to the remote end. The remote doctor plans the scalpel incision path through the three-dimensional path planning module, i.e., the three-dimensional software analysis and editing software, and transmits it back to the head-mounted three-dimensional augmented reality display device in the local augmented reality display module. The virtual scalpel in the head-mounted three-dimensional augmented reality display device will move according to the planned scalpel incision path to guide the operation of the local doctor. Similarly, the local binocular vision tracking module will transmit the calculated position of the trainee's scalpel to the local image processing module to calculate the position information of the local scalpel, and transmit it to the head-mounted three-dimensional augmented reality display device for comparison with the position of the scalpel planned by the remote doctor, to assist the local doctor in determining whether the operation is accurate.

[0032] Beneficial effects:

[0033] 1. The device of the present invention adopts an augmented reality display device, and the user can see the real world and the three-dimensional model integrated into the real world in his field of vision at the same time, which solves the problem in the prior art that doctors need to repeatedly switch the viewing angle between the display and the operating part, and helps to optimize the surgical path and improve the success rate of the operation.

[0034] 2. The augmented display device of the apparatus of the present invention displays the virtual model in the form of a hologram. Compared with the traditional planar display augmented reality display method, the three-dimensional holographic display can provide the user with depth perception, which is of great significance for improving surgical positioning accuracy and reducing surgical injuries.

[0035] 3. When the device of the present invention is used for surgical training, the trainer can not only provide real-time guidance and transmit the scalpel path, but also record and save the scalpel path in advance. In this way, the trainer's guidance will not be restricted by time and place. The trainees can be trained under the guidance of the scalpel path recorded in advance. At the same time, one trainer can guide multiple trainees at the same time, greatly improving the utilization rate of medical resources.

[0036] 4. The device of the present invention has two sets of binocular vision tracking systems. The surgical knife path of the local instructor will also be tracked simultaneously and fed back to the display module for comparison with the virtual path used for guidance. The operator can observe his own operation deviation in real time. Such a feedback mechanism can supervise and remind the operator, thereby improving the overall positioning accuracy.

[0037] 5. The invention adopts a head-mounted augmented reality display device, which is small in size, has long battery life and is easy to operate. The display device is not limited by the operating room space, the doctor's intraoperative operations and the patient's body parts. The virtual model moves in real time with the user's perspective and is aligned with the real world, which greatly improves the scope and flexibility of application and is easier to promote and use. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 is a schematic diagram of a remote tracker module and a local tracker module of the present invention;

[0039] Figure 2 is a schematic diagram of the device of the present invention used for surgical training;

[0040] Figure 3 is a schematic diagram of the device of the present invention used for remote surgical navigation;

[0041] Figure 4 It is a flow chart of the method of the present invention.

[0042] In the figure: 1 is a local tracker module, 2 is a remote tracker module, 3 is a local binocular vision tracking module, 4 is a remote binocular vision tracking module, 5 is a local image processing module, 6 is a remote image processing module, 7 is an augmented reality display module; 8 is a three-dimensional model scanning module, 9 is a three-dimensional model processing module, 10 is a three-dimensional path planning module, 11 is a bracket, 12 is a first LED light, 13 is a second LED light, and 14 is a third LED light. DETAILED DESCRIPTION

[0043] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of the present invention.

[0044] See also Figure 2 and Figure 3 As shown, the present invention provides a surgical training and surgical navigation device based on a head-mounted three-dimensional augmented reality display device. The device includes a local tracker module 1, a remote tracker module 2, a local binocular vision tracking module 3, a remote binocular vision tracking module 4, a local image processing module 5, a remote image processing module 6, an augmented reality display module 7, a three-dimensional model scanning module 8, a three-dimensional model processing module 9, and a three-dimensional path planning module 10.

[0045] The local binocular vision tracking module 3 is connected to the local image processing module 5, the remote binocular vision tracking module 4 is connected to the remote image processing module 6, and the three-dimensional model scanning module 8 is connected to the three-dimensional model processing module 9. The remote tracker module 2 has the same structure as the local tracker module 1. Figure 1 As shown, the remote tracker module 2 includes a bracket 11, a first LED light 12, a second LED light 13, and a third LED light 14. The three LED lights are located on the same plane, and the three LED lights are of different colors. The remote tracker module 2 and the local tracker module 1 are fixedly mounted on the scalpels at two locations. The bracket 11 extends three poles, each pole is equipped with an LED light, and the angles between every two poles are the same.

[0046] In the present invention, the local binocular vision tracking module 3 and the remote binocular vision tracking module 4 respectively include two color cameras, which respectively collect color pictures of the local tracker module 1 and the remote tracker module 2, and transmit the collected image data to the local image processing module 5 and the remote image processing module 6 respectively. The local image processing module 5 is a local computer, the remote image processing module 6 corresponds to a remote computer, the augmented reality display module 7 is a developable three-dimensional head-mounted augmented reality display device, the three-dimensional model scanning module 8 is a three-dimensional scanner, the three-dimensional model processing module 9, and the three-dimensional path planning module 10 are the aforementioned local and remote computers respectively.

[0047] One of the key applications of the above device is surgical training, which is a key step for medical students to obtain surgical physician certification and is of great help in improving the level of operators. Surgical operations require good spatial positioning skills. The use of the above device allows students to see the positioning information taught by the trainer in real time and intuitively throughout the whole process. It is easy to operate and is not limited by time and space.

[0048] The surgical training using the above device is divided into the following 7 steps: Figure 4 As shown:

[0049] 1. The trainer conducts real-time teaching at the remote end. The same teaching phantom model is used locally and remotely. The virtual model and the real world are integrated in advance in the local head-mounted 3D augmented reality display device.

[0050] 2. After the local and remote binocular cameras are calibrated with internal and external parameters, the remote binocular camera takes a picture of the trainer's scalpel in real time and transmits it to the remote image processing module;

[0051] 3. The remote image processing module identifies the luminous points on the image. To ensure the accuracy and stability of the identification, it takes n frames of image calculation values, and then calculates the coordinates of the scalpel tip according to the spatial structure design of the scalpel. The calculated coordinate data of the scalpel is transmitted to the local trainee's 3D augmented reality display device through the network;

[0052] 4. After the local head-mounted 3D augmented reality display device successfully receives the coordinate information of the scalpel sent remotely, it moves the position of the virtual scalpel according to the received coordinate information and draws the moving trajectory;

[0053] 5. The trainee at the local end sees the movement and trajectory of the virtual scalpel through the head-mounted 3D augmented reality display device, and locates according to the position of the virtual scalpel. During the operation, the scalpel in his hand follows the trajectory of the virtual scalpel;

[0054] 6. The local image processing module tracks the scalpel in the trainee's hand in the same way, calculates the coordinates of the tip of the real scalpel, and transmits the calculated coordinate data to the 3D augmented reality display device worn by the local trainee through the network;

[0055] 7. The head-mounted 3D augmented reality display device receives the position of the local trainee's scalpel, draws the trajectory of the local scalpel, and compares and calculates the positions of the trainer's and trainee's scalpels. When the deviation exceeds the threshold, image and voice prompts are given. The trainees use this as feedback to continuously adjust and improve positioning accuracy.

[0056] When performing remote surgical navigation, the three-dimensional model of the surgical scene is first scanned and optimized, and then transmitted to the remote doctor through the network. The remote doctor plans the scalpel incision trajectory through three-dimensional software, and transmits the three-dimensional model with the planned scalpel path back to the local head-mounted three-dimensional augmented reality display device. The local operator matches the returned virtual three-dimensional model with the real world and uses the virtual planned scalpel trajectory to guide the movement of the virtual scalpel. The device is then used in the same way as surgical training. The local operator operates under the guidance of the virtual scalpel and receives real-time position feedback, making continuous adjustments to improve positioning accuracy.

[0057] In some implementation processes, during surgical training, remote trainers can record teaching models in advance, the trajectory of the remote teacher's scalpel can be tracked and recorded, and local students can use the pre-recorded models multiple times at any time to train under the guidance of the virtual model of the head-mounted 3D augmented reality display device.

[0058] In some implementations, a head-mounted 3D augmented reality display device can share images, and a remote instructor can see the operator's field of view and provide voice guidance based on the images transmitted back.

[0059] Although the above describes the illustrative specific embodiments of the present invention to facilitate those skilled in the art to understand the present invention, and it should be clear that the present invention is not limited to the scope of the specific embodiments, for those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations using the concept of the present invention are protected.

Claims

1. A surgical training and navigation device based on a head-mounted three-dimensional augmented reality device, characterized in that: include: Local tracker module, remote tracker module, local binocular vision tracking module, remote binocular vision tracking module, local image processing module, remote image processing module, 3D model scanning module, 3D model processing module, 3D path planning module and augmented reality display module; The remote tracker module and the local tracker module have the same structure, respectively including a bracket and three colors of LED lights located on the same plane, and the remote tracker module and the local tracker module are respectively fixedly installed on the scalpels at two locations; the remote tracker module includes a bracket, a first LED light, a second LED light, and a third LED light, and the three LED lights are respectively located on the same plane, and the three LED lights are respectively of different colors, and the remote tracker module and the local tracker module are respectively fixedly installed on the scalpels at two locations, and the bracket extends three support rods, each of which is equipped with an LED light, and the angles between every two support rods are the same; the local binocular vision tracking module includes a binocular camera for collecting images of the local tracker module and transmitting them to the local image processing module in real time, and the local image processing module is used to extract the three-dimensional spatial coordinates of the LED lights of the local tracker module according to the color algorithm, and calculate and record the real-time trajectory of the local surgical tip according to the fixed spatial position relationship between the local tracker module and the surgical tip; The three-dimensional model scanning module is connected to the three-dimensional model processing module and the augmented reality display module; the three-dimensional path planning module is located at the remote end, and specifically includes the following operation process: Step 1: The trainer conducts real-time teaching at the remote end. The same teaching phantom model is used locally and remotely. The virtual model and the real world are integrated in advance in the local head-mounted 3D augmented reality display device. Step 2: After the local and remote binocular cameras are calibrated with internal and external parameters, the remote binocular camera takes a picture of the trainer's scalpel in real time and transmits it to the remote image processing module; Step 3, the remote image processing module identifies the luminous point on the image, takes n frames of images to calculate the average value, then calculates the coordinates of the scalpel tip according to the spatial structure of the scalpel, and transmits the calculated scalpel tip coordinate data to the local trainee's three-dimensional augmented reality display device through the network; Step 4: After the local head-mounted 3D augmented reality display device successfully receives the coordinate information of the scalpel tip sent remotely, it moves the position of the virtual scalpel according to the received coordinate information and draws a moving trajectory; Step 5: The trainee at the local end sees the movement and movement trajectory of the virtual scalpel through the head-mounted 3D augmented reality display device, and locates the scalpel according to the position of the virtual scalpel. During the operation, the scalpel in the hand follows the trajectory of the virtual scalpel; Step 6: The local image processing module tracks the scalpel in the trainee's hand in the same way, calculates the coordinates of the tip of the real scalpel, and transmits the calculated coordinate data to the three-dimensional augmented reality display device worn by the local trainee through the network; Step 7. The head-mounted 3D augmented reality display device receives the position of the local trainee's scalpel, draws the trajectory of the local scalpel, and compares and calculates the positions of the trainer's and trainee's scalpels. When the deviation exceeds a threshold, an image and voice prompt are given. The trainee uses this as feedback to continuously adjust and improve positioning accuracy.

2. A surgical training and navigation device based on a head-mounted three-dimensional augmented reality device according to claim 1, characterized in that: The augmented reality display module is a developable three-dimensional head-mounted augmented reality display device that displays a virtual model in the form of a hologram, and the three-dimensional model scanning module is a three-dimensional scanner.

3. The surgical training and navigation device based on a head-mounted three-dimensional augmented reality device according to claim 1, characterized in that: The three-dimensional model processing module and the three-dimensional path planning module are located on the aforementioned local and remote computers respectively.

4. A surgical training method based on a head-mounted three-dimensional augmented reality device according to claim 3, characterized in that: During surgical training, the remote trainer records the teaching model in advance, and the trajectory of the remote trainer's scalpel is tracked and recorded. Local students use the pre-recorded model to train under the guidance of the virtual model of the head-mounted 3D augmented reality display device.

5. The surgical training method based on a head-mounted three-dimensional augmented reality device according to claim 3, characterized in that: The head-mounted 3D augmented reality display device can share images, so the remote instructor can see the operator's field of view and provide voice guidance based on the images transmitted back.