Replaceable Tag device of auxiliary augmented reality head-mounted surgical navigation system

Abstract

The invention discloses a replaceable Tag device of an auxiliary augmented reality head-mounted surgical navigation system, and relates to the technical field of medical instruments. The device comprises a plate body at the upper half part and a cylinder body at the lower half part, the plate body is a carrier of an Apriltag visual identification code, and a tenon tongue is arranged on one side of the plate body; the cylinder body is used for providing holding and operation; a slot matched with the tenon tongue in an inserting manner is formed in one side of the cylinder body; a cuboid connecting piece is arranged on the other side of the cylinder body; when in use, the plate body printed with the visual identification code is inserted into a groove of a spine model or a simulated surgical instrument for connection. The method is suitable for nail placement simulation of an augmented reality head-mounted spinal surgical navigation system based on visual servo, meets the requirements of three-dimensional space registration and real-time optical tracking in surgical navigation application, facilitates efficient and convenient operation of a doctor in an operation, shortens the operation time, accelerates postoperative rehabilitation of a surgical patient, and improves the operation efficiency. And the operation safety of the pedicle screw placement surgery is ensured.

Description

Technical Field

[0001] The present invention belongs to the technical field of medical devices, and in particular relates to a replaceable Tag device for an assisted augmented reality head-mounted surgical navigation system. Background Art

[0002] The spine is the pillar of life for the human body and is closely related to normal human activities. Spinal diseases, such as slipped instability, spinal dislocation, lumbar fracture, scoliosis, etc., will hinder human movement and affect the patient's life safety. For such spinal problems, the fixation effect of pedicle screws is significant and provides strong technical support for restoring the normal physiological curvature of the spine. Therefore, pedicle screw placement surgery is considered to reconstruct and locally stabilize the lumbar sequence.

[0003] Pedicle screw placement in spinal surgery usually relies solely on the doctor's tactile feedback and anatomical markings to determine the screw entry point. Due to the lack of visualization of screw placement and the complexity of the screw placement angle, there is a certain error rate in manual screw placement. Therefore, a computer-assisted augmented reality surgical navigation system is introduced as a technical means to guide screw placement. Taking into account the existence of problems such as tracking line of sight occlusion and the smooth progress of normal clinical workflow, and supplemented by the consideration of conforming to human visual behavior habits and saving economic costs, a head-mounted device that simplifies the large-volume floor-standing surgical navigation system came into being. The head-mounted device has a built-in camera, and the spine model and surgical instruments are all attached with a square plate marked with the Apriltag visual identification code. Based on the visual servo algorithm, the camera is called to scan the identification code to achieve inside-out tracking, which can intuitively provide doctors operating the surgery with a three-dimensional spine image and screw guidance path with a free follow perspective.

[0004] In the existing augmented reality surgical navigation system, the main structure is a multifunctional optical tracker and navigation markers that can be attached to various surgical instruments (such as probes, drills, micro-debriders, etc.). The optical tracker usually uses a camera fixed on the platform to capture the optical navigation markers on the doctor's handheld instrument for real-time spatial position analysis and calculation, and obtain the real-time relative position between the marked instrument and the patient's spine in the same coordinate system, and then realize the tracking and navigation of the nail placement path. The device can achieve omnidirectional motion tracking, and the instrument path can be visualized in real time on the host screen, and the planning and navigation path can be presented to the doctor. Other advantages of the technology include sub-millimeter accuracy, stable performance, and reliable measurement quality. The above-mentioned existing technologies still have limitations: (1) surgical instruments with markers need to be used in conjunction with observation instruments that occupy a large area, which may hinder clinical workflow; (2) integrating large-scale machines into head-mounted devices greatly limits the use of some original functions, and instruments equipped with passive marker balls cannot project images onto head-mounted devices. Therefore, in response to the above problems, a replaceable Tag device for assisting augmented reality head-mounted surgical navigation system is provided. Summary of the invention

[0005] The present invention provides a replaceable Tag device for an assisted augmented reality head-mounted surgical navigation system, which has the function of easy disassembly and replacement, can be used in conjunction with the head-mounted surgical navigation system, and meets the requirements of various surgical instruments and visual path guidance for pedicle screw insertion surgery; in summary, the problems in the background technology are solved.

[0006] To solve the above technical problems, the present invention is achieved through the following technical solutions:

[0007] The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system of the present invention is used to replace surgical instruments of the assisted augmented reality head-mounted surgical navigation system in conjunction with the visual identification code, and the replaceable Tag device includes an upper plate body and a lower cylinder body;

[0008] The plate body is a carrier of the Apriltag visual logo code, which is used as the identification code part scanned by the optical perspective augmented reality display device and scanned and identified by the built-in camera of the head-mounted surgical navigation device; a tongue is provided on one side of the plate body;

[0009] The cylinder is a hand grip area for holding and operating; a slot for plugging with the tenon is provided on one side of the cylinder; a rectangular connecting piece is provided on the other side of the cylinder, the rectangular connecting piece constitutes a probe area, and a connecting groove for inserting a simulation screw with an opening facing outward is provided in the rectangular connecting piece, which is used to connect the simulation screw; when in use, the plate body printed with the visual identification code is inserted into the groove of the spinal model or the simulation surgical instrument for connection;

[0010] The plate body, cylinder body, cuboid connector and simulated screws are all made of PET or PP materials and can be made by 3D printing.

[0011] Furthermore, the tenon and the slot are detachable mortise and tenon structures, and the connection is tightly fitted.

[0012] Furthermore, the tenon and the slot are in a cross shape.

[0013] Furthermore, a visual identification code is provided on the plate body connected to the tenon, and after being scanned by an optical perspective augmented reality display device, a virtual spine model and a visualized guide path will be displayed.

[0014] Furthermore, the simulated screw is a cylinder, and the radius of the simulated screw is 3 mm and the height is 30 mm.

[0015] Furthermore, the length and width of the rectangular connecting piece are both 4 mm, and the height is 50 mm; the slot is a 20×20 mm cross-shaped slot.

[0016] Furthermore, the radius of the cylinder is 15 mm and the height is 100 mm.

[0017] Furthermore, the plate body is a square plate with a length, width and height of 120×120×5 mm respectively.

[0018] Compared with the prior art, the present invention has the following beneficial effects:

[0019] (1) The present invention adopts a detachable structure consisting of a plate body, a cylinder body, a rectangular parallelepiped connector and a simulated screw, which can conveniently disassemble and replace surgical instruments. When used in conjunction with a head-mounted surgical navigation system, it meets the requirements of various surgical instruments and visual path guidance for pedicle screw placement surgery;

[0020] (2) The tag device of the present invention enables the head-mounted device to call the built-in camera to scan and determine the position of the camera relative to the Apriltag code after capturing the visual recognition marker Apriltag code, accurately render the spatial position of the model in the doctor's perspective, and generate a three-dimensional spinal image from the first perspective in real time to complete spatial tracking, allowing the doctor to design the optimal pedicle screw placement path guidance based on the patient's actual anatomical structure, thereby improving the accuracy and safety of screw placement;

[0021] (3) The present invention is applicable to the simulation of screw placement in an augmented reality head-mounted spinal surgical navigation system based on visual servoing, which meets the requirements of three-dimensional spatial registration and real-time optical tracking in surgical navigation applications, facilitates doctors to operate efficiently and conveniently during surgery, shortens the operation time, accelerates the postoperative recovery of surgical patients, ensures the safety of pedicle screw placement surgery, and helps surgeons to further ensure medical safety;

[0022] (4) The present invention can also be used for clinical surgical training, and the present invention also complies with the requirements for replacement and use of different surgical instrument models, thereby saving experimental costs;

[0023] (5) The device mainly consists of two parts: a fixed part including a simulated screw, a probe area, and a hand-grip area, and a detachable part printed with an Apriltag identification code, which is used to replace a variety of surgical instruments in the future; when the instrument needs to be replaced, it can be replaced by removing the square plate, which greatly simplifies the use process and saves steps;

[0024] (6) The Tag device of the present invention is constructed using readily available materials, which saves costs and reduces energy consumption.

[0025] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0027] Figure 1 It is a schematic diagram of the structure of a replaceable Tag device of the augmented reality head-mounted surgical navigation system of the present invention;

[0028] In the accompanying drawings, the components represented by the reference numerals are listed as follows:

[0029] 1-simulated screw, 2-rectangular connecting piece, 3-cylinder, 31-slot, 4-plate, 41-tenon. DETAILED DESCRIPTION

[0030] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] In the description of the present invention, it should be understood that the terms "front end", "lower end", "groove" and the like indicating orientation or positional relationship are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the components or elements referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0032] See also Figure 1 As shown, the replaceable Tag device of the auxiliary augmented reality head-mounted surgical navigation system of the present invention is mainly used in the computer-assisted navigation system of spinal pedicle screw placement surgery, providing surgical path guidance for screw placement, which helps to improve the accuracy and safety of this surgery. The application scenario can also be expanded to other orthopedic and laparoscopic surgeries with insufficient visualization, and precise navigation can be achieved through three-dimensional reconstruction. The principle is to merge virtual images with real scenes based on augmented reality technology to avoid the doctor's line of sight switching back and forth between the surgical site and the flat-panel display, and to avoid visual occlusion and attention diversion problems. At the same time, optical tracking technology is also used. The optical perspective augmented reality head-mounted display device uses a translucent optical combiner to synthesize virtual information and real scene light sources through optical superposition and display within the doctor's field of view.

[0033] Due to its detachability, the device can meet the needs of using a variety of instruments in pedicle screw insertion surgery, and it is also convenient to replace different surgical instrument models in subsequent simulated surgeries, saving costs.

[0034] Figure 1 In the figure, the simulated screw 1 is a cylinder with a radius of 3 mm and a height of 30 mm, which is used for the simulation of pedicle screw placement, that is, to simulate the pedicle screw used in actual surgery. In surgical training, the simulated screw can be used as a teaching tool, allowing doctors to become familiar with the screw placement process and operation skills in a virtual environment, and plan the screw placement path during surgery to improve the accuracy and safety of the surgery. 3D printing technology is suitable for the manufacture of simulated medical devices due to its rapid prototyping and customized demand design. Therefore, the simulated screw can be manufactured using 3D printing technology to ensure its precise size and shape. Attention should be paid to selecting suitable biocompatible materials for printing to ensure the safe use of the simulated screw in a simulated surgical environment.

[0035] Figure 1 In the figure, the rectangular connector 2 is a rectangular parallelepiped with a length and width of 4 mm and a height of 50 mm, which constitutes the probe area and connects the simulated screw to the hand-holding area. The probe is moved, and the Apriltag visual mark code at the top is identified by the surgical navigation system. In the augmented reality scene, when the simulated screw at the front end of the probe coincides with the virtual mark point, the surgical navigation system will test and record its specific position, calculate and match the spatial position of the simulated screw and the patient's spinal model to verify the positioning accuracy of the simulated screw and the accuracy of the navigation system. In addition, the head-mounted surgical navigation system will display the nail placement path on the device based on augmented reality technology. The surgical navigation can be completed by simply moving the probe along the designed path. The probe area is manufactured using 3D printing technology to ensure its precise size and shape.

[0036] Figure 1 In the figure, the barrel 3 is a cylindrical body with a groove of 15mm radius and 100mm height. The yellow part is a cross-shaped groove of 20×20mm in size, and the countersunk depth is 30mm, which constitutes a hand-grip area for the operator to hold the simulation instrument, providing a better grip and operation convenience. The ergonomic design enables the operator to have more stable control and more precise positioning of the instrument during surgery. The design of the cross-shaped groove is matched with the cross-shaped protrusion at the lower end of the plate body 4 to achieve the function of detachability and reusability. The hand-grip area is manufactured by 3D printing technology to ensure its precise size and shape.

[0037] Figure 1In the figure, the plate body 4 is a 120×120×5mm square plate, which constitutes the part for the device to scan the identification code. As the carrier of the Apriltag visual logo code, it is used for scanning and identification by the built-in camera of the head-mounted surgical navigation device, which is the key to achieving real-time spatial positioning and tracking. Use SOLIDWORKS software to design the Apriltag visual logo code accessories in advance. After 3D printing the square plate, the paper-printed Apriltag code can be directly pasted to the surface of the square plate or the light-curing technology can be integrated into 3D printing to achieve the fixed connection between the two. Just scan the Apriltag code fixed on the square plate, and by establishing a coordinate conversion relationship, convert the data in the optical tracker coordinate system into the coordinate system required by the surgical navigation system, and then calculate the position and posture of the virtual model and the real physical model, providing intuitive and accurate surgical guidance.

[0038] The technical solution adopted is: after collecting image samples of the lesion in the anteroposterior and lateral positions, it is necessary to install a tracer at the appropriate position of the surgical instrument to determine the orientation, collect certain three-dimensional image data, and transmit it to the surgical navigation system by the shooting device, and select the corresponding vertebral body and screw specifications according to the patient's specific spinal condition and treatment needs, and the doctor will design the screw placement point and screw insertion angle and plan the complete surgical path. The traditional tracing method uses an optical tracker to capture the marker ball pre-fixed on the surgical instrument to measure the spatial position of the surgical instrument, and calculates the posture and displays it on the screen to obtain the path for the doctor's reference. Now use an alternative method to choose a more portable head-mounted device based on optical perspective. Apriltag code is a two-dimensional code system composed of black and white grids and used for computer vision and robot navigation. It can replace the passive marker ball as a visual recognition marker to perform three-dimensional spatial registration of the virtual model. The head-mounted device based on optical perspective has a built-in camera. After scanning and reading the specific pattern, it calculates the distribution of black and white grids to identify the number of the Apriltag code, analyzes the posture state through the size and angle of the Apriltag code, and the navigation system calculates and matches the spatial position of the virtual model and the real physical model in real time, and realizes spatial tracking through the visual servo ViSP algorithm.

[0039] A variety of medical instruments are required in the same pedicle screw placement surgery. When the present invention is applied, it is not realistic to redesign and print the visual identification code attachment using SOLIDWORKS software every time the instrument is replaced. Based on the need for repeatability, the present invention designs a device for detachable and replaceable visual identification codes, which allows the user to simply remove the original visual identification code attachment and then install a new attachment with a cross-shaped groove, which matches the simulated surgical instrument. In this way, there is no need to redesign and print each time, and only the corresponding attachment needs to be replaced to quickly adapt to the needs of different instruments in surgery, greatly improving the efficiency and convenience of surgical preparation.

[0040] The present invention can be divided into a fixed part and a replacement part.

[0041] In the fixed part, at the bottom is a cylinder with a radius of 3mm and a height of 30mm. Above it is a cuboid with a length and width of 4mm and a height of 50mm. The top of this part is a large cylinder with a radius of 15mm and a height of 100mm with a 20×20mm cross-shaped groove. The replacement part of the invention is a 120×120×5mm square plate with a cross-shaped tongue for the device to scan the identification code. It only needs to be designed with the SOLIDWORKS software visual identification code attachment and integrated with the spinal model and simulation instrument 3D printing to achieve a fixed connection.

[0042] Further experiments have shown that the actual application of simulated surgical navigation using the present invention is convenient and fast, with low errors, indicating that the invention not only ensures the desired effect but also takes into account the accuracy and ease of operation.

[0043] Compared with the prior art, the beneficial effect of the present invention is that it is constructed using readily available materials, which saves costs and reduces energy consumption. After the head-mounted device captures the visual recognition marker Apriltag code, it calls the built-in camera to scan and determine the camera's position relative to the Apriltag code, accurately renders the model's spatial position in the doctor's perspective, and generates a first-person three-dimensional spinal image in real time to complete spatial tracking, allowing the doctor to design the optimal pedicle screw placement path guidance based on the patient's actual anatomical structure, thereby improving the accuracy and safety of screw placement. When the instrument needs to be replaced, it can be replaced by disassembling the square plate, which greatly simplifies the use process and saves steps.

[0044] This example experiments on the nail placement accuracy of the surgical navigation system using the present invention.

[0045] (1) Spine model: Scan and preprocess the spinal CT images to obtain a medical imaging sequence containing only spinal bone tissue, reconstruct the voxelized 3D image based on the Marching Cubes algorithm of the VTK library, generate a 3D spine model, and obtain a physical model through 3D printing;

[0046] (2) An external optical tracking system was introduced, and the Vicon system and optical calibration marker balls were used to perform optical positioning and record the three-dimensional spatial coordinates of the apex of the simulated screw and the spinal model. The patient held the device in hand and wore the surgical navigation device on the head, and the whole process of real-time screw placement guidance was displayed from a first-person perspective. The simulated screw at the front end of the device was inserted into the spinal model according to the guided path, simulating the process of pedicle screw placement into the spinal model, and recording the device's entry point on the spinal model. A comparative experiment was conducted with the path of screw placement guided by a person wearing a head-mounted surgical navigation device.

[0047] (3) Data processing and analysis: Ten sets of data were obtained through repeated experiments. The error of the guided and planned spatial coordinates was calculated and the average was statistically averaged to obtain the result. The error was 2.18 mm.

[0048] Experimental conclusion: The present invention has better nail placement accuracy in the auxiliary head-mounted surgical navigation system.

[0049] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Within the spirit and principle of the present invention, any modification, equivalent replacement, improvement, etc. made shall be included in the protection scope of the present invention.

[0050] The preferred embodiments of the present invention disclosed above are only used to help illustrate the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A replaceable tag device for an augmented reality head-mounted surgical navigation system, used for replacing surgical instruments in conjunction with a visual identification code for an augmented reality head-mounted surgical navigation system, characterized in that: The replaceable Tag device comprises a plate body (4) at the upper part and a cylinder body (3) at the lower part; The plate body (4) is a carrier of the Apriltag visual logo code, used as the identification code part scanned by the optical perspective augmented reality display device, and provided for scanning and identification by the built-in camera of the head-mounted surgical navigation device; a tenon (41) is provided on one side of the plate body (4); The cylinder (3) is a hand-grip area for holding and operating; a slot (31) for plugging with the tenon (41) is provided on one side of the cylinder (3); a rectangular connecting piece (2) is provided on the other side of the cylinder (3); the rectangular connecting piece (2) constitutes a probe area; a connecting groove for inserting a simulation screw (1) with an opening facing outward is provided in the rectangular connecting piece (2), and is used to connect the simulation screw (1); when in use, the plate (4) printed with a visual identification code is inserted into the groove of a spinal model or a simulation surgical instrument for connection.

2. The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system according to claim 1, characterized in that: The tenon (41) and the slot (31) are detachable mortise and tenon structures, and the connection is tightly fitted.

3. The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system according to claim 2, characterized in that: The tenon (41) and the slot (31) are in a cross shape.

4. The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system according to claim 1, characterized in that: A visual identification code is provided on the plate body (4) connected to the tenon (41), and after being scanned by an optical perspective augmented reality display device, a virtual spine model and a visual guide path will be displayed.

5. The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system according to claim 1, characterized in that: The simulated screw (1) is a cylinder, and the radius of the simulated screw (1) is 3 mm and the height is 30 mm.

6. The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system according to claim 1, characterized in that: The length and width of the rectangular parallelepiped connector (2) are both 4 mm, and the height is 50 mm; the slot (31) is a 20×20 mm cross-shaped slot.

7. The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system according to claim 1, characterized in that: The cylinder (3) has a radius of 15 mm and a height of 100 mm.

8. The replaceable Tag device of the assisted augmented reality head-mounted surgical navigation system according to claim 1, characterized in that: The plate body (4) is a square plate with a length, width and height of 120×120×5 mm respectively.

Images