Operation guide plate position detection method and device and computer equipment

Abstract

The invention relates to a position detection method and device for an operation guide plate and computer equipment. The method comprises the following steps: displaying a three-dimensional virtual model of a target part; aligning the three-dimensional virtual model with the real scene image of the target part to obtain an aligned three-dimensional virtual model; displaying a virtual guide plate of the target part based on the aligned three-dimensional virtual model; and performing position detection on the real guide plate of the target part according to the virtual guide plate. By adopting the method, the virtual guide plate can be used for detecting whether the placement pose of the real guide plate is accurate or not, and the placement accuracy of the surgical guide plate is improved.

Description

Technical Field

[0001] This application relates to the field of intelligent medical technology, and particularly to a method, device, and computer device for detecting the position of a surgical guide plate. Background Art

[0002] As an auxiliary tool in surgery, a 3D printed guide plate can improve the accuracy and safety of surgery. In orthopedic surgery, the guide plate can help doctors accurately determine the entry point and entry angle of screws, reducing the risk of damage to other tissues during the operation. Based on this characteristic, the design of the guide plate needs to adapt to the actual anatomical structures of different patients to ensure that the planned surgical path is consistent with the actual needs.

[0003] Currently, the design of guide plates for intraoperative positioning and navigation is mainly based on preoperative Computed Tomography (CT) images of patients. A personalized model that adapts to the geometric characteristics of the surface of different patients' anatomical tissues is designed, and a suitable guide plate is printed through 3D printing technology. However, when placing the guide plate during the operation, there may be some deviations between the actual geometric features of the surface of the target tissue area and the model obtained from preoperative CT. For example, in spinal surgery, doctors must try to strip the soft tissues on the surface of the vertebrae before placing the guide plate. Otherwise, the remaining soft tissues may embed in the area where the guide plate is joined to the vertebrae, resulting in a deviation between the placement of the guide plate and the preoperative plan. Although this deviation is small, in a surgical scenario with extremely high precision requirements, it may also pose a risk to the health of the patient.

[0004] Therefore, there is currently a problem of low accuracy in placing the surgical guide plate during the application process. Summary of the Invention

[0005] Based on this, it is necessary to provide a method, device, computer device, computer-readable storage medium, and computer program product for detecting the position of a surgical guide plate that can improve the placement accuracy in response to the above technical problems.

[0006] In a first aspect, this application provides a method for detecting the position of a surgical guide plate, including:

[0007] Displaying a three-dimensional virtual model of the target site;

[0008] Aligning the three-dimensional virtual model with the real-scene image of the target site to obtain an aligned three-dimensional virtual model;

[0009] Based on the aligned three-dimensional virtual model, displaying a virtual guide plate of the target site;

[0010] Detecting the position of the real guide plate of the target site according to the virtual guide plate.

[0011] In a second aspect, the present application further provides a position detection device for a surgical guide plate, including:

[0012] A first display module for displaying a three-dimensional virtual model of a target site;

[0013] A position alignment module for aligning the three-dimensional virtual model with a real-scene image of the target site to obtain an aligned three-dimensional virtual model;

[0014] A second display module for displaying a virtual guide plate of the target site based on the aligned three-dimensional virtual model;

[0015] A position detection module for detecting the position of a real guide plate of the target site according to the virtual guide plate.

[0016] In a third aspect, the present application further provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0017] Display a three-dimensional virtual model of a target site;

[0018] Align the three-dimensional virtual model with a real-scene image of the target site to obtain an aligned three-dimensional virtual model;

[0019] Based on the aligned three-dimensional virtual model, display a virtual guide plate of the target site;

[0020] According to the virtual guide plate, detect the position of a real guide plate of the target site.

[0021] In a fourth aspect, the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0022] Display a three-dimensional virtual model of a target site;

[0023] Align the three-dimensional virtual model with a real-scene image of the target site to obtain an aligned three-dimensional virtual model;

[0024] Based on the aligned three-dimensional virtual model, display a virtual guide plate of the target site;

[0025] According to the virtual guide plate, detect the position of a real guide plate of the target site.

[0026] In a fifth aspect, the present application further provides a computer program product, including a computer program. When the computer program is executed by a processor, the following steps are implemented:

[0027] Display a three-dimensional virtual model of a target site;

[0028] Align the three-dimensional virtual model with the real-scene image of the target site to obtain an aligned three-dimensional virtual model;

[0029] Based on the aligned three-dimensional virtual model, display the virtual guide plate of the target site;

[0030] According to the virtual guide plate, perform position detection on the real guide plate of the target site.

[0031] The above method, device, computer device, computer-readable storage medium, and computer program product for position detection of the surgical guide plate display a three-dimensional virtual model of the target site, align the three-dimensional virtual model with the real-scene image of the target site to obtain an aligned three-dimensional virtual model, display the virtual guide plate of the target site based on the aligned three-dimensional virtual model, and perform position detection on the real guide plate of the target site according to the virtual guide plate; by means of Augmented Reality (AR) technology, the three-dimensional virtual model of the target site can be aligned with the real-scene image, and the pre-operative planned guide plate can be displayed on the aligned three-dimensional virtual model to form a virtual guide plate. Since the virtual guide plate is generated according to the pre-operative plan and has a correct pose, the virtual guide plate can be used to detect whether the pose of the real guide plate placement is accurate, thereby increasing the accuracy of the surgical guide plate placement. Description of the Drawings

[0032] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for the description in the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0033] Figure 1 It is a schematic flow chart of the method for position detection of the surgical guide plate in one embodiment;

[0034] Figure 2 It is a schematic diagram of the installation of the fiducial marker on the vertebra site in one embodiment;

[0035] Figure 3 It is a schematic diagram of the operation interface for manual fine-tuning alignment in one embodiment;

[0036] Figure 4 It is a schematic diagram of exposing the vertebra during the operation in one embodiment;

[0037] Figure 5 It is a schematic diagram of the initial registration of the virtual vertebra according to the fiducial marker installed on the spine in one embodiment;

[0038] Figure 6 Schematic diagram for aligning a virtual vertebra with a real vertebra in one embodiment;

[0039] Figure 7 Schematic diagram for comparing a virtual guide plate with a real guide plate and comparing a virtual nail track with a real nail track in one embodiment;

[0040] Figure 8 Flow schematic diagram of a method for detecting the position of a surgical guide plate in another embodiment;

[0041] Figure 9 Structural block diagram of a device for a method for detecting the position of a surgical guide plate in one embodiment. Detailed implementation manners

[0042] In order to make the objectives, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0043] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0044] Before introducing the specific embodiments of the present application, the professional terms involved in the present application will be explained first:

[0045] AR sensing device: A sensing device used to scan the real scene during surgery, on which sensors such as optical cameras, depth cameras, lidar, etc. are provided, and it can be but is not limited to various head-mounted devices, or other non-head-mounted devices with optical cameras, depth cameras, lidar, etc.;

[0046] AR display device: A device used to superimpose and display virtual objects and real scenes, on which a display is provided, and it can be but is not limited to various head-mounted display devices, or displays, mobile phones, tablets, etc.;

[0047] Preoperative image: A three-dimensional image scan performed on the surgical object before surgery, including but not limited to CT, Magnetic Resonance (MR), etc., which records the anatomical features of the target part of the surgical object;

[0048] Surgical guide: A guide used to assist doctors during surgery. It is a customized guide designed based on the anatomical features of the relevant part obtained from the preoperative images of the surgical object and is fabricated by 3D printing.

[0049] Reference marker and its installation and fixation: A reference marker is a bracket with a characteristic pattern that can be fixed to the surgical site for initial registration of the virtual model and the real scene during surgery. The reference marker bracket can be designed according to the anatomical features of the surgical site, formed by 3D printing, installed and fixed to the corresponding surgical site, and its geometric features are all known, such as the position and deflection angle of the reference marker plane, etc.

[0050] Reference marker tracking algorithm: Used to detect and track the pose of the reference marker in the real scene in real time.

[0051] Operation interface for manual fine-tuning and alignment: An interface for controlling the position and orientation of the virtual model to align the virtual object with the real scene. Exemplarily, the operation interface can be, but is not limited to, an application independently developed on a head-mounted display device, or on a desktop, mobile operating system, etc. The operation methods include, but are not limited to, using external devices such as gestures, gamepads, keyboards, and mice to control the components in the interface. The form of the interface is not limited to the window form and can also be a windowless form.

[0052] The following uses specific embodiments to elaborate in detail on the technical solutions of the present application and how the technical solutions of the present application solve the above technical problems. These several specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The following will describe the embodiments of the present application in conjunction with the accompanying drawings.

[0053] In an exemplary embodiment, as Figure 1 shown, a method for detecting the position of a surgical guide is provided. This embodiment takes the application of this method to an AR device as an example for illustration. Among them, the AR device includes an AR sensing device and an AR display device. It can be understood that this method can also be applied to other terminals, which can be, but are not limited to, various personal computers, laptop computers, smartphones, tablet computers, Internet of Things devices, and other wearable devices, etc., and can also be applied to a server or a system including a terminal and a server. In this embodiment, the method includes the following steps:

[0054] Step S102, display a three-dimensional virtual model of the target part.

[0055] Among them, the target part can be the part that needs to be operated on or the part that comes into contact with the surgical guide during the operation. The three-dimensional virtual model can be a model of the target part obtained based on the preoperative images and displayed on the AR device.

[0056] In a specific implementation, the AR device can obtain the preoperative image of the target part, obtain the three-dimensional virtual model of the target part based on the preoperative image, and display the three-dimensional virtual model.

[0057] In practical applications, before the operation, the preoperative image of the target part can be obtained, the three-dimensional virtual model of the target part can be obtained based on the preoperative image, the surgical guide plate can be designed and printed according to the three-dimensional virtual model, and the pose of the surgical guide plate placed on the target part can be planned, wherein the surgical guide plate is connected to the fiducial marker bracket, and the fiducial marker bracket is used to fix the fiducial marker. During the operation, the doctor can install the surgical guide plate, the fiducial marker bracket and the fiducial marker on the target part according to the preoperative plan. Affected by the residual soft tissue, the placement pose of the surgical guide plate may deviate from the preoperative plan at this time. The AR sensing device can scan the real scene, track the pose of the fiducial marker in the real scene, and use the relative spatial position relationship between the target part planned preoperatively and the fiducial marker to project the three-dimensional virtual model of the target part onto the real scene through the AR display device.

[0058] Step S104: Align the three-dimensional virtual model with the real scene image of the target part to obtain an aligned three-dimensional virtual model.

[0059] Among them, the real scene image can be the target part in the real scene displayed in the AR device.

[0060] In a specific implementation, the AR device can sense and display the real scene image of the target part in real time. If the real scene image is not aligned with the three-dimensional virtual model, the three-dimensional virtual model can be adjusted to align the three-dimensional virtual model with the real scene image to obtain an aligned three-dimensional virtual model. Otherwise, if the real scene image is aligned with the three-dimensional virtual model, the three-dimensional virtual model is directly used as the aligned three-dimensional virtual model. Among them, alignment can be understood as that the shape and position deviations between the three-dimensional virtual model and the real scene image are not greater than the preset threshold. On the contrary, non-alignment can be understood as that the shape deviation or position deviation between the three-dimensional virtual model and the real scene image is greater than the preset threshold.

[0061] In practical applications, using the fiducial marker planned preoperatively to project the three-dimensional virtual model of the target part onto the real scene can realize the initial registration of the three-dimensional virtual model and the real scene image of the target part. Due to the possible deviation of the placement pose of the surgical guide plate, which may further lead to the deviation of the fiducial marker, the three-dimensional virtual model obtained using the fiducial marker may not be aligned with the real scene image. At this time, the AR sensing device can sense the adjustment operation instruction of the user for the three-dimensional virtual model, and adjust the three-dimensional virtual model according to the adjustment operation instruction to make it aligned with the real scene image. Among them, the adjustment operation instruction includes but is not limited to rotation, translation, scaling, etc.

[0062] Step S106: Based on the aligned three-dimensional virtual model, display the virtual guide plate for the target site.

[0063] Among them, the virtual guide plate can be a surgical guide plate model generated according to the preoperative plan and displayed on the AR device.

[0064] In specific implementation, the AR device can, based on the aligned three-dimensional virtual model, generate a three-dimensional surgical guide plate model according to the preoperative plan, place the three-dimensional surgical guide plate model on the aligned three-dimensional virtual model to form the virtual guide plate for the target site. Since the three-dimensional virtual model is aligned with the real-scene image at this time, and the virtual guide plate is placed on the three-dimensional virtual model with the correct pose according to the preoperative plan, the pose of the real guide plate placement can be detected based on the virtual guide plate at this time.

[0065] Step S108: Detect the position of the real guide plate for the target site according to the virtual guide plate.

[0066] Among them, the real guide plate can be a surgical guide plate installed in the real scene.

[0067] In specific implementation, the AR device can compare whether the virtual guide plate and the real guide plate have the same pose. If they are the same, it means that the placement of the real guide plate conforms to the preoperative plan and there is no deviation in the placement pose. Otherwise, if the virtual guide plate and the real guide plate have different poses, it means that the placement of the real guide plate does not conform to the preoperative plan and there is a deviation in the placement pose. At this time, the real guide plate can be adjusted according to the virtual guide plate to reduce the deviation of the guide plate placement.

[0068] In practical applications, in addition to detecting the deviation of the guide plate placement, the deviation of the nail track placement can also be detected. After the three-dimensional virtual model is aligned with the real-scene image, the AR device can, based on the aligned three-dimensional virtual model, generate a virtual guide plate according to the preoperative plan, generate a virtual nail track on the basis of the virtual guide plate, compare whether the real guide plate is aligned with the virtual guide plate, and whether the real nail track is aligned with the virtual nail track. If they are aligned, no adjustment is required. Otherwise, if they are not aligned, the real guide plate can be adjusted according to the virtual guide plate, and the real nail track can be adjusted according to the virtual nail track.

[0069] The above-mentioned position detection method of the surgical guide plate displays a three-dimensional virtual model of the target site, aligns the three-dimensional virtual model with the real-scene image of the target site to obtain an aligned three-dimensional virtual model, and based on the aligned three-dimensional virtual model, displays a virtual guide plate of the target site. According to the virtual guide plate, the position of the real guide plate of the target site is detected. With the help of AR technology, the three-dimensional virtual model of the target site can be aligned with the real-scene image, and the guide plate planned before the operation is displayed on the aligned three-dimensional virtual model to form a virtual guide plate. Since the virtual guide plate is generated according to the preoperative plan and has the correct pose, the virtual guide plate can be used to detect whether the pose of the real guide plate placement is accurate, increasing the accuracy of the surgical guide plate placement.

[0070] In an exemplary embodiment, the above step S102 may specifically include: identifying a reference mark installed on the target site; and displaying a three-dimensional virtual model of the target site according to a pre-determined first position relationship between the reference mark and the target site.

[0071] Wherein, the first position relationship may be the relative spatial position relationship between the reference mark planned before the operation and the target site.

[0072] In specific implementation, the first position relationship between the reference mark and the target site may be determined in advance before the operation. After the reference mark is installed on the target site during the operation, the AR device can identify the reference mark and display the three-dimensional virtual model of the target site with the reference mark as a reference according to the first position relationship.

[0073] In practical applications, the surgical guide plate is used to be installed on the target site, and the reference mark bracket is connected to the surgical guide plate to fix the reference mark. It can be seen that there is a relative spatial position relationship between the surgical guide plate and the reference mark. Before the operation, the surgical guide plate and the reference mark bracket can be designed and printed according to the preoperative image, and the pose of the surgical guide plate placed on the target site can be planned, thereby obtaining the relative spatial position relationship between the surgical guide plate and the target site. Since there is also a relative spatial position relationship between the surgical guide plate and the reference mark, the relative spatial position relationship between the reference mark and the target site, that is, the first position relationship, can be obtained. During the operation, after the surgical guide plate, the reference mark bracket and the reference mark are installed on the target site, the AR device can identify the reference mark, determine the display position of the three-dimensional virtual model of the target site according to the relative spatial position relationship between the reference mark and the target site, and display the three-dimensional virtual model according to this position.

[0074] In this embodiment, by identifying the reference markers installed at the target site and according to the pre-determined first positional relationship between the reference markers and the target site, a three-dimensional virtual model of the target site can be displayed. The three-dimensional virtual model of the target site can be displayed based on the preoperative planning of the surgical guide plate, so that the displayed three-dimensional virtual model is as close as possible to the real-scene image of the target site, reducing the adjustment amount of the three-dimensional virtual model and improving the alignment efficiency of the three-dimensional virtual model.

[0075] In an exemplary embodiment, the above step S104 may specifically include: identifying the first feature points of the three-dimensional virtual model and the second feature points corresponding to the first feature points in the real-scene image; adjusting the first feature points to align with the second feature points to obtain an aligned three-dimensional virtual model.

[0076] Among them, the first feature points may be the feature points in the three-dimensional virtual model. The second feature points may be the feature points corresponding to the first feature points in the real-scene image. The first feature points and the second feature points may correspond to the same tissue site.

[0077] In specific implementation, the AR device may identify the first feature points in the three-dimensional virtual model, obtain the second feature points corresponding to the first feature points in the real-scene image, and determine whether it is necessary to align the three-dimensional virtual model with the real-scene image according to the first feature points and the second feature points. If alignment is required, the first feature points may be adjusted to align with the second feature points to obtain an aligned three-dimensional virtual model. Otherwise, if alignment is not required, the first feature points do not need to be adjusted.

[0078] In practical applications, taking the target site as the vertebra as an example, the top of the vertebral spinous process can be used as the feature point. By adjusting the three-dimensional virtual model, the top of the vertebral spinous process (the first feature point) in the three-dimensional virtual model is aligned with the top of the vertebral spinous process (the second feature point) in the real-scene image to obtain an aligned three-dimensional virtual model.

[0079] In this embodiment, by identifying the first feature points of the three-dimensional virtual model and the second feature points corresponding to the first feature points in the real-scene image, and adjusting the first feature points to align with the second feature points to obtain an aligned three-dimensional virtual model, the three-dimensional virtual model can be aligned with the real-scene image by aligning the feature points, improving the alignment accuracy of the three-dimensional virtual model.

[0080] In an exemplary embodiment, the above step S104 may specifically further include: receiving an adjustment operation instruction from the user for the three-dimensional virtual model; the adjustment operation instruction includes at least one of rotation, translation, and scaling; in response to the adjustment operation instruction, displaying the adjusted three-dimensional virtual model; when the adjusted three-dimensional virtual model is aligned with the real-scene image, obtaining an aligned three-dimensional virtual model.

[0081] Among them, the adjustment operation instruction can be information indicating operations such as rotating, translating, and scaling a three-dimensional virtual model, including but not limited to gesture information in the three-dimensional space received by the AR perception device, as well as user instructions received by devices such as keyboards, mice, touchscreens, microphones, and gamepads.

[0082] In specific implementation, when it is necessary to align the three-dimensional virtual model with the real scene image, the user can issue an adjustment operation instruction, and the AR device adjusts the three-dimensional virtual model in real time according to the received adjustment operation instruction to obtain an adjusted three-dimensional virtual model. When the adjusted three-dimensional virtual model is aligned with the real scene image, the current adjusted three-dimensional virtual model can be used as the aligned three-dimensional virtual model.

[0083] For example, the user can indicate operations such as rotating, translating, and scaling the three-dimensional virtual model through gesture information in the three-dimensional space until the three-dimensional virtual model is aligned with the real scene image, and can also indicate the rotation, translation, scaling, etc. of the three-dimensional virtual model through a keyboard, mouse, touchscreen, microphone, joystick, etc. until the three-dimensional virtual model is aligned with the real scene image.

[0084] In this embodiment, by receiving the adjustment operation instruction of the user for the three-dimensional virtual model, in response to the adjustment operation instruction, the adjusted three-dimensional virtual model is displayed. When the adjusted three-dimensional virtual model is aligned with the real scene image, the aligned three-dimensional virtual model is obtained, and the three-dimensional virtual model can be aligned with the real scene image through a human-computer interaction method to ensure the reliability of the alignment of the three-dimensional virtual model.

[0085] In an exemplary embodiment, the above step S106 may specifically include: generating a virtual guide plate according to a pre-determined guide plate model; and displaying the virtual guide plate according to a pre-determined second positional relationship between the real guide plate and the target part and the aligned three-dimensional virtual model.

[0086] Among them, the guide plate model can be a three-dimensional surgical guide plate model planned before surgery. The second positional relationship can be the relative spatial positional relationship between the surgical guide plate planned before surgery and the target part.

[0087] In specific implementation, the relative spatial positional relationship between the surgical guide plate and the target part can be determined through preoperative planning to obtain the second positional relationship. After aligning the three-dimensional virtual model with the real scene image of the target part, the AR device can form a virtual guide plate according to the pre-planned guide plate model and place the virtual guide plate on the aligned three-dimensional virtual model for display according to the second positional relationship.

[0088] In this embodiment, a virtual guide plate is generated according to a pre-determined guide plate model. Based on the pre-determined second positional relationship between the real guide plate and the target site, and the aligned three-dimensional virtual model, the virtual guide plate is displayed. The virtual guide plate can be displayed on the three-dimensional virtual model aligned with the real-scene image, obtaining a virtual guide plate that conforms to the preoperative plan. Using this virtual guide plate, it is possible to detect whether the placement pose of the real guide plate conforms to the preoperative plan.

[0089] In an exemplary embodiment, before the above step S102, it may specifically further include: obtaining the preoperative image of the target site; determining the guide plate model and the second positional relationship between the real guide plate and the target site according to the preoperative image.

[0090] In specific implementation, the AR device can obtain the preoperative image of the target site, plan the shape of the surgical guide plate and the pose of the surgical guide plate placed on the target site based on the preoperative image. The guide plate model of the target site can be obtained according to the shape of the surgical guide plate, and the second positional relationship between the real guide plate and the target site can be obtained according to the pose of the surgical guide plate placed on the target site.

[0091] It can be understood that by performing 3D printing according to the guide plate model, a real guide plate for performing the surgery can be obtained.

[0092] In this embodiment, by obtaining the preoperative image of the target site, determining the guide plate model and the second positional relationship between the real guide plate and the target site according to the preoperative image, the shape and the placement pose of the surgical guide plate can be preoperatively planned, ensuring the reliability of the application of the surgical guide plate.

[0093] In an exemplary embodiment, the above step S108 may specifically include: comparing the real guide plate with the virtual guide plate to obtain the position deviation of the real guide plate, and adjusting the actual position of the real guide plate placed on the target site according to the position deviation.

[0094] Among them, the position deviation may be the deviation in position between the real guide plate and the virtual guide plate. The actual position may be the position and pose of the current real guide plate placed on the target site.

[0095] In specific implementation, the AR device can sense and display in real time the real guide plate installed on the target site, compare the real guide plate with the virtual guide plate generated based on the aligned three-dimensional virtual model, and determine the position deviation between the real guide plate and the virtual guide plate. If the position deviation is not greater than the preset threshold, there is no need to adjust the real guide plate. Otherwise, if the position deviation is greater than the preset threshold, it is necessary to adjust the actual placement position of the real guide plate on the target site. The actual placement position of the real guide plate can be automatically adjusted by the AR device or manually adjusted by a human, and this application does not limit this.

[0096] In practical applications, since it is necessary to insert a Kirschner wire into the nail track sleeve of the surgical guide plate during the operation, the nail track of the Kirschner wire can also be planned before the operation. During the operation, after aligning the three-dimensional virtual model with the real-scene image, the AR device can generate a virtual guide plate and a virtual nail track based on the pre-operative plan on the aligned three-dimensional virtual model. The AR device can sense and display in real time the real guide plate and the real nail track installed at the target site, compare the real guide plate with the virtual guide plate, and compare the real nail track with the virtual nail track to obtain the position deviation between the real guide plate and the virtual guide plate, and the position deviation between the real nail track and the virtual nail track. According to the position deviation between the real guide plate and the virtual guide plate, and the position deviation between the real nail track and the virtual nail track, adjust the actual placement positions of the surgical guide plate and the Kirschner wire. For example, the position deviation between the real guide plate and the virtual guide plate, and the position deviation between the real nail track and the virtual nail track can be gradually reduced until both position deviations are less than a preset threshold.

[0097] In this embodiment, by comparing the real guide plate with the virtual guide plate to obtain the position deviation of the real guide plate, and adjusting the actual position of the real guide plate placed at the target site according to the position deviation, it can be determined whether the real guide plate and nail track are aligned with the virtual guide plate and nail track. When they are not aligned, the positions of the guide plate and nail track are adjusted to improve the accuracy of the placement positions of the guide plate and nail track.

[0098] To facilitate those skilled in the art to deeply understand the embodiments of the present application, the following will be described with a specific example.

[0099] The present application provides a method for verifying the accuracy of guide plate placement in spinal surgery, which can help doctors judge whether the position and posture of the guide plate placement are consistent with the pre-operative plan during the process of placing the guide plate in orthopedic surgery. This method projects the tissue segmentation model and the guide plate design model obtained based on the pre-operative CT images onto the actual intraoperative position through the AR display device and presents them to the doctor. At the same time, an interactive operation interface is provided to achieve fine alignment with the actual tissue anatomical structure. When placing the guide plate, by comparing whether the real guide plate is aligned with the virtual guide plate, it can be evaluated whether the guide plate is accurately placed during the operation.

[0100] This method provides a method for doctors to help them judge whether the guide plate is accurately placed during the operation, which can increase the accuracy of guide plate placement, thereby improving the accuracy and efficiency of the operation. At the same time, for surgeries such as spinal surgery that require X-ray fluoroscopy guidance, it can effectively reduce the number of times doctors use X-ray fluoroscopy and reduce the risk of ionizing radiation exposure for patients and doctors.

[0101] This application integrates the virtual guide and the real scene through a two-step registration method, including an automatic initial registration step and a manual fine-tuning alignment step. This method ensures that the virtual object can be accurately integrated with the real scene: the initial registration step can quickly place the virtual object in a relatively reasonable position based on the spatial information in the preoperative image, and the manual fine-tuning alignment provides a means for the user to interact with the virtual object, further ensuring the adaptability to different scene changes and improving the robustness of the system.

[0102] Among them, the initial registration method automatically registers and integrates the virtual object with the intraoperative real scene by tracking the pose of the fiducial markers installed on the patient's bone in the three-dimensional space and using the spatial information contained in the preoperative image; the manual fine-tuning alignment can be used after the initial registration step to further reduce the registration error caused by the inconsistency between the intraoperative scene and the preoperative image. The system provides an operation interface for the user, and the user can manually adjust the position and pose of the virtual object according to the real-time visual feedback in the system display interface to achieve "hand-eye coordination" and finally realize the complete alignment of the virtual object and the real object.

[0103] The method for verifying the accuracy of the guide placement in spinal surgery described above may include the following steps:

[0104] Before surgery:

[0105] Step S201, obtain the preoperative image of the target part of the surgical object;

[0106] Step S202, obtain the three-dimensional virtual model of the target part or the part in contact with the guide from the preoperative image;

[0107] Step S203, design and print the surgical guide and the fiducial marker bracket according to the preoperative image.

[0108] During surgery:

[0109] Step S204, install the fiducial markers at the positions designed before surgery; Figure 2 A schematic diagram of the intraoperative installation of the fiducial markers on the vertebra is provided. Among them, the surgical guide 302 is installed on the vertebra, the surgical guide 302 is connected to the fiducial marker bracket 304, and the fiducial marker bracket 304 is used to support the fiducial marker 306. The doctor can install the surgical guide 302 on the vertebra according to the position and pose planned before surgery; it should be noted that in addition to the pattern shown in this embodiment, the fiducial markers can also be, but are not limited to, other two-dimensional patterns or three-dimensional objects that can be used for tracking;

[0110] Step S205, scan the intraoperative real scene through the AR perception device and use the fiducial marker tracking algorithm to determine the pose of the fiducial markers in the three-dimensional space of the intraoperative real scene;

[0111] Step S206: Using the relative spatial position relationship (the first position relationship) between the target site and the reference marker in the preoperative image, project the virtual object onto the real scene through the AR display device to form a three-dimensional virtual model of the target site. This step is the initial registration of the three-dimensional virtual model and the real object (real scene image) during the operation. At this time, it is possible to determine whether the two are aligned by comparing the feature points of the three-dimensional virtual model and the real object during the operation, or to determine the error size between the two;

[0112] Step S207: If the initial registration does not align the three-dimensional virtual model with the real object during the operation, the position and pose of the three-dimensional virtual model can be finely adjusted to achieve alignment using the operation interface for manual fine-tuning alignment; Figure 3 A schematic diagram of the operation interface for manual fine-tuning alignment is provided. The operation interface for manual fine-tuning alignment can be used to adjust the position and pose of the three-dimensional virtual model in three different directions (head-tail direction, vertical direction, horizontal direction); Figure 3 Taking the head-tail direction as an example, it is used to adjust the rotation angle and displacement in the head-tail direction;

[0113] Step S208: After determining that the three-dimensional virtual model is aligned with the real object during the operation, the real guide plate can be installed at the pre-designed position, and it can be judged whether the placement of the real guide plate is consistent with the preoperative plan by comparing the poses of the virtual guide plate and the real guide plate in the AR display device.

[0114] In one embodiment, a process of using reference markers for initial registration and manual fine-tuning alignment of vertebrae during the operation is provided. Figure 4 A schematic diagram of the exposed spine 401 during the operation is given. Refer to Figure 5 , reference markers can be installed on the spine 401, and the virtual vertebra 402 can be initially registered according to the reference markers. Refer to Figure 6 , the virtual vertebra 402 can be outlined and displayed to form a virtual vertebra contour 403. The virtual vertebra contour 403 includes the top 404 of the vertebral spinous process. The virtual vertebra contour 403 is finely adjusted and aligned according to the top 404 of the vertebral spinous process. Specifically, the top 404 of the vertebral spinous process can be aligned with the spine 401. According to Figure 7 , the guide plate is installed at the pre-designed position of the vertebra, and a Kirschner wire is inserted into the nail track sleeve of the guide plate. The virtual guide plate 405 and the virtual nail track 406 are displayed, and it is compared whether the virtual guide plate 405 and the virtual nail track 406 are aligned with the real guide plate and nail track. It should be noted that for the convenience of observation, Figures 5 to 7 the reference marker bracket and the real guide plate installed below the reference marker during the operation are omitted.

[0115] In one embodiment, as Figure 8As shown, a method for detecting the position of a surgical guide is provided. Taking the application of this method to an augmented reality device as an example, it includes the following steps:

[0116] Step S501, obtain the preoperative image of the target site, determine the guide model according to the preoperative image, and the positional relationship between the real guide and the target site, and generate the real guide according to the guide model;

[0117] Step S502, identify the fiducial markers installed on the target site, and display the three-dimensional virtual model of the target site according to the pre-determined positional relationship between the fiducial markers and the target site;

[0118] Step S503, identify the first feature points of the three-dimensional virtual model, and the second feature points corresponding to the first feature points in the real scene image, and adjust the first feature points to align with the second feature points to obtain an aligned three-dimensional virtual model;

[0119] Step S504, generate a virtual guide according to the guide model, and display the virtual guide according to the positional relationship between the real guide and the target site, and the aligned three-dimensional virtual model;

[0120] Step S505, compare the real guide with the virtual guide to obtain the position deviation of the real guide, and adjust the actual position of the real guide placed on the target site according to the position deviation.

[0121] In specific implementation, before the operation, the AR device can obtain the preoperative image of the target site, plan the surgical guide according to the preoperative image, including determining the guide model of the surgical guide, the fiducial marker bracket model, and determining the pose of the real guide placed on the target site, and perform 3D printing on the guide model and the fiducial marker bracket model respectively to obtain the real guide and the fiducial marker bracket. During the operation, the doctor can place the real guide and the fiducial marker bracket on the target site according to the preoperative plan, install the fiducial markers on the fiducial marker bracket. Since there may be residual soft tissues, etc. on the target site, the positions where the real guide and the fiducial marker bracket are placed may have deviations. The AR device can identify the fiducial markers on the fiducial marker bracket, perform initial registration on the three-dimensional virtual model according to the relative positional relationship between the pre-planned fiducial markers and the target site, display the three-dimensional virtual model of the target site, align the three-dimensional virtual model with the real scene image according to the feature points in the three-dimensional virtual model. Taking the aligned three-dimensional virtual model as the reference, display the virtual guide and the virtual nail track of the target site according to the relative positional relationship between the real guide and the target site. By comparing the real guide and the virtual guide, and comparing the real nail track and the virtual nail track, the position deviations of the guide and the nail track can be obtained, and the actual position of the real guide placed on the target site can be adjusted according to the position deviations of the guide and the nail track.

[0122] The above method for detecting the position of the surgical guide plate can utilize AR technology to align the three-dimensional virtual model of the target site with the real-scene image, and display the pre-operative planned guide plate on the aligned three-dimensional virtual model to form a virtual guide plate. Since the virtual guide plate is generated according to the pre-operative plan and has the correct pose, the virtual guide plate can be used to detect whether the pose of the real guide plate placed is accurate, thereby increasing the accuracy of placing the surgical guide plate.

[0123] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily have to be executed in the order indicated by the arrows. Unless there is a clear indication in this article, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily have to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily have to be sequential, but can be executed alternately or in turn with at least some of the steps or stages in other steps or other steps.

[0124] Based on the same inventive concept, the embodiments of the present application also provide a position detection device for a surgical guide plate for implementing the above-mentioned method for detecting the position of the surgical guide plate. The solution provided by this device to solve the problem is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the position detection device for the surgical guide plate provided below can refer to the limitations on the method for detecting the position of the surgical guide plate in the above text, and will not be repeated here.

[0125] In an exemplary embodiment, as Figure 9 shown, a position detection device for a surgical guide plate is provided, including: a first display module 602, a position alignment module 604, a second display module 606, and a position detection module 608, where:

[0126] The first display module 602 is configured to display a three-dimensional virtual model of the target site;

[0127] The position alignment module 604 is configured to align the three-dimensional virtual model with the real-scene image of the target site to obtain an aligned three-dimensional virtual model;

[0128] The second display module 606 is configured to display the virtual guide plate of the target site based on the aligned three-dimensional virtual model;

[0129] The position detection module 608 is configured to perform position detection on the real guide plate of the target site according to the virtual guide plate.

[0130] In an exemplary embodiment, the above-mentioned first display module 602 is further configured to identify a reference mark installed on the target part; and display the three-dimensional virtual model of the target part according to a pre-determined first positional relationship between the reference mark and the target part.

[0131] In an exemplary embodiment, the above-mentioned position alignment module 604 is further configured to identify a first feature point of the three-dimensional virtual model and a second feature point corresponding to the first feature point in the real scene image; and adjust the first feature point to be aligned with the second feature point to obtain the aligned three-dimensional virtual model.

[0132] In an exemplary embodiment, the above-mentioned position alignment module 604 is further configured to receive an adjustment operation instruction from the user for the three-dimensional virtual model; the adjustment operation instruction includes at least one of rotation, translation, and scaling; in response to the adjustment operation instruction, display the adjusted three-dimensional virtual model; and when the adjusted three-dimensional virtual model is aligned with the real scene image, obtain the aligned three-dimensional virtual model.

[0133] In an exemplary embodiment, the above-mentioned second display module 606 is further configured to generate the virtual guide plate according to a pre-determined guide plate model; and display the virtual guide plate according to a pre-determined second positional relationship between the real guide plate and the target part and the aligned three-dimensional virtual model.

[0134] In an exemplary embodiment, the position detection device of the above-mentioned surgical guide plate further includes a preoperative planning module, configured to obtain a preoperative image of the target part; and determine the guide plate model and the second positional relationship between the real guide plate and the target part according to the preoperative image.

[0135] In an exemplary embodiment, the above-mentioned position detection module 608 is further configured to compare the real guide plate with the virtual guide plate to obtain a position deviation of the real guide plate; and adjust the actual position of the real guide plate placed on the target part according to the position deviation.

[0136] Each module in the above-mentioned position detection device of the surgical guide plate can be implemented in whole or in part by software, hardware, and their combination. The above-mentioned modules can be embedded in the processor of the computer device in the form of hardware or be independent of it, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned respective modules.

[0137] In an exemplary embodiment, a computer device is provided, which may be an augmented reality device. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit, and an input device. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface, the display unit, and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, near field communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a method for detecting the position of a surgical guide. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the computer device housing, or an external keyboard, touchpad, or mouse, etc.

[0138] Those skilled in the art can understand that the above structure is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0139] In an embodiment, a computer device is further provided, which may be but is not limited to an augmented reality device, including a memory and a processor. A computer program is stored in the memory, and when the processor executes the computer program, the steps in the above method embodiments are implemented.

[0140] In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0141] In an embodiment, a computer program product is provided, including a computer program. When the computer program is executed by the processor, the steps in the above method embodiments are implemented.

[0142] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with relevant regulations.

[0143] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in this application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., and are not limited thereto. The processors involved in the embodiments provided in this application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., and are not limited thereto.

[0144] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this application.

[0145] The above-described embodiments merely represent several implementation manners of this application. The description is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of this application. It should be noted that for those of ordinary skill in the art, without departing from the concept of this application, several variations and improvements can still be made, and these all belong to the protection scope of this application. Therefore, the protection scope of this application shall be subject to the appended claims.

Claims

1. A method for detecting the position of a surgical guide plate, characterized in that, The method includes: Displaying a three-dimensional virtual model of a target site; Aligning the three-dimensional virtual model with a real-scene image of the target site to obtain an aligned three-dimensional virtual model; Based on the aligned three-dimensional virtual model, displaying a virtual guide plate of the target site; According to the virtual guide plate, performing position detection on a real guide plate of the target site.

2. The method according to claim 1, wherein The displaying of the three-dimensional virtual model of the target site includes: Identifying fiducial marks installed on the target site; According to a pre-determined first positional relationship between the fiducial marks and the target site, displaying the three-dimensional virtual model of the target site.

3. The method according to claim 1, wherein The aligning of the three-dimensional virtual model with the real-scene image of the target site to obtain an aligned three-dimensional virtual model includes: Identifying first feature points of the three-dimensional virtual model and second feature points corresponding to the first feature points in the real-scene image; Adjusting the first feature points to be aligned with the second feature points to obtain the aligned three-dimensional virtual model.

4. The method according to claim 1, wherein The aligning of the three-dimensional virtual model with the real-scene image of the target site to obtain an aligned three-dimensional virtual model includes: Receiving an adjustment operation instruction for the three-dimensional virtual model; the adjustment operation instruction includes at least one of rotation, translation, and scaling; In response to the adjustment operation instruction, displaying an adjusted three-dimensional virtual model; When the adjusted three-dimensional virtual model is aligned with the real-scene image, obtaining the aligned three-dimensional virtual model.

5. The method according to claim 1, wherein The displaying of the virtual guide plate of the target site based on the aligned three-dimensional virtual model includes: Generating the virtual guide plate according to a pre-determined guide plate model; According to a pre-determined second positional relationship between the real guide plate and the target site and the aligned three-dimensional virtual model, displaying the virtual guide plate.

6. The method according to claim 5, wherein Before displaying the three-dimensional virtual model of the target site, it further includes: Obtaining a preoperative image of the target site; According to the preoperative image, determining the guide plate model and the second positional relationship between the real guide plate and the target site.

7. The method according to claim 1, wherein The performing of position detection on the real guide plate of the target site according to the virtual guide plate includes: Comparing the real guide plate with the virtual guide plate to obtain a position deviation of the real guide plate; According to the position deviation, adjusting the actual position where the real guide plate is placed on the target site.

8. A position detection device for a surgical guide plate, characterized in that, The device includes: A first display module for displaying a three-dimensional virtual model of a target site; A position alignment module for aligning the three-dimensional virtual model with a real-scene image of the target site to obtain an aligned three-dimensional virtual model; A second display module for displaying a virtual guide plate of the target site based on the aligned three-dimensional virtual model; A position detection module for performing position detection on a real guide plate of the target site according to the virtual guide plate.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images