A method, device and system for navigation and positioning of a surgical robot

Abstract

This invention provides a surgical robot navigation and positioning method, apparatus, and system, relating to the field of data processing technology. The method includes: acquiring two two-dimensional images captured by an image acquisition device, and obtaining object pose data and tracer pose data; planning screw pose data in the two-dimensional images; for each two-dimensional image, identifying markers in the image, and calculating a first transformation relationship based on the identified image position and the position of the marker in the tracer coordinate system; calculating a second transformation relationship based on the object pose data and the tracer pose data; converting the screw pose data into target pose data based on the first and second transformation relationships; and navigating and positioning the surgical robot based on the target pose data. Applying the surgical robot navigation and positioning scheme provided by this invention enables surgical robot navigation and positioning.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus and system for navigation and positioning of a surgical robot. Background Technology

[0002] When performing surgeries involving inserting screws into the bones of humans, animals, or other objects, surgeons can use surgical robots. Because surgical robots can typically perform more precise and delicate operations, they can improve the success rate of surgeries. During the procedure, the surgical robot's pose is usually planned first, then adjusted to the planned pose to achieve navigation and positioning, and finally, the surgery is performed based on this navigation and positioning.

[0003] Therefore, a navigation and positioning scheme for surgical robots is needed to navigate and position them. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, and system for navigation and positioning of a surgical robot. The specific technical solution is as follows:

[0005] In a first aspect, embodiments of the present invention provide a surgical robot navigation and positioning method, the method comprising:

[0006] Two two-dimensional images are acquired by an image acquisition device, and object pose data of the target object and tracer pose data of the first tracer are obtained in the same preset coordinate system. The field of view of the image acquisition device includes the target object and a registration plate with markers. The image acquisition device acquires each two-dimensional image at a different acquisition angle, and the first tracer is mounted on the registration plate.

[0007] Screw pose data for the screws used to place the screws on the target object are planned from the two obtained two-dimensional images;

[0008] For each two-dimensional image, the marker in the two-dimensional image is identified. Based on the image position of the identified marker and the pre-obtained position of the marker in the tracer coordinate system, a first transformation relationship between the image coordinate system of the two-dimensional image and the tracer coordinate system is calculated. The tracer coordinate system is a coordinate system established based on the first tracer.

[0009] Based on the object pose data and the tracer pose data, a second transformation relationship between the object coordinate system and the tracer coordinate system is calculated, wherein the object coordinate system is a coordinate system established based on the target object;

[0010] Based on the first and second transformation relationships, the screw pose data is converted into target pose data in the object coordinate system;

[0011] The surgical robot is navigated and positioned based on the target pose data.

[0012] In one embodiment of the present invention, when the surgical robot is equipped with a robotic arm and a locator is fixed to the end of the robotic arm, the step of navigating and locating the surgical robot based on the target pose data includes:

[0013] Obtain the current posture data of the end effector of the robotic arm in the base coordinate system and the current position data of the positioner in the object coordinate system, wherein the base coordinate system is a coordinate system established based on the base of the robotic arm;

[0014] Based on the target pose data, current pose data, third transformation relationship, and locator pose data, the surgical pose data of the robotic arm end effector in the base coordinate system is calculated when the locator points to the target pose data. The third transformation relationship is the transformation relationship between the locator coordinate system and the robotic arm end effector coordinate system. The base coordinate system is a coordinate system established based on the base of the robotic arm. The robotic arm end effector coordinate system is a coordinate system established based on the end effector of the robotic arm. The robotic arm end effector pose data is the pose data of the robotic arm end effector coordinate system in the base coordinate system. The locator pose data is the pose data of the locator in the object coordinate system.

[0015] The surgical robot is navigated and positioned based on the surgical pose data.

[0016] In one embodiment of the present invention, before converting the screw pose data into target pose data in the object coordinate system, the method further includes:

[0017] Show users the various 2D images after the planning;

[0018] Obtain the pose adjustment information input by the user based on the displayed two-dimensional image;

[0019] The screw pose data is adjusted based on the pose adjustment information.

[0020] In one embodiment of the present invention, the step of navigating and locating the surgical robot based on the target pose data includes:

[0021] Obtain the first pose data of the surgical robot in the object coordinate system;

[0022] The surgical robot is navigated and positioned based on the first pose data and the target pose data.

[0023] After navigation and positioning, the surgical robot obtains its second pose data in the object coordinate system.

[0024] Based on the second pose data, calculate the fifth pose data of the screw for placement based on the current pose of the surgical robot;

[0025] Calculate the pose difference between the fifth pose data and the target pose data;

[0026] If the pose difference is greater than a preset difference threshold, the second pose data is used as the first pose data, and the process of obtaining the navigation and positioning steps for the surgical robot is returned until the calculated pose difference is less than or equal to the difference threshold.

[0027] In one embodiment of the present invention, after navigating and locating the surgical robot based on the target pose data, the method further includes:

[0028] Real-time acquisition of surgical instrument pose data in the object coordinate system during surgery on the target object;

[0029] Based on the first transformation relationship and the second transformation relationship, the instrument pose data is converted into the sixth pose data in the image coordinate system;

[0030] The system displays a two-dimensional image containing the sixth pose data to the user in real time.

[0031] Secondly, embodiments of the present invention also provide a surgical robot navigation and positioning device, the device comprising:

[0032] The data acquisition module is used to acquire two two-dimensional images acquired by the image acquisition device, and to acquire the object pose data of the target object and the tracer pose data of the first tracer in the same preset coordinate system. The field of view of the image acquisition device includes the target object and a registration plate with markers. The image acquisition device acquires each two-dimensional image at a different acquisition angle. The first tracer is mounted on the registration plate.

[0033] The screw planning module is used to plan the screw pose data for placing the screw on the target object from the two obtained two-dimensional images.

[0034] The first calculation module is used to identify the marker in each two-dimensional image, and calculate a first transformation relationship between the image coordinate system of the two-dimensional image and the tracer coordinate system based on the image position of the identified marker and the pre-obtained position of the marker in the tracer coordinate system. The tracer coordinate system is a coordinate system established based on the first tracer.

[0035] The second calculation module is used to calculate a second transformation relationship between the object coordinate system and the tracer coordinate system based on the object pose data and the tracer pose data, wherein the object coordinate system is a coordinate system established based on the target object;

[0036] The first conversion module is used to convert the screw pose data into target pose data in the object coordinate system according to the first conversion relationship and the second conversion relationship.

[0037] The navigation and positioning module is used to navigate and position the surgical robot based on the target pose data.

[0038] In one embodiment of the present invention, when the surgical robot is equipped with a robotic arm and a locator is fixed to the end of the robotic arm, the navigation and positioning module includes:

[0039] The data acquisition submodule is used to acquire the current posture data of the current end effector of the robotic arm in the base coordinate system and the current positioner pose data of the current positioner in the object coordinate system, wherein the base coordinate system is a coordinate system established based on the base of the robotic arm;

[0040] The pose calculation submodule is used to calculate the surgical pose data of the robotic arm end effector in the base coordinate system when the locator points to the target pose data, based on the target pose data, current pose data, third transformation relationship, and locator pose data. The third transformation relationship is the transformation relationship between the locator coordinate system and the robotic arm end effector coordinate system. The base coordinate system is a coordinate system established based on the base of the robotic arm. The robotic arm end effector coordinate system is a coordinate system established based on the end effector of the robotic arm. The robotic arm end effector pose data is the pose data of the robotic arm end effector coordinate system in the base coordinate system. The locator pose data is the pose data of the locator in the object coordinate system.

[0041] The navigation and positioning submodule is used to navigate and position the surgical robot based on the surgical pose data.

[0042] In one embodiment of the present invention, the apparatus further includes:

[0043] The image display module is used to display the planned two-dimensional images to the user before converting the screw pose data into target pose data in the object coordinate system;

[0044] The information acquisition module is used to acquire the pose adjustment information input by the user based on the displayed two-dimensional image;

[0045] The pose adjustment module is used to adjust the screw pose data according to the pose adjustment information.

[0046] In one embodiment of the present invention, the navigation and positioning module is specifically used for:

[0047] Obtain the first pose data of the surgical robot in the object coordinate system;

[0048] The surgical robot is navigated and positioned based on the first pose data and the target pose data.

[0049] After navigation and positioning, the surgical robot obtains its second pose data in the object coordinate system.

[0050] Based on the second pose data, calculate the fifth pose data of the screw for placement based on the current pose of the surgical robot;

[0051] Calculate the pose difference between the fifth pose data and the target pose data;

[0052] If the pose difference is greater than a preset difference threshold, the second pose data is used as the first pose data, and the process of obtaining the navigation and positioning steps for the surgical robot is returned until the calculated pose difference is less than or equal to the difference threshold.

[0053] In one embodiment of the present invention, the apparatus further includes:

[0054] The pose acquisition module is used to obtain, in real time, the instrument pose data of the surgical instruments in the object coordinate system when performing surgery on the target object after the surgical robot is navigated and positioned according to the target pose data;

[0055] The second conversion module is used to convert the instrument pose data into a sixth pose data in the image coordinate system according to the first conversion relationship and the second conversion relationship.

[0056] The real-time display module is used to display a two-dimensional image containing the sixth pose data to the user in real time.

[0057] Thirdly, embodiments of the present invention also provide a surgical robot navigation and positioning system, the system comprising a three-dimensional navigation and tracking device, a surgical robot with a robotic arm, an object tracer for displaying the pose data of a target object, a locator, a first tracer, a registration plate with markers, an image acquisition device, and a host computer;

[0058] The locator is mounted on the end of the robotic arm of the surgical robot, and the first tracer is mounted on the registration plate;

[0059] The image acquisition device is used to acquire two two-dimensional images and send the acquired two-dimensional images to the host computer. The field of view of the image acquisition device includes the target object and the registration plate. The acquisition angle of each two-dimensional image is different.

[0060] The three-dimensional navigation and tracking device is used to obtain the object pose data shown by the object tracker and the tracker pose data of the first tracker in the same preset coordinate system, and to send the object pose data and the tracker pose data to the host computer.

[0061] The host computer is used to control the surgical robot to perform navigation and positioning according to the steps of the method described in the first aspect above.

[0062] Fourthly, embodiments of the present invention also provide an electronic device, including a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0063] Memory, used to store computer programs;

[0064] When a processor executes a program stored in memory, it implements the steps of the method described in the first aspect above.

[0065] Fifthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0066] Beneficial effects of the embodiments of the present invention:

[0067] As can be seen from the above, when using the solution provided in the embodiments of the present invention to navigate and locate a surgical robot, a first tracer is mounted on the registration plate. Based on the positions of the markers in the registration plate in the image coordinate system and the tracer coordinate system of each two-dimensional image, the first transformation relationship between the image coordinate system and the tracer coordinate system of each two-dimensional image can be accurately calculated. Based on the object pose data and the tracer pose data, the second transformation relationship between the object coordinate system and the tracer coordinate system can be accurately calculated. Thus, based on the first transformation relationship and the second transformation relationship, the screw pose data of the screw planned in the two two-dimensional images can be converted into target pose data in the object coordinate system. Then, based on the target pose data, navigation and positioning of the surgical robot can be achieved.

[0068] Furthermore, since the image acquisition device captures two 2D images from different angles, the two images can reflect the 3D information of the target object. Therefore, planning the screw pose data from these two 2D images can reflect the 3D pose of the inserted screw. This screw pose data can then be converted into target pose data, enabling accurate navigation and positioning of the surgical robot. Therefore, the surgical robot navigation and positioning scheme provided in this invention can improve the accuracy of surgical robot navigation and positioning. Attached Figure Description

[0069] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other embodiments can be obtained based on these drawings.

[0070] Figure 1a A flowchart illustrating the first surgical robot navigation and positioning method provided in an embodiment of the present invention;

[0071] Figure 1b This is a partial structural diagram of a C-arm machine provided in an embodiment of the present invention;

[0072] Figure 2a This is a schematic diagram of the structure of a robotic arm end effector provided in an embodiment of the present invention;

[0073] Figure 2b This is a schematic diagram of the structure of a locator provided in an embodiment of the present invention;

[0074] Figure 2c This is a schematic diagram of a screw placement surgery scenario provided by an embodiment of the present invention;

[0075] Figure 2d This is a flowchart illustrating the second surgical robot navigation and positioning method provided in an embodiment of the present invention.

[0076] Figure 3 A flowchart illustrating the third surgical robot navigation and positioning method provided in this embodiment of the invention;

[0077] Figure 4 A flowchart illustrating the fourth surgical robot navigation and positioning method provided in this embodiment of the invention;

[0078] Figure 5 A flowchart illustrating the fifth surgical robot navigation and positioning method provided in this embodiment of the invention;

[0079] Figure 6 This is a schematic diagram of the structure of a first surgical robot navigation and positioning device provided in an embodiment of the present invention;

[0080] Figure 7 This is a schematic diagram of the structure of a second surgical robot navigation and positioning device provided in an embodiment of the present invention;

[0081] Figure 8 This is a schematic diagram of the structure of a third type of surgical robot navigation and positioning device provided in an embodiment of the present invention;

[0082] Figure 9 This is a schematic diagram of the structure of the fourth surgical robot navigation and positioning device provided in an embodiment of the present invention;

[0083] Figure 10 This is a schematic diagram of the structure of a surgical robot navigation and positioning system provided in an embodiment of the present invention;

[0084] Figure 11 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0085] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art based on the present invention are within the scope of protection of the present invention.

[0086] Nowadays, when surgeons use surgical robots to perform screw insertion surgeries, they typically need to acquire CT images or multiple X-ray images of the patient, such as humans or animals, during the procedure. The surgery is then performed based on the acquired data, images, and the surgical robot. Both CT and X-ray image acquisition devices use X-rays for image acquisition. When these devices acquire CT or X-ray images, both the surgeon and the patient are exposed to X-ray radiation emitted by the acquisition device.

[0087] To minimize the amount of X-ray radiation received by the surgeon or patient during surgery, embodiments of the present invention provide a surgical robot navigation and positioning method and apparatus. These will be described in detail below.

[0088] See Figure 1a , Figure 1a This is a flowchart illustrating the first surgical robot navigation and positioning method provided in an embodiment of the present invention. The method includes the following steps S101-S106.

[0089] Step S101: Obtain two two-dimensional images acquired by the image acquisition device, and obtain the object pose data of the target object and the tracer pose data of the first tracer in the same preset coordinate system.

[0090] The field of view of the image acquisition device includes the target object and the registration plate with markers.

[0091] The image acquisition device described above can be a two-dimensional imaging device. For example, the image acquisition device described above can be a C-arm machine.

[0092] In one embodiment of the present invention, the above-mentioned two-dimensional image is an X-ray image.

[0093] Image acquisition devices capture two-dimensional images from different angles. For example, a two-dimensional image can be captured from the front of the target object, or it can be captured from the side of the target object.

[0094] The first tracer mentioned above is mounted on the registration plate.

[0095] The aforementioned registration plate can be installed on the image acquisition device and located within the field of view of the image acquisition device. The aforementioned registration plate can also be placed in other locations within the field of view of the image acquisition device.

[0096] For example, see Figure 1b , Figure 1b This is a partial structural diagram of a C-arm crane with a registration plate installed. Figure 1b In the image, the cylindrical area at the top represents the C-arm image intensifier. Below the C-arm image intensifier are two rings, which represent the double-layer registration plate. The cross area on the left side of the registration plate and the four circular areas adjacent to the cross area represent the tracer mounted on the registration plate, i.e., the first tracer.

[0097] When the registration plate described above can be mounted on an image acquisition device, the acquisition angle changes when the image acquisition device acquires a two-dimensional image. Therefore, the pose of the registration plate also changes, and the pose of the first tracer mounted on the registration plate also changes. In this case, when obtaining the object pose data and the tracer pose data, the object pose data and the tracer pose data at the acquisition time of the two two-dimensional images can be obtained respectively.

[0098] The aforementioned object pose data may include the position data and attitude data of the target object in a preset coordinate system, and the aforementioned tracer pose data may include the position data and attitude data of the first tracer in the preset coordinate system.

[0099] In one embodiment of the present invention, pose data of the target object and the first tracer acquired by the binocular camera can be obtained. The acquired pose data is pose data in a camera coordinate system established based on the binocular camera. Based on this, the aforementioned camera coordinate system can be used as the aforementioned preset coordinate system, thereby obtaining the pose data of the target object acquired by the binocular camera at the time of acquiring the aforementioned two-dimensional image, as object pose data, and obtaining the pose data of the first tracer acquired by the binocular camera at the time of acquiring the aforementioned two-dimensional image, as tracer pose data.

[0100] In addition, by adjusting the camera parameters of the binocular camera, the camera coordinate system can be made to be the same as the coordinate system established by the surgical robot. In this way, the pose data of the target object and the first tracer acquired by the binocular camera at the time of acquisition of the two-dimensional image are obtained, and the object pose data and tracer pose data in the coordinate system established by the surgical robot are obtained.

[0101] The aforementioned preset coordinate system can also be other coordinate systems that can be set manually. In this way, the above object pose data and tracer pose data can also be obtained in other ways. The embodiments of the present invention will not be described in detail here.

[0102] In addition, a second tracer can be placed on the body of the target object. In this way, when the object pose data of the target object is obtained, the pose data of the placed second tracer can be obtained and the pose data can be used as the object pose data.

[0103] Step S102: Plan the screw pose data of the screw that will be used to place the screw on the target object from the two obtained two-dimensional images.

[0104] The screw pose data mentioned above may include the positions of the two ends of the screw in the image coordinate system where the two-dimensional image is located, and may also include data such as the length and diameter of the screw.

[0105] Specifically, the screw pose can be planned in each two-dimensional image to obtain the screw pose data planned in the image coordinate system of each two-dimensional image. Since the two two-dimensional images are acquired from different angles, they can reflect the three-dimensional object information of the target object. Thus, the screw pose data planned in the two two-dimensional images can be understood as the three-dimensional pose data of the screw.

[0106] Screw pose planning in the two two-dimensional images above can be achieved using existing planning techniques, which will not be detailed here.

[0107] Step S103: For each two-dimensional image, identify the markers in the two-dimensional image, and calculate the first transformation relationship between the image coordinate system and the tracer coordinate system based on the image position of the identified markers and the pre-obtained position of the markers in the tracer coordinate system.

[0108] The tracer coordinate system is defined as the coordinate system established based on the first tracer.

[0109] Since the first tracer is mounted on the registration plate, the relative position between the first tracer and the marker in the registration plate is fixed. Thus, after establishing the tracer coordinate system, the position of the marker in the tracer coordinate system can be calibrated.

[0110] Specifically, it can identify the pixels corresponding to the markers in a two-dimensional image, obtain the position of the identified pixels in the two-dimensional image, and then determine the image position of the markers in the image coordinate system based on the obtained position.

[0111] Identifying markers in two-dimensional images can be achieved using existing object recognition technologies, which will not be detailed here.

[0112] Since there are usually multiple markers in the registration plate, and these multiple markers are not coplanar, the image positions of these multiple markers can be identified when recognizing the markers in the two-dimensional image. Thus, when calculating the first transformation relationship mentioned above, the first transformation relationship between the image coordinate system and the tracer coordinate system can be calculated based on the image positions of these multiple markers and their positions in the tracer coordinate system.

[0113] Step S104: Calculate the second transformation relationship between the object coordinate system and the tracer coordinate system based on the object pose data and the tracer pose data.

[0114] The object coordinate system is a coordinate system established based on the target object.

[0115] The above object pose data reflects the pose of the target object in a preset coordinate system. The object coordinate system is a coordinate system established based on the target object. Knowing the pose of the target object in the preset coordinate system means knowing the pose of the object coordinate system in the preset human coordinate system. Therefore, the above object pose data can be understood as the pose data of the object coordinate system in the preset coordinate system. Similarly, the above tracer pose data can be understood as the pose data of the tracer coordinate system in the preset coordinate system.

[0116] Specifically, the second transformation relationship described above can be calculated using either of the following two methods.

[0117] In the first implementation, the relative positional relationship between the target object and the first tracer can be determined based on the object pose data and the tracer pose data. This relative positional relationship can also be regarded as the relative positional relationship between the object coordinate system and the tracer coordinate system. In this way, the relative positional relationship can be converted into a second transformation relationship between the object coordinate system and the tracer coordinate system.

[0118] In the second implementation, since the object pose data can be understood as the pose data of the object coordinate system under the preset coordinate system, the transformation relationship between the object coordinate system and the preset coordinate system can be calculated based on the object pose data. Similarly, the transformation relationship between the tracer coordinate system and the preset coordinate system can be calculated based on the tracer pose data. Thus, based on these two calculated transformation relationships, the second transformation relationship between the object coordinate system and the tracer coordinate system can be determined.

[0119] Furthermore, when the registration plate is installed on the image acquisition device, the object pose data and tracer pose data at the acquisition time of the two two-dimensional images can be obtained respectively. In this way, when calculating the second transformation relationship, one second transformation relationship can be calculated based on the object pose data and tracer pose data corresponding to one acquisition time, and another second transformation relationship can be calculated based on the object pose data and tracer pose data corresponding to another acquisition time.

[0120] Step S105: Based on the first transformation relationship and the second transformation relationship, convert the screw pose data into target pose data in the object coordinate system.

[0121] In one embodiment of the present invention, the screw pose data can be converted by either of the following two implementation methods.

[0122] In the first implementation, the screw pose data is pose data in the image coordinate system. Therefore, according to the first transformation relationship between each image coordinate system and the tracer coordinate system, the screw pose data in the image coordinate system can be converted into the screw pose data in the tracer coordinate system. According to the second transformation relationship between the object coordinate system and the tracer coordinate system, the screw pose data in the tracer coordinate system can be converted into the screw pose data in the object coordinate system again.

[0123] In the second implementation, the first transformation relationship is the transformation relationship between the image coordinate system and the tracer coordinate system, and the second transformation relationship is the transformation relationship between the object coordinate system and the tracer coordinate system. Therefore, based on the first and second transformation relationships, the transformation relationship between the image coordinate system and the object coordinate system can be determined. Thus, based on the determined transformation relationship, the screw pose data in the image coordinate system can be converted into the target pose data in the object coordinate system.

[0124] Step S106: Navigate and locate the surgical robot based on the target pose data.

[0125] The aforementioned target pose data can be understood as the pose data of the screw that is used to place the screw on the target object, planned in the object coordinate system. The object coordinate system is a coordinate system established based on the target object. Based on the target pose data, the surgical robot can be navigated and positioned so that the doctor can use the surgical robot after navigation and positioning to place the screw into the bone of the target object, and make the pose data of the placed screw in the object coordinate system the target pose data.

[0126] Navigating and locating the surgical robot based on the target pose data can be achieved using existing navigation and positioning technologies, which will not be detailed here.

[0127] Specifically, the navigation pose data of the surgical robot in the object coordinate system can be calculated based on the target pose data, and the pose of the surgical robot can be adjusted according to the calculated navigation pose data, thereby realizing the navigation and positioning of the surgical robot.

[0128] When calculating the above navigation pose data, since the relative position between the surgical robot and the inserted screw is usually known, the relative positional relationship between the surgical robot and the inserted screw can be obtained in advance. Thus, after obtaining the above target pose data, the navigation pose data of the surgical robot in the object coordinate system can be calculated based on the obtained relative positional relationship.

[0129] In one embodiment of the present invention, when adjusting the pose of the surgical robot, the current pose data of the surgical robot can be obtained, and the movement path of the surgical robot can be planned based on the current pose data of the surgical robot and the calculated navigation pose data, and then the pose of the surgical robot can be adjusted according to the planned movement path.

[0130] The surgical robot mentioned above can be a robot with a robotic arm. When calculating the navigation pose data, the pose data of the robotic arm of the surgical robot after navigation and positioning in the object coordinate system can be calculated based on the target pose data. Then, the pose of the robotic arm can be adjusted according to the calculated pose data, thereby realizing the navigation and positioning of the surgical robot.

[0131] When a surgical robot is equipped with a robotic arm, the specific implementation method for navigation and positioning of the surgical robot will be discussed later. Figure 2d Steps S106A-S106C in the illustrated embodiment will not be described in detail here.

[0132] As can be seen from the above, when using the solution provided in the embodiments of the present invention to navigate and locate a surgical robot, a first tracer is mounted on the registration plate. Based on the positions of the markers in the registration plate in the image coordinate system and the tracer coordinate system of each two-dimensional image, the first transformation relationship between the image coordinate system and the tracer coordinate system of each two-dimensional image can be accurately calculated. Based on the object pose data and the tracer pose data, the second transformation relationship between the object coordinate system and the tracer coordinate system can be accurately calculated. Thus, based on the first transformation relationship and the second transformation relationship, the screw pose data of the screw planned in the two two-dimensional images can be converted into target pose data in the object coordinate system. Then, based on the target pose data, navigation and positioning of the surgical robot can be achieved.

[0133] Furthermore, since the image acquisition device captures two 2D images from different angles, the two images can reflect the 3D information of the target object. Therefore, planning the screw pose data from these two 2D images can reflect the 3D pose of the inserted screw. This screw pose data can then be converted into target pose data, enabling accurate navigation and positioning of the surgical robot. Therefore, the surgical robot navigation and positioning scheme provided in this invention can improve the accuracy of surgical robot navigation and positioning.

[0134] When applying the surgical robot navigation and positioning scheme provided in this embodiment of the invention, the image acquisition device only needs to acquire two two-dimensional images of the target object, eliminating the need for multiple acquisitions. This minimizes the X-ray radiation received by the surgeon and the target object, especially when the two-dimensional images are X-ray images. Furthermore, in existing surgical robot navigation and positioning schemes, the screw pose data is typically planned only in the three-dimensional image of the target object, requiring the surgeon to manually operate the surgical robot for navigation and positioning. In this scheme, the screw pose data for placing the screw on the target object can be planned in the two two-dimensional images of the target object, and the screw pose data can be converted into target pose data in the object coordinate system. Based on the target pose data, the surgical robot is navigated and positioned. This eliminates the need for surgeon intervention, directly providing the surgical robot with access information. Therefore, applying the surgical robot navigation and positioning scheme provided in this embodiment of the invention reduces the workload of the surgeon and avoids the influence of subjective factors such as the surgeon's experience and condition on the surgery, thereby improving the success rate of the surgery.

[0135] In one embodiment of the present invention, multiple two-dimensional images acquired by the image acquisition device can also be obtained. By planning screw pose data in multiple two-dimensional images, the accuracy of the planned screw pose data can be improved, thereby improving the accuracy of the target pose data converted from the screw pose data. Navigating and positioning the surgical robot based on the more accurate target pose data can improve the accuracy of the surgical robot's navigation and positioning.

[0136] Surgical robots can be equipped with robotic arms, which can be multi-axis collaborative robotic arms such as 4-axis, 5-axis, 6-axis, and 7-axis, or they can be collaborative robotic arms in series or parallel configurations.

[0137] When the surgical robot is equipped with a robotic arm, the registration plate with the first tracer can be installed at the end of the robotic arm. This allows the robotic arm to be positioned so that the registration plate installed at the end of the robotic arm is within the field of view of the image acquisition device when the image acquisition device acquires the first image.

[0138] like Figure 2a As shown, Figure 2a This is a schematic diagram of the end effector of the robotic arm. Figure 2a The two parallel rectangular areas on the right represent the registration plate with a double-layer structure. The first tracer can be mounted on the registration plate. The T-shaped area to the right of these two rectangular areas represents the end effector of the robotic arm, which is equipped with bolts for fixing to other instruments.

[0139] In addition, doctors usually need to use a variety of instruments when performing pin placement surgery on the target object, such as locators, probes, calibrators, puncture tools, and host computers.

[0140] The probe tail can be equipped with a tracer, and the pose of the probe tip and the orientation of the probe axis can be obtained by detecting the pose data of the tracer.

[0141] The aforementioned puncture tools can be manual or electric.

[0142] Manual puncture tools can be openers with tracers. By identifying the pose data of the tracer, the position and axial direction of the opener tip can be obtained.

[0143] The electric puncture tool can be a bone drill with a tracer, which can be equipped with a guide needle or Kirschner wire. By identifying the positional data of the tracer, the position and axial direction of the tip of the installed guide needle or Kirschner wire can be obtained.

[0144] The calibrator can be used during surgery to detect whether there is a deviation in the relative position between the tip and axis of the puncture tool and the tracer mounted on the puncture tool.

[0145] Doctors can combine surgical robots with robotic arms and other instruments to perform surgery.

[0146] In one embodiment of the present invention, a positioner may be fixed to the end of the robotic arm.

[0147] See Figure 2b , Figure 2b This is a schematic diagram of a locator with a tracer. Figure 2b The cross-shaped area and four circular areas on the right side of the instrument shown represent the tracer. The tracer is fixedly connected to the channel of the locator, and instruments such as probes, puncture tools, sleeves, guide needles, and bone drills can be placed in this channel.

[0148] like Figure 2c As shown, Figure 2c This is a schematic diagram of a screw placement surgery procedure, located in... Figure 2c The instruments in the left area represent surgical robots with robotic arms. A positioner is installed at the end of the robotic arm. Figure 2c The equipment in the middle area is a C-arm machine. The target subject can lie flat on the C-arm machine. Figure 2c The instruments in the right-hand area represent a three-dimensional navigation and tracking system, which can track the pose of the target object and instruments such as surgical robots and locators in real time. The aforementioned three-dimensional navigation and tracking system can be any of a variety of navigation systems, such as infrared optical navigation system, visible light optical navigation system, magnetic navigation system, and electric navigation system.

[0149] The following explains how to navigate and locate a surgical robot when it has a robotic arm.

[0150] In one embodiment of the present invention, see Figure 2d The present invention provides a flowchart of a second surgical robot navigation and positioning method. In this embodiment, when the surgical robot is equipped with a robotic arm, the above step S106 can be achieved through the following steps S106A-S106C.

[0151] Step S106A: Obtain the current posture data of the current robotic arm end effector in the base coordinate system and the current positioner pose data of the current positioner in the object coordinate system, wherein the base coordinate system is a coordinate system established based on the base of the robotic arm.

[0152] Specifically, when the first current posture data is obtained, the posture data of the robotic arm can be monitored in real time. The robotic arm includes a base and an end effector. Therefore, based on the posture data of the robotic arm, the relative position between the end effector and the base of the robotic arm can be determined. Based on this relative position, the first current posture data of the end effector in the base coordinate system can be calculated.

[0153] When obtaining the above locator pose data, tracers can be deployed on the locator and the target object. The pose of the tracer on the locator can represent the pose of the locator, and the pose of the tracer on the target object can represent the pose of the target object. In this way, the pose data of the two tracers in the binocular camera coordinate system can be obtained, thereby obtaining the relative position between the current locator and the target object. Based on this relative position, the locator pose data in the object coordinate system can be calculated.

[0154] Step S106B: Based on the target pose data, current pose data, third transformation relationship, and locator pose data, calculate the surgical pose data of the robotic arm end effector in the base coordinate system when the locator points to the target pose data.

[0155] The third transformation relationship is the transformation relationship between the positioner coordinate system and the robotic arm end effector coordinate system.

[0156] The coordinate system of the robotic arm's end effector is: a coordinate system established based on the end effector of the robotic arm.

[0157] Specifically, the current attitude data reflects the transformation relationship between the current end-effector coordinate system and the base coordinate system. The third transformation relationship is the transformation relationship between the end-effector coordinate system and the locator coordinate system. The locator pose data reflects the transformation relationship between the locator coordinate system and the object coordinate system. Therefore, based on the current attitude data, the third transformation relationship, and the locator pose data, the transformation relationship between the base coordinate system and the object coordinate system can be calculated, thus determining the relative positional relationship between the robot arm base and the target object. Based on the calculated transformation relationship between the base coordinate system and the object coordinate system, the target pose data can be transformed into the base coordinate system.

[0158] In addition, when the doctor uses the locator to perform screw placement surgery, the axis of the locator channel is the same straight line as the axis of the inserted screw. Therefore, after obtaining the target pose data in the base coordinate system, the pose data of the axis of the inserted screw in the base coordinate system can be determined. This pose data is used as the pose data of the axis of the locator channel in the base coordinate system. Based on the pose data of the locator channel axis, the pose data of the locator in the base coordinate system is calculated. Combined with the third transformation relationship mentioned above, the pose data of the robotic arm end effector in the base coordinate system is calculated. The calculated pose data is the surgical pose data mentioned above.

[0159] Step S106C: Navigate and position the surgical robot based on the surgical pose data.

[0160] Specifically, surgical pose data can be understood as the pose data of the robotic arm end effector planned in the base coordinate system. After obtaining the surgical pose data, the pose of the robotic arm can be adjusted according to the surgical pose data so that the pose data of the robotic arm end effector is the planned pose, thereby realizing the navigation and positioning of the surgical robot.

[0161] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, if the surgical robot has a robotic arm and a locator is fixed at the end of the robotic arm, the axis of the locator's channel needs to be aligned with the axis of the inserted screw during surgery. Therefore, obtaining the target pose data of the screw also yields the planned pose data of the locator. Since the locator is fixed at the end of the robotic arm, the pose data of the robotic arm's end can be obtained based on the positional relationship between the locator and the robotic arm's end. Thus, based on the pose data of the robotic arm's end, accurate navigation and positioning of the surgical robot can be achieved. Therefore, the surgical robot navigation and positioning solution provided in the embodiments of the present invention can improve the accuracy of surgical robot navigation and positioning.

[0162] In one embodiment of the present invention, after navigating and locating the surgical robot, the doctor can insert a puncture tool into the target body through the locator channel. By identifying the pose data of the tracer installed on the puncture tool in the object coordinate system, the doctor can obtain pose data such as the tip position and axis orientation of the puncture tool in the object coordinate system. Based on the transformation relationship between the image coordinate system and the object coordinate system, the obtained pose data of the puncture tool is converted into pose data in the image coordinate system and displayed in real time on a two-dimensional image for the doctor to observe.

[0163] Furthermore, as can be seen from the above embodiments, a second tracer can be placed on the target object's body, and the pose data of the placed second tracer can be used as the target object's pose data. This allows for real-time capture of the target object's position information and display in a two-dimensional image. When the target object moves, the target object's pose data can be updated in the two-dimensional image, and the position of the puncture tool can be compensated based on the positional difference before and after the target object's movement, thereby ensuring navigation and positioning accuracy.

[0164] The screw pose data planned in the two 2D images above for placing the screw on the target object may not be the optimal pose data for the placed screw. Therefore, after obtaining the screw pose data, the following can be used... Figure 3 In the illustrated embodiment, steps S107-S109 adjust the planned screw pose data to improve the accuracy of the screw pose data.

[0165] In one embodiment of the present invention, see Figure 3 The present invention provides a flowchart of a third surgical robot navigation and positioning method. In this embodiment, before converting the screw pose data into target pose data in the object coordinate system, the above method further includes the following steps S107-S109.

[0166] Step S107: Show the user the planned 2D images.

[0167] Specifically, based on the screw pose data obtained from the planning, the pixel points of the inserted screw can be determined in the two-dimensional image. This allows for pixel value adjustment, marking, and other processing of the determined pixels, and the processed two-dimensional image can be displayed to the user so that the user can know the position of the inserted screw in the two-dimensional image.

[0168] Step S108: Obtain pose adjustment information input by the user based on the displayed two-dimensional image.

[0169] After viewing the displayed 2D image, users can determine whether the planned screw pose data meets their expectations. If not, they can input pose adjustment information to adjust the screw pose data.

[0170] Step S109: Adjust the screw pose data according to the pose adjustment information.

[0171] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, the planned two-dimensional images can be displayed to the user. The user can view the two-dimensional images and input pose adjustment information. Based on the pose adjustment information input by the user, the screw pose data can be adjusted, which can improve the accuracy of the planned screw pose data and make the adjusted screw pose data more in line with the user's surgical method, thereby improving the success rate of the surgery.

[0172] In one embodiment of the present invention, multiple two-dimensional images acquired by an image acquisition device can be obtained. When planning the screw pose, two two-dimensional images can be randomly selected from the multiple acquired two-dimensional images for screw pose planning. After obtaining the screw pose data, three-dimensional screw pose data can be synthesized based on the planned screw pose data. Then, based on the acquisition angle of each two-dimensional image, the synthesized three-dimensional screw pose data is projected onto each acquired two-dimensional image, and the projected two-dimensional image is displayed to the user so that the user can view whether the planned screw pose meets expectations from multiple angles.

[0173] When navigating and locating a surgical robot, mechanical or other errors may exist, causing the robot's pose after navigation and positioning to be different from the expected pose.

[0174] Therefore, in one embodiment of the present invention, see... Figure 4 The present invention provides a flowchart of a fourth surgical robot navigation and positioning method. In this embodiment, the above step S106 can be achieved through the following steps S106E-S106K.

[0175] Step S106E: Obtain the first pose data of the surgical robot in the object coordinate system.

[0176] Specifically, the binocular camera can observe the poses of the surgical robot and the target object, thus obtaining the relative positional relationship between the surgical robot and the target object. Based on this relative positional relationship, the first pose data of the surgical robot in the object coordinate system can be calculated.

[0177] Trackers can also be deployed on the surgical robot and the target object. This allows us to obtain the pose data of the tracker on the surgical robot as the pose data of the surgical robot, and obtain the pose data of the tracker on the target object as the pose data of the target object.

[0178] Step S106F: Navigate and locate the surgical robot based on the first pose data and the target pose data.

[0179] Specifically, as can be seen from step S106 above, the relative position between the surgical robot and the inserted screw is usually known. Therefore, the relative positional relationship between the surgical robot and the inserted screw can be obtained in advance. After obtaining the target pose data, the pose data of the surgical robot in the object coordinate system can be calculated based on the obtained relative positional relationship. This pose data can be understood as the navigation pose data of the surgical robot planned in the object coordinate system. After calculating the pose data, the movement path and attitude change data of the surgical robot can be planned based on the first pose data and the calculated pose data. Therefore, the surgical robot can be controlled to perform navigation and positioning based on the planned movement path and attitude change data.

[0180] Step S106G: Obtain the second pose data of the surgical robot in the object coordinate system after navigation and positioning.

[0181] The implementation method for obtaining the second pose data can be found in the above embodiments, and will not be repeated here.

[0182] Step S106H: Based on the second pose data, calculate the fifth pose data of the screw for placement based on the current pose of the surgical robot.

[0183] After obtaining the second pose data, based on the relative positional relationship between the surgical robot and the inserted screw, the pose data of the screw used to insert the screw into the target object can be calculated when the pose data of the surgical robot is the second pose data. This pose data is the fifth pose data mentioned above.

[0184] Step S106I: Calculate the pose difference between the fifth pose data and the target pose data.

[0185] The pose differences mentioned above can be represented in various ways, and the methods for calculating pose differences under different representations are different.

[0186] In one embodiment of the present invention, the above pose difference can be calculated by any one of the following three implementation methods.

[0187] In the first implementation, the region of the screw in the object coordinate system can be determined based on the fifth pose data and the target pose data, and the non-overlapping region between the two determined regions can be identified, and the non-overlapping region can be used as the pose difference mentioned above.

[0188] In the second implementation, after identifying the non-overlapping region, the ratio of the non-overlapping region to the entire screw region can be calculated, and this ratio can be used as the pose difference.

[0189] In the third implementation, the axis of the screw can be calculated based on the fifth pose data and the target pose data, and the included angle between the two screw axes can be calculated, and the included angle can be used as the pose difference mentioned above.

[0190] In addition, the above pose differences can also be represented in other ways, and similarly, the above pose differences can be calculated in other ways, which will not be detailed here.

[0191] Step S106J: Determine whether the pose difference is greater than the preset difference threshold. If yes, use the second pose data as the first pose data and return to step S106F above; if no, execute step S106K.

[0192] The aforementioned difference threshold can be set manually.

[0193] If the pose difference is greater than the preset difference threshold, it means that the pose of the surgical robot after navigation and positioning is significantly different from the expected pose. In this case, navigation and positioning need to be re-performed based on the current pose data of the surgical robot. Since the current pose data of the surgical robot is the second pose data, the second pose data can be used as the first pose data, and the process returns to step S106F. If the pose difference is less than or equal to the difference threshold, it means that the pose of the surgical robot after navigation and positioning is relatively small compared with the expected pose. It can be considered that the surgical robot after navigation and positioning has been navigated and positioned to the expected pose. In this case, step S106K is executed.

[0194] Step S106K: End surgical robot navigation and positioning.

[0195] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, it is determined whether the pose difference between the fifth pose data and the target pose data is greater than a preset difference threshold. If the pose difference is greater than the difference threshold, the surgical robot is navigated and positioned again until the pose difference is less than or equal to the difference threshold. This can eliminate the influence of the above error on the surgical robot navigation and positioning, thereby improving the accuracy of navigation and positioning of the surgical robot.

[0196] After navigating and locating the surgical robot, the user can use the located surgical robot to perform surgery on the target object. During the surgery, the user can also obtain the pose data of the surgical instruments used in the image coordinate system of the first image in real time, and display the first image containing the pose data to the user in real time to realize the visualization of pin placement.

[0197] In one embodiment of the present invention, see Figure 5The present invention provides a flowchart of a fifth surgical robot navigation and positioning method. In this embodiment, after navigating and positioning the surgical robot according to the target pose data, the present invention further includes the following steps S110-S112.

[0198] Step S110: Obtain the instrument pose data in the object coordinate system in real time when performing surgery on the target object.

[0199] Specifically, a tracer can be installed on the aforementioned surgical instruments. This way, when performing surgery on the target object, the pose data of the tracer installed on the surgical instruments in the object coordinate system can be obtained in real time, serving as the instrument pose data of the surgical instruments in the object coordinate system.

[0200] Step S111: Based on the first transformation relationship and the second transformation relationship, convert the instrument pose data into the sixth pose data in the image coordinate system.

[0201] Specifically, according to the second transformation relationship, the instrument pose data in the object coordinate system can be transformed to the tracer coordinate system. According to the first transformation relationship, the instrument pose data in the tracer coordinate system can be transformed to the image coordinate system, thereby obtaining the sixth pose data of the surgical instrument in the image coordinate system.

[0202] Step S112: Display a two-dimensional image containing the sixth pose data to the user in real time.

[0203] This step is similar to step S107 above, and will not be described in detail here.

[0204] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, after the surgical robot navigation and positioning is completed, the position and pose data of the surgical instruments can be displayed in real time in a two-dimensional image. In this way, the user can view the displayed position and pose data and make real-time adjustments based on the displayed position and pose data. Therefore, the surgical navigation and positioning solution provided in the embodiments of the present invention can improve the safety and accuracy of the surgery.

[0205] Corresponding to the above-described surgical robot navigation and positioning method, this embodiment of the invention also provides a surgical robot navigation and positioning device.

[0206] In one embodiment of the present invention, see Figure 6 A schematic diagram of the structure of a first surgical robot navigation and positioning device is provided, the device comprising:

[0207] The data acquisition module 601 is used to acquire two two-dimensional images acquired by the image acquisition device, and to acquire the object pose data of the target object and the tracer pose data of the first tracer in the same preset coordinate system. The field of view of the image acquisition device includes the target object and a registration plate with markers. The image acquisition device acquires each two-dimensional image at a different acquisition angle. The first tracer is mounted on the registration plate.

[0208] The screw planning module 602 is used to plan the screw pose data of the screw that will be used to place the screw on the target object from the two obtained two-dimensional images.

[0209] The first calculation module 603 is used to identify the marker in each two-dimensional image, and calculate a first transformation relationship between the image coordinate system of the two-dimensional image and the tracer coordinate system based on the image position of the identified marker and the pre-obtained position of the marker in the tracer coordinate system. The tracer coordinate system is a coordinate system established based on the first tracer.

[0210] The second calculation module 604 is used to calculate a second transformation relationship between the object coordinate system and the tracer coordinate system based on the object pose data and the tracer pose data, wherein the object coordinate system is a coordinate system established based on the target object;

[0211] The first conversion module 605 is used to convert the screw pose data into target pose data in the object coordinate system according to the first conversion relationship and the second conversion relationship.

[0212] The navigation and positioning module 606 is used to navigate and position the surgical robot based on the target pose data.

[0213] As can be seen from the above, when using the solution provided in the embodiments of the present invention to navigate and locate a surgical robot, a first tracer is mounted on the registration plate. Based on the positions of the markers in the registration plate in the image coordinate system and the tracer coordinate system of each two-dimensional image, the first transformation relationship between the image coordinate system and the tracer coordinate system of each two-dimensional image can be accurately calculated. Based on the object pose data and the tracer pose data, the second transformation relationship between the object coordinate system and the tracer coordinate system can be accurately calculated. Thus, based on the first transformation relationship and the second transformation relationship, the screw pose data of the screw planned in the two two-dimensional images can be converted into target pose data in the object coordinate system. Then, based on the target pose data, navigation and positioning of the surgical robot can be achieved.

[0214] Furthermore, since the image acquisition device captures two 2D images from different angles, the two images can reflect the 3D information of the target object. Therefore, planning the screw pose data from these two 2D images can reflect the 3D pose of the inserted screw. This screw pose data can then be converted into target pose data, enabling accurate navigation and positioning of the surgical robot. Therefore, the surgical robot navigation and positioning scheme provided in this invention can improve the accuracy of surgical robot navigation and positioning.

[0215] When applying the surgical robot navigation and positioning scheme provided in this embodiment of the invention, the image acquisition device only needs to acquire two two-dimensional images of the target object, eliminating the need for multiple acquisitions. This minimizes the X-ray radiation received by the surgeon and the target object, especially when the two-dimensional images are X-ray images. Furthermore, in existing surgical robot navigation and positioning schemes, the screw pose data is typically planned only in the three-dimensional image of the target object, requiring the surgeon to manually operate the surgical robot for navigation and positioning. In this scheme, the screw pose data for placing the screw on the target object can be planned in the two two-dimensional images of the target object, and the screw pose data can be converted into target pose data in the object coordinate system. Based on the target pose data, the surgical robot is navigated and positioned. This eliminates the need for surgeon intervention, directly providing the surgical robot with access information. Therefore, applying the surgical robot navigation and positioning scheme provided in this embodiment of the invention reduces the workload of the surgeon and avoids the influence of subjective factors such as the surgeon's experience and condition on the surgery, thereby improving the success rate of the surgery.

[0216] In one embodiment of the present invention, see Figure 7 A schematic diagram of a second type of surgical robot navigation and positioning device is provided. In this embodiment, when the surgical robot has a robotic arm and a locator is fixed at the end of the robotic arm, the navigation and positioning module 606 includes:

[0217] The data acquisition submodule 606A is used to acquire the current posture data of the end effector of the robotic arm in the base coordinate system and the current positioner pose data of the positioner in the object coordinate system, wherein the base coordinate system is a coordinate system established based on the base of the robotic arm;

[0218] The pose calculation submodule 606B is used to calculate the surgical pose data of the robotic arm end effector in the base coordinate system when the locator points to the target pose data, based on the target pose data, current pose data, third transformation relationship, and locator pose data. The third transformation relationship is the transformation relationship between the locator coordinate system and the robotic arm end effector coordinate system. The base coordinate system is a coordinate system established based on the base of the robotic arm. The robotic arm end effector coordinate system is a coordinate system established based on the end effector of the robotic arm. The robotic arm end effector pose data is the pose data of the robotic arm end effector coordinate system in the base coordinate system. The locator pose data is the pose data of the locator in the object coordinate system.

[0219] The navigation and positioning submodule 606C is used to navigate and position the surgical robot based on the surgical pose data.

[0220] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, if the surgical robot has a robotic arm and a locator is fixed at the end of the robotic arm, the axis of the locator's channel needs to be aligned with the axis of the inserted screw during surgery. Therefore, obtaining the target pose data of the screw also yields the planned pose data of the locator. Since the locator is fixed at the end of the robotic arm, the pose data of the robotic arm's end can be obtained based on the positional relationship between the locator and the robotic arm's end. Thus, based on the pose data of the robotic arm's end, accurate navigation and positioning of the surgical robot can be achieved. Therefore, the surgical robot navigation and positioning solution provided in the embodiments of the present invention can improve the accuracy of surgical robot navigation and positioning.

[0221] In one embodiment of the present invention, see Figure 8 A schematic diagram of a third type of surgical robot navigation and positioning device is provided. In this embodiment, the device further includes:

[0222] Image display module 607 is used to display planned two-dimensional images to the user before converting screw pose data into target pose data in the object coordinate system;

[0223] The information acquisition module 608 is used to acquire the pose adjustment information input by the user based on the displayed two-dimensional image;

[0224] The pose adjustment module 609 is used to adjust the screw pose data according to the pose adjustment information.

[0225] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, the planned two-dimensional images can be displayed to the user. The user can view the two-dimensional images and input pose adjustment information. Based on the pose adjustment information input by the user, the screw pose data can be adjusted, which can improve the accuracy of the planned screw pose data and make the adjusted screw pose data more in line with the user's surgical method, thereby improving the success rate of the surgery.

[0226] In one embodiment of the present invention, the navigation and positioning module 606 is specifically used for:

[0227] Obtain the first pose data of the surgical robot in the object coordinate system;

[0228] The surgical robot is navigated and positioned based on the first pose data and the target pose data.

[0229] After navigation and positioning, the surgical robot obtains its second pose data in the object coordinate system.

[0230] Based on the second pose data, calculate the fifth pose data of the screw for placement based on the current pose of the surgical robot;

[0231] Calculate the pose difference between the fifth pose data and the target pose data;

[0232] If the pose difference is greater than a preset difference threshold, the second pose data is used as the first pose data, and the process of obtaining the navigation and positioning steps for the surgical robot is returned until the calculated pose difference is less than or equal to the difference threshold.

[0233] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, it is determined whether the pose difference between the fifth pose data and the target pose data is greater than a preset difference threshold. If the pose difference is greater than the difference threshold, the surgical robot is navigated and positioned again until the pose difference is less than or equal to the difference threshold. This can eliminate the influence of the above error on the surgical robot navigation and positioning, thereby improving the accuracy of navigation and positioning of the surgical robot.

[0234] In one embodiment of the present invention, see Figure 9 A schematic diagram of a fourth type of surgical robot navigation and positioning device is provided. In this embodiment, the device further includes:

[0235] The pose acquisition module 610 is used to obtain, in real time, the instrument pose data of the surgical instruments in the object coordinate system when performing surgery on the target object after the surgical robot is navigated and positioned according to the target pose data.

[0236] The second conversion module 611 is used to convert the instrument pose data into a sixth pose data in the image coordinate system according to the first conversion relationship and the second conversion relationship.

[0237] The real-time display module 612 is used to display a two-dimensional image containing the sixth pose data to the user in real time.

[0238] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, after the surgical robot navigation and positioning is completed, the position and pose data of the surgical instruments can be displayed in real time in a two-dimensional image. In this way, the user can view the displayed position and pose data and make real-time adjustments based on the displayed position and pose data. Therefore, the surgical navigation and positioning solution provided in the embodiments of the present invention can improve the safety and accuracy of the surgery.

[0239] Corresponding to the above-described surgical robot navigation and positioning method, this embodiment of the invention also provides a surgical robot navigation and positioning system.

[0240] In one embodiment of the present invention, see Figure 10 A schematic diagram of a surgical robot navigation and positioning system is provided. The system includes a three-dimensional navigation and tracking device 1001, a surgical robot 1002 with a robotic arm, an object tracer 1003 for displaying the pose data of the target object, a locator 1004, a first tracer 1005, a registration plate with markers 1006, an image acquisition device 1007, and a host computer 1008.

[0241] The locator 1004 is installed at the end of the robotic arm of the surgical robot 1002, and the first tracer 1005 is installed on the registration plate 1006.

[0242] The image acquisition device 1007 is used to acquire two two-dimensional images and send the acquired two-dimensional images to the host computer 1008. The field of view of the image acquisition device 1007 includes the target object and the registration plate 1006. The acquisition angle of each two-dimensional image acquired by the image acquisition device 1007 is different.

[0243] The registration plate 1006 is mounted on the image acquisition device 1007 and is located within the field of view of the image acquisition device.

[0244] In addition, the registration plate 1006 can also be installed at the end of the robotic arm of the surgical robot 1002, so that when the image acquisition device acquires images, the position of the end of the robotic arm of the surgical robot 1002 can be adjusted so that the registration plate 1006 is within the field of view of the image acquisition device 1007.

[0245] The three-dimensional navigation and tracking device 1001 is used to obtain the object pose data shown by the object tracer 1003 and the tracer pose data of the first tracer 1005 in the same preset coordinate system, and to send the object pose data and the tracer pose data to the host computer 1008.

[0246] The host computer 1008 is used to control the surgical robot 1002 to perform navigation and positioning according to the method steps mentioned in the above method embodiments.

[0247] In one embodiment of the present invention, the above-mentioned three-dimensional navigation tracking device 1001 may be a visible light optical navigation device, a magnetic navigation device, an electric navigation device, or other three-dimensional navigation devices.

[0248] In one embodiment of the present invention, the image acquisition device 1007 is a C-arm camera.

[0249] As can be seen from the above, when using the solution provided in the embodiments of the present invention for surgical robot navigation and positioning, if a locator is installed at the end of the surgical robot's robotic arm, the axis of the locator's channel needs to be aligned with the axis of the inserted screw during surgery. Therefore, after obtaining the target pose data of the screw, the locator's pose data can be accurately calculated based on the target pose data and the positional relationship between the locator channel and the screw. Combined with the first positional relationship between the locator and the end of the robotic arm, the end pose data of the robotic arm can be accurately calculated. Based on the relatively accurate end pose data and the base pose data, the surgical pose data can be calculated. The accuracy of navigation and positioning of the surgical robot can be improved based on the calculated surgical pose data.

[0250] This invention also provides an electronic device, such as... Figure 11 As shown, it includes a processor 1101, a communication interface 1102, a memory 1103, and a communication bus 1104. The processor 1101, communication interface 1102, and memory 1103 communicate with each other via the communication bus 1104.

[0251] Memory 1103 is used to store computer programs;

[0252] When processor 1101 executes the program stored in memory 1103, it performs the following steps:

[0253] Two two-dimensional images are acquired by an image acquisition device, and object pose data of the target object and tracer pose data of the first tracer are obtained in the same preset coordinate system. The field of view of the image acquisition device includes the target object and a registration plate with markers. The image acquisition device acquires each two-dimensional image at a different acquisition angle, and the first tracer is mounted on the registration plate.

[0254] Screw pose data for the screws used to place the screws on the target object are planned from the two obtained two-dimensional images;

[0255] For each two-dimensional image, the marker in the two-dimensional image is identified. Based on the image position of the identified marker and the pre-obtained position of the marker in the tracer coordinate system, a first transformation relationship between the image coordinate system of the two-dimensional image and the tracer coordinate system is calculated. The tracer coordinate system is a coordinate system established based on the first tracer.

[0256] Based on the object pose data and the tracer pose data, a second transformation relationship between the object coordinate system and the tracer coordinate system is calculated, wherein the object coordinate system is a coordinate system established based on the target object;

[0257] Based on the first and second transformation relationships, the screw pose data is converted into target pose data in the object coordinate system;

[0258] The surgical robot is navigated and positioned based on the target pose data.

[0259] Other schemes for the surgical robot navigation and positioning implemented by the processor 1101 executing the program stored in the memory 1103 are the same as those mentioned in the aforementioned method embodiments, and will not be repeated here.

[0260] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0261] The communication interface is used for communication between the aforementioned electronic devices and other devices.

[0262] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0263] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0264] In another embodiment of the present invention, a computer-readable storage medium is also provided, which stores a computer program that, when executed by a processor, implements the steps of any of the above-described surgical robot navigation and positioning methods.

[0265] In another embodiment of the present invention, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to execute any of the surgical robot navigation and positioning methods described above.

[0266] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid state disk (SSD)).

[0267] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0268] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments of apparatus, electronic devices, computer-readable storage media, and computer program products are basically similar to the method embodiments, and therefore the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0269] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A navigation and positioning method for a surgical robot, characterized in that, The method includes: Two two-dimensional images are acquired by an image acquisition device, and object pose data of the target object and tracer pose data of a first tracer are obtained in the same preset coordinate system. The field of view of the image acquisition device includes the target object and a registration plate with markers. The image acquisition device acquires each two-dimensional image at a different angle. The first tracer is mounted on the registration plate. The relative position between the first tracer and the markers on the registration plate is fixed. Screw pose data for the screws used to place the screws on the target object are planned from the two obtained two-dimensional images; For each two-dimensional image, the marker in the two-dimensional image is identified. Based on the image position of the identified marker and the pre-obtained position of the marker in the tracer coordinate system, a first transformation relationship between the image coordinate system of the two-dimensional image and the tracer coordinate system is calculated. The tracer coordinate system is a coordinate system established based on the first tracer. Based on the object pose data and the tracer pose data, a second transformation relationship between the object coordinate system and the tracer coordinate system is calculated, wherein the object coordinate system is a coordinate system established based on the target object; Based on the first and second transformation relationships, the screw pose data is converted into target pose data in the object coordinate system; The surgical robot is navigated and positioned based on the target pose data.

2. The method according to claim 1, characterized in that, When the surgical robot is equipped with a robotic arm and a locator is fixed to the end of the robotic arm, the navigation and positioning of the surgical robot based on the target pose data includes: Obtain the current posture data of the end effector of the robotic arm in the base coordinate system and the current position data of the positioner in the object coordinate system, wherein the base coordinate system is a coordinate system established based on the base of the robotic arm; Based on the target pose data, current pose data, third transformation relationship, and locator pose data, the surgical pose data of the robotic arm end effector in the base coordinate system when the locator points to the target pose data is calculated. The third transformation relationship is the transformation relationship between the locator coordinate system and the robotic arm end effector coordinate system. The robotic arm end effector coordinate system is a coordinate system established based on the end effector of the robotic arm. The robotic arm end effector pose data is the pose data of the robotic arm end effector coordinate system in the base coordinate system. The locator pose data is the pose data of the locator in the object coordinate system. The surgical robot is navigated and positioned based on the surgical pose data.

3. The method according to claim 1 or 2, characterized in that, Before converting the screw pose data into target pose data in the object coordinate system, the method further includes: Show users the various 2D images after the planning; Obtain the pose adjustment information input by the user based on the displayed two-dimensional image; The screw pose data is adjusted based on the pose adjustment information.

4. The method according to claim 1, characterized in that, The step of navigating and locating the surgical robot based on the target pose data includes: Obtain the first pose data of the surgical robot in the object coordinate system; The surgical robot is navigated and positioned based on the first pose data and the target pose data. After navigation and positioning, the surgical robot obtains its second pose data in the object coordinate system. Based on the second pose data, calculate the fifth pose data of the screw for placement based on the current pose of the surgical robot; Calculate the pose difference between the fifth pose data and the target pose data; If the pose difference is greater than a preset difference threshold, the second pose data is used as the first pose data, and the process of obtaining the navigation and positioning steps for the surgical robot is returned until the calculated pose difference is less than or equal to the difference threshold.

5. The method according to claim 1 or 2, characterized in that, After navigating and locating the surgical robot based on the target pose data, the method further includes: Real-time acquisition of surgical instrument pose data in the object coordinate system during surgery on the target object; Based on the first transformation relationship and the second transformation relationship, the instrument pose data is converted into the sixth pose data in the image coordinate system; The system displays a two-dimensional image containing the sixth pose data to the user in real time.

6. A surgical robot navigation and positioning device, characterized in that, The device includes: The data acquisition module is used to acquire two two-dimensional images captured by the image acquisition device, and to acquire the object pose data of the target object and the tracer pose data of the first tracer in the same preset coordinate system. The field of view of the image acquisition device includes the target object and a registration plate with markers. The image acquisition device acquires each two-dimensional image at a different angle. The first tracer is mounted on the registration plate. The relative position between the first tracer and the markers on the registration plate is fixed. The screw planning module is used to plan the screw pose data for placing the screw on the target object from the two obtained two-dimensional images. The first calculation module is used to identify the marker in each two-dimensional image, and calculate a first transformation relationship between the image coordinate system of the two-dimensional image and the tracer coordinate system based on the image position of the identified marker and the pre-obtained position of the marker in the tracer coordinate system. The tracer coordinate system is a coordinate system established based on the first tracer. The second calculation module is used to calculate a second transformation relationship between the object coordinate system and the tracer coordinate system based on the object pose data and the tracer pose data, wherein the object coordinate system is a coordinate system established based on the target object; The first conversion module is used to convert the screw pose data into target pose data in the object coordinate system according to the first conversion relationship and the second conversion relationship. The navigation and positioning module is used to navigate and position the surgical robot based on the target pose data.

7. The apparatus according to claim 6, characterized in that, In the case where the surgical robot is equipped with a robotic arm and a locator is fixed to the end of the robotic arm, the navigation and positioning module includes: The data acquisition submodule is used to acquire the current posture data of the current end effector of the robotic arm in the base coordinate system and the current positioner pose data of the current positioner in the object coordinate system, wherein the base coordinate system is a coordinate system established based on the base of the robotic arm; The pose calculation submodule is used to calculate the surgical pose data of the robotic arm end effector in the base coordinate system when the locator points to the target pose data, based on the target pose data, current pose data, third transformation relationship, and locator pose data. The third transformation relationship is the transformation relationship between the locator coordinate system and the robotic arm end effector coordinate system. The robotic arm end effector coordinate system is a coordinate system established based on the end effector of the robotic arm. The robotic arm end effector pose data is the pose data of the robotic arm end effector coordinate system in the base coordinate system. The locator pose data is the pose data of the locator in the object coordinate system. The navigation and positioning submodule is used to navigate and position the surgical robot based on the surgical pose data.

8. The apparatus according to claim 6 or 7, characterized in that, The device further includes: The image display module is used to display the planned two-dimensional images to the user before converting the screw pose data into target pose data in the object coordinate system; The information acquisition module is used to acquire the pose adjustment information input by the user based on the displayed two-dimensional image; The pose adjustment module is used to adjust the screw pose data according to the pose adjustment information.

9. The apparatus according to claim 6, characterized in that, The navigation and positioning module is specifically used for: Obtain the first pose data of the surgical robot in the object coordinate system; The surgical robot is navigated and positioned based on the first pose data and the target pose data. After navigation and positioning, the surgical robot obtains its second pose data in the object coordinate system. Based on the second pose data, calculate the fifth pose data of the screw for placement based on the current pose of the surgical robot; Calculate the pose difference between the fifth pose data and the target pose data; If the pose difference is greater than a preset difference threshold, the second pose data is used as the first pose data, and the process of obtaining the navigation and positioning steps for the surgical robot is returned until the calculated pose difference is less than or equal to the difference threshold.

10. The apparatus according to claim 6 or 7, characterized in that, The device further includes: The pose acquisition module is used to obtain, in real time, the instrument pose data of the surgical instruments in the object coordinate system when performing surgery on the target object after the surgical robot is navigated and positioned according to the target pose data; The second conversion module is used to convert the instrument pose data into a sixth pose data in the image coordinate system according to the first conversion relationship and the second conversion relationship. The real-time display module is used to display a two-dimensional image containing the sixth pose data to the user in real time.

11. A surgical robot navigation and positioning system, characterized in that, The system includes a three-dimensional navigation and tracking device, a surgical robot with a robotic arm, an object tracer for displaying the pose data of the target object, a locator, a first tracer, a registration plate with markers, an image acquisition device, and a host computer. The locator is mounted on the end of the robotic arm of the surgical robot, and the first tracer is mounted on the registration plate; the relative position between the first tracer and the markers in the registration plate is fixed. The image acquisition device is used to acquire two two-dimensional images and send the acquired two-dimensional images to the host computer. The field of view of the image acquisition device includes the target object and the registration plate. The acquisition angle of each two-dimensional image is different. The three-dimensional navigation and tracking device is used to obtain the object pose data shown by the object tracker and the tracker pose data of the first tracker in the same preset coordinate system, and to send the object pose data and the tracker pose data to the host computer. The host computer is used to control the surgical robot to perform navigation and positioning according to any one of the method steps in claims 2-5.

12. An electronic device, characterized in that, It includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the steps of the method described in any one of claims 1-5.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1-5.

Images