Robot system guiding method, device, computer equipment and storage medium

Abstract

The present application relates to a robot system guidance method, device, computer equipment, storage medium and computer program product. The method comprises: determining whether the tracker obtains the first real-time posture of the first mark set at the end of the robot arm; if the first real-time posture is not obtained, determining whether the tracker obtains the fixed posture of the second mark set on the trolley; if the fixed posture is obtained, entering the first navigation mode, in which the second real-time posture of the end of the robot arm is re-obtained according to the fixed posture, and a first update path is formed; if the fixed posture is not obtained, entering the second navigation mode, in which the second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment; sending the first update path or the second update path to the controller to control the movement of the robot arm. The use of this method can avoid interruption of robot arm control.

Description

Technical Field

[0001] The present application relates to the field of robot control technology, and in particular to a robot system guidance method, device, computer equipment, storage medium and computer program product. Background Art

[0002] The existing robot system guidance method uses a tracker to capture the optical marker at the end of the robot arm. The robot system calculates the position and posture of the robot arm end based on the position and posture information of the optical marker, and then controls the movement of the robot arm based on the position and posture of the robot arm end and the planned action.

[0003] However, when the space of the target object is narrow, the optical marker at the end of the robot arm may be blocked easily during the movement of the robot arm. The robot system loses the position information of the optical marker and cannot calculate the position of the end of the robot arm, which will cause the control interruption of the robot arm. Summary of the invention

[0004] Based on this, it is necessary to provide a robot system guidance method, device, computer equipment, computer-readable storage medium and computer program product that can solve the problem of robot arm control interruption mentioned above.

[0005] In a first aspect, the present application provides a method for guiding a robot system. The robot system includes a robot arm, a trolley, a controller and a tracker, and the robot arm is arranged on the trolley. The method includes:

[0006] Determine whether the tracker obtains a first real-time position and posture of a first marker set at the end of the robotic arm;

[0007] If the first real-time position and posture is not obtained, determining whether the tracker obtains the fixed position and posture of the second marker set on the trolley;

[0008] If a fixed posture is obtained, the first navigation mode is entered. In the first navigation mode, the second real-time posture of the end of the robot arm is re-obtained according to the fixed posture, and a first update path is formed;

[0009] If the fixed posture is not obtained, the second navigation mode is entered. In the second navigation mode, a second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment;

[0010] The first updated path or the second updated path is sent to the controller to control the movement of the robot arm.

[0011] In one embodiment, the robot system guidance method further includes:

[0012] determining whether the tracker obtains a third real-time position and posture of a third marker set on the target object;

[0013] If the third real-time posture is not obtained, the robot arm is controlled to stop moving;

[0014] If the third real-time posture is obtained and the first real-time posture is obtained, the third navigation mode is entered. In the third navigation mode, the movement of the robotic arm is controlled according to the expected posture planned before surgery and the first real-time posture; wherein the expected posture is obtained according to the planning in the medical image.

[0015] In one embodiment, before the step of determining whether the tracker has acquired the first real-time position and posture of the first marker set at the end of the robot arm, the step further includes:

[0016] The desired posture is registered in the target object coordinate system through the tracker and the pre-registered probe to obtain the true posture of the desired posture in the target object coordinate system.

[0017] In one embodiment, a mapping device is pre-installed in a target area of ​​a target object, the mapping device includes a first feature point and a first registration point, and a fixed positional relationship exists between any first registration point and any first feature point; the medical image includes the first feature point; the desired posture is registered in the target object coordinate system through a tracker and a pre-registered probe to obtain a real posture of the desired posture in the target object coordinate system, including:

[0018] When the tip of the probe contacts the first registration point and the tracker tracks the marker of the probe and the third marker respectively, the position and posture of the first registration point and the static position and posture of the third marker are respectively obtained;

[0019] Establishing a target region coordinate system according to the position and posture of the first registration point, and establishing a target object coordinate system according to the static position and posture of the third marker, and calculating a first transformation matrix from the target region coordinate system to the target object coordinate system;

[0020] Calculate the position and posture of the first feature point in the target area coordinate system according to the position and posture of the first registration point;

[0021] Obtaining the position and posture of the first feature point in the medical image coordinate system, and calculating a second transformation matrix from the medical image coordinate system to the target region coordinate system according to the position and posture of the first feature point in the target region coordinate system and the position and posture of the first feature point in the medical image coordinate system;

[0022] The actual posture of the desired posture in the target object coordinate system is calculated according to the desired posture planned before surgery, the second transformation matrix and the first transformation matrix.

[0023] In one embodiment, if the fixed posture is not obtained, the robot system guidance method further includes:

[0024] Determine the pose of the desired pose in the tracker coordinate system at the current moment according to the third real-time pose of the third marker set on the target object and the real pose of the desired pose in the target object coordinate system; wherein the third real-time pose of the third marker represents the real-time pose of the target object in the tracker coordinate system at the current moment;

[0025] Acquire the online pose of the third marker on the target object acquired by the tracker at the last moment, and determine the pose of the expected pose at the last moment of the online pose in the tracker coordinate system according to the online pose of the third marker and the real pose of the expected pose in the target object coordinate system;

[0026] Obtain the pose of the end of the manipulator in the manipulator base coordinate system at the last moment, and obtain the third transformation matrix between the tracker coordinate system and the manipulator base coordinate system according to the pose of the end of the manipulator in the manipulator base coordinate system at the last moment and the pose of the expected pose in the tracker coordinate system at the last moment;

[0027] According to the third transformation matrix and the pose of the desired pose at the current moment in the tracker coordinate system, a first target pose of the desired pose at the current moment in the robot base coordinate system is calculated.

[0028] In one embodiment, forming a second update path according to the online posture of the end of the robot arm acquired by the tracker at the last moment includes:

[0029] The second updated path is calculated according to the first target posture and the online posture of the end of the manipulator acquired by the tracker at the last moment.

[0030] In one embodiment, a second registration point is provided on the trolley, and the second registration point is located on the surface of the robot arm base; if a fixed posture is obtained, the robot system guidance method further includes:

[0031] The tracker obtains the fixed position of the second marker and establishes a trolley coordinate system according to the fixed position;

[0032] When the tip of the probe touches the second registration point on the trolley and the tracker tracks the mark of the probe, the position and posture of the second registration point is obtained, the robot base coordinate system is established according to the position and posture of the second registration point, and the intermediate conversion matrix from the trolley coordinate system to the robot base coordinate system is calculated;

[0033] The fourth transformation matrix from the target object coordinate system to the trolley coordinate system is calculated, and the second target posture of the real posture in the robot base coordinate system is calculated according to the fourth transformation matrix and the intermediate transformation matrix in sequence.

[0034] In one embodiment, forming the first update path includes:

[0035] A first updated path is calculated according to the second real-time pose and the second target pose.

[0036] In one embodiment, an end effector is installed at the end of the robot arm, and the end effector includes a third registration point. If the first real-time posture is obtained, the robot system guidance method further includes:

[0037] When the tip of the probe contacts the third registration point and the tracker tracks the probe's mark and the first mark respectively, the pose of the third registration point and the static pose of the first mark are obtained respectively, an end effector coordinate system is established according to the pose of the third registration point, and a robot base coordinate system is established according to the static pose of the first mark, and the conversion relationship from the end effector coordinate system to the robot base coordinate system is calculated.

[0038] In a second aspect, the present application also provides a robot arm guidance system, the system comprising:

[0039] A trolley, wherein a second mark is provided on the trolley;

[0040] A mechanical arm, the mechanical arm is arranged on the trolley, and a first mark is arranged at the end of the mechanical arm;

[0041] A tracker is used to track a first marker disposed at the end of the robot arm to obtain a first real-time position and posture of the first marker, and to track a second marker disposed on the trolley to obtain a fixed position and posture of the second marker;

[0042] The host computer is used to determine whether the tracker has acquired the first real-time posture of the first mark set at the end of the robot arm. If the first real-time posture is not acquired, it is determined whether the tracker has acquired the fixed posture of the second mark set on the trolley; if the fixed posture is acquired, the first navigation mode is entered. In the first navigation mode, the second real-time posture of the end of the robot arm is re-acquired according to the fixed posture, and a first update path is formed; if the fixed posture is not acquired, the second navigation mode is entered. In the second navigation mode, a second update path is formed according to the online posture of the end of the robot arm acquired by the tracker at the last moment, and the first update path or the second update path is sent to the controller;

[0043] The controller is used to control the movement of the robot arm according to the first update path or the second update path.

[0044] In a third aspect, the present application also provides a robot system guiding device. The robot system includes a robot arm, a trolley controller and a tracker, the robot arm is arranged on the trolley, and the device includes:

[0045] The robot system includes a robotic arm, a trolley, a controller and a tracker. The robotic arm is arranged on the trolley. The device includes:

[0046] A first judgment module is used to judge whether the tracker obtains a first real-time position and posture of a first marker set at the end of the robot arm;

[0047] A second judgment module is used to judge whether the tracker obtains a fixed posture of a second marker set on the trolley if the first real-time posture is not obtained;

[0048] A first path acquisition module, for entering a first navigation mode if a fixed posture is acquired, and in the first navigation mode, reacquiring a second real-time posture of the end of the robotic arm according to the fixed posture and forming a first updated path;

[0049] A second path acquisition module is used to enter a second navigation mode if a fixed posture is not obtained. In the second navigation mode, a second update path is formed according to the online posture of the end of the manipulator obtained by the tracker at the last moment;

[0050] The control module is used to send the first update path or the second update path to the controller to control the movement of the robot arm.

[0051] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0052] Determine whether the tracker obtains a first real-time position and posture of a first marker set at the end of the robotic arm;

[0053] If the first real-time position and posture is not obtained, determining whether the tracker obtains the fixed position and posture of the second marker set on the trolley;

[0054] If a fixed posture is obtained, the first navigation mode is entered. In the first navigation mode, the second real-time posture of the end of the robot arm is re-obtained according to the fixed posture, and a first update path is formed;

[0055] If the fixed posture is not obtained, the second navigation mode is entered. In the second navigation mode, a second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment;

[0056] The first updated path or the second updated path is sent to the controller to control the movement of the robot arm.

[0057] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0058] Determine whether the tracker obtains a first real-time position and posture of a first marker set at the end of the robotic arm;

[0059] If the first real-time position and posture is not obtained, determining whether the tracker obtains the fixed position and posture of the second marker set on the trolley;

[0060] If a fixed posture is obtained, the first navigation mode is entered. In the first navigation mode, the second real-time posture of the end of the robot arm is re-obtained according to the fixed posture, and a first update path is formed;

[0061] If the fixed posture is not obtained, the second navigation mode is entered. In the second navigation mode, a second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment;

[0062] The first updated path or the second updated path is sent to the controller to control the movement of the robot arm.

[0063] The robot system guidance method, device, computer equipment, storage medium and computer program product determine whether the tracker obtains the first real-time posture of the first mark set at the end of the robot arm; if the first real-time posture is not obtained, determine whether the tracker obtains the fixed posture of the second mark set on the trolley, if the fixed posture is obtained, enter the first navigation mode, in the first navigation mode, re-acquire the second real-time posture of the end of the robot arm according to the fixed posture, and form a first update path, to ensure that when the first mark is blocked, the first update path can be formed according to the fixed posture of the second mark obtained by the tracker, and the controller controls the movement of the robot arm according to the first update path, thereby avoiding interruption of the control of the robot arm; if the fixed posture is not obtained, enter the second navigation mode, in the second navigation mode, form a second update path according to the online posture of the end of the robot arm obtained by the tracker at the last moment, to ensure that when both the first mark and the second mark are blocked, the second update path can be formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment, and the controller controls the movement of the robot arm according to the second update path, thereby avoiding interruption of the control of the robot arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A diagram of an application environment of a robot system guidance method in one embodiment;

[0065] Figure 2 FIG. 1 is a flow chart of a robot system guidance method in one embodiment. Figure 1 ;

[0066] Figure 3 FIG. 1 is a flow chart of a robot system guidance method in one embodiment. Figure 2 ;

[0067] Figure 4 FIG. 1 is a flow chart of a robot system guidance method in one embodiment. Figure 3 ;

[0068] Figure 5 FIG. 1 is a flow chart of a robot system booting method in one embodiment. Figure 4 ;

[0069] Figure 6 Schematic diagram of a robot system guidance method in one embodiment Figure 5 ;

[0070] Figure 7 is an application environment diagram of a robot system guidance method in a specific embodiment;

[0071] Figure 8 A schematic diagram of the structure of a robot arm guidance system in one embodiment;

[0072] Fig. 9 A schematic diagram of each mark in an embodiment;

[0073] Fig.10 A flowchart of a navigation mode selection method in one embodiment;

[0074] Fig.11 It is a schematic diagram of the flow chart of a robot system guidance method in a second navigation mode in one embodiment;

[0075] Fig.12 It is a schematic diagram of the flow chart of a robot system guidance method in a first navigation mode in an embodiment;

[0076] Fig.13 It is a schematic diagram of a flow chart of a robot system guidance method in a third navigation mode in one embodiment;

[0077] Fig.14 A schematic diagram of a process of outputting alarm information in an embodiment;

[0078] Fig.15 A schematic diagram of a registration process of a robot system guidance method in one embodiment;

[0079] Fig.16 A schematic diagram of the structure of a probe in one embodiment;

[0080] Fig.17 A schematic diagram of a method for probe registration in one embodiment;

[0081] Fig.18 A schematic diagram of a flow chart of a probe registration method in one embodiment;

[0082] Fig.19 A schematic diagram of a process of a trolley registration method in one embodiment;

[0083] Fig. 20 is a schematic flow chart of an instrument registration method in one embodiment;

[0084] Fig.21 is a schematic flow chart of a surgical area registration method in one embodiment;

[0085] Fig. 22 is a schematic flow chart of a robot system guiding method in a specific embodiment;

[0086] Fig.23 is a structural block diagram of a robot system guiding device in one embodiment; DETAILED DESCRIPTION

[0087] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0088] The robot system guidance method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. The host computer 101 interacts with the robot system 102. Among them, the robot system 102 includes a robot arm 103, a trolley 104 and a tracker 105, and the robot arm 103 is set on the trolley 104. The first marker 106 is set at the end of the robot arm 103, and the second marker 107 is set on the trolley 104. The tracker 105 is used to obtain the first real-time posture of the first marker 106 set at the end of the robot arm 103 or the fixed posture of the second marker 107 set on the trolley 104. The robot system also includes a controller 108, and the controller 108 is used to control the robot arm 103 to move according to the first update path or the second update path. The host computer 101 determines whether the tracker 105 obtains the first real-time posture of the first marker 106 set at the end of the robot arm 103; if the first real-time posture is not obtained, it is determined whether the tracker 105 obtains the fixed posture of the second marker 107 set on the trolley 104; if the fixed posture is obtained, it enters the first navigation mode. In the first navigation mode, the second real-time posture of the end of the robot arm 103 is re-obtained according to the fixed posture, and a first update path is formed according to the expected posture planned before the operation and the second real-time posture of the end of the robot arm 103; if the fixed posture is not obtained, it enters the second navigation mode. In the second navigation mode, a second update path is formed according to the online posture of the end of the robot arm 103 obtained by the tracker 105 at the last moment and the expected posture planned before the operation; the first update path or the second update path is sent to the controller 108 to control the movement of the robot arm 103.

[0089] In one embodiment, Figure 2 As shown, a robot system guidance method is provided, which is applied to Figure 1 Taking the host computer 101 in the example as an example, the following steps are included:

[0090] S201, determining whether a tracker has acquired a first real-time position and posture of a first marker disposed at an end of a robotic arm.

[0091] Specifically, the host computer determines whether the tracker obtains the first real-time position and posture of the first marker set at the end of the robot arm.

[0092] Among them, the robotic arm is a programmable robotic arm, which is connected by joints and can rotate or translate. The first mark refers to a reflective mark set at the end of the robotic arm. Different types of marks can be reflective mark points in the mark with different arrangement orders, sizes or shapes. Posture includes position and attitude, and the change of posture includes two motion processes of translation and rotation, among which the change of position is the translation process, and the change of attitude is the rotation process. The tracker is a device that can measure the position and direction of an object. Exemplarily, the tracker can be an optical tracker. The first real-time posture is the real-time posture of the first marker obtained by the tracker.

[0093] When the first marker is not blocked, the tracker can obtain the first real-time pose, and the host computer determines that the tracker has obtained the first real-time pose of the first marker set at the end of the robotic arm. When the first marker is blocked, the tracker cannot obtain the first real-time pose, and the host computer determines that the tracker has not obtained the first real-time pose of the first marker set at the end of the robotic arm.

[0094] S202: If the first real-time posture is not obtained, determine whether the tracker obtains the fixed posture of the second marker set on the trolley.

[0095] The robot arm is mounted on a trolley. The second mark is a reflective mark set on the trolley. Specifically, when the host computer determines that the tracker has not obtained the first real-time position and posture of the first mark set at the end of the robot arm, it continues to determine whether the tracker has obtained the fixed position and posture of the second mark set on the trolley.

[0096] When the second marker is not blocked, the tracker can obtain the fixed posture of the second marker, and the host computer determines that the tracker has obtained the fixed posture of the second marker set on the trolley. When the second marker is blocked, the tracker cannot obtain the fixed posture of the second marker, and the host computer determines that the tracker has not obtained the fixed posture of the second marker set on the trolley.

[0097] S203: If a fixed posture is obtained, enter the first navigation mode. In the first navigation mode, the second real-time posture of the end of the robot arm is re-obtained according to the fixed posture, and a first update path is formed.

[0098] If the host computer determines that the tracker obtains the fixed position of the second marker set on the trolley, it enters the first navigation mode, that is, the first navigation mode is a navigation mode when the first marker is blocked and the second marker is not blocked.

[0099] Since the second marker is set on the trolley, the robotic arm is installed on the trolley, and the end of the robotic arm has a fixed posture relationship with the second marker, the host computer can obtain the second real-time posture of the end of the robotic arm based on the fixed posture of the second marker and the fixed posture relationship between the end of the robotic arm and the second marker.

[0100] The second real-time posture of the end of the robot arm is the real-time posture of the end of the robot arm. As the robot arm moves, the second real-time posture of the end of the robot arm is continuously updated to form a first update path. The first update path is the movement path of the end of the robot arm in the first navigation mode.

[0101] S204: If the fixed posture is not obtained, enter the second navigation mode. In the second navigation mode, a second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment.

[0102] If the host computer determines that the tracker has not obtained the fixed position of the second marker set on the trolley, it enters the second navigation mode, that is, the second navigation mode is a navigation mode when both the first marker and the second marker are blocked.

[0103] The online pose of the end of the manipulator refers to the real-time pose of the end of the manipulator when the first marker is not blocked. The last moment refers to the moment when the first marker changes from being unblocked to being blocked.

[0104] When both the first marker and the second marker are blocked, the host computer obtains the online posture of the end of the robot arm obtained by the tracker at the last moment.

[0105] The host computer uses the online posture of the end of the robot arm obtained at the last moment as the starting point of the end of the robot arm, and forms a second update path as the robot arm moves. The second update path is the movement path of the end of the robot arm in the second navigation mode.

[0106] S205: Send the first updated path or the second updated path to the controller to control the movement of the robot arm.

[0107] The movement of the robot arm is controlled by a controller in the robot system. In the first navigation mode, the host computer sends the first update path to the controller, and the controller controls the robot arm to move according to the first update path. In the second navigation mode, the host computer sends the second update path to the controller, and the controller controls the robot arm to move according to the second update path.

[0108] In the above robot system guidance method, it is determined whether the tracker obtains the first real-time posture of the first mark set at the end of the robot arm; if the first real-time posture is not obtained, it is determined whether the tracker obtains the fixed posture of the second mark set on the trolley. If the fixed posture is obtained, the first navigation mode is entered. In the first navigation mode, the second real-time posture of the end of the robot arm is re-acquired according to the fixed posture, and a first update path is formed, which ensures that when the first mark is blocked, the first update path can be formed according to the fixed posture of the second mark obtained by the tracker, and the controller controls the movement of the robot arm according to the first update path, thereby avoiding interruption of the control of the robot arm; if the fixed posture is not obtained, the second navigation mode is entered. In the second navigation mode, the second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment, thereby ensuring that when both the first mark and the second mark are blocked, the second update path can be formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment, and the controller controls the movement of the robot arm according to the second update path, thereby avoiding interruption of the control of the robot arm.

[0109] In one embodiment, Figure 3 As shown, the robot system guidance method also includes:

[0110] S301, determining whether the tracker obtains a third real-time position and posture of a third marker set on the target object.

[0111] The third mark refers to a reflective mark set on the target object. Specifically, the host computer determines whether the tracker obtains the third real-time pose of the third mark set on the target object. When the third mark is not blocked, the tracker can obtain the third real-time pose, and the host computer determines that the tracker obtains the third real-time pose of the third mark set on the target object. When the third mark is blocked, the tracker cannot obtain the third real-time pose, and the host computer determines that the tracker does not obtain the third real-time pose of the third mark set on the target object.

[0112] S302: If the third real-time posture is not obtained, the robot arm is controlled to stop moving.

[0113] If the host computer determines that the tracker has not obtained the third real-time posture of the third mark set on the target object, it sends an instruction to the controller to stop the movement of the robot arm. After receiving the instruction, the controller controls the robot arm to stop moving. That is, when the third mark is blocked, the host computer will control the robot arm to stop moving through the controller to avoid safety problems caused by the continued movement of the robot arm. At the same time, the host computer can also send an alarm instruction to the alarm device to control the alarm device to give an alarm prompt according to the alarm instruction.

[0114] S303, if the third real-time posture is obtained and the first real-time posture is obtained, then enter the third navigation mode. In the third navigation mode, the movement of the robotic arm is controlled according to the expected posture planned before surgery and the first real-time posture; wherein the expected posture is planned according to the medical image.

[0115] If the host computer determines that the tracker obtains the third real-time pose of the third marker set on the target object, and determines that the tracker obtains the first real-time pose of the first marker set at the end of the robot arm, the third navigation mode is entered. That is, the third navigation mode is a navigation mode when neither the third marker nor the first marker is blocked.

[0116] The desired posture planned before surgery is planned in the medical image. The host computer takes the desired posture as the end point of the end of the robot arm and the first real-time posture of the first mark as the starting point. The host computer forms a third update path based on the desired posture planned before surgery and the first real-time posture. The third update path is the movement path of the robot arm formed from the starting posture to the end posture in the third navigation mode. The host computer sends the third update path to the controller, and the controller controls the robot arm to move according to the third update path.

[0117] In this embodiment, by judging whether the tracker obtains the third real-time posture of the third mark set on the target object, when the third mark is blocked, the robot arm is controlled to stop moving, so as to avoid the safety problem caused by the continued movement of the robot arm. When both the third mark and the first mark are not blocked, the third navigation mode is entered. In the third navigation mode, the robot arm is controlled to move according to the expected posture planned before the operation and the first real-time posture, so as to ensure the stable movement of the robot arm and avoid the interruption of the control of the robot arm.

[0118] In one embodiment, before the step of determining whether the tracker has acquired the first real-time posture of the first marker set at the end of the robotic arm, it also includes: aligning the expected posture in the target object coordinate system through the tracker and a pre-registered probe to obtain the actual posture of the expected posture in the target object coordinate system.

[0119] Among them, the probe is a detection device with a tip. A probe marker is deployed on the probe, and the probe marker is a reflective mark. A pre-registered probe refers to a probe whose relative posture relationship between the tip of the probe and the probe marker is known. The tracker obtains the real-time posture of the probe marker, and the real-time posture of the tip of the probe can be obtained according to the real-time posture of the probe marker and the relative posture relationship between the tip of the probe and the probe marker. The target object coordinate system refers to the coordinate system established by the host computer on the target object. Registration refers to the process of obtaining various transformation relationships in the process of guiding the robot system. Specifically, the host computer converts the expected posture in the medical image into the target object coordinate system through registration through the tracker and the pre-registered probe, and obtains the real posture of the expected posture in the target object coordinate system.

[0120] In this embodiment, the desired posture is aligned in the target object coordinate system through the tracker and the pre-registered probe to obtain the real posture of the desired posture in the target object coordinate system. The real posture of the desired posture in the target object coordinate system is obtained by alignment before controlling the movement of the manipulator. The manipulator is guided to move based on the real posture of the aligned desired posture in the target object coordinate system, which can ensure the stable operation of the manipulator and avoid interruption of the control of the manipulator.

[0121] In one embodiment, Figure 4 As shown, a mapping device is pre-installed in a target area of ​​a target object, the mapping device includes a first feature point and a first registration point, and a fixed positional relationship exists between any first registration point and any first feature point; the medical image includes the first feature point; the desired posture is registered in the target object coordinate system through a tracker and a pre-registered probe, and the real posture of the desired posture in the target object coordinate system is obtained, including:

[0122] S401 , when the tip of the probe contacts the first registration point and the tracker tracks the marker of the probe and the third marker respectively, the position and posture of the first registration point and the static position and posture of the third marker are respectively obtained.

[0123] The mapping device is pre-installed in the target area of ​​the target object. The mapping device includes a first feature point and a first registration point. Exemplarily, the first feature point may be a metal ball on the mapping device. The first registration point is at least one pose point selected on the mapping device. There is a fixed positional relationship between any first registration point and any first feature point.

[0124] When the tip of the probe contacts the first registration point, the tip of the probe, the marker of the probe, the first registration point and the third marker are all relatively still. When the tip of the probe contacts the first registration point, the tracker tracks the marker of the probe and obtains the real-time position and posture of the marker of the probe. Based on the real-time position and posture of the marker of the probe and the relative position and posture relationship between the tip of the probe and the marker of the probe, the real-time position and posture of the tip of the probe is obtained. The real-time position and posture of the tip of the probe is also the position and posture of the first registration point.

[0125] The tracker tracks the third marker to obtain a third static position of the third marker. When the tip of the probe contacts the first registration point, the tip of the probe, the marker of the probe, the first registration point and the third marker are all relatively static.

[0126] S402, establishing a target region coordinate system according to the position and posture of the first registration point, and establishing a target object coordinate system according to the static position and posture of the third marker, and calculating a first transformation matrix from the target region coordinate system to the target object coordinate system.

[0127] Among them, the target area is the area on the target object. The host computer uses the posture of the first registration point as the center of the coordinate circle to establish a target area coordinate system according to the target area, and uses the static posture of the third marker as the center of the coordinate circle to establish a target object coordinate system according to the target object. Since the posture of the first registration point is obtained based on the real-time posture of the marker of the probe acquired by the tracker, and the static posture of the third marker is acquired by the tracker, the host computer performs coordinate conversion from the target area coordinate system to the target object coordinate system to obtain the first conversion matrix. The first conversion matrix is ​​used to indicate the conversion relationship from the target area coordinate system to the target object coordinate system.

[0128] S403: Calculate the position and pose of the first feature point in the target area coordinate system according to the position and pose of the first registration point.

[0129] Among them, since the target area coordinate system is established according to the position and posture of the first registration point, the host computer can obtain the position and posture of the first registration point in the target area coordinate system. The position and posture of the first registration point in the target area coordinate system is added to the fixed positional relationship between the first registration point and the first feature point to obtain the position and posture of the first feature point in the target area coordinate system.

[0130] S404, obtaining the position and posture of the first feature point in the medical image coordinate system, and calculating a second transformation matrix from the medical image coordinate system to the target area coordinate system according to the position and posture of the first feature point in the target area coordinate system and the position and posture of the first feature point in the medical image coordinate system.

[0131] The medical image coordinate system is a coordinate system established based on the medical image. The medical image includes a first feature point. The host computer obtains the position and posture of the first feature point in the medical image coordinate system. The host computer can obtain a second transformation matrix from the medical image coordinate system to the target area coordinate system through coordinate transformation based on the position and posture of the first feature point in the target area coordinate system and the position and posture of the first feature point in the medical image coordinate system.

[0132] S405 , calculating the actual posture of the desired posture in the target object coordinate system according to the desired posture planned before surgery, the second transformation matrix, and the first transformation matrix.

[0133] The second conversion matrix indicates the conversion relationship from the medical image coordinate system to the target region coordinate system, and the first conversion matrix indicates the conversion relationship from the target region coordinate system to the target object coordinate system. The host computer multiplies the second conversion matrix with the first conversion matrix to obtain the conversion matrix from the medical image coordinate system to the target object coordinate system. The host computer multiplies the desired posture planned before surgery with the conversion matrix from the medical image coordinate system to the target object coordinate system to obtain the actual posture of the desired posture in the target object coordinate system.

[0134] In this embodiment, a first transformation matrix and a second transformation matrix are obtained by aligning a probe and a tracker, and then the real posture of the desired posture in the target object coordinate system is obtained. Since the real posture of the desired posture in the target object coordinate system is obtained by converting the desired posture from the medical image coordinate system to the target object coordinate system, it is aligned before controlling the movement of the robotic arm. Controlling the movement of the robotic arm based on the real posture of the desired posture in the target object coordinate system can improve the control efficiency of the robotic arm and avoid interruption of robotic arm control.

[0135] In one embodiment, Figure 5 As shown, if the fixed posture is not obtained, the robot system guidance method also includes:

[0136] S501, determining the pose of the desired pose in the tracker coordinate system at the current moment according to the third real-time pose of the third marker set on the target object and the actual pose of the desired pose in the target object coordinate system; wherein the third real-time pose of the third marker represents the real-time pose of the target object in the tracker coordinate system at the current moment.

[0137] The third real-time pose is the real-time pose of the third marker obtained by the tracker tracking the third marker set on the target object. The tracker coordinate system is a coordinate system established according to the tracker. The third real-time pose is the real-time pose of the third marker in the tracker coordinate system. Since the third marker is set on the target object, the third real-time pose represents the real-time pose of the target object in the tracker coordinate system at the current moment.

[0138] The host computer multiplies the third real-time posture with the actual posture of the desired posture in the target object coordinate system to obtain the posture of the desired posture in the tracker coordinate system at the current moment, thereby realizing the conversion of the desired posture from the target object coordinate system to the tracker coordinate system.

[0139] In some embodiments, the target object is movable, the third marker set on the target object is movable, and the third real-time posture of the third marker changes in real time, that is, the real-time posture of the target object in the tracker coordinate system at the current moment changes in real time. The real posture of the desired posture obtained by registration in the target object coordinate system is fixed, and based on the fixed real posture and the changing third real-time posture, the posture of the desired posture at the current moment in the tracker coordinate system changes in real time. The upper computer can track the real-time posture changes of the third marker through the tracker, and combine the real posture corresponding to the desired posture obtained by registration to obtain the posture of the desired posture that changes in real time in the tracker coordinate system.

[0140] S502, obtaining an online pose of a third marker on the target object acquired by the tracker at the last moment, and determining the pose of the expected pose at the last moment of the online pose in the tracker coordinate system according to the online pose of the third marker and the real pose of the expected pose in the target object coordinate system.

[0141] The online position and posture of the third marker refers to the real-time position and posture of the third marker when the first marker is not blocked.

[0142] When the first marker and the second marker are both blocked, but the third marker is not blocked, the host computer obtains the online position and posture of the third marker acquired by the tracker at the last moment.

[0143] The online pose of the third marker represents the real-time pose of the target object in the tracker coordinate system at the last moment.

[0144] The host computer multiplies the online pose of the third marker and the actual pose of the expected pose in the target object coordinate system to obtain the pose of the expected pose at the last moment of the online pose in the tracker coordinate system.

[0145] S503, obtaining the pose of the end of the robot arm in the robot arm base coordinate system at the last moment, and obtaining the third transformation matrix between the tracker coordinate system and the robot arm base coordinate system according to the pose of the end of the robot arm in the robot arm base coordinate system at the last moment and the pose of the expected pose in the tracker coordinate system at the last moment.

[0146] The robot arm is mounted on the trolley through the robot arm base, and the robot arm base coordinate system is a coordinate system established based on the robot arm base. At the last moment, the end of the robot arm coincides with the desired posture. In some embodiments, the posture of the target object changes at the current moment relative to the last moment, and the desired posture changes. The purpose of the robot system guidance method is to control the end of the robot arm to move from the posture at the last moment to the desired posture at the current moment.

[0147] The host computer multiplies the pose of the end of the manipulator in the manipulator base coordinate system at the last moment by the pose of the expected pose in the tracker coordinate system at the last moment to obtain the third transformation matrix between the tracker coordinate system and the manipulator base coordinate system.

[0148] S504: Calculate a first target pose of the desired pose at the current moment in the robot base coordinate system according to the third transformation matrix and the pose of the desired pose at the current moment in the tracker coordinate system.

[0149] Among them, the host computer multiplies the third transformation matrix and the posture of the desired posture at the current moment in the tracker coordinate system to obtain the first target posture of the desired posture at the current moment in the robot base coordinate system. The desired posture is converted from the tracker coordinate system to the robot base coordinate system. The first target posture represents the posture to which the robot needs to move at the current moment. In some embodiments, the desired posture changes in real time in the tracker coordinate system, and the first target posture of the desired posture in the robot base coordinate system also changes in real time. In other embodiments, the desired posture is unchanged in the tracker coordinate system, and the first target posture of the desired posture in the robot base coordinate system is also unchanged.

[0150] In some embodiments, the calculation formula for the first target pose is:

[0151]

[0152] in, represents the first target pose, inv(T) represents the inversion of matrix T, Indicates the position of the end of the robot arm in the robot arm base coordinate system at the last moment. It represents the online pose of the third marker on the target object acquired by the tracker at the last moment. represents the actual pose of the desired pose in the target object coordinate system, represents the third real-time pose of the third marker, Represents the actual pose of the desired pose in the target object's coordinate system.

[0153] In this embodiment, the first target pose of the desired pose at the current moment in the coordinate system of the robot base is obtained through the third real-time pose, the real pose of the desired pose in the target object coordinate system, the online pose of the third marker, and the pose of the end of the robot arm in the coordinate system of the robot base at the last moment, so that the desired pose is converted from the target object coordinate system to the tracker coordinate system, and then to the robot base coordinate system. It is beneficial to control the robot arm to move to the first target pose based on the first target pose in the second navigation mode, thereby improving the control efficiency of the robot arm. At the same time, it avoids the interruption of robot arm control when both the first marker and the second marker are blocked.

[0154] In one embodiment, forming a second update path according to the online posture of the end of the robot arm acquired by the tracker at the last moment includes: calculating the second update path according to the first target posture and the online posture of the end of the robot arm acquired by the tracker at the last moment.

[0155] Among them, the host computer takes the first target posture as the end point of the robot end, and takes the online posture of the robot end obtained by the tracker at the last moment as the starting point of the robot end, and the second update path can be determined from the starting point to the end point.

[0156] In this embodiment, the host computer calculates the second update path according to the first target posture and the online posture of the end of the robot arm obtained by the tracker at the last moment, so as to ensure that in the second navigation mode, when the first mark and the second mark are both blocked, the second update path can be formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment and the first target posture to control the movement of the robot arm, thereby avoiding interruption of the control of the robot arm.

[0157] In one embodiment, Figure 6 As shown, a second registration point is provided on the trolley, and the second registration point is located on the surface of the robot arm base; if a fixed posture is obtained, the robot system guidance method further includes:

[0158] S601, obtaining a fixed position and posture of the second marker via a tracker, and establishing a trolley coordinate system according to the fixed position and posture.

[0159] The tracker tracks the second marker, and the host computer obtains the fixed position of the second marker obtained by the tracker tracking the second marker. The host computer takes the fixed position of the second marker as the center of the circle and establishes a trolley coordinate system according to the trolley.

[0160] S602, when the tip of the probe touches the second registration point on the trolley and the tracker tracks the mark of the probe, the posture of the second registration point is obtained, the robot base coordinate system is established according to the posture of the second registration point, and the intermediate transformation matrix from the trolley coordinate system to the robot base coordinate system is calculated.

[0161] When the tip of the probe contacts the second registration point on the trolley, the second registration point, the tip of the probe and the mark of the probe are relatively still. When the tip of the probe contacts the second registration point, the tracker tracks the mark of the probe and obtains the real-time position and posture of the mark of the probe. According to the real-time position and posture of the mark of the probe and the relative position and posture relationship between the tip of the probe and the mark of the probe, the real-time position and posture of the tip of the probe is obtained. The real-time position and posture of the tip of the probe is also the position and posture of the second registration point.

[0162] The host computer takes the position of the second registration point as the center and establishes the robot base coordinate system according to the robot base. The host computer performs coordinate conversion between the trolley coordinate system and the robot base coordinate system to obtain an intermediate conversion matrix, which is used to indicate the conversion relationship from the trolley coordinate system to the robot base coordinate system.

[0163] S603, calculating the fourth transformation matrix from the target object coordinate system to the trolley coordinate system, and calculating the second target posture of the real posture in the robot base coordinate system according to the fourth transformation matrix and the intermediate transformation matrix in sequence.

[0164] Among them, the host computer performs coordinate conversion on the target object coordinate system and the trolley coordinate system to obtain a fourth conversion matrix, and the fourth conversion matrix is ​​used to indicate the conversion relationship from the target object coordinate system to the trolley coordinate system. The host computer multiplies the fourth conversion matrix and the intermediate conversion matrix to obtain the conversion relationship from the target object coordinate system to the robot base coordinate system. The host computer multiplies the actual posture of the desired posture in the target object coordinate system with the conversion relationship from the target object coordinate system to the robot base coordinate system to obtain the actual posture of the desired posture in the robot base coordinate system, and uses the actual posture of the desired posture in the robot base coordinate system as the second target posture.

[0165] In this embodiment, by establishing a trolley coordinate system and a robot base coordinate system, and tracking the alignment of the probe through the probe and tracker, an intermediate transformation matrix and a fourth transformation matrix are obtained, and then the second target posture is obtained. Since the second target posture is obtained by transforming the desired posture from the target object coordinate system to the robot base coordinate system, and is aligned before controlling the movement of the robot, controlling the movement of the robot based on the second target posture can improve the control efficiency of the robot and avoid interruption of the robot control.

[0166] In one embodiment, a second registration point is provided on the trolley, and the second registration point is located on the surface of the robot base; if a fixed posture is obtained, the robot system guidance method also includes: obtaining the real-time posture of the second marker obtained by the tracker tracking the second marker, and the third real-time posture of the third marker obtained by the tracker tracking the third marker; wherein the third real-time posture of the third marker represents the real-time posture of the target object in the tracker coordinate system at the current moment; according to the third real-time posture of the third marker and the real posture of the expected posture in the target object coordinate system, the posture of the expected posture at the current moment in the tracker coordinate system is determined; according to the real-time posture of the second marker and the intermediate transformation matrix from the trolley coordinate system to the robot base coordinate system, the transformation matrix from the tracker coordinate system to the robot base coordinate system is obtained; according to the posture of the expected posture at the current moment in the tracker coordinate system and the transformation matrix from the tracker coordinate system to the robot base coordinate system, the second target posture of the expected posture at the current moment in the robot base coordinate system is obtained.

[0167] Among them, the host computer obtains the real-time posture of the second marker obtained by the tracker tracking the second marker, and the third real-time posture of the third marker obtained by the tracker tracking the third marker. Since the third marker is set on the target object, the third real-time posture represents the real-time posture of the target object in the tracker coordinate system at the current moment.

[0168] The host computer multiplies the third real-time posture with the actual posture of the desired posture in the target object coordinate system to obtain the posture of the desired posture in the tracker coordinate system at the current moment, thereby realizing the conversion of the desired posture from the target object coordinate system to the tracker coordinate system.

[0169] The host computer multiplies the real-time position and posture of the second marker with the intermediate conversion matrix from the trolley coordinate system to the robot base coordinate system to obtain the conversion matrix from the tracker coordinate system to the robot base coordinate system.

[0170] The host computer multiplies the current desired posture in the tracker coordinate system by the conversion matrix from the tracker coordinate system to the robot base coordinate system to obtain the second target posture of the current desired posture in the robot base coordinate system.

[0171] In some embodiments, the calculation formula for the second target posture is:

[0172]

[0173] in, represents the second target pose, inv(T) represents the inversion of matrix T, Represents the intermediate transformation matrix from the trolley coordinate system to the robot base coordinate system, represents the real-time pose of the second marker, represents the third real-time pose of the third marker, Represents the actual pose of the desired pose in the target object's coordinate system.

[0174] In this embodiment, the second target pose of the desired pose at the current moment in the robot base coordinate system is obtained through the real-time pose of the second marker, the third real-time pose of the third marker, the intermediate conversion matrix from the trolley coordinate system to the robot base coordinate system, and the real pose of the desired pose in the target object coordinate system, so that the desired pose is converted from the target object coordinate system to the tracker coordinate system, and then to the robot base coordinate system. It is beneficial to control the robot to move to the second target pose based on the second target pose in the first navigation mode, thereby improving the control efficiency of the robot. At the same time, it avoids the interruption of robot control when the first marker is blocked.

[0175] In one embodiment, forming the first updated path includes: calculating the first updated path according to the second real-time pose and the second target pose.

[0176] The host computer uses the second real-time posture of the end of the robot arm as the starting point of the end of the robot arm, and uses the second target posture as the end point of the end of the robot arm. The first update path can be confirmed from the starting point to the end point.

[0177] In this embodiment, the host computer calculates the second update path according to the second real-time posture and the second target posture, which can ensure that in the first navigation mode, when the first marker is blocked, the first update path can be formed according to the real-time posture of the second marker obtained by the tracker to control the movement of the robotic arm and avoid interruption of the robotic arm control.

[0178] In one embodiment, an end effector is installed at the end of the robot arm, and the end effector includes a third registration point. If the first real-time posture is obtained, the robot system guidance method also includes: when the tip of the probe contacts the third registration point, and the tracker tracks the mark of the probe and the first mark respectively, the posture of the third registration point and the static posture of the first mark are respectively obtained, an end effector coordinate system is established according to the posture of the third registration point, and a robot arm base coordinate system is established according to the static posture of the first mark, and the conversion relationship from the end effector coordinate system to the robot arm base coordinate system is calculated.

[0179] The end effector is a device installed at the end of the robot arm. The third configuration point is a posture point on the end effector. When the tip of the probe contacts the third registration point, the end effector, the tip of the probe, the mark of the probe and the first mark are relatively stationary.

[0180] The tip of the probe contacts the third registration point, and the tracker tracks the marker of the probe to obtain the real-time position and posture of the marker of the probe. The real-time position and posture of the tip of the probe is obtained based on the real-time position and posture of the marker of the probe and the relative position and posture relationship between the tip of the probe and the marker of the probe. The real-time position and posture of the tip of the probe is also the position and posture of the third registration point.

[0181] The tip of the probe contacts the first marker, and the static pose of the first marker is obtained.

[0182] The host computer uses the pose of the third registration point as the center of the coordinate circle to establish the end effector coordinate system according to the end effector. The host computer uses the static pose of the first mark as the center of the coordinate circle to establish the robot base coordinate system according to the robot base. The host computer performs coordinate conversion between the end effector coordinate system and the robot base coordinate system to obtain the conversion relationship from the end effector coordinate system to the robot base coordinate system.

[0183] In some embodiments, the host computer multiplies the first target posture by the conversion relationship from the end effector coordinate system to the robot base coordinate system to obtain the actual posture of the desired posture at the current moment in the end effector coordinate system. The host computer uses the actual posture of the current desired posture in the end effector coordinate system as the end point of the end of the robot arm to control the movement of the robot arm.

[0184] In some embodiments, the host computer multiplies the second real-time posture of the end of the robot arm by the conversion relationship from the end effector coordinate system to the robot arm base coordinate system to obtain the real posture of the end of the robot arm in the end effector coordinate system.

[0185] In this embodiment, the probe and the tracker are aligned to establish the end effector coordinate system and the robot base coordinate system, and then the conversion relationship from the end effector coordinate system to the robot base coordinate system is obtained. The conversion relationship from the end effector coordinate system to the robot base coordinate system can be used to transform the desired posture into the end effector coordinate system to control the movement of the robot. Since the conversion relationship from the end effector coordinate system to the robot base coordinate system is obtained by alignment before controlling the movement of the robot, controlling the movement of the robot based on the actual posture of the desired posture in the end effector coordinate system can improve the control efficiency of the robot and avoid interruption of robot control.

[0186] To explain the robot system guidance method and effect in detail, the following is a most detailed embodiment:

[0187] refer to Figure 7In the application scenario, the robot system includes a robot arm, a trolley, a tracker and a controller, and the robot arm is set on the trolley. The first marker is set at the end of the robot arm, and the second marker is set on the trolley. The tracker is used to obtain a first real-time pose of the first marker set at the end of the robot arm, a fixed pose of the second marker set on the trolley, or a third real-time pose of the third marker set on the target object. The controller is used to control the robot arm to move according to the first update path, the second update path, or the third path. The host computer interacts with the controller in the robot system. The host computer determines whether the tracker obtains a first real-time posture of a first marker set at the end of the robot arm; if the first real-time posture is not obtained, it determines whether the tracker obtains a fixed posture of a second marker set on the trolley; if the fixed posture is obtained, it enters a first navigation mode, in which the second real-time posture of the end of the robot arm is re-obtained according to the fixed posture, and a first update path is formed according to the expected posture planned before the operation and the second real-time posture of the end of the robot arm; if the fixed posture is not obtained, it enters a second navigation mode, in which the second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment and the expected posture planned before the operation; the first update path or the second update path is sent to the controller to control the movement of the robot arm.

[0188] refer to Figure 8 The robot guidance system in the invention comprises a host computer, a trolley, a tracker and a robot. The trolley is provided with a second marker. The robot is provided on the trolley, and a first marker is provided at the end of the robot. The tracker is used to track the first marker provided at the end of the robot to obtain the first real-time posture of the first marker, and to track the second marker provided on the trolley to obtain the fixed posture of the second marker. The host computer is used to determine whether the tracker obtains the first real-time posture of the first marker provided at the end of the robot. If the first real-time posture is not obtained, it is determined whether the tracker obtains the fixed posture of the second marker provided on the trolley; if the fixed posture is obtained, the first navigation mode is entered. In the first navigation mode, the second real-time posture of the end of the robot is re-obtained according to the fixed posture, and a first update path is formed; if the fixed posture is not obtained, the second navigation mode is entered. In the second navigation mode, the second update path is formed according to the online posture of the end of the robot obtained by the tracker at the last moment, and the first update path or the second update path is sent to the controller. The controller is used to control the movement of the robot according to the first update path or the second update path.

[0189] like Fig. 9 Shown is a schematic diagram of each marker, where Fig. 9 (a) is a first marking schematic diagram, Fig. 9 (b) is a schematic diagram of the second mark, Fig. 9(c) is a schematic diagram of the third mark, Fig. 9 (d) is a schematic diagram of probe labeling. Fig.10 Shown is a schematic diagram of navigation mode selection for the robot system guidance method.

[0190] like Fig.11 The figure shows a flow chart of the robot system guidance method in the second navigation mode. In the second navigation mode, both the first marker and the second marker are blocked, and the host computer calculates the second update path according to the first target pose and the online pose of the end of the robot arm obtained by the tracker at the last moment, where the calculation formula of the first target pose is:

[0191]

[0192] in, represents the first target pose, inv(T) represents the inversion of matrix T, Indicates the position of the end of the robot arm in the robot arm base coordinate system at the last moment. It represents the online pose of the third marker on the target object acquired by the tracker at the last moment. represents the actual pose of the desired pose in the target object coordinate system, It represents the online pose of the third marker on the target object acquired by the tracker at the last moment. Represents the actual pose of the desired pose in the target object's coordinate system.

[0193] like Fig.12 The figure shows a schematic flow chart of the robot system guidance method in the first mode. In the first navigation mode, the first marker is blocked, the second marker is not blocked, and the host computer calculates the first update path according to the second real-time posture and the second target posture.

[0194] Among them, the calculation formula of the second target posture is:

[0195]

[0196] in, represents the second target pose, inv(T) represents the inversion of matrix T, Represents the intermediate transformation matrix from the trolley coordinate system to the robot base coordinate system, represents the real-time pose of the second marker, represents the third real-time pose of the third marker, Represents the actual pose of the desired pose in the target object's coordinate system.

[0197] like Fig.13The figure shows a flow chart of the robot system guidance method in the third navigation mode. The host computer determines whether the tracker obtains the third real-time posture of the third mark set on the target object; if the third real-time posture is not obtained, the robot arm is controlled to stop moving; if the third real-time posture is obtained and the first real-time posture is obtained, the third navigation mode is entered. In the third navigation mode, the third target posture of the end of the robot arm is obtained according to the expected posture of the preoperative planning and the first real-time posture, wherein the calculation formula of the third target posture of the end of the robot arm can be:

[0198]

[0199] in, represents the third target pose, Indicates the current position of the end of the robot arm in the robot arm base coordinate system. Represents the position and posture of the end of the robot arm in the end effector coordinate system. represents the first real-time pose of the first marker, represents the third real-time pose of the third marker, Represents the actual pose of the desired pose in the target object's coordinate system.

[0200] like Fig.14 The figure shows the basic framework process of outputting alarm information in the robot system guidance method. According to whether the first mark, the second optical mark, and the third mark are blocked, the corresponding navigation mode is entered and the corresponding alarm information is output. Table 1 shows the relationship between the alarm information rules and the types of each mark.

[0201] Table 1 Relationship between alarm information rules and marking types

[0202]

[0203] like Fig.15 The figure shows the registration process of the robot system guidance method, including probe registration, instrument registration, surgical area registration, and trolley registration. Fig.16 The figure shows the schematic diagram of the structure of the probe, wherein the lower end of the probe is a probe tip, and the upper end of the probe has a probe mark, which can be recognized by the tracker. Fig.17 The diagram shows a method for probe registration. The probe can take a fixed point relatively still with respect to the tracker in at least one different posture. Fig.18 FIG. 4 is a flow chart of the probe registration method. Fig.19 The figure shows a schematic diagram of the trolley registration method. Fig. 20 The following is a schematic diagram of the instrument registration method. Fig.21 The figure shows a schematic diagram of the method for surgical area registration. Fig. 22The figure shows a schematic diagram of the process of applying the robot system guidance method to install implants in the target area, wherein the robot system guidance method is mainly used for robot arm positioning, robot arm automatic hole preparation and robot arm assisted hole preparation.

[0204] The above-mentioned robot system guidance method determines whether the tracker obtains the first real-time posture of the first mark set at the end of the robot arm; if the first real-time posture is not obtained, it is determined whether the tracker obtains the fixed posture of the second mark set on the trolley. If the fixed posture is obtained, the first navigation mode is entered. In the first navigation mode, the second real-time posture of the end of the robot arm is re-acquired according to the fixed posture, and a first update path is formed, which ensures that when the first mark is blocked, the first update path can be formed according to the fixed posture of the second mark obtained by the tracker, and the controller controls the movement of the robot arm according to the first update path, thereby avoiding interruption of the control of the robot arm; if the fixed posture is not obtained, the second navigation mode is entered. In the second navigation mode, the second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment, thereby ensuring that when both the first mark and the second mark are blocked, the second update path can be formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment, and the controller controls the movement of the robot arm according to the second update path, thereby avoiding interruption of the control of the robot arm.

[0205] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0206] Based on the same inventive concept, the embodiment of the present application also provides a robot system guiding device for implementing the robot system guiding method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more robot system guiding device embodiments provided below can refer to the limitations of the robot system guiding method above, and will not be repeated here.

[0207] In one embodiment, Fig.23As shown, a robot system guiding device 100 is provided. The robot system includes a robot arm, a trolley and a tracker. The robot arm is arranged on the trolley. The device includes: a first judgment module 110, a second judgment module 120, a first path acquisition module 130, a second path acquisition module 140 and a control module 150, wherein:

[0208] The first determination module 110 is used to determine whether the tracker obtains a first real-time position and posture of a first marker set at the end of the robot arm.

[0209] The second judgment module 120 is used to judge whether the tracker obtains the fixed posture of the second marker set on the trolley if the first real-time posture is not obtained.

[0210] The first path acquisition module 130 is used to enter the first navigation mode if a fixed posture is acquired. In the first navigation mode, the second real-time posture of the end of the robot arm is re-acquired according to the fixed posture, and a first updated path is formed.

[0211] The second path acquisition module 140 is used to enter the second navigation mode if the fixed posture is not acquired. In the second navigation mode, a second update path is formed according to the online posture of the end of the robot arm acquired by the tracker at the last moment.

[0212] The control module 150 is used to send the first update path or the second update path to the controller to control the movement of the robot arm.

[0213] The above-mentioned robot system guiding device determines whether the tracker obtains the first real-time posture of the first mark set at the end of the robot arm; if the first real-time posture is not obtained, it determines whether the tracker obtains the fixed posture of the second mark set on the trolley. If the fixed posture is obtained, it enters the first navigation mode. In the first navigation mode, the second real-time posture of the end of the robot arm is re-obtained according to the fixed posture, and a first update path is formed, which ensures that when the first mark is blocked, the first update path can be formed according to the fixed posture of the second mark obtained by the tracker, and the controller controls the movement of the robot arm according to the first update path, thereby avoiding interruption of the control of the robot arm; if the fixed posture is not obtained, it enters the second navigation mode. In the second navigation mode, the second update path is formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment, thereby ensuring that when both the first mark and the second mark are blocked, the second update path can be formed according to the online posture of the end of the robot arm obtained by the tracker at the last moment, and the controller controls the movement of the robot arm according to the second update path, thereby avoiding interruption of the control of the robot arm.

[0214] In one embodiment, the robot system guidance device 100 is also used to: determine whether the tracker obtains the third real-time posture of the third marker set on the target object; if the third real-time posture is not obtained, control the robot arm to stop moving; if the third real-time posture is obtained and the first real-time posture is obtained, enter the third navigation mode, and in the third navigation mode, control the movement of the robot arm according to the expected posture planned before surgery and the first real-time posture; wherein the expected posture is obtained according to the planning in the medical image.

[0215] In one embodiment, the first determination module 110 is further configured to: register the desired posture in the target object coordinate system via the tracker and the pre-registered probe to obtain the actual posture of the desired posture in the target object coordinate system.

[0216] In one embodiment, a mapping device is pre-installed in a target area of ​​a target object, the mapping device includes a first feature point and a first registration point, and there is a fixed positional relationship between any first registration point and any first feature point; the medical image includes a first feature point; the desired posture is registered in the target object coordinate system through a tracker and a pre-registered probe to obtain the real posture of the desired posture in the target object coordinate system, and the first judgment module 110 is also used to: when the tip of the probe contacts the first registration point, the tracker tracks the marker and the third marker of the probe respectively, to obtain the posture of the first registration point and the static posture of the third marker respectively; according to the first registration point A target area coordinate system is established according to the static posture of the third marker, and a target object coordinate system is established, and a first transformation matrix from the target area coordinate system to the target object coordinate system is calculated; the posture of the first feature point in the target area coordinate system is calculated according to the posture of the first registration point; the posture of the first feature point in the medical image coordinate system is obtained, and the second transformation matrix from the medical image coordinate system to the target area coordinate system is calculated according to the posture of the first feature point in the target area coordinate system and the posture of the first feature point in the medical image coordinate system; the actual posture of the expected posture in the target object coordinate system is calculated according to the expected posture of preoperative planning, the second transformation matrix and the first transformation matrix.

[0217] In one embodiment, when a fixed posture is not obtained, the robot system guidance device 100 is further used to: determine the posture of the desired posture at the current moment in the tracker coordinate system according to the third real-time posture of the third marker set on the target object and the real posture of the desired posture in the target object coordinate system; wherein the third real-time posture of the third marker represents the real-time posture of the target object in the tracker coordinate system at the current moment; obtain the online posture of the third marker on the target object obtained by the tracker at the last moment, and determine the desired posture in the target object according to the online posture of the third marker and the actual posture of the desired posture in the target object coordinate system. The actual posture in the coordinate system is determined to determine the posture of the expected posture at the last moment of the online posture in the tracker coordinate system; the posture of the end of the manipulator in the manipulator base coordinate system at the last moment is obtained, and the third transformation matrix between the tracker coordinate system and the manipulator base coordinate system is obtained according to the posture of the end of the manipulator in the manipulator base coordinate system at the last moment and the posture of the expected posture at the last moment in the tracker coordinate system; according to the third transformation matrix and the posture of the expected posture at the current moment in the tracker coordinate system, the first target posture of the expected posture at the current moment in the manipulator base coordinate system is calculated.

[0218] In one embodiment, in forming a second updated path according to the online posture of the end of the robotic arm acquired by the tracker at the last moment, the second path acquisition module 140 is further used to: calculate the second updated path according to the first target posture and the online posture of the end of the robotic arm acquired by the tracker at the last moment.

[0219] In one embodiment, a second registration point is provided on the trolley, and the second registration point is located on the surface of the robot arm base; when a fixed posture is obtained, the robot system guiding device 100 is also used to: obtain the fixed posture of the second mark through the tracker, and establish the trolley coordinate system according to the fixed posture; when the tip of the probe contacts the second registration point on the trolley and the tracker tracks the mark of the probe, the posture of the second registration point is obtained, the robot arm base coordinate system is established according to the posture of the second registration point, and the intermediate conversion matrix from the trolley coordinate system to the robot arm base coordinate system is calculated; the fourth conversion matrix from the target object coordinate system to the trolley coordinate system is calculated, and the second target posture of the real posture in the robot arm base coordinate system is calculated according to the fourth conversion matrix and the intermediate conversion matrix.

[0220] In one embodiment, in forming the first updated path, the first path acquisition module 130 is further used to: calculate the first updated path according to the second real-time posture and the second target posture.

[0221] In one embodiment, an end effector is installed at the end of the robot arm, and the end effector includes a third registration point. When the first real-time posture is obtained, the robot system guidance device 100 is also used to: when the tip of the probe contacts the third registration point, and the tracker tracks the mark of the probe and the first mark respectively, obtain the posture of the third registration point and the static posture of the first mark respectively, establish an end effector coordinate system according to the posture of the third registration point, and establish a robot arm base coordinate system according to the static posture of the first mark, and calculate the conversion relationship from the end effector coordinate system to the robot arm base coordinate system.

[0222] Each module in the above robot system guiding device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0223] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0224] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0225] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above-mentioned embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of the present specification. The above-mentioned embodiments only express several implementation methods of the present application, and the description is more specific and detailed, but it cannot be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for those of ordinary skill in the art, without departing from the concept of the present application, several deformations and improvements can be made, which all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A robot system guidance method, characterized in that: The robot system comprises a robot arm, a trolley, a controller and a tracker, wherein the robot arm is arranged on the trolley, and the method comprises: Determining whether the tracker obtains a first real-time position and posture of a first marker disposed at the end of the robotic arm; If the first real-time posture is not obtained, determining whether the tracker obtains a fixed posture of a second marker disposed on the trolley; If the fixed posture is obtained, a first navigation mode is entered, in which a second real-time posture of the end of the robotic arm is re-obtained according to the fixed posture, and a first update path is formed; If the fixed posture is not obtained, entering the second navigation mode, in which a second update path is formed according to the online posture of the end of the robotic arm obtained by the tracker at the last moment; The first updated path or the second updated path is sent to the controller to control the movement of the robot arm.

2. The method according to claim 1, characterized in that The method further comprises: determining whether the tracker acquires a third real-time position and posture of a third marker disposed on the target object; If the third real-time posture is not obtained, controlling the robotic arm to stop moving; If the third real-time posture is acquired and the first real-time posture is acquired, a third navigation mode is entered, in which the movement of the robotic arm is controlled according to the desired posture planned before surgery and the first real-time posture; Wherein, the desired posture is obtained according to planning in the medical image.

3. The method according to claim 2, characterized in that Before the step of determining whether the tracker has acquired the first real-time posture of the first marker set at the end of the robotic arm, the method further includes: The desired posture is registered in the target object coordinate system via the tracker and the pre-registered probe to obtain the true posture of the desired posture in the target object coordinate system.

4. The method according to claim 3, characterized in that A mapping device is pre-installed in the target area of ​​the target object, the mapping device includes a first feature point and a first registration point, and there is a fixed positional relationship between any of the first registration points and any of the first feature points; the medical image includes the first feature point; The method of registering the desired posture in a target object coordinate system via the tracker and the pre-registered probe to obtain a true posture of the desired posture in the target object coordinate system includes: When the tip of the probe contacts the first registration point and the tracker tracks the marker of the probe and the third marker respectively, the position and posture of the first registration point and the static position and posture of the third marker are respectively acquired; Establishing a target area coordinate system according to the position and posture of the first registration point, and establishing a target object coordinate system according to the static position and posture of the third marker, and calculating a first transformation matrix from the target area coordinate system to the target object coordinate system; Calculate the position and posture of the first feature point in the target area coordinate system according to the position and posture of the first registration point; Acquire the position and posture of the first feature point in the medical image coordinate system, and calculate a second transformation matrix from the medical image coordinate system to the target region coordinate system according to the position and posture of the first feature point in the target region coordinate system and the position and posture of the first feature point in the medical image coordinate system; The actual posture of the expected posture in the target object coordinate system is calculated according to the expected posture planned before surgery, the second transformation matrix and the first transformation matrix.

5. The method according to claim 4, characterized in that If the fixed posture is not obtained, the method further includes: Determine the pose of the desired pose in the tracker coordinate system at the current moment according to a third real-time pose of a third marker set on the target object and the real pose of the desired pose in the target object coordinate system; wherein the third real-time pose of the third marker represents the real-time pose of the target object in the tracker coordinate system at the current moment; Acquire the online pose of the third marker on the target object acquired by the tracker at the last moment, and determine the pose of the expected pose in the tracker coordinate system at the last moment of the online pose according to the online pose of the third marker and the real pose of the expected pose in the target object coordinate system; Acquire the posture of the end of the manipulator in the manipulator base coordinate system at the last moment, and obtain a third transformation matrix between the tracker coordinate system and the manipulator base coordinate system according to the posture of the end of the manipulator in the manipulator base coordinate system at the last moment and the posture of the expected posture in the tracker coordinate system at the last moment; According to the third transformation matrix and the pose of the desired pose at the current moment in the tracker coordinate system, a first target pose of the desired pose at the current moment in the robot base coordinate system is calculated.

6. The method according to claim 5, characterized in that: Forming a second update path according to the online posture of the end of the robotic arm acquired by the tracker at the last moment, including: The second update path is calculated according to the first target posture and the online posture of the end of the robotic arm acquired by the tracker at the last moment.

7. The method according to claim 4, characterized in that A second registration point is provided on the trolley, and the second registration point is located on the surface of the robot arm base; if the fixed posture is obtained, the method further includes: Acquiring a fixed position and posture of the second marker via the tracker, and establishing a trolley coordinate system according to the fixed position and posture; When the tip of the probe contacts a second registration point on the trolley and the tracker tracks the mark of the probe, the position and posture of the second registration point are acquired, a robot base coordinate system is established according to the position and posture of the second registration point, and an intermediate conversion matrix from the trolley coordinate system to the robot base coordinate system is calculated; A fourth transformation matrix from the target object coordinate system to the trolley coordinate system is calculated, and a second target posture of the real posture in the robot base coordinate system is calculated according to the fourth transformation matrix and the intermediate transformation matrix in sequence.

8. The method according to claim 7, characterized in that The forming of the first update path comprises: The first updated path is calculated according to the second real-time pose and the second target pose.

9. The method according to claim 5 or 7, characterized in that: An end effector is installed at the end of the robot arm, and the end effector includes a third registration point. If the first real-time posture is obtained, the method further includes: When the tip of the probe contacts the third registration point and the tracker tracks the mark of the probe and the first mark respectively, the pose of the third registration point and the static pose of the first mark are respectively acquired, an end effector coordinate system is established according to the pose of the third registration point, and a robot base coordinate system is established according to the static pose of the first mark, and the conversion relationship from the end effector coordinate system to the robot base coordinate system is calculated.

10. A robot arm guidance system, characterized in that: The system comprises: A trolley, wherein a second mark is provided on the trolley; A mechanical arm, wherein the mechanical arm is disposed on the trolley, and a first mark is disposed at the end of the mechanical arm; A tracker, used for tracking a first marker disposed at the end of the robot arm to obtain a first real-time position and posture of the first marker, and tracking a second marker disposed on the trolley to obtain a fixed position and posture of the second marker; A host computer, used to determine whether the tracker has acquired a first real-time posture of a first mark set at the end of the robot arm, and if the first real-time posture is not acquired, determine whether the tracker has acquired a fixed posture of a second mark set on the trolley; if the fixed posture is acquired, enter a first navigation mode, in which the second real-time posture of the end of the robot arm is reacquired according to the fixed posture, and a first update path is formed; if the fixed posture is not acquired, enter a second navigation mode, in which a second update path is formed according to the online posture of the end of the robot arm acquired by the tracker at the last moment, and the first update path or the second update path is sent to the controller; A controller is used to control the movement of the robotic arm according to the first update path or the second update path.

11. A robot system guiding device, characterized in that: The robot system includes a mechanical arm, a trolley, a controller and a tracker, wherein the mechanical arm is arranged on the trolley, and the device includes: A first judgment module, used to judge whether the tracker obtains a first real-time posture of a first marker set at the end of the robot arm; a second judgment module, configured to judge whether the tracker has acquired a fixed posture of a second marker disposed on the trolley if the first real-time posture is not acquired; A first path acquisition module, configured to enter a first navigation mode if the fixed posture is acquired, and in the first navigation mode, reacquire a second real-time posture of the end of the robotic arm according to the fixed posture, and form a first updated path; A second path acquisition module, configured to enter a second navigation mode if the fixed posture is not acquired, and form a second update path in the second navigation mode according to the online posture of the end of the robotic arm acquired by the tracker at the last moment; A control module is used to send the first update path or the second update path to the controller to control the movement of the robotic arm.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images