Interactive man-machine cooperative path planning and mechanical arm follow-up control system and method

Abstract

The invention discloses an interactive man-machine cooperative path planning and mechanical arm follow-up control system and method, and relates to the technical field of medical robot path planning control. The control system comprises an optical positioning camera, a mechanical arm, a patient tracer, an interaction device and a controller; the mechanical arm comprises a mechanical arm main control computer, a mechanical arm tail end tracer and a mechanical arm tail end guider; the interaction device comprises an interaction device tracer; the optical positioning camera is used for acquiring poses of the interaction device tracer, the mechanical arm tail end tracer and the patient tracer in a camera coordinate system; the mechanical arm main control computer is used for acquiring the pose of the tail end of the mechanical arm in a mechanical arm base coordinate system; and the controller is used for realizing follow-up display of the two-dimensional section of the three-dimensional image along with the screw placement position of the interaction device according to the real-time position mapping and controlling the mechanical arm to aim at the screw placement path. According to the invention, more visual, more efficient and safer non-contact man-machine interaction is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of medical robot path planning and control, and in particular to an interactive human-machine collaborative path planning and robot arm follow-up control system and method. Background Art

[0002] In minimally invasive orthopedic surgery, robot-assisted navigation and positioning often uses an interactive control method based on medical image planning. That is, after completing the preoperative registration work, the positional relationship between the robot arm operation space, the three-dimensional anatomical space of the affected area, and the medical image space is obtained. The doctor manually plans the position of the nail in the two-dimensional medical image, maps the nail position to the three-dimensional anatomical space through the spatial registration matrix, and controls the robot arm to move to the nail position in the robot arm operation space.

[0003] However, the planning information of the above method is not intuitive enough. Since the incision of minimally invasive surgery is small, the doctor is required to have rich anatomical knowledge and surgical experience to confirm the precise position of the nail in a single two-dimensional medical image; secondly, this method requires a sufficient learning curve. The doctor needs to gradually adjust the parameters such as the nail angle, entry and exit points in the software planning interface, which requires the doctor to have a certain level of proficiency in software operation; in addition, the doctor needs to walk back and forth between the surgical area and the equipment, which not only reduces the efficiency of the operation, but also is not conducive to protecting the sterile environment of the operation.

[0004] Therefore, in robot-assisted nail placement surgery, it is necessary to study a new interactive control strategy to achieve more intuitive, efficient and safer intelligent interaction between doctors and robotic systems. Summary of the invention

[0005] In view of the above-mentioned defects or deficiencies in the prior art, the present invention provides an interactive human-machine collaborative path planning and robot arm follow-up control system and method, which can fully or partially solve the above-mentioned technical problems.

[0006] One aspect of the present invention provides an interactive human-machine collaborative path planning and robot arm follow-up control system, comprising: an optical positioning camera, a robot arm, a patient tracer, an interactor and a controller; the robot arm comprises a robot arm master control machine, a robot arm end tracer and a robot arm end guide, and the interactor comprises an interactor tracer; the optical positioning camera is used to obtain the position and posture of the interactor tracer, the robot arm end tracer and the patient tracer in the camera coordinate system; the robot arm master control machine is used to obtain the position and posture of the end of the robot arm in the robot arm base coordinate system; the controller , which is used to calculate the real-time position mapping of the interactor pinning line segment in the three-dimensional image space according to the positions of the interactor tracer, the robot arm end tracer and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and the position of the end of the current robot arm in the robot arm base coordinate system obtained by the robot arm master control computer, and realize the display of the two-dimensional section of the three-dimensional image following the pinning position of the interactor according to the real-time position mapping, and control the movement of the robot arm end guide until the pinning line segment of the robot arm end guide and the interactor pinning line segment coincide with each other in the robot arm base coordinate system.

[0007] In another aspect of the present invention, a control method based on the above-mentioned interactive human-machine collaborative path planning and robot arm follow-up control device is provided, including: commanding an optical positioning camera to obtain the position and posture of an interactor tracer, a robot arm end tracer and a patient tracer in a camera coordinate system; commanding a robot arm master control machine to obtain the position and posture of the end of the robot arm in a robot arm base coordinate system; commanding a controller to calculate a real-time position mapping of an interactor pin placement line segment in a three-dimensional image space according to the position and posture of the interactor tracer, the robot arm end tracer and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and the position and posture of the end of the current robot arm in the robot arm base coordinate system obtained by the robot arm master control machine, and according to the real-time position mapping, realize the display of a two-dimensional section of the three-dimensional image following the pin placement position of the interactor according to the real-time position mapping, and control the movement of a robot arm end guide until the pin placement line segment of the robot arm end guide and the interactor pin placement line segment coincide with each other in the robot arm base coordinate system.

[0008] The interactive human-machine collaborative path planning and robotic arm follow-up control system and method provided by the present invention can, on the one hand, realize the real-time follow-up display of the two-dimensional cross-sectional view of the medical image of the anatomical tissue of the affected part along the coordinate system of the interactor, thereby enhancing the intuitiveness of the affected part information during the operation; on the other hand, it can realize the nail placement path planning along the axial direction of the interactor, and cooperate with the remote device to control the robotic arm to execute to the position of the interactor, so as to realize the contactless interaction between the doctor and the robotic arm and improve the safety of the sterile surgical environment; the third aspect is that the reading of the affected part information, the planning of the nail placement, and the control and execution of the robotic arm do not require manual operation of the software, which shortens the doctor's learning curve and improves the efficiency of the operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Other features, objects and advantages of the present application will become more apparent by reading the detailed description of non-limiting embodiments made with reference to the following drawings: Figure 1 It is a real-life schematic diagram of an interactive human-machine collaborative path planning and a robot arm follow-up control system provided by an embodiment of the present application; Figure 2 It is a logical principle diagram of an interactive human-machine collaborative path planning and robot arm follow-up control system provided by an embodiment of the present application; Figure 3 It is a schematic diagram of the coordinate system and coordinate transformation of an interactive human-machine collaborative path planning and robot arm follow-up control system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0010] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0011] The terms used in the embodiments of the present invention are only for the purpose of describing specific embodiments, and are not intended to limit the present invention. The singular forms "a", "said" and "the" used in the embodiments of the present invention and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0012] It should be understood that although the terms first, second, third, etc. may be used to describe the acquisition modules in the embodiments of the present invention, the acquisition modules should not be limited to these terms. These terms are only used to distinguish the acquisition modules from each other.

[0013] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0014] It should be noted that the directional words such as "upper", "lower", "left", and "right" described in the embodiments of the present invention are described at the angles shown in the drawings and should not be understood as limiting the embodiments of the present invention. In addition, in the context, it should also be understood that when it is mentioned that an element is formed "on" or "under" another element, it can not only be formed directly "on" or "under" another element, but also be formed "on" or "under" another element indirectly through an intermediate element.

[0015] At present, orthopedic surgical robot-assisted minimally invasive nail placement surgery has the problem of poor human-computer interaction. Doctors need to operate the software to determine the nail placement position in the two-dimensional medical image. This method of planning information is not intuitive, the learning curve is long, the surgical efficiency is low, and the sterile environment protection is poor.

[0016] In order to solve the above technical problems existing in the prior art, see Figure 1 An embodiment of the present application provides an interactive human-machine collaborative path planning and robot arm servo control system 100, comprising: Optical positioning camera 101, robotic arm 102, patient tracer 103, interactor 104 and controller (not shown in the figure). Among them, robotic arm 102 includes robotic arm master 1021, robotic arm end tracer 1022 and robotic arm end guide 1023. Interactor 104 includes interactor tracer.

[0017] See also Figure 2 , the basic operation process of the control system 100 is as follows: The first step is to collect and register preoperative medical images, and then calibrate the interactor 104 to obtain the position conversion relationship between the interactor space and the medical three-dimensional image space; Step 2: The doctor holds the interactive device 104 and moves it freely on the patient's body (not limited to the adjustment of position, angle, and depth), and the two-dimensional image information of the affected area under the current section where the interactive device 104 is located can be displayed in real time on the software interface.

[0018] Step 3: Save the current position and posture of the interactor 104 as the position and posture parameters of the pin placement, and control the robot arm to execute to the pin placement position, thereby achieving rapid planning and navigation control.

[0019] The following describes in detail the connection relationship, structure, function and role of each component in the interactive human-machine collaborative path planning and robot arm servo control system 100.

[0020] The optical positioning camera 101 is used to obtain the positions and postures of the interactor tracer, the robot end tracer 1022 and the patient tracer 103 in the camera coordinate system.

[0021] The robot arm master control machine 1021 is used to obtain the position and posture of the end of the robot arm in the robot arm base coordinate system.

[0022] The controller is used to calculate the real-time position mapping of the interactor pinning line segment in the three-dimensional image space according to the positions of the interactor tracer, the robot arm end tracer 1022 and the patient tracer 103 in the camera coordinate system obtained by the optical positioning camera 101, and the position of the end of the current robot arm in the robot arm base coordinate system obtained by the robot arm master control computer 1021, and realize the display of the two-dimensional section of the three-dimensional image following the pinning position of the interactor according to the real-time position mapping, and control the movement of the robot arm end guide 1023 until the pinning line segment of the robot arm end guide and the interactor pinning line segment coincide with each other in the robot arm base coordinate system.

[0023] See also Figure 3 The controller is specifically used to: locate the interactor tracer, the end tracer 1022 of the robot arm and the patient tracer 103 in the camera coordinate system according to the optical positioning camera 101 The camera coordinate system is calculated based on the pose To the interactor tracer coordinate system The homogeneous transformation matrix , Camera coordinate system To the end coordinate system of the robot arm The homogeneous transformation matrix and the camera coordinate system To the patient tracer coordinate system The homogeneous transformation matrix In addition, the end of the current robotic arm is obtained by the robotic arm master control computer 1021 in the robotic arm base coordinate system. The pose under the robot arm base coordinate system is calculated To the end coordinate system of the robot arm The homogeneous transformation matrix .

[0024] See also Figure 3 The controller is further used to realize the display of the two-dimensional slice of the three-dimensional image following the position of the interactive device pin: Move the Interactor axis in the direction The end point on the , determine the starting point of the nail placement according to the nail placement length , get the interactor tracer coordinate system Lower interactor pin segment ; The patient tracer coordinate system is calculated according to the following formula: To the interactor tracer coordinate system The homogeneous transformation matrix : = ; in, is the camera coordinate system to the patient tracer coordinate system The homogeneous transformation matrix The inverse matrix of .

[0025] The patient tracer coordinate system is calculated according to the following formula: The interactor pins the segment below Starting point and end point Location: = ; = ; The interactive device pinning line segment in the three-dimensional image space is calculated according to the following formula Starting point and end point Location: = ; = ; in, The coordinate system of the patient tracer is obtained in advance To the 3D image space coordinate system The space transformation matrix It should be pointed out that the spatial transformation matrix can be calculated by the point cloud registration algorithm of the iterative closest point (ICP) for preoperative scanning of three-dimensional medical images. , I will not go into details here.

[0026] Finally, according to the starting point and end point The real-time position of the interactive device can be realized by image processing algorithm, and the two-dimensional section of the three-dimensional image can be displayed in accordance with the position of the interactive device pin.

[0027] See also Figure 3 , the controller is further used to realize nail placement planning and robot arm follow-up control: Calculate the robot arm base coordinate system according to the following formula To the interactor tracer coordinate system The homogeneous transformation matrix : = ; in, is the camera coordinate system To the end coordinate system of the robot arm The homogeneous transformation matrix The inverse matrix of Translate the interactor tracer coordinate system Lower interactor pin segment Convert to the robot base coordinate system The interactor pins the segment below : = ; = ; In the axial direction of the guide at the end of the robot arm On the top, the lower edge of the guide 1023 at the end of the robot arm is offset by a preset distance and set as the starting point for nail placement. , preferably at an offset of 80mm, the starting point of the nail The offset nail long distance is set as the nail placement end point , get the end coordinate system of the robot arm The nail placement line segment of the robot arm end guide ; The robot end coordinate system is calculated by the following formula The nail placement line segment of the robot arm end guide Convert to the robot base coordinate system The nail placement line segment of the robot arm end guide : = ; = ; Set the robot base coordinate system The interactor pins the segment below As the target position, the robot base coordinate system The nail placement line segment of the robot arm end guide As the current position, the robot arm inverse kinematics solution algorithm is used to control the movement of the robot arm end guide 1023 until the robot arm end guide nail placement line segment and interactor pin segments In the robot base coordinate system The nail placement planning of the position specified by the interactor 104 and the follow-up control of the robot arm 102 are realized.

[0028] The present embodiment provides an interactive human-machine collaborative path planning and robotic arm follow-up control system 100, which uses an interactor 104 to realize intuitive display of three-dimensional medical image sections, real-time planning of nail placement positions and follow-up control of the robotic arm, thereby achieving more intuitive, more efficient and safer contactless human-machine interaction.

[0029] Another embodiment of the present application also provides an interactive human-machine collaborative path planning and robot arm follow-up control method, which is based on the control system in the above-mentioned device embodiment and realizes the planning and control of the robot arm nail placement path.

[0030] Specifically, the interactive human-machine collaborative path planning and robot arm follow-up control method of this embodiment includes the following steps: Step S101, instructing the optical positioning camera to obtain the positions and postures of the interactor tracer, the robot arm end tracer and the patient tracer in the camera coordinate system; Step S102, instructing the robot arm master control machine to obtain the position and posture of the end of the robot arm in the robot arm base coordinate system; Step S103, instructs the controller to calculate the real-time position mapping of the interactor pinning line segment in the three-dimensional image space according to the positions of the interactor tracer, the robot arm end tracer and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and the position of the end of the current robot arm in the robot arm base coordinate system obtained by the robot arm master control computer, and realize the display of the two-dimensional section of the three-dimensional image following the pinning position of the interactor according to the real-time position mapping, and control the movement of the robot arm end guide until the pinning line segment of the robot arm end guide and the interactor pinning line segment coincide with each other in the robot arm base coordinate system.

[0031] Furthermore, step S103 further includes the following step S1031: The controller is instructed to calculate the homogeneous transformation matrix from the camera coordinate system to the interactor tracer coordinate system according to the poses of the interactor tracer, the end tracer of the robotic arm, and the patient tracer in the camera coordinate system obtained by the optical positioning camera. , the homogeneous transformation matrix from the camera coordinate system to the robot end coordinate system and the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system .

[0032] Furthermore, step S103 further includes the following step S1032: The controller calculates the homogeneous transformation matrix from the robot base coordinate system to the robot end coordinate system according to the current posture of the robot end in the robot base coordinate system obtained by the robot master. .

[0033] Furthermore, step S103 further includes the following step S1033: Instruct the controller to move the interactive axis in the direction The end point on the , determine the starting point of the nail placement according to the nail placement length , get the interactive device pinning line segment in the interactive device tracer coordinate system ; The homogeneous transformation matrix from the patient tracer coordinate system to the interactor tracer coordinate system is calculated according to the following formula: : = ; in, is the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system The inverse matrix of .

[0034] The interactive device pinning line segment in the patient tracer coordinate system is calculated according to the following formula Starting point and end point Location: = ; = ; The interactive device pinning line segment in the three-dimensional image space is calculated according to the following formula Starting point and end point Location: = ; = ; in, The spatial transformation matrix from the previously acquired patient tracer coordinate system to the 3D image space coordinate system The inverse matrix of According to the starting point and end point The real-time position of the interactive device can be realized, and the two-dimensional section of the three-dimensional image can be displayed following the position of the interactive device pin.

[0035] Furthermore, step S103 also includes the following step S1034: Let the controller calculate the homogeneous transformation matrix from the robot base coordinate system to the interactor tracer coordinate system according to the following formula: : = ; in, is the homogeneous transformation matrix from the camera coordinate system to the robot end coordinate system The inverse matrix of Place the interactor pin segment in the interactor tracer coordinate system The interactive device pinning line segment converted to the robot base coordinate system : = ; = ; In the axial direction of the guide at the end of the robot arm On the left, set the preset distance of the bottom edge of the guide at the end of the robot arm as the starting point for nail placement. , the starting point of the nail The offset nail long distance is set as the nail placement end point , get the nail placement line segment of the manipulator end guide in the manipulator end coordinate system ; The nail line segment of the manipulator end guide in the manipulator end coordinate system is calculated by the following formula The nail placement line segment of the end guide of the manipulator converted into the manipulator base coordinate system : = ; = ; Place the interactor in the robot base coordinate system As the target position, place the nail line segment of the end guide of the robot arm in the robot arm base coordinate system As the current position, control the movement of the end guide of the robot arm until the end guide of the robot arm places the nail line segment and interactor pin segments Overlap in the robot base coordinate system.

[0036] It should be noted that the interactive human-machine collaborative path planning and robotic arm follow-up control method provided in this embodiment can realize the operation of the control system 100 in the device embodiment through computer programs or field programmable logic hardware. Its implementation principle and technical effects are similar to the functions of each component in the device embodiment, and will not be repeated here.

[0037] The above description is only a preferred embodiment of the present invention. Those skilled in the art should understand that the scope of disclosure involved in the present invention is not limited to the technical solution formed by a specific combination of the above technical features, but also should cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present invention (but not limited to) to form a technical solution.

Claims

1. An interactive human-machine collaborative path planning and robot arm servo control system, characterized in that: include: optical positioning cameras, robotic arms, patient trackers, interactors, and controllers; The robot arm includes a robot arm master, a robot arm end tracer and a robot arm end guide, and the interactor includes an interactor tracer; The optical positioning camera is used to obtain the position and posture of the interactor tracer, the robot end tracer and the patient tracer in the camera coordinate system; The robot arm master control machine is used to obtain the position and posture of the end of the robot arm in the robot arm base coordinate system; The controller is used to calculate the real-time position mapping of the interactor pinning line segment in the three-dimensional image space according to the positions of the interactor tracer, the robot arm end tracer and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and the position of the end of the current robot arm in the robot arm base coordinate system obtained by the robot arm master control computer, realize the display of the two-dimensional section of the three-dimensional image following the pinning position of the interactor according to the real-time position mapping, and control the movement of the robot arm end guide until the pinning line segment of the robot arm end guide and the interactor pinning line segment coincide with each other in the robot arm base coordinate system.

2. The interactive human-machine collaborative path planning and robot arm servo control system according to claim 1, characterized in that: The controller is used to: According to the poses of the interactor tracer, the end tracer of the robotic arm and the patient tracer in the camera coordinate system obtained by the optical positioning camera, the homogeneous transformation matrix from the camera coordinate system to the interactor tracer coordinate system is calculated. , the homogeneous transformation matrix from the camera coordinate system to the robot end coordinate system and the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system .

3. The interactive human-machine collaborative path planning and robot arm servo control system according to claim 2, characterized in that: The controller is used to: According to the current position of the end of the robot arm in the robot arm base coordinate system obtained by the robot arm master, the homogeneous transformation matrix from the robot arm base coordinate system to the robot arm end coordinate system is calculated .

4. The interactive human-machine collaborative path planning and robot arm servo control system according to claim 3, characterized in that: The controller is used to: Set the end point of the interactor in the axial direction as the pinning end point , determine the starting point of the nail placement according to the nail placement length , get the interactive device pinning line segment in the interactive device tracer coordinate system ; The homogeneous transformation matrix from the patient tracer coordinate system to the interactor tracer coordinate system is calculated according to the following formula: : = ； in, is the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system The inverse matrix of The interactive device pinning line segment in the patient tracer coordinate system is calculated according to the following formula Starting point and end point Location: = ； = ； The interactive device pinning line segment in the three-dimensional image space is calculated according to the following formula Starting point and end point Location: = ； = ； in, The spatial transformation matrix from the previously acquired patient tracer coordinate system to the 3D image space coordinate system The inverse matrix of According to the starting point and end point The real-time position of the interactive device can be realized, and the two-dimensional section of the three-dimensional image can be displayed following the position of the interactive device pin.

5. The interactive human-machine collaborative path planning and robot arm servo control system according to claim 4, characterized in that: The controller is used to: The homogeneous transformation matrix from the robot base coordinate system to the interactor tracer coordinate system is calculated according to the following formula: : = ； in, is the homogeneous transformation matrix from the camera coordinate system to the robot end coordinate system The inverse matrix of Place the interactor pin segment in the interactor tracer coordinate system The interactive device pinning line segment converted to the robot base coordinate system : = ； = ； In the axial direction of the guide at the end of the robot arm, set the preset distance offset from the lower edge of the guide at the end of the robot arm as the starting point for nail placement. , the starting point of the nail The offset nail long distance is set as the nail placement end point , get the nail placement line segment of the manipulator end guide in the manipulator end coordinate system ; The nail line segment of the manipulator end guide in the manipulator end coordinate system is calculated by the following formula The nail placement line segment of the end guide of the manipulator converted into the manipulator base coordinate system : = ； = ； Place the interactor in the robot base coordinate system As the target position, place the nail line segment of the end guide of the robot arm in the robot arm base coordinate system As the current position, control the movement of the end guide of the robot arm until the end guide of the robot arm places the nail line segment and interactor pin segments Overlap in the robot base coordinate system.

6. A control method for interactive human-machine collaborative path planning and robot arm servo control system according to any one of claims 1 to 5, characterized in that: include: Using an optical positioning camera to obtain the positions and postures of the interactor tracer, the end tracer of the robotic arm, and the patient tracer in the camera coordinate system; Instruct the robot arm master to obtain the position and posture of the end of the robot arm in the robot arm base coordinate system; The controller is instructed to calculate the real-time position mapping of the interactor pinning line segment in the three-dimensional image space according to the positions of the interactor tracer, the robot arm end tracer and the patient tracer in the camera coordinate system obtained by the optical positioning camera, and the position of the end of the current robot arm in the robot arm base coordinate system obtained by the robot arm master control computer. According to the real-time position mapping, the two-dimensional section of the three-dimensional image is displayed following the pinning position of the interactor, and the movement of the robot arm end guide is controlled until the pinning line segment of the robot arm end guide and the interactor pinning line segment coincide with each other in the robot arm base coordinate system.

7. The control method according to claim 6, characterized in that: Also includes: The controller is instructed to calculate the homogeneous transformation matrix from the camera coordinate system to the interactor tracer coordinate system according to the poses of the interactor tracer, the end tracer of the robotic arm, and the patient tracer in the camera coordinate system obtained by the optical positioning camera. , the homogeneous transformation matrix from the camera coordinate system to the robot end coordinate system and the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system .

8. The control method according to claim 7, characterized in that: Also includes: The controller calculates the homogeneous transformation matrix from the robot base coordinate system to the robot end coordinate system according to the current posture of the robot end in the robot base coordinate system obtained by the robot master. .

9. The control method according to claim 8, characterized in that: Also includes: Let the controller take the end point of the interactor in the axial direction as the pinning end point , determine the starting point of nail placement according to the nail placement length , get the interactive device pinning line segment in the interactive device tracer coordinate system ; The homogeneous transformation matrix from the patient tracer coordinate system to the interactor tracer coordinate system is calculated according to the following formula: : = ； in, is the homogeneous transformation matrix from the camera coordinate system to the patient tracer coordinate system The inverse matrix of The interactive device pinning line segment in the patient tracer coordinate system is calculated according to the following formula Starting point and end point Location: = ； = ； The interactive device pinning line segment in the three-dimensional image space is calculated according to the following formula Starting point and end point Location: = ； = ； in, The spatial transformation matrix from the previously acquired patient tracer coordinate system to the 3D image space coordinate system The inverse matrix of According to the starting point and end point The real-time position of the interactive device can be realized, and the two-dimensional section of the three-dimensional image can be displayed following the position of the interactive device pin.

10. The control method according to claim 9, characterized in that: Also includes: Let the controller calculate the homogeneous transformation matrix from the robot base coordinate system to the interactor tracer coordinate system according to the following formula: : = ； in, is the homogeneous transformation matrix from the camera coordinate system to the robot end coordinate system The inverse matrix of Place the interactor pin segment in the interactor tracer coordinate system The interactive device pinning line segment converted to the robot base coordinate system : = ； = ； In the axial direction of the guide at the end of the robot arm, set the preset distance offset from the lower edge of the guide at the end of the robot arm as the starting point for nail placement. , the starting point of the nail The offset nail long distance is set as the nail placement end point , get the nail placement line segment of the manipulator end guide in the manipulator end coordinate system ; The nail line segment of the manipulator end guide in the manipulator end coordinate system is calculated by the following formula The nail placement line segment of the end guide of the manipulator converted into the manipulator base coordinate system : = ； = ； Place the interactor in the robot base coordinate system As the target position, place the nail line segment of the end guide of the robot arm in the robot arm base coordinate system As the current position, control the movement of the end guide of the robot arm until the end guide of the robot arm places the nail line segment and interactor pin segments Overlap in the robot base coordinate system.

Images